1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179
|
Batch Management
================
For more information on the Azure Batch service, check out the `Batch Documentation <https://azure.microsoft.com/en-us/documentation/services/batch/>`__.
For working samples, see the `Batch samples repo <https://github.com/Azure/azure-batch-samples/tree/master/Python>`__.
Create the Batch Management client
----------------------------------
The following code creates an instance of the management client.
You will need to provide your ``subscription_id`` which can be retrieved
from `your subscription list <https://manage.windowsazure.com/#Workspaces/AdminTasks/SubscriptionMapping>`__.
See :doc:`Resource Management Authentication <resourcemanagementauthentication>`
for details on handling Azure Active Directory authentication with the Python SDK, and creating a ``Credentials`` instance.
.. code:: python
from azure.mgmt.batch import BatchManagementClient
from azure.common.credentials import UserPassCredentials
# Replace this with your subscription id
subscription_id = '33333333-3333-3333-3333-333333333333'
# See above for details on creating different types of AAD credentials
credentials = UserPassCredentials(
'user@domain.com', # Your user
'my_password', # Your password
)
batch_client = BatchManagementClient(
credentials,
subscription_id
)
Registration
------------
Some operations in the ARM APIs require a one-time registration of the provider with your subscription.
Use the following code to do the registration. You can use the same
credentials you created in the previous section.
.. code:: python
from azure.mgmt.resource.resources import ResourceManagementClient
resource_client = ResourceManagementClient(
credentials,
subscription_id
)
resource_client.providers.register('Microsoft.Batch')
Create a Batch Account
----------------------
A Batch Account will need to be created in a specified location and resource group.
The default Batch Account quota is 1 per location per subscription, but can be increased to a maximum of 50.
Please contact support if you require a quota increase.
For more information on resource groups and resource management, see :doc:`Resource Management<resourcemanagement>`.
In order to make use of Application Packages, a storage account will need to be linked to the Batch Account. A linked storage account is known as 'auto-storage'.
A storage account can be created with the :doc:`Storage Resource Management Client <resourcemanagementstorage>`.
.. code:: python
# Create a Resource Group (or use an existing one)
import azure.mgmt.resource
RESOURCE_GROUP = 'python_sdk'
LOCATION = 'westus'
resource_client = = azure.mgmt.storage.ResourceManagementClient(
azure.mgmt.resources.ResourceManagementClientConfiguration(
credentials, # See section above
subscription_id
)
)
group = azure.mgmt.resource.resources.models.ResourceGroup(
name=RESOURCE_GROUP,
location=LOCATION
)
resource_client.resource_groups.create_or_update(
RESOURCE_GROUP,
group,
)
# Create a storage account for 'auto-storage' (or use an existing one)
import azure.mgmt.storage
storage_client = azure.mgmt.storage.StorageManagementClient(
azure.mgmt.storage.StorageManagementClientConfiguration(
credentials, # See section above
subscription_id
)
)
storage_params = azure.mgmt.storage.models.StorageAccountCreateParameters(
location=LOCATION,
account_type=azure.mgmt.storage.models.AccountType.standard_lrs
)
creating = storage_client.storage_accounts.create(
RESOURCE_GROUP,
'pythonstorageaccount',
storage_params
)
creating.wait()
# Create a Batch Account, specifying the storage account we want to link
storage_resource = '/subscriptions/{}/resourceGroups/{}/providers/Microsoft.Storage/storageAccounts/{}'.format(
subscription_id,
RESOURCE_GROUP,
'pythonstorageaccount'
)
batch_account = azure.mgmt.batch.models.BatchAccountCreateParameters(
location=LOCATION,
auto_storage=azure.mgmt.batch.models.AutoStorageBaseProperties(storage_resource)
)
creating = batch_client.account.create('MyBatchAccount', LOCATION, batch_account)
creating.wait()
Account keys (used for authenticating the :doc:`Batch Client <batch>`) can be retrieved or regenerated.
.. code:: python
batch_client.account.regenerate_key(
RESOURCE_GROUP,
'MyBatchAccount',
'Primary'
)
accounts_keys = batch_client.account.list_keys(RESOURCE_GROUP, 'MyBatchAccount')
print('Updated primary key: {}'.format(accounts_keys.primary))
Application Packages
--------------------
Application packages can be configured to be used by the the :doc:`Batch Client <batch>` for running tasks.
An Application can have multiple versioned packages (zipped directories containing the application to be executed on the Compute Node) associated with it.
You can find an overview of this feature in this article on `application deployment with Azure Batch Applications <https://azure.microsoft.com/en-us/documentation/articles/batch-application-packages/>`__.
.. code:: python
# Create Application reference
batch_client.application.add(
RESOURCE_GROUP,
'MyBatchAccount',
'MyApplicationId'
allow_updates=True,
display_name='Test App v1'
)
# Add a new package to the application
package_ref = batch_client.application.add_application_package(
RESOURCE_GROUP,
'MyBatchAccount',
'MyApplicationId',
'v1.0'
)
# Upload a zip directory for the created package reference
import requests
with open('my_application.zip', 'rb') as app_data:
headers = {'x-ms-blob-type': 'BlockBlob'}
requests.put(package_ref.storage_url, headers=headers, data=app_data.read())
# In order to use the application in a job, the package must be activated
batch_client.application.activate_application_package(
RESOURCE_GROUP,
'MyBatchAccount',
'MyApplicationId',
'v1.0',
'zip'
)
|