| title | description | ms.topic | ms.date |
|---|---|---|---|
Migrate Batch pool configuration from Cloud Services to Virtual Machines |
Learn how to update your pool configuration to the latest and recommended configuration |
how-to |
09/03/2021 |
Currently, Batch pools can be created using either virtualMachineConfiguration or cloudServiceConfiguration. We recommend using Virtual Machine Configuration only, as this configuration supports all Batch capabilities.
Cloud Services Configuration pools don't support some of the current Batch features, and won't support any newly-added features. You won't be able to create new 'CloudServiceConfiguration' pools or add new nodes to existing pools after February 29, 2024.
If your Batch solutions currently use 'cloudServiceConfiguration' pools, we recommend changing to 'virtualMachineConfiguration' as soon as possible. This will enable you to benefit from all Batch capabilities, such as an expanded selection of VM series, Linux VMs, containers, Azure Resource Manager virtual networks, and node disk encryption.
You can't switch an existing active pool that uses 'cloudServiceConfiguration' to use 'virtualMachineConfiguration'. Instead, you'll need to create new pools. Once you've created your new 'virtualMachineConfiguration' pools and replicated all of your jobs and tasks, you can delete the old 'cloudServiceConfiguration'pools that you're no longer using.
All Batch APIs, command-line tools, the Azure portal, and the Batch Explorer UI let you create pools using 'virtualMachineConfiguration'.
For a walkthrough of the process of creating pools that use 'virtualMachineConfiguration, see the .NET tutorial or the Python tutorial.
Some of the key differences between the two configurations include:
-
'cloudServiceConfiguration' pool nodes only use Windows OS. 'virtualMachineConfiguration' pools can use either Linux or Windows OS.
-
Compared to 'cloudServiceConfiguration' pools, 'virtualMachineConfiguration' pools have a richer set of capabilities, such as container support, data disks, and disk encryption.
-
Pool and node startup and delete times may differ slightly between 'cloudServiceConfiguration' pools and 'virtualMachineConfiguration' pools.
-
'virtualMachineConfiguration' pool nodes utilize managed OS disks. The managed disk type that is used for each node depends on the VM size chosen for the pool. If a 's' VM size is specified for the pool, for example 'Standard_D2s_v3', then a premium SSD is used. If a 'non-s' VM size is specified, for example 'Standard_D2_v3', then a standard HDD is used.
[!IMPORTANT] As with Virtual Machines and Virtual Machine Scale Sets, the OS managed disk used for each node incurs a cost, which is additional to the cost of the VMs. 'virtualMachineConfiguration' pools can use ephemeral OS disks, which create the OS disk on the VM cache or temporary SSD, to avoid extra costs associated with managed disks.There is no OS disk cost for 'cloudServiceConfiguration' nodes, as the OS disk is created on the nodes local SSD.
Azure Batch pools can be used to run Data Factory custom activities. Any 'cloudServiceConfiguration' pools used to run custom activities will need to be deleted and new 'virtualMachineConfiguration' pools created.
When creating your new pools to run Data Factory custom activities, follow these practices:
- Pause all pipelines before creating the new pools and deleting the old ones to ensure no executions will be interrupted.
- The same pool id can be used to avoid linked service configuration changes.
- Resume pipelines when new pools have been created.
For more information about using Azure Batch to run Data Factory custom activities, see Azure Batch linked service and Custom activities in a Data Factory pipeline
- Learn more about pool configurations.
- Learn more about pool best practices.
- See the REST API reference for pool addition and virtualMachineConfiguration.