| title | titleSuffix | description | services | ms.service | ms.subservice | ms.topic | author | ms.author | ms.reviewer | ms.date | ms.custom |
|---|---|---|---|---|---|---|---|---|---|---|---|
How to use batch endpoints in studio |
Azure Machine Learning |
In this article, learn how to create a batch endpoint in Azure Machine Learning studio. Batch endpoints are used to continuously batch score large data. |
machine-learning |
machine-learning |
mlops |
how-to |
shivanissambare |
ssambare |
larryfr |
04/26/2022 |
how-to, studio, managed-batch-endpoints, event-tier1-build-2022 |
In this article, you learn how to use batch endpoints to do batch scoring in Azure Machine Learning studio. For more, see What are Azure Machine Learning endpoints?.
In this article, you learn about:
[!div class="checklist"]
- Create a batch endpoint with a no-code experience for MLflow model
- Check batch endpoint details
- Start a batch scoring job
- Overview of batch endpoint features in Azure machine learning studio
-
An Azure subscription - If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.
-
The example repository - Clone the AzureML Example repository. This article uses the assets in
/cli/endpoints/batch. -
A compute target where you can run batch scoring workflows. For more information on creating a compute target, see Create compute targets in Azure Machine Learning studio.
-
Register machine learning model.
There are two ways to create Batch Endpoints in Azure Machine Learning studio:
-
From the Endpoints page, select Batch Endpoints and then select + Create.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/create-batch-endpoints.png" alt-text="Screenshot of creating a batch endpoint/deployment from Endpoints page":::
OR
-
From the Models page, select the model you want to deploy and then select Deploy to batch endpoint.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/models-page-deployment.png" alt-text="Screenshot of creating a batch endpoint/deployment from Models page":::
Tip
If you're using an MLflow model, you can use no-code batch endpoint creation. That is, you don't need to prepare a scoring script and environment, both can be auto generated. For more, see Train and track ML models with MLflow and Azure Machine Learning.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/mlflow-model-wizard.png" alt-text="Screenshot of deploying an MLflow model":::
Complete all the steps in the wizard to create a batch endpoint and deployment.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/review-batch-wizard.png" alt-text="Screenshot of batch endpoints/deployment review screen":::
After a batch endpoint is created, select it from the Endpoints page to view the details.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/batch-endpoint-details.png" alt-text="Screenshot of the check batch endpoints and deployment details":::
A batch scoring workload runs as an offline job. By default, batch scoring stores the scoring outputs in blob storage. You can also configure the outputs location and overwrite some of the settings to get the best performance.
-
Select + Create job:
:::image type="content" source="media/how-to-use-batch-endpoints-studio/create-batch-job.png" alt-text="Screenshot of the create job option to start batch scoring":::
-
You can update the default deployment while submitting a job from the drop-down:
:::image type="content" source="media/how-to-use-batch-endpoints-studio/job-setting-batch-scoring.png" alt-text="Screenshot of using the deployment to submit a batch job":::
Some settings can be overwritten when you start a batch scoring job. For example, you might overwrite settings to make better use of the compute resource, or to improve performance. To override settings, select Override deployment settings and provide the settings. For more information, see Use batch endpoints.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/overwrite-setting.png" alt-text="Screenshot of the overwrite setting when starting a batch job":::
You have two options to specify the data inputs in Azure machine learning studio:
-
Use a registered dataset:
[!NOTE] During Preview, only FileDataset is supported.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/select-dataset-for-job.png" alt-text="Screenshot of selecting registered dataset as an input option":::
OR
-
Use a datastore:
You can specify AML registered datastore or if your data is publicly available, specify the public path.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/select-datastore-job.png" alt-text="Screenshot of selecting datastore as an input option":::
By default, the batch scoring results are stored in the default blob store for the workspace. Results are in a folder named after the job name (a system-generated GUID).
To change where the results are stored, providing a blob store and output path when you start a job.
Important
You must use a unique output location. If the output file exists, the batch scoring job will fail.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/configure-output-location.png" alt-text="Screenshot of optionally configuring output location":::
To see a summary of all the submitted jobs for an endpoint, select the endpoint and then select the Runs tab.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/summary-jobs.png" alt-text="Screenshot of summary of jobs submitted to a batch endpoint":::
To learn how to view the scoring results, see Use batch endpoints.
In Azure machine learning studio, there are two ways to add a deployment to an existing batch endpoint:
-
From the Endpoints page, select the batch endpoint to add a new deployment to. Select + Add deployment, and complete the wizard to add a new deployment.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/add-deployment-option.png" alt-text="Screenshot of add new deployment option":::
OR
-
From the Models page, select the model you want to deploy. Then select Deploy to batch endpoint option from the drop-down. In the wizard, on the Endpoint screen, select Existing. Complete the wizard to add the new deployment.
:::image type="content" source="media/how-to-use-batch-endpoints-studio/add-deployment-models-page.png" alt-text="Screenshot of selecting an existing batch endpoint to add new deployment":::
If an endpoint has multiple deployments, one of the deployments is the default. The default deployment receives 100% of the traffic to the endpoint. To change the default deployment, use the following steps:
- Select the endpoint from the Endpoints page.
- Select Update default deployment. From the Details tab, select the deployment you want to set as default and then select Update. :::image type="content" source="media/how-to-use-batch-endpoints-studio/update-default-deployment.png" alt-text="Screenshot of updating default deployment":::
To delete an endpoint, select the endpoint from the Endpoints page and then select delete.
Warning
Deleting an endpoint also deletes all deployments to that endpoint.
To delete a deployment, select the endpoint from the Endpoints page, select the deployment, and then select delete.
In this article, you learned how to create and call batch endpoints. See these other articles to learn more about Azure Machine Learning: