Skip to content

Files

Latest commit

alexbuckgitalexbuckgit
[BULK UPDATE] DocuTune - Apply event-tier1-build-2022 ms.custom tag t…
May 23, 2022
55762f7 · May 23, 2022

History

History
176 lines (117 loc) · 10.2 KB

how-to-use-managed-online-endpoint-studio.md

File metadata and controls

176 lines (117 loc) · 10.2 KB
title titleSuffix description services ms.service ms.subservice ms.topic ms.custom ms.author author ms.reviewer ms.date
Use managed online endpoints in the studio
Azure Machine Learning
Learn how to create and use managed online endpoints using the Azure Machine Learning studio.
machine-learning
machine-learning
mlops
how-to
how-to, managed online endpoints, devplatv2, studio, event-tier1-build-2022
ssambare
shivanissambare
laobri
10/21/2021

Create and use managed online endpoints in the studio

Learn how to use the studio to create and manage your managed online endpoints in Azure Machine Learning. Use managed online endpoints to streamline production-scale deployments. For more information on managed online endpoints, see What are endpoints.

In this article, you learn how to:

[!div class="checklist"]

  • Create a managed online endpoint
  • View managed online endpoints
  • Add a deployment to a managed online endpoint
  • Update managed online endpoints
  • Delete managed online endpoints and deployments

Prerequisites

Create a managed online endpoint

Use the studio to create a managed online endpoint directly in your browser. When you create a managed online endpoint in the studio, you must define an initial deployment. You cannot create an empty managed online endpoint.

  1. Go to the Azure Machine Learning studio.
  2. In the left navigation bar, select the Endpoints page.
  3. Select + Create.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/endpoint-create-managed-online-endpoint.png" lightbox="media/how-to-create-managed-online-endpoint-studio/endpoint-create-managed-online-endpoint.png" alt-text="A screenshot for creating managed online endpoint from the Endpoints tab.":::

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/online-endpoint-wizard.png" lightbox="media/how-to-create-managed-online-endpoint-studio/online-endpoint-wizard.png" alt-text="A screenshot of a managed online endpoint create wizard.":::

Follow the setup wizard to configure your managed online endpoint.

  1. You can use our sample model and scoring script from https://github.com/Azure/azureml-examples/tree/main/cli/endpoints/online/model-1
  2. On the Environment step of the wizard, you can select the AzureML-sklearn-0.24.1-ubuntu18.04-py37-cpu-inference curated environment.

You can also create a managed online endpoint from the Models page in the studio. This is an easy way to add a model to an existing managed online deployment.

  1. Go to the Azure Machine Learning studio.
  2. In the left navigation bar, select the Models page.
  3. Select a model by checking the circle next to the model name.
  4. Select Deploy > Deploy to real-time endpoint.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/deploy-from-models-page.png" lightbox="media/how-to-create-managed-online-endpoint-studio/deploy-from-models-page.png" alt-text="A screenshot of creating a managed online endpoint from the Models UI.":::

View managed online endpoints

You can view your managed online endpoints in the Endpoints page. Use the endpoint details page to find critical information including the endpoint URI, status, testing tools, activity monitors, deployment logs, and sample consumption code:

  1. In the left navigation bar, select Endpoints.
  2. (Optional) Create a Filter on Compute type to show only Managed compute types.
  3. Select an endpoint name to view the endpoint detail page.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/managed-endpoint-details-page.png" lightbox="media/how-to-create-managed-online-endpoint-studio/managed-endpoint-details-page.png" alt-text="Screenshot of managed endpoint details view.":::

Test

Use the Test tab in the endpoints details page to test your managed online deployment. Enter sample input and view the results.

  1. Select the Test tab in the endpoint's detail page.
  2. Use the dropdown to select the deployment you want to test.
  3. Enter sample input.
  4. Select Test.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/test-deployment.png" lightbox="media/how-to-create-managed-online-endpoint-studio/test-deployment.png" alt-text="A screenshot of testing a deployment by providing sample data, directly in your browser.":::

Monitoring

Use the Monitoring tab to see high-level activity monitor graphs for your managed online endpoint.

To use the monitoring tab, you must select "Enable Application Insight diagnostic and data collection" when you create your endpoint.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/monitor-endpoint.png" lightbox="media/how-to-create-managed-online-endpoint-studio/monitor-endpoint.png" alt-text="A screenshot of monitoring endpoint-level metrics in the studio.":::

For more information on how viewing additional monitors and alerts, see How to monitor managed online endpoints.

Add a deployment to a managed online endpoint

You can add a deployment to your existing managed online endpoint.

From the Endpoint details page

  1. Select + Add Deployment button in the endpoint details page.
  2. Follow the instructions to complete the deployment.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/add-deploy-option-from-endpoint-page.png" lightbox="media/how-to-create-managed-online-endpoint-studio/add-deploy-option-from-endpoint-page.png" alt-text="A screenshot of Add deployment option from Endpoint details page.":::

Alternatively, you can use the Models page to add a deployment:

  1. In the left navigation bar, select the Models page.
  2. Select a model by checking the circle next to the model name.
  3. Select Deploy > Deploy to real-time endpoint.
  4. Choose to deploy to an existing managed online endpoint.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/select-existing-managed-endpoints.png" lightbox="media/how-to-create-managed-online-endpoint-studio/select-existing-managed-endpoints.png" alt-text="A screenshot of Add deployment option from Models page.":::

Note

You can adjust the traffic balance between deployments in an endpoint when adding a new deployment.

:::image type="content" source="media/how-to-create-managed-online-endpoint-studio/adjust-deployment-traffic.png" lightbox="media/how-to-create-managed-online-endpoint-studio/adjust-deployment-traffic.png" alt-text="A screenshot of how to use sliders to control traffic distribution across multiple deployments.":::

Update managed online endpoints

You can update deployment traffic percentage and instance count from Azure Machine Learning studio.

Update deployment traffic allocation

Use deployment traffic allocation to control the percentage of incoming of requests going to each deployment in an endpoint.

  1. In the endpoint details page, Select Update traffic.
  2. Adjust your traffic and select Update.

Tip

The Total traffic percentage must sum to either 0% (to disable traffic) or 100% (to enable traffic).

Update deployment instance count

Use the following instructions to scale an individual deployment up or down by adjusting the number of instances:

  1. In the endpoint details page. Find the card for the deployment you want to update.
  2. Select the edit icon in the deployment detail card.
  3. Update the instance count.
  4. Select Update.

Delete managed online endpoints and deployments

Learn how to delete an entire managed online endpoint and it's associated deployments. Or, delete an individual deployment from a managed online endpoint.

Delete a managed online endpoint

Deleting a managed online endpoint also deletes any deployments associated with it.

  1. Go to the Azure Machine Learning studio.
  2. In the left navigation bar, select the Endpoints page.
  3. Select an endpoint by checking the circle next to the model name.
  4. Select Delete.

Alternatively, you can delete a managed online endpoint directly in the endpoint details page.

Delete an individual deployment

Use the following steps to delete an individual deployment from a managed online endpoint. This does affect the other deployments in the managed online endpoint:

Note

You cannot delete a deployment that has allocated traffic. You must first set traffic allocation for the deployment to 0% before deleting it.

  1. Go to the Azure Machine Learning studio.
  2. In the left navigation bar, select the Endpoints page.
  3. Select your managed online endpoint.
  4. In the endpoint details page, find the deployment you want to delete.
  5. Select the delete icon.

Next steps

In this article, you learned how to use Azure Machine Learning managed online endpoints. See these next steps: