| title | titleSuffix | description | services | ms.service | ms.subservice | ms.author | ms.reviewer | author | ms.date | ms.topic | ms.custom |
|---|---|---|---|---|---|---|---|---|---|---|---|
Access Azure resources from an online endpoint |
Azure Machine Learning |
Securely access Azure resources for your machine learning model deployment from an online endpoint with a system-assigned or user-assigned managed identity. |
machine-learning |
machine-learning |
core |
seramasu |
larryfr |
rsethur |
04/07/2022 |
how-to |
devplatv2, cliv2, event-tier1-build-2022 |
[!INCLUDE cli v2]
Learn how to access Azure resources from your scoring script with an online endpoint and either a system-assigned managed identity or a user-assigned managed identity.
Managed endpoints allow Azure Machine Learning to manage the burden of provisioning your compute resource and deploying your machine learning model. Typically your model needs to access Azure resources such as the Azure Container Registry or your blob storage for inferencing; with a managed identity you can access these resources without needing to manage credentials in your code. Learn more about managed identities.
This guide assumes you don't have a managed identity, a storage account or an online endpoint. If you already have these components, skip to the give access permission to the managed identity section.
-
To use Azure Machine Learning, you must have an Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.
-
Install and configure the Azure CLI and ML (v2) extension. For more information, see Install, set up, and use the 2.0 CLI.
-
An Azure Resource group, in which you (or the service principal you use) need to have
User Access AdministratorandContributoraccess. You'll have such a resource group if you configured your ML extension per the above article. -
An Azure Machine Learning workspace. You'll have a workspace if you configured your ML extension per the above article.
-
A trained machine learning model ready for scoring and deployment. If you are following along with the sample, a model is provided.
-
If you haven't already set the defaults for the Azure CLI, save your default settings. To avoid passing in the values for your subscription, workspace, and resource group multiple times, run this code:
az account set --subscription <subscription ID> az configure --defaults workspace=<Azure Machine Learning workspace name> group=<resource group> -
To follow along with the sample, clone the samples repository
git clone https://github.com/Azure/azureml-examples --depth 1 cd azureml-examples/cli
- The identity for an endpoint is immutable. During endpoint creation, you can associate it with a system-assigned identity (default) or a user-assigned identity. You can't change the identity after the endpoint has been created.
To deploy an online endpoint with the CLI, you need to define the configuration in a YAML file. For more information on the YAML schema, see online endpoint YAML reference document.
The YAML files in the following examples are used to create online endpoints.
The following YAML example is located at endpoints/online/managed/managed-identities/1-sai-create-endpoint. The file,
- Defines the name by which you want to refer to the endpoint,
my-sai-endpoint. - Specifies the type of authorization to use to access the endpoint,
auth-mode: key.
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/1-sai-create-endpoint.yml":::
This YAML example, 2-sai-deployment.yml,
- Specifies that the type of endpoint you want to create is an
onlineendpoint. - Indicates that the endpoint has an associated deployment called
blue. - Configures the details of the deployment such as, which model to deploy and which environment and scoring script to use.
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/2-sai-deployment.yml":::
The following YAML example is located at endpoints/online/managed/managed-identities/1-uai-create-endpoint. The file,
- Defines the name by which you want to refer to the endpoint,
my-uai-endpoint. - Specifies the type of authorization to use to access the endpoint,
auth-mode: key. - Indicates the identity type to use,
type: user_assigned
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/1-uai-create-endpoint.yml":::
This YAML example, 2-sai-deployment.yml,
- Specifies that the type of endpoint you want to create is an
onlineendpoint. - Indicates that the endpoint has an associated deployment called
blue. - Configures the details of the deployment such as, which model to deploy and which environment and scoring script to use.
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/managed-identities/2-uai-deployment.yml":::
Configure the variable names for the workspace, workspace location, and the endpoint you want to create for use with your deployment.
The following code exports these values as environment variables in your endpoint:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="set_variables" :::
Next, specify what you want to name your blob storage account, blob container, and file. These variable names are defined here, and are referred to in az storage account create and az storage container create commands in the next section.
The following code exports those values as environment variables:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="configure_storage_names" :::
After these variables are exported, create a text file locally. When the endpoint is deployed, the scoring script will access this text file using the system-assigned managed identity that's generated upon endpoint creation.
Decide on the name of your endpoint, workspace, workspace location and export that value as an environment variable:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="set_variables" :::
Next, specify what you want to name your blob storage account, blob container, and file. These variable names are defined here, and are referred to in az storage account create and az storage container create commands in the next section.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="configure_storage_names" :::
After these variables are exported, create a text file locally. When the endpoint is deployed, the scoring script will access this text file using the user-assigned managed identity used in the endpoint.
Decide on the name of your user identity name, and export that value as an environment variable:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="set_user_identity_name" :::
To access Azure resources, create a system-assigned or user-assigned managed identity for your online endpoint.
When you create an online endpoint, a system-assigned managed identity is automatically generated for you, so no need to create a separate one.
To create a user-assigned managed identity, use the following:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_user_identity" :::
For this example, create a blob storage account and blob container, and then upload the previously created text file to the blob container. This is the storage account and blob container that you'll give the online endpoint and managed identity access to.
First, create a storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="create_storage_account" :::
Next, create the blob container in the storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="create_storage_container" :::
Then, upload your text file to the blob container.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="upload_file_to_storage" :::
First, create a storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_storage_account" :::
You can also retrieve an existing storage account ID with the following.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_storage_account_id" :::
Next, create the blob container in the storage account.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_storage_container" :::
Then, upload file in container.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="upload_file_to_storage" :::
The following code creates an online endpoint without specifying a deployment.
Warning
The identity for an endpoint is immutable. During endpoint creation, you can associate it with a system-assigned identity (default) or a user-assigned identity. You can't change the identity after the endpoint has been created.
When you create an online endpoint, a system-assigned managed identity is created for the endpoint by default.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="create_endpoint" :::
Check the status of the endpoint with the following.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="check_endpoint_Status" :::
If you encounter any issues, see Troubleshooting online endpoints deployment and scoring.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_endpoint" :::
Check the status of the endpoint with the following.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_endpoint_Status" :::
If you encounter any issues, see Troubleshooting online endpoints deployment and scoring.
Important
Online endpoints require Azure Container Registry pull permission, AcrPull permission, to the container registry and Storage Blob Data Reader permission to the default datastore of the workspace.
You can allow the online endpoint permission to access your storage via its system-assigned managed identity or give permission to the user-assigned managed identity to access the storage account created in the previous section.
Retrieve the system-assigned managed identity that was created for your endpoint.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="get_system_identity" :::
From here, you can give the system-assigned managed identity permission to access your storage.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="give_permission_to_user_storage_account" :::
Retrieve user-assigned managed identity client ID.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_user_identity_client_id" :::
Retrieve the user-assigned managed identity ID.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_user_identity_id" :::
Get the container registry associated with workspace.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_container_registry_id" :::
Retrieve the default storage of the workspace.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="get_workspace_storage_id" :::
Give permission of storage account to the user-assigned managed identity.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="give_permission_to_user_storage_account" :::
Give permission of container registry to user assigned managed identity.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="give_permission_to_container_registry" :::
Give permission of default workspace storage to user-assigned managed identity.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="give_permission_to_workspace_storage_account" :::
Refer to the following script to understand how to use your identity token to access Azure resources, in this scenario, the storage account created in previous sections.
:::code language="python" source="~/azureml-examples-main/cli/endpoints/online/model-1/onlinescoring/score_managedidentity.py":::
Create a deployment that's associated with the online endpoint. Learn more about deploying to online endpoints.
Warning
This deployment can take approximately 8-14 minutes depending on whether the underlying environment/image is being built for the first time. Subsequent deployments using the same environment will go quicker.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="deploy" :::
Note
The value of the --name argument may override the name key inside the YAML file.
Check the status of the deployment.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="check_deploy_Status" :::
To refine the above query to only return specific data, see Query Azure CLI command output.
Note
The init method in the scoring script reads the file from your storage account using the system assigned managed identity token.
To check the init method output, see the deployment log with the following code.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="check_deployment_log" :::
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="create_endpoint" :::
Note
The value of the --name argument may override the name key inside the YAML file.
Once the command executes, you can check the status of the deployment.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_endpoint_Status" :::
To refine the above query to only return specific data, see Query Azure CLI command output.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_deployment_log" :::
Note
The init method in the scoring script reads the file from your storage account using the system assigned managed identity token.
To check the init method output, see the deployment log with the following code.
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="check_deployment_log" :::
When your deployment completes, the model, the environment, and the endpoint are registered to your Azure Machine Learning workspace.
Once your online endpoint is deployed, confirm its operation. Details of inferencing vary from model to model. For this guide, the JSON query parameters look like:
:::code language="json" source="~/azureml-examples-main/cli/endpoints/online/model-1/sample-request.json" :::
To call your endpoint, run:
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="test_endpoint" :::
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="test_endpoint" :::
If you don't plan to continue using the deployed online endpoint and storage, delete them to reduce costs. When you delete the endpoint, all of its associated deployments are deleted as well.
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="delete_endpoint" :::
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-sai.sh" id="delete_storage_account" :::
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="delete_endpoint" :::
::: code language="azurecli" source="/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="delete_storage_account" :::
::: code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-access-resource-uai.sh" id="delete_user_identity" :::
- Deploy and score a machine learning model by using a online endpoint.
- For more on deployment, see Safe rollout for online endpoints.
- For more information on using the CLI, see Use the CLI extension for Azure Machine Learning.
- To see which compute resources you can use, see Managed online endpoints SKU list.
- For more on costs, see View costs for an Azure Machine Learning managed online endpoint.
- For information on monitoring endpoints, see Monitor managed online endpoints.
- For limitations for managed endpoints, see Manage and increase quotas for resources with Azure Machine Learning.