Skip to content

Files

Latest commit

7e303ca · Mar 18, 2022

History

History
226 lines (173 loc) · 9.11 KB

how-to-github-actions-machine-learning.md

File metadata and controls

226 lines (173 loc) · 9.11 KB
title titleSuffix description services ms.service ms.subservice author ms.author ms.date ms.topic ms.custom
GitHub Actions for CI/CD
Azure Machine Learning
Learn about how to create a GitHub Actions workflow to train a model on Azure Machine Learning
machine-learning
machine-learning
mlops
juliakm
jukullam
10/21/2021
how-to
github-actions-azure

Use GitHub Actions with Azure Machine Learning

Get started with GitHub Actions to train a model on Azure Machine Learning.

Note

GitHub Actions for Azure Machine Learning are provided as-is, and are not fully supported by Microsoft. If you encounter problems with a specific action, open an issue in the repository for the action. For example, if you encounter a problem with the aml-deploy action, report the problem in the https://github.com/Azure/aml-deploy repo.

Prerequisites

Workflow file overview

A workflow is defined by a YAML (.yml) file in the /.github/workflows/ path in your repository. This definition contains the various steps and parameters that make up the workflow.

The file has four sections:

Section Tasks
Authentication 1. Define a service principal.
2. Create a GitHub secret.
Connect 1. Connect to the machine learning workspace.
2. Connect to a compute target.
Run 1. Submit a training run.
Deploy 1. Register model in Azure Machine Learning registry. 1. Deploy the model.

Create repository

Create a new repository off the ML Ops with GitHub Actions and Azure Machine Learning template.

  1. Open the template on GitHub.

  2. Select Use this template.

    :::image type="content" source="media/how-to-github-actions-machine-learning/gh-actions-use-template.png" alt-text="Select use this template":::

  3. Create a new repository from the template. Set the repository name to ml-learning or a name of your choice.

Generate deployment credentials

You can create a service principal with the az ad sp create-for-rbac command in the Azure CLI. Run this command with Azure Cloud Shell in the Azure portal or by selecting the Try it button.

az ad sp create-for-rbac --name "myML" --role contributor \
                            --scopes /subscriptions/<subscription-id>/resourceGroups/<group-name> \
                            --sdk-auth

In the example above, replace the placeholders with your subscription ID, resource group name, and app name. The output is a JSON object with the role assignment credentials that provide access to your App Service app similar to below. Copy this JSON object for later.

  {
    "clientId": "<GUID>",
    "clientSecret": "<GUID>",
    "subscriptionId": "<GUID>",
    "tenantId": "<GUID>",
    (...)
  }

Configure the GitHub secret

  1. In GitHub, browse your repository, select Settings > Secrets > Add a new secret.

  2. Paste the entire JSON output from the Azure CLI command into the secret's value field. Give the secret the name AZURE_CREDENTIALS.

Connect to the workspace

Use the Azure Machine Learning Workspace action to connect to your Azure Machine Learning workspace.

    - name: Connect/Create Azure Machine Learning Workspace
      id: aml_workspace
      uses: Azure/aml-workspace@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}

By default, the action expects a workspace.json file. If your JSON file has a different name, you can specify it with the parameters_file input parameter. If there is not a file, a new one will be created with the repository name.

    - name: Connect/Create Azure Machine Learning Workspace
      id: aml_workspace
      uses: Azure/aml-workspace@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}
          parameters_file: "alternate_workspace.json"

The action writes the workspace Azure Resource Manager (ARM) properties to a config file, which will be picked by all future Azure Machine Learning GitHub Actions. The file is saved to GITHUB_WORKSPACE/aml_arm_config.json.

Connect to a Compute Target in Azure Machine Learning

Use the Azure Machine Learning Compute action to connect to a compute target in Azure Machine Learning. If the compute target exists, the action will connect to it. Otherwise the action will create a new compute target. The AML Compute action only supports the Azure ML compute cluster and Azure Kubernetes Service (AKS).

    - name: Connect/Create Azure Machine Learning Compute Target
      id: aml_compute_training
      uses: Azure/aml-compute@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}

Submit training run

Use the Azure Machine Learning Training action to submit a ScriptRun, an Estimator or a Pipeline to Azure Machine Learning.

    - name: Submit training run
      id: aml_run
      uses: Azure/aml-run@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}

Register model in registry

Use the Azure Machine Learning Register Model action to register a model to Azure Machine Learning.

    - name: Register model
      id: aml_registermodel
      uses: Azure/aml-registermodel@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}
          run_id:  ${{ steps.aml_run.outputs.run_id }}
          experiment_name: ${{ steps.aml_run.outputs.experiment_name }}

Deploy model to Azure Machine Learning to ACI

Use the Azure Machine Learning Deploy action to deploys a model and create an endpoint for the model. You can also use the Azure Machine Learning Deploy to deploy to Azure Kubernetes Service. See this sample workflow for a model that deploys to Azure Kubernetes Service.

    - name: Deploy model
      id: aml_deploy
      uses: Azure/aml-deploy@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}
          model_name:  ${{ steps.aml_registermodel.outputs.model_name }}
          model_version: ${{ steps.aml_registermodel.outputs.model_version }}

Complete example

Train your model and deploy to Azure Machine Learning.

# Actions train a model on Azure Machine Learning
name: Azure Machine Learning training and deployment
on:
  push:
    branches:
      - master
    # paths:
    #   - 'code/*'
jobs:
  train:
    runs-on: ubuntu-latest
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - name: Check Out Repository
      id: checkout_repository
      uses: actions/checkout@v2
        
    # Connect or Create the Azure Machine Learning Workspace
    - name: Connect/Create Azure Machine Learning Workspace
      id: aml_workspace
      uses: Azure/aml-workspace@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}
    
    # Connect or Create a Compute Target in Azure Machine Learning
    - name: Connect/Create Azure Machine Learning Compute Target
      id: aml_compute_training
      uses: Azure/aml-compute@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}
    
    # Submit a training run to the Azure Machine Learning
    - name: Submit training run
      id: aml_run
      uses: Azure/aml-run@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}

    # Register model in Azure Machine Learning model registry
    - name: Register model
      id: aml_registermodel
      uses: Azure/aml-registermodel@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}
          run_id:  ${{ steps.aml_run.outputs.run_id }}
          experiment_name: ${{ steps.aml_run.outputs.experiment_name }}

    # Deploy model in Azure Machine Learning to ACI
    - name: Deploy model
      id: aml_deploy
      uses: Azure/aml-deploy@v1
      with:
          azure_credentials: ${{ secrets.AZURE_CREDENTIALS }}
          model_name:  ${{ steps.aml_registermodel.outputs.model_name }}
          model_version: ${{ steps.aml_registermodel.outputs.model_version }}

Clean up resources

When your resource group and repository are no longer needed, clean up the resources you deployed by deleting the resource group and your GitHub repository.

Next steps

[!div class="nextstepaction"] Create and run machine learning pipelines with Azure Machine Learning SDK