Skip to content

Files

Latest commit

dksimpsondksimpson
Fix code tags for Azure CLI code blocks - batch 5
Mar 2, 2022
ec937a5 · Mar 2, 2022

History

History
260 lines (209 loc) · 11.9 KB

tutorial-deploy-azure-resources.md

File metadata and controls

260 lines (209 loc) · 11.9 KB
title description keywords services ms.service ms.topic author ms.author ms.reviewer ms.date
Set up resources for deploying an ML model in Azure SQL Edge
In part one of this three-part Azure SQL Edge tutorial for predicting iron ore impurities, you'll install the prerequisite software and set up required Azure resources for deploying a machine learning model in Azure SQL Edge.
sql-edge
sql-edge
tutorial
kendalvandyke
kendalv
jroth
05/19/2020

Install software and set up resources for the tutorial

In this three-part tutorial, you'll create a machine learning model to predict iron ore impurities as a percentage of Silica, and then deploy the model in Azure SQL Edge. In part one, you'll install the required software and deploy Azure resources.

Prerequisites

  1. If you don't have an Azure subscription, create a free account.
  2. Install Visual Studio 2019 with
    • Azure IoT Edge tools
    • .NET core cross-platform development
    • Container development tools
  3. Install Azure Data Studio
  4. Open Azure Data Studio and configure Python for notebooks. For details, see Configure Python for Notebooks. This step can take several minutes.
  5. Install the latest version of Azure CLI. The following scripts require that AZ PowerShell be the latest version (3.5.0, Feb 2020).
  6. Set up the environment to debug, run, and test IoT Edge solution by installing Azure IoT EdgeHub Dev Tool.
  7. Install Docker.

Deploy Azure resources using PowerShell Script

Deploy the Azure resources required by this Azure SQL Edge tutorial. These can be deployed either by using a PowerShell script or through the Azure portal. This tutorial uses a PowerShell script.

  1. Import the modules needed to run the PowerShell script in this tutorial.

    Import-Module Az.Accounts -RequiredVersion 1.7.3
Import-Module -Name Az -RequiredVersion 3.5.0
Import-Module Az.IotHub -RequiredVersion 2.1.0
Import-Module Az.Compute -RequiredVersion 3.5.0
az extension add --name azure-iot
az extension add --name azure-cli-ml

    [!INCLUDE iot-hub-cli-version-info]

  2. Declare the variables required by the PowerShell script.

    $ResourceGroup = "<name_of_the_resource_group>"
$IoTHubName = "<name_of_the_IoT_hub>"
$location = "<location_of_your_Azure_Subscription>"
$SubscriptionName = "<your_azure_subscription>"
$NetworkSecGroup = "<name_of_your_network_security_group>"
$StorageAccountName = "<name_of_your_storage_account>"

  3. Declare the rest of the variables.

    $IoTHubSkuName = "S1"
$IoTHubUnits = 4
$EdgeDeviceId = "IronOrePredictionDevice"
$publicIpName = "VMPublicIP"
$imageOffer = "iot_edge_vm_ubuntu"
$imagePublisher = "microsoft_iot_edge"
$imageSku = "ubuntu_1604_edgeruntimeonly"
$AdminAcc = "iotadmin"
$AdminPassword = ConvertTo-SecureString "IoTAdmin@1234" -AsPlainText -Force
$VMSize = "Standard_DS3"
$NetworkName = "MyNet"
$NICName = "MyNIC"
$SubnetName = "MySubnet"
$SubnetAddressPrefix = "10.0.0.0/24"
$VnetAddressPrefix = "10.0.0.0/16"
$MyWorkSpace = "SQLDatabaseEdgeDemo"
$containerRegistryName = $ResourceGroup + "ContRegistry"

  4. To begin creating assets, log in to Azure.

    Login-AzAccount

az login

  5. Set the Azure subscription ID.

    Select-AzSubscription -Subscription $SubscriptionName
az account set --subscription $SubscriptionName

  6. Create the resource group if it doesn't already exist.

    $rg = Get-AzResourceGroup -Name $ResourceGroup
if($rg -eq $null)
{
    Write-Output("Resource Group $ResourceGroup does not exist, creating Resource Gorup")
    New-AzResourceGroup -Name $ResourceGroup -Location $location
}
else
{
    Write-Output ("Resource Group $ResourceGroup exists")
}

  7. Create the storage account and storage account container in the resource group.

    $sa = Get-AzStorageAccount -ResourceGroupName $ResourceGroup -Name $StorageAccountName 
if ($sa -eq $null)
{
    New-AzStorageAccount -ResourceGroupName $ResourceGroup -Name $StorageAccountName -SkuName Standard_LRS -Location $location -Kind Storage
    $sa = Get-AzStorageAccount -ResourceGroupName $ResourceGroup -Name $StorageAccountName 
    $storagekey = Get-AzStorageAccountKey -ResourceGroupName $ResourceGroup -Name $StorageAccountName 
    $storageContext = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $storagekey[0].Value
    New-AzStorageContainer -Name "sqldatabasedacpac" -Context $storageContext 
}
else
{
   Write-Output ("Storage Account $StorageAccountName exists in Resource Group $ResourceGroup")     
}

  8. Upload the database dacpac file to the storage account and generate a SAS URL for the blob. Make a note of the SAS URL for the database dacpac blob.

    $file = Read-Host "Please Enter the location to the zipped Database DacPac file:"
Set-AzStorageBlobContent -File $file -Container "sqldatabasedacpac" -Blob "SQLDatabasedacpac.zip" -Context $sa.Context
$DacpacFileSASURL = New-AzStorageBlobSASToken -Container "sqldatabasedacpac" -Blob "SQLDatabasedacpac.zip" -Context $sa.Context -Permission r -StartTime (Get-Date).DateTime -ExpiryTime (Get-Date).AddMonths(12) -FullUri

  9. Create an Azure container registry within this resource group.

    $containerRegistry = Get-AzContainerRegistry -ResourceGroupName $ResourceGroup -Name $containerRegistryName 
if ($containerRegistry -eq $null)
{
    New-AzContainerRegistry -ResourceGroupName $ResourceGroup -Name $containerRegistryName -Sku Standard -Location $location -EnableAdminUser 
    $containerRegistry = Get-AzContainerRegistry -ResourceGroupName $ResourceGroup -Name $containerRegistryName 
}
else
{
    Write-Output ("Container Registry $containerRegistryName exists in Resource Group $ResourceGroup")
}

  10. Create the network security group within the resource group.

    $nsg = Get-AzNetworkSecurityGroup -ResourceGroupName $ResourceGroup -Name $NetworkSecGroup 
if($nsg -eq $null)
{
    Write-Output("Network Security Group $NetworkSecGroup does not exist in the resource group $ResourceGroup")

    $rule1 = New-AzNetworkSecurityRuleConfig -Name "SSH" -Description "Allow SSH" -Access Allow -Protocol Tcp -Direction Inbound -Priority 100 -SourceAddressPrefix Internet -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 22
    $rule2 = New-AzNetworkSecurityRuleConfig -Name "SQL" -Description "Allow SQL" -Access Allow -Protocol Tcp -Direction Inbound -Priority 101 -SourceAddressPrefix Internet -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 1600
    New-AzNetworkSecurityGroup -Name $NetworkSecGroup -ResourceGroupName $ResourceGroup -Location $location -SecurityRules $rule1, $rule2

    $nsg = Get-AzNetworkSecurityGroup -ResourceGroupName $ResourceGroup -Name $NetworkSecGroup
}
else
{
    Write-Output ("Network Security Group $NetworkSecGroup exists in the resource group $ResourceGroup")
}

  11. Create an Azure virtual machine enabled with SQL Edge. This VM will act as an Edge device.

    $AzVM = Get-AzVM -ResourceGroupName $ResourceGroup -Name $EdgeDeviceId
If($AzVM -eq $null)
{
    Write-Output("The Azure VM with Name- $EdgeVMName is not present in the Resource Group- $ResourceGroup ")

    $SingleSubnet = New-AzVirtualNetworkSubnetConfig -Name $SubnetName -AddressPrefix $SubnetAddressPrefix
    $Vnet = New-AzVirtualNetwork -Name $NetworkName -ResourceGroupName $ResourceGroup -Location $location -AddressPrefix $VnetAddressPrefix -Subnet $SingleSubnet
    $publicIp = New-AzPublicIpAddress -Name $publicIpName -ResourceGroupName $ResourceGroup -AllocationMethod Static -Location $location  
    $NIC = New-AzNetworkInterface -Name $NICName -ResourceGroupName $ResourceGroup -Location $location -SubnetId $Vnet.Subnets[0].Id -NetworkSecurityGroupId $nsg.Id -PublicIpAddressId $publicIp.Id
     
    ##Set-AzNetworkInterfaceIpConfig -Name "ipconfig1"  -NetworkInterface $NIC -PublicIpAddress $publicIp

    $Credential = New-Object System.Management.Automation.PSCredential ($AdminAcc, $AdminPassword);

    $VirtualMachine = New-AzVMConfig -VMName $EdgeDeviceId -VMSize $VMSize
    $VirtualMachine = Set-AzVMOperatingSystem -VM $VirtualMachine -Linux -ComputerName $EdgeDeviceId -Credential $Credential
    $VirtualMachine = Add-AzVMNetworkInterface -VM $VirtualMachine -Id $NIC.Id  
    $VirtualMachine = Set-AzVMSourceImage -VM $VirtualMachine -PublisherName $imagePublisher -Offer $imageOffer -Skus $imageSku -Version latest 
    $VirtualMachine = Set-AzVMPlan -VM $VirtualMachine -Name $imageSku -Publisher $imagePublisher -Product $imageOffer

    $AzVM = New-AzVM -ResourceGroupName $ResourceGroup -Location $location -VM $VirtualMachine -Verbose
    $AzVM = Get-AzVM -ResourceGroupName $ResourceGroup -Name $EdgeDeviceId
      
}
else
{
    Write-Output ("The Azure VM with Name- $EdgeDeviceId is present in the Resource Group- $ResourceGroup ")
}

  12. Create an IoT hub within the resource group.

    $iotHub = Get-AzIotHub -ResourceGroupName $ResourceGroup -Name $IoTHubName
If($iotHub -eq $null)
{
    Write-Output("IoTHub $IoTHubName does not exists, creating The IoTHub in the resource group $ResourceGroup")
    New-AzIotHub -ResourceGroupName $ResourceGroup -Name $IoTHubName -SkuName $IoTHubSkuName -Units $IoTHubUnits -Location $location -Verbose
}
else
{
    Write-Output ("IoTHub $IoTHubName present in the resource group $ResourceGroup") 
}

  13. Add an Edge device to the IoT hub. This step only creates the device digital identity.

    $deviceIdentity = Get-AzIotHubDevice -ResourceGroupName $ResourceGroup -IotHubName $IoTHubName -DeviceId $EdgeDeviceId
If($deviceIdentity -eq $null)
{
    Write-Output("The Edge Device with DeviceId- $EdgeDeviceId is not registered to the IoTHub- $IoTHubName ")
    Add-AzIotHubDevice -ResourceGroupName $ResourceGroup -IotHubName $IoTHubName -DeviceId $EdgeDeviceId -EdgeEnabled  
}
else
{
    Write-Output ("The Edge Device with DeviceId- $EdgeDeviceId is registered to the IoTHub- $IoTHubName")
}
$deviceIdentity = Get-AzIotHubDevice -ResourceGroupName $ResourceGroup -IotHubName $IoTHubName -DeviceId $EdgeDeviceId

  14. Get the device primary connection string. This will be needed later for the VM. The following command uses Azure CLI for deployments.

    $deviceConnectionString = az iot hub device-identity show-connection-string --device-id $EdgeDeviceId --hub-name $IoTHubName --resource-group $ResourceGroup --subscription $SubscriptionName
$connString = $deviceConnectionString[1].Substring(23,$deviceConnectionString[1].Length-24)
$connString

  15. Update the connection string in the IoT Edge configuration file on the Edge device. The following commands use Azure CLI for deployments.

    $script = "/etc/iotedge/configedge.sh '" + $connString + "'"
az vm run-command invoke -g $ResourceGroup -n $EdgeDeviceId  --command-id RunShellScript --script $script

  16. Create an Azure Machine Learning workspace within the resource group.

    az ml workspace create -w $MyWorkSpace -g $ResourceGroup

Next Steps