quick-create-azure-cli.md

title	description	services	ms.service	author	ms.author	ms.reviewer	ms.workload	ms.topic	ms.custom	ms.date
Quickstart - Create an Azure Stream Analytics job using the Azure CLI	This quickstart shows how to use the Azure CLI to create an Azure Stream Analytics job.	stream-analytics	stream-analytics	sidramadoss	sidram	jasonh	big-data	quickstart	mvc, devx-track-azurecli, mode-api	07/01/2020

Quickstart: Create an Azure Stream Analytics job using the Azure CLI

In this quickstart, you use the Azure CLI to define a Stream Analytics job that filters real-time sensor messages with a temperature reading greater than 27. Your Stream Analytics job will read data from IoT Hub, transform the data, and write the data back to a container in blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.

Before you begin

[!INCLUDE quickstarts-free-trial-note]

[!INCLUDE azure-cli-prepare-your-environment.md]

• Create a resource group. All Azure resources must be deployed into a resource group. Resource groups allow you to organize and manage related Azure resources.

  For this quickstart, create a resource group named streamanalyticsrg in the eastus location with the following az group create command:

  az group create --name streamanalyticsrg --location eastus
  

Prepare the input data

Before you define the Stream Analytics job, prepare the data that's used for the job's input.

The following Azure CLI code blocks are commands that prepare the input data required by the job. Review the sections to understand the code.

1. Create an IoT Hub using the az iot hub create command. This example creates an IoT Hub called MyASAIoTHub. Because IoT Hub names are unique, you need to come up with your own IoT Hub name. Set the SKU to F1 to use the free tier if it is available with your subscription. If not, choose the next lowest tier.

   az iot hub create --name "MyASAIoTHub" --resource-group streamanalyticsrg --sku S1
   

   Once the IoT hub has been created, get the IoT Hub connection string using the az iot hub show-connection-string command. Copy the entire connection string and save it for when you add the IoT Hub as input to your Stream Analytics job.

   az iot hub show-connection-string --hub-name "MyASAIoTHub"
   

2. Add a device to IoT Hub using the az iothub device-identity create command. This example creates a device called MyASAIoTDevice.

   az iot hub device-identity create --hub-name "MyASAIoTHub" --device-id "MyASAIoTDevice"
   

3. Get the device connection string using the az iot hub device-identity show-connection-string command. Copy the entire connection string and save it for when you create the Raspberry Pi simulator.

   az iot hub device-identity show-connection-string --hub-name "MyASAIoTHub" --device-id "MyASAIoTDevice" --output table
   

   Output example:

   HostName=MyASAIoTHub.azure-devices.net;DeviceId=MyASAIoTDevice;SharedAccessKey=a2mnUsg52+NIgYudxYYUNXI67r0JmNubmfVafojG8=
   

Create a blob storage account

The following Azure CLI code blocks create a blob storage account that's used for job output. Review the sections to understand the code.

1. Create a general-purpose storage account with the az storage account create command. The general-purpose storage account can be used for all four services: blobs, files, tables, and queues.

   Remember to replace placeholder values in angle brackets with your own values:

   az storage account create \
       --name <storage-account> \
       --resource-group streamanalyticsrg \
       --location eastus \
       --sku Standard_ZRS \
       --encryption-services blob
   

2. Get the key for your storage account by running the az storage account keys list command. Save this key to use in the next step.

   az storage account keys list -g streamanalyticsrg -n <storage-account>
   

3. Create a container for storing blobs with the az storage container create command. You use the storage account key to authorize the operation to create the container. For more information about authorizing data operations with Azure CLI, see Authorize access to blob or queue data with Azure CLI.

   az storage container create \
       --account-name <storage-account> \
       --name sample-container \
       --account-key <key>
       --auth-mode key
   

Create a Stream Analytics job

The following Azure CLI code blocks create a Stream Analytics job. Review the sections to understand the code

1. Create a Stream Analytics job with the az stream-analytics job create command.

az stream-analytics job create \
    --resource-group streamanalyticsrg 
    --name streamanalyticsjob \
    --location eastus \
    --output-error-policy "Drop" \
    --events-outoforder-policy "Drop" \
    --events-outoforder-max-delay 5 \
    --events-late-arrival-max-delay 16 \     
    --data-locale "en-US"

Configure input to the job

Add an input to your job by using the az stream-analytics input cmdlet. This cmdlet takes the job name, job input name, resource group name, and the job input definition as parameters. The job input definition is a JSON file that contains the properties required to configure the job's input. In this example, you'll create an IoT Hub as an input.

On your local machine, create a file named datasource.json and add the following JSON data to it. Make sure to replace the value for sharedAccessPolicyKey with the SharedAccessKey portion of the IoT Hub connection string you saved in a previous section.

{
    "type": "Microsoft.Devices/IotHubs",
    "properties": {
        "iotHubNamespace": "iothub",
        "sharedAccessPolicyName": "iothubowner",
        "sharedAccessPolicyKey": "sharedAccessPolicyKey=",
        "consumerGroupName": "sdkconsumergroup",
        "endpoint": "messages/events"
    }
}

On your local machine, create a file named serialization.json and add the following JSON data to it.

{
     "type": "Json",
     "properties": {
         "encoding": "UTF8"
     }
}

Next, run the az stream-analytics input create cmdlet. Be sure to replace the value of datasource variable with the path where you've stored the job input definition JSON file, and the value of serialization variable with the path where you've stored the serialization JSON file.

az stream-analytics input create \
    --resource-group streamanalyticsrg 
    --job-name streamanalyticsjob \
    --name asaiotinput \
    --type Stream \
    --datasource datasource.json \
    --serialization serialization.json

Configure output to the job

Add an output to your job by using the az stream-analytics output create cmdlet. This cmdlet takes the job name, job output name, resource group name, and the job output definition as parameters. The job output definition is a JSON file that contains the properties required to configure job's output. This example uses blob storage as output.

On your local machine, create a file named datasink.json, and add the following JSON data to it. Make sure to replace the value for accountKey with your storage account's access key that is the value stored in $storageAccountKey value.

{
    "type": "Microsoft.Storage/Blob",
    "properties": {
        "storageAccounts": [
            {
                "accountName": "<storage-account>",
                "accountKey": "accountKey=="
            }
        ],
        "container": "state",
        "pathPattern": "{date}/{time}",
        "dateFormat": "yyyy/MM/dd",
        "timeFormat": "HH"
    }
}

Next, run the az stream-analytics output cmdlet. Be sure to replace the value of datasource variable with the path where you've stored the job output definition JSON file, and the value of serialization variable with the path where you've stored the serialization JSON file.

az stream-analytics output create \
    --resource-group streamanalyticsrg \
    --job-name streamanalyticsjob \
    --name asabloboutput \
    --datasource datasink.json \
    --serialization serialization.json

Define the transformation query

Add a transformation your job by using the az stream-analytics transformation create cmdlet. This cmdlet takes the job name, job transformation name, resource group name, and the job transformation definition as parameters.

Run the az stream-analytics transformation create cmdlet.

az stream-analytics transformation create \
    --resource-group streamanalyticsrg \
    --job-name streamanalyticsjob \
    --name Transformation \
    --streaming-units "6" \
    --transformation-query "SELECT * INTO asabloboutput FROM asaiotinput WHERE Temperature > 27"

Run the IoT simulator

1. Open the Raspberry Pi Azure IoT Online Simulator.

2. Replace the placeholder in Line 15 with the entire Azure IoT Hub Device connection string you saved in a previous section.

3. Click Run. The output should show the sensor data and messages that are being sent to your IoT Hub.

   

Start the Stream Analytics job and check the output

Start the job by using the az stream-analytics job start cmdlet. This cmdlet takes the job name, resource group name, output start mode, and start time as parameters. OutputStartMode accepts values of JobStartTime, CustomTime, or LastOutputEventTime.

After you run the following cmdlet, it returns True as output if the job starts. In the storage container, an output folder is created with the transformed data.

az stream-analytics job start \
    --resource-group streamanalyticsrg \
    --name streamanalyticsjob \
    --output-start-mode JobStartTime

Clean up resources

When no longer needed, delete the resource group, the streaming job, and all related resources. Deleting the job avoids billing the streaming units consumed by the job. If you're planning to use the job in future, you can skip deleting it, and stop the job for now. If you aren't going to continue to use this job, delete all resources created by this quickstart by running the following cmdlet:

az group delete \
    --name streamanalyticsrg \
    --no-wait

Next steps

In this quickstart, you deployed a simple Stream Analytics job using Azure CLI. You can also deploy Stream Analytics jobs using the Azure portal and Visual Studio.

To learn about configuring other input sources and performing real-time detection, continue to the following article:

[!div class="nextstepaction"] Real-time fraud detection using Azure Stream Analytics