Skip to content

Files

Latest commit

greg-lindsaygreg-lindsay
update author
May 24, 2022
5def238 · May 24, 2022

History

History
311 lines (224 loc) · 12.5 KB

monitor-load-balancer.md

File metadata and controls

311 lines (224 loc) · 12.5 KB
title description author ms.author ms.service ms.topic ms.custom ms.date ms.devlang
Monitoring Azure Load Balancer
Start here to learn how to monitor load balancer.
greg-lindsay
greglin
load-balancer
how-to
subject-monitoring, devx-track-azurecli
06/29/2021
azurecli

Monitoring load balancer

When you have critical applications and business processes relying on Azure resources, you want to monitor those resources for their availability, performance, and operation.

This article describes the monitoring data generated by Load Balancer. Load Balancer uses Azure Monitor. If you are unfamiliar with the features of Azure Monitor common to all Azure services that use it, read Monitoring Azure resources with Azure Monitor.

Load balancer insights

Some services in Azure have a special focused pre-built monitoring dashboard in the Azure portal that provides a starting point for monitoring your service. These special dashboards are called "insights".

Load Balancer insights provide:

  • Functional dependency view
  • Metrics dashboard
  • Overview tab
  • Frontend and Backend Availability tab
  • Data Throughput tab
  • Flow Distribution
  • Connection Monitors
  • Metric Definitions

For more information on Load Balancer insights, see Using Insights to monitor and configure your Azure Load Balancer

Monitoring data

Load Balancer collects the same kinds of monitoring data as other Azure resources that are described in Monitoring data from Azure resources.

See Monitoring Load Balancer data reference for detailed information on the metrics and logs metrics created by Load Balancer.

Load Balancer provides additional monitoring data through:

Collection and routing

Platform metrics and the Activity log are collected and stored automatically, but can be routed to other locations by using a diagnostic setting.

Resource Logs are not collected and stored until you create a diagnostic setting and route them to one or more locations.

Creating a diagnostic setting

You can create a diagnostic setting by using the Azure portal, PowerShell, or the Azure CLI.

For general guidance, see Create diagnostic setting to collect platform logs and metrics in Azure.

When you create a diagnostic setting, you specify which categories of logs to collect. The category for Load Balancer is AllMetrics

Portal

  1. Sign in to the Azure portal

  2. In the search box at the top of the portal, enter Load balancer.

  3. Select Load balancers in the search results.

  4. Select your load balancer. For this example, myLoadBalancer is used.

  5. In the Monitoring section of myLoadBalancer, select Diagnostic settings.

  6. In Diagnostic settings, select + Add diagnostic setting.

  7. Enter or select the following information in Diagnostic setting.

    Setting Value
    Diagnostic setting name Enter a name for the diagnostic setting.
    Category details
    metric Select AllMetrics.

  8. Select the Destination details. Some of the destinations options are:

    • Send to Log Analytics
      • Select the Subscription and Log Analytics workspace.
    • Archive to a storage account
      • Select the Subscription and the Storage Account.
    • Stream to an event hub
      • Select the Subscription, Event hub namespace, Event hub name (optional), and Event hub policy name

  9. Select Save.

PowerShell

Sign in to Azure PowerShell:

Connect-AzAccount

Log analytics workspace

To enable Diagnostic Logs for a Log Analytics workspace, enter these commands. Replace the bracketed values with your values:

## Place the load balancer in a variable. ##
$lbpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-load-balancer-name>
}
$lb = Get-AzLoadBalancer @lbpara
    
## Place the workspace in a variable. ##
$wspara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-log-analytics-workspace-name>
}
$ws = Get-AzOperationalInsightsWorkspace @wspara
    
## Enable the diagnostic setting. ##
Set-AzDiagnosticSetting `
    -ResourceId $lb.id `
    -Name <your-diagnostic-setting-name> `
    -Enabled $true `
    -MetricCategory 'AllMetrics' `
    -WorkspaceId $ws.ResourceId

Storage account

To enable Diagnostic Logs in a storage account, enter these commands. Replace the bracketed values with your values:

## Place the load balancer in a variable. ##
$lbpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-load-balancer-name>
}
$lb = Get-AzLoadBalancer @lbpara
    
## Place the storage account in a variable. ##
$storpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-storage-account-name>
}
$storage = Get-AzStorageAccount @storpara
    
## Enable the diagnostic setting. ##
Set-AzDiagnosticSetting `
    -ResourceId $lb.id `
    -Name <your-diagnostic-setting-name> `
    -StorageAccountId $storage.id `
    -Enabled $true `
    -MetricCategory 'AllMetrics'

Event hub

To enable Diagnostic Logs for an event hub namespace, enter these commands. Replace the bracketed values with your values:

## Place the load balancer in a variable. ##
$lbpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-load-balancer-name>
}
$lb = Get-AzLoadBalancer @lbpara
    
## Place the event hub in a variable. ##
$hubpara = @{
    ResourceGroupName = <your-resource-group-name>
    Name = <your-event-hub-name>
}
$eventhub = Get-AzEventHubNamespace @hubpara

## Place the event hub authorization rule in a variable. ##    
$hubrule = @{
    ResourceGroupName = 'myResourceGroup'
    Namespace = 'myeventhub8675'
}
$eventhubrule = Get-AzEventHubAuthorizationRule @hubrule

## Enable the diagnostic setting. ##
Set-AzDiagnosticSetting `
    -ResourceId $lb.Id `
    -Name 'myDiagSetting-event'`
    -EventHubName $eventhub.Name `
    -EventHubAuthorizationRuleId $eventhubrule.Id `
    -Enabled $true `
    -MetricCategory 'AllMetrics'

Azure CLI

Sign in to Azure CLI:

az login

Log analytics workspace

To enable Diagnostic Logs for a Log Analytics workspace, enter these commands. Replace the bracketed values with your values:

lbid=$(az network lb show \
    --name <your-load-balancer-name> \
    --resource-group <your-resource-group> \
    --query id \
    --output tsv)

wsid=$(az monitor log-analytics workspace show \
    --resource-group <your-resource-group> \
    --workspace-name <your-log-analytics-workspace-name> \
    --query id \
    --output tsv)
    
az monitor diagnostic-settings create \
    --name <your-diagnostic-setting-name> \
    --resource $lbid \
    --metrics '[{"category": "AllMetrics","enabled": true}]' \
    --workspace $wsid

Storage account

To enable Diagnostic Logs in a storage account, enter these commands. Replace the bracketed values with your values:

lbid=$(az network lb show \
    --name <your-load-balancer-name> \
    --resource-group <your-resource-group> \
    --query id \
    --output tsv)

storid=$(az storage account show \
        --name <your-storage-account-name> \
        --resource-group <your-resource-group> \
        --query id \
        --output tsv)
    
az monitor diagnostic-settings create \
    --name <your-diagnostic-setting-name> \
    --resource $lbid \
    --metrics '[{"category": "AllMetrics","enabled": true}]' \
    --storage-account $storid

Event hub

To enable Diagnostic Logs for an event hub namespace, enter these commands. Replace the bracketed values with your values:

lbid=$(az network lb show \
    --name <your-load-balancer-name> \
    --resource-group <your-resource-group> \
    --query id \
    --output tsv)

az monitor diagnostic-settings create \
    --name myDiagSetting-event \
    --resource $lbid \
    --metrics '[{"category": "AllMetrics","enabled": true}]' \
    --event-hub-rule /subscriptions/<your-subscription-id>/resourceGroups/<your-resource-group>/providers/Microsoft.EventHub/namespaces/<your-event-hub-namespace>/authorizationrules/RootManageSharedAccessKey

The metrics and logs you can collect are discussed in the following sections.

Analyzing metrics

You can analyze metrics for Load Balancer with metrics from other Azure services using metrics explorer by opening Metrics from the Azure Monitor menu. See Getting started with Azure Metrics Explorer for details on using this tool.

For a list of the platform metrics collected for Load Balancer, see Monitoring Load Balancer data reference metrics

For reference, you can see a list of all resource metrics supported in Azure Monitor.

Analyzing logs

Data in Azure Monitor Logs is stored in tables where each table has its own set of unique properties.

The Activity log is a type of platform log that provides insight into subscription-level events. You can view it independently or route it to Azure Monitor Logs, where you can do much more complex queries using Log Analytics.

For a list of the tables used by Azure Monitor Logs and queryable by Log Analytics, see Monitoring Load Balancer data reference

Sample Kusto queries

Note

There is currently an issue with Kusto queries that prevents data from being retrieved from load balancer logs.

Alerts

Azure Monitor alerts proactively notify you when important conditions are found in your monitoring data. They allow you to identify and address issues in your system before your customers notice them. You can set alerts on metrics, logs, and the activity log. Different types of alerts have benefits and drawbacks

If you are creating or running an application, which run on Load Balancer Azure Monitor Application Insights may offer additional types of alerts.

The following table lists common and recommended alert rules for Load Balancer.

Alert type Condition Description
Load balancing rule unavailable due to unavailable VMs If data path availability split by Frontend IP address and Frontend Port (all known and future values) is equal to zero and health probe status is equal to zero, then fire alerts This alert determines if the data path availability for any configured load balancing rules is not servicing traffic due to all VMs in the associated backend pool being probed down by the configured health probe. Review load balancer troubleshooting guide to investigate the potential root cause.
VM availability significantly low If health probe status split by Backend IP and Backend Port is equal to user defined probed-up percentage of total pool size (i.e. 25% are probed up), then fire alert This alert determines if there are less than needed VMs available to serve traffic
Outbound connections to internet endpoint failing If SNAT Connection Count filtered to Connection State = Failed is greater than zero, then fire alert This alert fires when SNAT ports are exhausted and VMs are failing to initiate outbound connections.
Approaching SNAT exhaustion If Used SNAT Ports is greater than user defined number, then fire alert This alert requires a static outbound configuration where the same number of ports are always allocated. It then fires when a percentage of the allocated ports is used.

Next steps