title | description | ms.topic | ms.date |
---|---|---|---|
Use managed identities in Azure Kubernetes Service |
Learn how to use managed identities in Azure Kubernetes Service (AKS) |
article |
06/01/2022 |
Currently, an Azure Kubernetes Service (AKS) cluster (specifically, the Kubernetes cloud provider) requires an identity to create additional resources like load balancers and managed disks in Azure. This identity can be either a managed identity or a service principal. If you use a service principal, you must either provide one or AKS creates one on your behalf. If you use managed identity, this will be created for you by AKS automatically. Clusters using service principals eventually reach a state in which the service principal must be renewed to keep the cluster working. Managing service principals adds complexity, which is why it's easier to use managed identities instead. The same permission requirements apply for both service principals and managed identities.
Managed identities are essentially a wrapper around service principals, and make their management simpler. Credential rotation for MI happens automatically every 46 days according to Azure Active Directory default. AKS uses both system-assigned and user-assigned managed identity types. These identities are currently immutable. To learn more, read about managed identities for Azure resources.
You must have the following resource installed:
- The Azure CLI, version 2.23.0 or later
Note
AKS will create a kubelet MI in the Node resource group if you do not bring your own kubelet MI.
- Tenants move / migrate of managed identity enabled clusters isn't supported.
- If the cluster has
aad-pod-identity
enabled, Node-Managed Identity (NMI) pods modify the nodes' iptables to intercept calls to the Azure Instance Metadata endpoint. This configuration means any request made to the Metadata endpoint is intercepted by NMI even if the pod doesn't useaad-pod-identity
. AzurePodIdentityException CRD can be configured to informaad-pod-identity
that any requests to the Metadata endpoint originating from a pod that matches labels defined in CRD should be proxied without any processing in NMI. The system pods withkubernetes.azure.com/managedby: aks
label in kube-system namespace should be excluded inaad-pod-identity
by configuring the AzurePodIdentityException CRD. For more information, see Disable aad-pod-identity for a specific pod or application. To configure an exception, install the mic-exception YAML.
AKS uses several managed identities for built-in services and add-ons.
| Identity | Name | Use case | Default permissions | Bring your own identity |----------------------------|-----------|----------| | Control plane | AKS Cluster Name | Used by AKS control plane components to manage cluster resources including ingress load balancers and AKS managed public IPs, Cluster Autoscaler, Azure Disk & File CSI drivers | Contributor role for Node resource group | Supported | Kubelet | AKS Cluster Name-agentpool | Authentication with Azure Container Registry (ACR) | NA (for kubernetes v1.15+) | Supported | Add-on | AzureNPM | No identity required | NA | No | Add-on | AzureCNI network monitoring | No identity required | NA | No | Add-on | azure-policy (gatekeeper) | No identity required | NA | No | Add-on | azure-policy | No identity required | NA | No | Add-on | Calico | No identity required | NA | No | Add-on | Dashboard | No identity required | NA | No | Add-on | HTTPApplicationRouting | Manages required network resources | Reader role for node resource group, contributor role for DNS zone | No | Add-on | Ingress application gateway | Manages required network resources| Contributor role for node resource group | No | Add-on | omsagent | Used to send AKS metrics to Azure Monitor | Monitoring Metrics Publisher role | No | Add-on | Virtual-Node (ACIConnector) | Manages required network resources for Azure Container Instances (ACI) | Contributor role for node resource group | No | OSS project | aad-pod-identity | Enables applications to access cloud resources securely with Azure Active Directory (AAD) | NA | Steps to grant permission at https://github.com/Azure/aad-pod-identity#role-assignment.
You can now create an AKS cluster with managed identities by using the following CLI commands.
First, create an Azure resource group:
# Create an Azure resource group
az group create --name myResourceGroup --location westus2
Then, create an AKS cluster:
az aks create -g myResourceGroup -n myManagedCluster --enable-managed-identity
Once the cluster is created, you can then deploy your application workloads to the new cluster and interact with it just as you've done with service-principal-based AKS clusters.
Finally, get credentials to access the cluster:
az aks get-credentials --resource-group myResourceGroup --name myManagedCluster
You can now update an AKS cluster currently working with service principals to work with managed identities by using the following CLI commands.
az aks update -g <RGName> -n <AKSName> --enable-managed-identity
Note
An update will only work if there is an actual VHD update to consume. If you are running the latest VHD, you will need to wait till the next VHD is available in order to do the actual update.
Note
After updating, your cluster's control plane and addon pods will switch to use managed identity, but kubelet will KEEP USING SERVICE PRINCIPAL until you upgrade your agentpool. Perform an az aks nodepool upgrade --node-image-only
on your nodes to complete the update to managed identity.
If your cluster was using --attach-acr to pull from image from Azure Container Registry, after updating your cluster to Managed Identity, you need to rerun az aks update --attach-acr <ACR Resource ID>
to let the newly created kubelet used for managed identity get the permission to pull from ACR. Otherwise you will not be able to pull from ACR after the upgrade.
The Azure CLI will ensure your addon's permission is correctly set after migrating, if you're not using the Azure CLI to perform the migrating operation, you will need to handle the addon identity's permission by yourself. Here is one example using ARM.
Warning
Nodepool upgrade will cause downtime for your AKS cluster as the nodes in the nodepools will be cordoned/drained and then reimaged.
Confirm your AKS cluster is using managed identity with the following CLI command:
az aks show -g <RGName> -n <ClusterName> --query "servicePrincipalProfile"
If the cluster is using managed identities, you will see a clientId
value of "msi". A cluster using a Service Principal instead will instead show the object ID. For example:
{
"clientId": "msi"
}
After verifying the cluster is using managed identities, you can find the control plane system-assigned identity's object ID with the following command:
az aks show -g <RGName> -n <ClusterName> --query "identity"
{
"principalId": "<object-id>",
"tenantId": "<tenant-id>",
"type": "SystemAssigned",
"userAssignedIdentities": null
},
Note
For creating and using your own VNet, static IP address, or attached Azure disk where the resources are outside of the worker node resource group, CLI will add the role assignement automatically. If you are using ARM template or other clients, you need to use the PrincipalID of the cluster System Assigned Managed Identity to perform a role assignment. For more information on role assignment, see Delegate access to other Azure resources.
Permission grants to cluster Managed Identity used by Azure Cloud provider may take up 60 minutes to populate.
A custom control plane identity enables access to be granted to the existing identity prior to cluster creation. This feature enables scenarios such as using a custom VNET or outboundType of UDR with a pre-created managed identity.
You must have the Azure CLI, version 2.15.1 or later installed.
- USDOD Central, USDOD East, USGov Iowa in Azure Government aren't currently supported.
If you don't have a managed identity yet, you should go ahead and create one for example by using the az identity command.
az identity create --name myIdentity --resource-group myResourceGroup
Azure CLI will automatically add required role assignment for control plane MI. If you are using ARM template or other clients, you need to create the role assignment manually.
az role assignment create --assignee <control-plane-identity-object-id> --role "Managed Identity Operator" --scope <kubelet-identity-resource-id>
If your managed identity is part of your subscription, you can use az identity CLI command to query it.
az identity list --query "[].{Name:name, Id:id, Location:location}" -o table
Now you can use the following command to create your cluster with your existing identity:
az aks create \
--resource-group myResourceGroup \
--name myManagedCluster \
--network-plugin azure \
--vnet-subnet-id <subnet-id> \
--docker-bridge-address 172.17.0.1/16 \
--dns-service-ip 10.2.0.10 \
--service-cidr 10.2.0.0/24 \
--enable-managed-identity \
--assign-identity <identity-id>
A successful cluster creation using your own managed identities contains this userAssignedIdentities profile information:
"identity": {
"principalId": null,
"tenantId": null,
"type": "UserAssigned",
"userAssignedIdentities": {
"/subscriptions/<subscriptionid>/resourcegroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myIdentity": {
"clientId": "<client-id>",
"principalId": "<principal-id>"
}
}
},
A Kubelet identity enables access to be granted to the existing identity prior to cluster creation. This feature enables scenarios such as connection to ACR with a pre-created managed identity.
Warning
Updating kubelet MI will upgrade Nodepool, which causes downtime for your AKS cluster as the nodes in the nodepools will be cordoned/drained and then reimaged.
- You must have the Azure CLI, version 2.26.0 or later installed.
- Only works with a User-Assigned Managed cluster.
- China East, China North in Azure China 21Vianet aren't currently supported.
If you don't have a control plane managed identity yet, you should go ahead and create one. The following example uses the az identity create command:
az identity create --name myIdentity --resource-group myResourceGroup
The result should look like:
{
"clientId": "<client-id>",
"clientSecretUrl": "<clientSecretUrl>",
"id": "/subscriptions/<subscriptionid>/resourcegroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myIdentity",
"location": "westus2",
"name": "myIdentity",
"principalId": "<principalId>",
"resourceGroup": "myResourceGroup",
"tags": {},
"tenantId": "<tenant-id>",
"type": "Microsoft.ManagedIdentity/userAssignedIdentities"
}
If you don't have a kubelet managed identity yet, you should go ahead and create one. The following example uses the az identity create command:
az identity create --name myKubeletIdentity --resource-group myResourceGroup
The result should look like:
{
"clientId": "<client-id>",
"clientSecretUrl": "<clientSecretUrl>",
"id": "/subscriptions/<subscriptionid>/resourcegroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myKubeletIdentity",
"location": "westus2",
"name": "myKubeletIdentity",
"principalId": "<principalId>",
"resourceGroup": "myResourceGroup",
"tags": {},
"tenantId": "<tenant-id>",
"type": "Microsoft.ManagedIdentity/userAssignedIdentities"
}
If your existing managed identity is part of your subscription, you can use the az identity list command to query it:
az identity list --query "[].{Name:name, Id:id, Location:location}" -o table
Now you can use the following command to create your cluster with your existing identities. Provide the control plane identity resource ID via assign-identity
and the kubelet managed identity via assign-kubelet-identity
:
az aks create \
--resource-group myResourceGroup \
--name myManagedCluster \
--network-plugin azure \
--vnet-subnet-id <subnet-id> \
--docker-bridge-address 172.17.0.1/16 \
--dns-service-ip 10.2.0.10 \
--service-cidr 10.2.0.0/24 \
--enable-managed-identity \
--assign-identity <identity-resource-id> \
--assign-kubelet-identity <kubelet-identity-resource-id>
A successful cluster creation using your own kubelet managed identity contains the following output:
"identity": {
"principalId": null,
"tenantId": null,
"type": "UserAssigned",
"userAssignedIdentities": {
"/subscriptions/<subscriptionid>/resourcegroups/resourcegroups/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myIdentity": {
"clientId": "<client-id>",
"principalId": "<principal-id>"
}
}
},
"identityProfile": {
"kubeletidentity": {
"clientId": "<client-id>",
"objectId": "<object-id>",
"resourceId": "/subscriptions/<subscriptionid>/resourcegroups/resourcegroups/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myKubeletIdentity"
}
},
Update kubelet identity on an existing cluster with your existing identities.
# Check the version of Azure CLI modules
az version
# Upgrade the version to make sure it is 2.37.0 or later
az upgrade
Now you can use the following command to update your cluster with your existing identities. Provide the control plane identity resource ID via assign-identity
and the kubelet managed identity via assign-kubelet-identity
:
az aks update \
--resource-group myResourceGroup \
--name myManagedCluster \
--enable-managed-identity \
--assign-identity <identity-resource-id> \
--assign-kubelet-identity <kubelet-identity-resource-id>
A successful cluster update using your own kubelet managed identity contains the following output:
"identity": {
"principalId": null,
"tenantId": null,
"type": "UserAssigned",
"userAssignedIdentities": {
"/subscriptions/<subscriptionid>/resourcegroups/resourcegroups/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myIdentity": {
"clientId": "<client-id>",
"principalId": "<principal-id>"
}
}
},
"identityProfile": {
"kubeletidentity": {
"clientId": "<client-id>",
"objectId": "<object-id>",
"resourceId": "/subscriptions/<subscriptionid>/resourcegroups/resourcegroups/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myKubeletIdentity"
}
},
- Use Azure Resource Manager templates to create Managed Identity enabled clusters.