Skip to content

Files

Latest commit

miwithromiwithro
Update http-proxy.md
May 24, 2022
0689d11 · May 24, 2022

History

History
136 lines (101 loc) · 6.13 KB

http-proxy.md

File metadata and controls

136 lines (101 loc) · 6.13 KB
title description services author ms.topic ms.date ms.author
Configuring Azure Kubernetes Service (AKS) nodes with an HTTP proxy
Use the HTTP proxy configuration feature for Azure Kubernetes Service (AKS) nodes.
container-service
nickomang
article
05/23/2022
nickoman

HTTP proxy support in Azure Kubernetes Service

Azure Kubernetes Service (AKS) clusters, whether deployed into a managed or custom virtual network, have certain outbound dependencies necessary to function properly. Previously, in environments requiring internet access to be routed through HTTP proxies, this was a problem. Nodes had no way of bootstrapping the configuration, environment variables, and certificates necessary to access internet services.

This feature adds HTTP proxy support to AKS clusters, exposing a straightforward interface that cluster operators can use to secure AKS-required network traffic in proxy-dependent environments.

Some more complex solutions may require creating a chain of trust to establish secure communications across the network. The feature also enables installation of a trusted certificate authority onto the nodes as part of bootstrapping a cluster.

Limitations and other details

The following scenarios are not supported:

  • Different proxy configurations per node pool
  • Updating proxy settings post cluster creation
  • User/Password authentication
  • Custom CAs for API server communication
  • Windows-based clusters
  • Node pools using Virtual Machine Availability Sets (VMAS)

By default, httpProxy, httpsProxy, and trustedCa have no value.

Prerequisites

Configuring an HTTP proxy using Azure CLI

Using AKS with an HTTP proxy is done at cluster creation, using the az aks create command and passing in configuration as a JSON file.

The schema for the config file looks like this:

{
  "httpProxy": "string",
  "httpsProxy": "string",
  "noProxy": [
    "string"
  ],
  "trustedCa": "string"
}

httpProxy: A proxy URL to use for creating HTTP connections outside the cluster. The URL scheme must be http. httpsProxy: A proxy URL to use for creating HTTPS connections outside the cluster. If this is not specified, then httpProxy is used for both HTTP and HTTPS connections. noProxy: A list of destination domain names, domains, IP addresses or other network CIDRs to exclude proxying. trustedCa: A string containing the base64 encoded alternative CA certificate content. For now we only support PEM format. Another thing to note is that, for compatibility with Go-based components that are part of the Kubernetes system, the certificate MUST support Subject Alternative Names(SANs) instead of the deprecated Common Name certs.

Example input: Note the CA cert should be the base64 encoded string of the PEM format cert content.

{
  "httpProxy": "http://myproxy.server.com:8080/", 
  "httpsProxy": "https://myproxy.server.com:8080/", 
  "noProxy": [
    "localhost",
    "127.0.0.1"
  ],
  "trustedCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUgvVENDQmVXZ0F3SUJB...b3Rpbk15RGszaWFyCkYxMFlscWNPbWVYMXVGbUtiZGkvWG9yR2xrQ29NRjNURHg4cm1wOURCaUIvCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0="
}

Create a file and provide values for httpProxy, httpsProxy, and noProxy. If your environment requires it, also provide a trustedCa value. Next, deploy a cluster, passing in your filename via the http-proxy-config flag.

az aks create -n $clusterName -g $resourceGroup --http-proxy-config aks-proxy-config.json

Your cluster will initialize with the HTTP proxy configured on the nodes.

Configuring an HTTP proxy using Azure Resource Manager (ARM) templates

Deploying an AKS cluster with an HTTP proxy configured via ARM template is straightforward. The same schema used for CLI deployment exists in the Microsoft.ContainerService/managedClusters definition under properties:

"properties": {
    ...,
    "httpProxyConfig": {
        "httpProxy": "string",
        "httpsProxy": "string",
        "noProxy": [
            "string"
        ],
        "trustedCa": "string"
    }
}

In your template, provide values for httpProxy, httpsProxy, and noProxy. If necessary, also provide a value for `trustedCa. Deploy the template, and your cluster should initialize with your HTTP proxy configured on the nodes.

Handling CA rollover

Values for httpProxy, httpsProxy, and noProxy cannot be changed after cluster creation. However, to support rolling CA certs, the value for trustedCa can be changed and applied to the cluster with the az aks update command.

For example, assuming a new file has been created with the base64 encoded string of the new CA cert called aks-proxy-config-2.json, the following action will update the cluster:

az aks update -n $clusterName -g $resourceGroup --http-proxy-config aks-proxy-config-2.json

Monitoring add-on configuration

When using the HTTP proxy with the Monitoring add-on, the following configurations are supported:

  • Outbound proxy without authentication
  • Outbound proxy with username & password authentication
  • Outbound proxy with trusted cert for Log Analytics endpoint

The following configurations are not supported:

  • The Custom Metrics and Recommended Alerts features are not supported when using proxy with trusted cert
  • Outbound proxy is not supported with Azure Monitor Private Link Scope (AMPLS)

Next steps