Advanced NGINX ingress controller and ingress configurations with the application routing add-on for Azure Kubernetes Service (AKS)

Configuration properties for NGINX ingress controllers

The application routing add-on uses a Kubernetes custom resource definition (CRD) called NginxIngressController to configure NGINX ingress controllers. You can create more ingress controllers or modify existing configurations.

The following table lists properties you can set to configure an NginxIngressController:

Control the default NGINX ingress controller configuration

When you enable the application routing add-on with NGINX, it creates an ingress controller called default in the app-routing-namespace configured with a public facing Azure load balancer. That ingress controller uses an ingress class name of webapprouting.kubernetes.azure.com.

You can also control if the default gets a public or an internal IP, or if it gets created at all when enabling the add-on.

Possible configuration options include:

• None: The default NGINX ingress controller isn't created and isn't deleted if it already exists. You should manually delete the default NginxIngressController custom resource if desired.
• Internal: The default NGINX ingress controller is created with an internal load balancer. Any annotation changes on the NginxIngressController custom resource to make it external are overwritten.
• External: The default NGINX ingress controller created with an external load balancer. Any annotation changes on the NginxIngressController custom resource to make it internal are overwritten.
• AnnotationControlled (default): The default NGINX ingress controller is created with an external load balancer. You can edit the default NginxIngressController custom resource to configure load balancer annotations.)

Control the default ingress controller configuration on a new cluster

Update the default ingress controller configuration on an existing cluster

Create another public facing NGINX ingress controller

1. Copy the following YAML manifest into a new file named nginx-public-controller.yaml and save the file to your local computer.

   apiVersion: approuting.kubernetes.azure.com/v1alpha1
   kind: NginxIngressController
   metadata:
     name: nginx-public
   spec:
     ingressClassName: nginx-public
     controllerNamePrefix: nginx-public
   

2. Create the NGINX ingress controller resources using the kubectl apply command.

   kubectl apply -f nginx-public-controller.yaml
   

   The following example output shows the created resource:

   nginxingresscontroller.approuting.kubernetes.azure.com/nginx-public created
   

Create an internal NGINX ingress controller with a private IP address

1. Copy the following YAML manifest into a new file named nginx-internal-controller.yaml and save the file to your local computer.

   apiVersion: approuting.kubernetes.azure.com/v1alpha1
   kind: NginxIngressController
   metadata:
     name: nginx-internal
   spec:
     ingressClassName: nginx-internal
     controllerNamePrefix: nginx-internal
     loadBalancerAnnotations: 
       service.beta.kubernetes.io/azure-load-balancer-internal: "true"
   

2. Create the NGINX ingress controller resources using the kubectl apply command.

   kubectl apply -f nginx-internal-controller.yaml
   

   The following example output shows the created resource:

   nginxingresscontroller.approuting.kubernetes.azure.com/nginx-internal created
   

Create an NGINX ingress controller with a static IP address

1. Create an Azure resource group using the az group create command.

   az group create --name $NETWORK_RESOURCE_GROUP --___location $LOCATION
   

2. Create a static public IP address using the az network public ip create command.

   az network public-ip create \
     --resource-group $NETWORK_RESOURCE_GROUP \
     --name $PUBLIC_IP_NAME \
     --sku Standard \
     --allocation-method static
   

   Note

   If you're using a Basic SKU load balancer in your AKS cluster, use Basic for the --sku parameter when defining a public IP. Only Basic SKU IPs work with the Basic SKU load balancer and only Standard SKU IPs work with the Standard SKU load balancers.

3. Ensure the cluster identity used by the AKS cluster has delegated permissions to the public IP's resource group using the az role assignment create command.

   CLIENT_ID=$(az aks show --name $CLUSTER_NAME --resource-group $RESOURCE_GROUP --query identity.principalId -o tsv)
   RG_SCOPE=$(az group show --name $NETWORK_RESOURCE_GROUP --query id -o tsv)
   az role assignment create \
     --assignee ${CLIENT_ID} \
     --role "Network Contributor" \
     --scope ${RG_SCOPE}
   

4. Copy the following YAML manifest into a new file named nginx-staticip-controller.yaml and save the file to your local computer.

   Note

   You can either use service.beta.kubernetes.io/azure-pip-name for public IP name, or use service.beta.kubernetes.io/azure-load-balancer-ipv4 for an IPv4 address and service.beta.kubernetes.io/azure-load-balancer-ipv6 for an IPv6 address, as shown in the example YAML. Adding the service.beta.kubernetes.io/azure-pip-name annotation ensures the most efficient Load Balancer creation and is highly recommended to avoid potential throttling.

   apiVersion: approuting.kubernetes.azure.com/v1alpha1
   kind: NginxIngressController
   metadata:
     name: nginx-static
   spec:
     ingressClassName: nginx-static
     controllerNamePrefix: nginx-static
     loadBalancerAnnotations: 
       service.beta.kubernetes.io/azure-pip-name: "$PUBLIC_IP_NAME"
       service.beta.kubernetes.io/azure-load-balancer-resource-group: "$NETWORK_RESOURCE_GROUP"
   

5. Create the NGINX ingress controller resources using the kubectl apply command.

   kubectl apply -f nginx-staticip-controller.yaml
   

   The following example output shows the created resource:

   nginxingresscontroller.approuting.kubernetes.azure.com/nginx-static created
   

Verify the ingress controller was created

• Verify the status of the NGINX ingress controller using the kubectl get nginxingresscontroller command.

  kubectl get nginxingresscontroller --name $INGRESS_CONTROLLER_NAME
  

  The following example output shows the created resource. It may take a few minutes for the controller to be available:

  NAME           INGRESSCLASS   CONTROLLERNAMEPREFIX   AVAILABLE
  nginx-public   nginx-public   nginx                  True
  

View the conditions of the ingress controller

• View the conditions of the ingress controller to troubleshoot any issues using the kubectl get nginxingresscontroller command.

  kubectl get nginxingresscontroller --name $INGRESS_CONTROLLER_NAME -o jsonpath='{range .items[*].status.conditions[*]}{.lastTransitionTime}{"\t"}{.status}{"\t"}{.type}{"\t"}{.message}{"\n"}{end}'
  

  The following example output shows the conditions of a healthy ingress controller:

  2023-11-29T19:59:24Z    True    IngressClassReady       Ingress Class is up-to-date
  2023-11-29T19:59:50Z    True    Available               Controller Deployment has minimum availability and IngressClass is up-to-date
  2023-11-29T19:59:50Z    True    ControllerAvailable     Controller Deployment is available
  2023-11-29T19:59:25Z    True    Progressing             Controller Deployment has successfully progressed
  

Use the ingress controller in an ingress

1. Copy the following YAML manifest into a new file named ingress.yaml and save the file to your local computer.

   Note

   Update <HostName> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: aks-helloworld
     namespace: hello-web-app-routing
   spec:
     ingressClassName: <IngressClassName>
     rules:
     - host: <HostName>
       http:
         paths:
         - backend:
             service:
               name: aks-helloworld
               port:
                 number: 80
           path: /
           pathType: Prefix
   

2. Create the cluster resources using the kubectl apply command.

   kubectl apply -f ingress.yaml --namespace hello-web-app-routing
   

   The following example output shows the created resource:

   ingress.networking.k8s.io/aks-helloworld created
   

Verify the managed ingress was created

• Verify the managed ingress was created using the kubectl get ingress command.

  kubectl get ingress --namespace hello-web-app-routing
  

  Your output should resemble the following example output:

  NAME             CLASS                                HOSTS               ADDRESS       PORTS     AGE
  aks-helloworld   webapprouting.kubernetes.azure.com   myapp.contoso.com   20.51.92.19   80, 443   4m
  

Remove ingress controllers

• Remove the NGINX ingress controller using the kubectl delete nginxingresscontroller command.

  kubectl delete nginxingresscontroller --name $INGRESS_CONTROLLER_NAME
  

Configuration per ingress resource through annotations

The NGINX ingress controller supports adding annotations to specific ingress objects to customize their behavior.

You can annotate the ingress object by adding the respective annotation in the metadata.annotations field.

The following sections provide examples for common configurations. For a full list, see the NGINX ingress annotations documentation.

Custom max body size

For NGINX, a 413 error is returned to the client when the size in a request exceeds the maximum allowed size of the client request body. To override the default value, use the annotation:

nginx.ingress.kubernetes.io/proxy-body-size: 4m

Here's an example ingress configuration using this annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 4m
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <HostName>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Custom connection timeout

You can change the timeout that the NGINX ingress controller waits to close a connection with your workload. All timeout values are unitless and in seconds. To override the default timeout, use the following annotation to set a valid 120-seconds proxy read timeout:

nginx.ingress.kubernetes.io/proxy-read-timeout: "120"

Review custom timeouts for other configuration options.

Here's an example ingress configuration using this annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <HostName>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Backend protocol

The NGINX ingress controller uses HTTP to reach the services by default. To configure alternative backend protocols such as HTTPS or GRPC, use one of the following annotations:

# HTTPS annotation
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

# GRPC annotation
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"

Review backend protocols for other configuration options.

Here's an example ingress configuration using this annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <HostName>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Cross-Origin Resource Sharing (CORS)

To enable Cross-Origin Resource Sharing (CORS) in an Ingress rule, use the following annotation:

nginx.ingress.kubernetes.io/enable-cors: "true"

Review enable CORS for other configuration options.

Here's an example ingress configuration using this annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <HostName>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Disable SSL redirect

The controller redirects (308) to HTTPS if TLS is enabled for an ingress by default. To disable this feature for specific ingress resources, use the following annotation:

nginx.ingress.kubernetes.io/ssl-redirect: "false"

Review server-side HTTPS enforcement through redirect for other configuration options.

Here's an example ingress configuration using this annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <HostName>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

URL rewriting

In some scenarios, the exposed URL in the backend service differs from the specified path in the ingress rule. Without a rewrite any request returns 404. This configuration is useful with path-based routing where you can serve two different web applications under the same ___domain. You can set path expected by the service using the following annotation:

nginx.ingress.kubernetes.io/rewrite-target: /$2

Here's an example ingress configuration using this annotation:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <HostName>
    http:
      paths:
      - path: /app-one(/|$)(.*)
        pathType: Prefix 
        backend:
          service:
            name: app-one
            port:
              number: 80
      - path: /app-two(/|$)(.*)
        pathType: Prefix 
        backend:
          service:
            name: app-two
            port:
              number: 80

NGINX health probe path update

The default health probe path for the Azure Load Balancer associated with the NGINX ingress controller must be set to "/healthz". To ensure correct health checks, verify that the ingress controller service has the following annotation:

metadata:
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path: "/healthz"

If you're using Helm to manage your NGINX ingress controller, you can define the Azure Load Balancer health-probe annotation in a values file and apply it during an upgrade:

controller:
  service:
    annotations:
      service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path: "/healthz"

This configuration helps maintain service availability and avoids unexpected traffic disruption during upgrades.