Edit

Share via

Advanced NGINX ingress controller and ingress configurations with the application routing add-on

  • Article

The application routing add-on supports two ways to configure ingress controllers and ingress objects:

Prerequisites

An AKS cluster with the application routing add-on.

Connect to your AKS cluster

To connect to the Kubernetes cluster from your local computer, you use kubectl, the Kubernetes command-line client. You can install it locally using the az aks install-cli command. If you use the Azure Cloud Shell, kubectl is already installed.

Configure kubectl to connect to your Kubernetes cluster using the az aks get-credentials command.

az aks get-credentials --resource-group <ResourceGroupName> --name <ClusterName>

Configuration of the NGINX ingress controller

The application routing add-on uses a Kubernetes custom resource definition (CRD) called NginxIngressController to configure NGINX ingress controllers. You can create more ingress controllers or modify existing configuration.

This table shows a reference to properties you can set to configure an NginxIngressController.

Field Type Description Required Default
controllerNamePrefix string Name for the managed NGINX Ingress Controller resources. Yes nginx
customHTTPErrors array Array of error codes to be sent to the default backend in case of an error. No
defaultBackendService object Service to route unmatched HTTP traffic. Contains nested properties: No
 name string Service name. Yes
 namespace string Service namespace. Yes
defaultSSLCertificate object Contains the default certificate for accessing the default backend service. Contains nested properties: No
 forceSSLRedirect boolean Forces HTTPS redirection when a certificate is set. No false
 keyVaultURI string URI for a Key Vault secret storing the certificate. No
 secret object Holds secret information for the default SSL certificate. Contains nested properties: No
  name string Secret name. Yes
  namespace string Secret namespace. Yes
httpDisabled boolean Flag to disable HTTP traffic to the controller. No
ingressClassName string IngressClass name used by the controller. Yes nginx.approuting.kubernetes.azure.com
loadBalancerAnnotations object A map of annotations to control the behavior of the NGINX ingress controller's service by setting load balancer annotations. No
scaling object Configuration for scaling the controller. Contains nested properties: No
 maxReplicas integer Upper limit for replicas. No 100
 minReplicas integer Lower limit for replicas. No 2
 threshold string Scaling threshold defining how aggressively to scale. rapid scales quickly for sudden spikes, steady favors cost-effectiveness, and balanced is a mix. No balanced

Common configurations

Control the default NGINX ingress controller configuration

When you enable the application routing add-on with NGINX, it creates an ingress controller called default in the app-routing-namespace configured with a public facing Azure load balancer. That ingress controller uses an ingress class name of webapprouting.kubernetes.azure.com.

You can also control if the default gets a public or an internal IP, or if it gets created at all when enabling the add-on.

Here are the possible configuration options:

  • None: The default Nginx ingress controller is not created and will not be deleted if it already exists. Users should delete the default NginxIngressController custom resource manually if desired.
  • Internal: The default Nginx ingress controller is created with an internal load balancer. Any annotations changes on the NginxIngressController custom resource to make it external will be overwritten.
  • External: The default Nginx ingress controller created with an external load balancer. Any annotations changes on the NginxIngressController custom resource to make it internal will be overwritten.
  • AnnotationControlled (default): The default Nginx ingress controller is created with an external load balancer. Users can edit the default NginxIngressController custom resource to configure load balancer annotations.

Control the default ingress controller configuration when creating the cluster

To enable application routing on a new cluster, use the az aks create command, specifying the --enable-app-routing and the --app-routing-default-nginx-controller flags. You need to set the <DefaultIngressControllerType> to one of the configuration options described earlier.

az aks create \
--resource-group <ResourceGroupName> \
--name <ClusterName> \
--___location <Location> \
--enable-app-routing \
--app-routing-default-nginx-controller <DefaultIngressControllerType>

Update the default ingress controller configuration on an existing cluster

To update the application routing default ingress controller configuration on an existing cluster, use the az aks approuting update command, specifying the --nginx flag. You need to set the <DefaultIngressControllerType> to one of the configuration options described earlier.

az aks approuting update --resource-group <ResourceGroupName> --name <ClusterName> --nginx <DefaultIngressControllerType>

Create another public facing NGINX ingress controller

To create another NGINX ingress controller with a public facing Azure Load Balancer:

  1. Copy the following YAML manifest into a new file named nginx-public-controller.yaml and save the file to your local computer.

    apiVersion: approuting.kubernetes.azure.com/v1alpha1
kind: NginxIngressController
metadata:
  name: nginx-public
spec:
  ingressClassName: nginx-public
  controllerNamePrefix: nginx-public

  2. Create the NGINX ingress controller resources using the kubectl apply command.

    kubectl apply -f nginx-public-controller.yaml

    The following example output shows the created resource:

    nginxingresscontroller.approuting.kubernetes.azure.com/nginx-public created

Create an internal NGINX ingress controller with a private IP address

To create an NGINX ingress controller with an internal facing Azure Load Balancer with a private IP address:

  1. Copy the following YAML manifest into a new file named nginx-internal-controller.yaml and save the file to your local computer.

    apiVersion: approuting.kubernetes.azure.com/v1alpha1
kind: NginxIngressController
metadata:
  name: nginx-internal
spec:
  ingressClassName: nginx-internal
  controllerNamePrefix: nginx-internal
  loadBalancerAnnotations: 
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"

  2. Create the NGINX ingress controller resources using the kubectl apply command.

    kubectl apply -f nginx-internal-controller.yaml

    The following example output shows the created resource:

    nginxingresscontroller.approuting.kubernetes.azure.com/nginx-internal created

Create an NGINX ingress controller with a static IP address

To create an NGINX ingress controller with a static IP address on the Azure Load Balancer:

  1. Create an Azure resource group using the az group create command.

    az group create --name myNetworkResourceGroup --___location eastus

  2. Create a static public IP address using the az network public ip create command.

    az network public-ip create \
    --resource-group myNetworkResourceGroup \
    --name myIngressPublicIP \
    --sku Standard \
    --allocation-method static

    Note

    If you're using a Basic SKU load balancer in your AKS cluster, use Basic for the --sku parameter when defining a public IP. Only Basic SKU IPs work with the Basic SKU load balancer and only Standard SKU IPs work with Standard SKU load balancers.

  3. Ensure the cluster identity used by the AKS cluster has delegated permissions to the public IP's resource group using the az role assignment create command.

    Note

    Update <ClusterName> and <ClusterResourceGroup> with your AKS cluster's name and resource group name.

    CLIENT_ID=$(az aks show --name <ClusterName> --resource-group <ClusterResourceGroup> --query identity.principalId -o tsv)
RG_SCOPE=$(az group show --name myNetworkResourceGroup --query id -o tsv)
az role assignment create \
    --assignee ${CLIENT_ID} \
    --role "Network Contributor" \
    --scope ${RG_SCOPE}

  4. Copy the following YAML manifest into a new file named nginx-staticip-controller.yaml and save the file to your local computer.

    Note

    You can either use service.beta.kubernetes.io/azure-pip-name for public IP name, or use service.beta.kubernetes.io/azure-load-balancer-ipv4 for an IPv4 address and service.beta.kubernetes.io/azure-load-balancer-ipv6 for an IPv6 address, as shown in the example YAML. Adding the service.beta.kubernetes.io/azure-pip-name annotation ensures the most efficient LoadBalancer creation and is highly recommended to avoid potential throttling.

    apiVersion: approuting.kubernetes.azure.com/v1alpha1
kind: NginxIngressController
metadata:
  name: nginx-static
spec:
  ingressClassName: nginx-static
  controllerNamePrefix: nginx-static
  loadBalancerAnnotations: 
    service.beta.kubernetes.io/azure-pip-name: "myIngressPublicIP"
    service.beta.kubernetes.io/azure-load-balancer-resource-group: "myNetworkResourceGroup"

  5. Create the NGINX ingress controller resources using the kubectl apply command.

    kubectl apply -f nginx-staticip-controller.yaml

    The following example output shows the created resource:

    nginxingresscontroller.approuting.kubernetes.azure.com/nginx-static created

Verify the ingress controller was created

You can verify the status of the NGINX ingress controller using the kubectl get nginxingresscontroller command.

Note

Update <IngressControllerName> with name you used when creating the `NginxIngressController``.

kubectl get nginxingresscontroller -n <IngressControllerName>

The following example output shows the created resource. It may take a few minutes for the controller to be available:

NAME           INGRESSCLASS   CONTROLLERNAMEPREFIX   AVAILABLE
nginx-public   nginx-public   nginx                  True

You can also view the conditions to troubleshoot any issues:

kubectl get nginxingresscontroller -n <IngressControllerName> -o jsonpath='{range .items[*].status.conditions[*]}{.lastTransitionTime}{"\t"}{.status}{"\t"}{.type}{"\t"}{.message}{"\n"}{end}'

The following example output shows the conditions of a healthy ingress controller:

2023-11-29T19:59:24Z    True    IngressClassReady       Ingress Class is up-to-date
2023-11-29T19:59:50Z    True    Available               Controller Deployment has minimum availability and IngressClass is up-to-date
2023-11-29T19:59:50Z    True    ControllerAvailable     Controller Deployment is available
2023-11-29T19:59:25Z    True    Progressing             Controller Deployment has successfully progressed

Use the ingress controller in an ingress

  1. Copy the following YAML manifest into a new file named ingress.yaml and save the file to your local computer.

    Note

    Update <Hostname> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <Hostname>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

  2. Create the cluster resources using the kubectl apply command.

    kubectl apply -f ingress.yaml -n hello-web-app-routing

    The following example output shows the created resource:

    ingress.networking.k8s.io/aks-helloworld created

Verify the managed Ingress was created

You can verify the managed Ingress was created using the kubectl get ingress command.

kubectl get ingress -n hello-web-app-routing

The following example output shows the created managed Ingress. The ingress class, host, and IP address may be different:

NAME             CLASS                                HOSTS               ADDRESS       PORTS     AGE
aks-helloworld   webapprouting.kubernetes.azure.com   myapp.contoso.com   20.51.92.19   80, 443   4m

Clean up of ingress controllers

You can remove the NGINX ingress controller using the kubectl delete nginxingresscontroller command.

Note

Update <IngressControllerName> with name you used when creating the NginxIngressController.

kubectl delete nginxingresscontroller -n <IngressControllerName>

Configuration per ingress resource through annotations

The NGINX ingress controller supports adding annotations to specific Ingress objects to customize their behavior.

You can annotate the ingress object by adding the respective annotation in the metadata.annotations field.

Note

Annotation keys and values can only be strings. Other types, such as boolean or numeric values must be quoted, i.e. "true", "false", "100".

Here are some examples annotations for common configurations. Review the NGINX ingress annotations documentation for a full list.

Custom max body size

For NGINX, a 413 error is returned to the client when the size in a request exceeds the maximum allowed size of the client request body. To override the default value, use the annotation:

nginx.ingress.kubernetes.io/proxy-body-size: 4m

Here's an example ingress configuration using this annotation:

Note

Update <Hostname> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 4m
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <Hostname>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Custom connection time out

You can change the time out that the NGINX ingress controller waits to close a connection with your workload. All time out values are unitless and in seconds. To override the default time out, use the following annotation to set a valid 120-seconds proxy read time out:

nginx.ingress.kubernetes.io/proxy-read-timeout: "120"

Review custom time outs for other configuration options.

Here's an example ingress configuration using this annotation:

Note

Update <Hostname> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <Hostname>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Backend protocol

By default the NGINX ingress controller uses HTTP to reach the services. To configure alternative backend protocols such as HTTPS or GRPC, use the annotation:

nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

or

nginx.ingress.kubernetes.io/backend-protocol: "GRPC"

Review backend protocols for other configuration options.

Here's an example ingress configuration using this annotation:

Note

Update <Hostname> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <Hostname>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Cross-Origin Resource Sharing (CORS)

To enable Cross-Origin Resource Sharing (CORS) in an Ingress rule, use the annotation:

nginx.ingress.kubernetes.io/enable-cors: "true"

Review enable CORS for other configuration options.

Here's an example ingress configuration using this annotation:

Note

Update <Hostname> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <Hostname>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

Disable SSL redirect

By default the controller redirects (308) to HTTPS if TLS is enabled for an ingress. To disable this feature for specific ingress resources, use the annotation:

nginx.ingress.kubernetes.io/ssl-redirect: "false"

Review server-side HTTPS enforcement through redirect for other configuration options.

Here's an example ingress configuration using this annotation:

Note

Update <Hostname> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <Hostname>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix

URL rewriting

In some scenarios, the exposed URL in the backend service differs from the specified path in the Ingress rule. Without a rewrite any request returns 404. This configuration is useful with path based routing where you can serve two different web applications under the same ___domain. You can set path expected by the service using the annotation:

nginx.ingress.kubernetes.io/rewrite-target": /$2

Here's an example ingress configuration using this annotation:

Note

Update <Hostname> with your DNS host name. The <IngressClassName> is the one you defined when creating the NginxIngressController.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  ingressClassName: <IngressClassName>
  rules:
  - host: <Hostname>
    http:
      paths:
      - path: /app-one(/|$)(.*)
        pathType: Prefix 
        backend:
          service:
            name: app-one
            port:
              number: 80
      - path: /app-two(/|$)(.*)
        pathType: Prefix 
        backend:
          service:
            name: app-two
            port:
              number: 80

Next steps

Learn about monitoring the ingress-nginx controller metrics included with the application routing add-on with with Prometheus in Grafana as part of analyzing the performance and usage of your application.