Affinity In Nginx Ingress Controller-1
What is session affinity
Let’s assume you have a stateful application that you have deployed in some container orchestration environment, let’s say Kubernetes. These kinds of applications direct traffic from a client to the same pod. This is usually important when a client makes the first request to the server, a session object is created. This object may be stored in server RAM, a file on the server, passed back to the client in an HTTP cookie, stored in a database, or stored in the client. All subsequent requests use the same session object. When you store the session object in a single server’s RAM or file system, then the only way for the client to continue a session in the next request is for the browser to return to the same server instance.
How K8s Handle Routing Traffic Without Session Affinity
As explained in their official documentation[1] Kubernetes uses iptables to distribute traffic across a set of pods. To explain this simply I will use an example. Let’s say there is an application pod named demo-app. And there are two replicas of this demo-app.
For these two replicas Kubernetes assigns two real ip addresses as 10.100.1.164, 10.100.1.165. For these two pods we create a service pointing to both of these pods. When creating this service Kubernetes creates a virtual Cluster IP address for this service. Which is 172.20.86.5.
$ kubectl get service foo
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
foo ClusterIP 172.20.86.5 <none> 443:30937/TCP 12m
This IP address will be used in the map in Nginx Ingress Controller. When a request is made to a host and a path Ingress controller maps the request to a service. Controllers maintain a map for cluster ip and service name.
When the request is routed to the service level the Kubernetes documentation says the load balancing method by default is round robin, but this is not entirely accurate. If you look at iptables on any of the worker nodes, you can see that for the service demo app with ClusterIP of 172.20.86.5 and 2 pods.
The service uses statistical mode random probability to randomly send traffic to one of the two pods defined here; each pod has a 50% chance of being chosen to receive traffic.
How do we configure session affinity in Kubernetes
We can use an ingress template as below to configure session affinity. Here we define a cookie that is used from nginx controller for handling session affinity.
apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: demo-ing annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/affinity: "cookie" nginx.ingress.kubernetes.io/session-cookie-name: "afc"spec: rules: - host: demo.com http: paths: - path: / backend: serviceName: demo-app servicePort: 80
How this cookie is handling the session affinity. After this is configured the deployment changes as below
Here as you can see ingress controller maintain a continuous session with one application pod.
Life Cycle of a Session Cookie
The session cookie gets created in the nginx ingress controller. The process get triggered when a requests hits the controller. The expiration time of this cookie can be configured from the ingress annotations. [1]
nginx.ingress.kubernetes.io/session-cookie-max-age
The browser may store the cookie and send it back to the same server with later requests.
Session cookies are deleted when the current session ends. The browser defines when the “current session” ends, and some browsers use session restoring when restarting. This can cause session cookies to last indefinitely. [3]
Format of the Session Cookie
Format of the session cookie is defined in [2],
ngx.now() + i, ngx.worker.pid(), math_random(999999)
- ngx.now() — Time stamp
- i- Iterative value from 1 to 20
- ngx.worker.pid() — Process ID of the nginx worker
Example for this is as follows 1668359498.145.369
So this blog basically explained what is session affinity, why we need session affinity, how to configure session affinity ,and the basic details about the approach that used by nginx ingress controller.
There is a more interesting story on how this cookie is used internally in nginx ingress controller to identify the upstream. This involves going through the lua scripts in the nginx controller. That will be discussed in the part two of this blog.
[1]. https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/
