Can we do a header based routing using session affinity or sticky sessions using kubernetes itself with an arbitrary header instead of cookies or clientIP ?
The first request to the service should be randomly(or by a load balancing algorithm) routed to a pod and the mapping ‘value_of_certain_header -> pod_location’ should be saved somehow so next request will be routed to specific pod.
It’s going to depend on the controller for the resource itself. Are you referring to ingress or are you asking about a service load balancer?
I am looking towards using either nginx-ingress or kubernetes service itself
I don’t think Service or Ingress objects provide that functionality themselves in Kubernetes. The question hinges on the underlying hardware or application that will be routing the traffic.
The only thing I could find for the Nginx Ingress Controller was related to using cookies: Sticky Sessions - NGINX Ingress Controller
If you need a lot of flexibility, Traefik might be worth looking into. You can do matches on headers, prefixes, and hostnames to route as you want. If you install it, use the traefik helm chart to get v2 and when referencing does keep in mind that it is not backwards compatible with v1.