Hello,
I have a bare-metal K8s cluster with 3 nodes, with 1 master. My goal is to run multiple applications with replicas, being accessible outside the cluster, with source IP preserved.
For example, I would like to run an SSH-accessible container, and a webserver. One pod of each is running on each worker node. I’m looking for the possibility to have one external IP, load-balacing by IP session affinity, depending on the source port : 22 redirected to SSH pods, 80 to the webservers.
I looked metallb which can handle the load-balancing part apparently, however the Local externalTrafficPolicy doesn’t allow cross-nodes communications, so I do not know how I could, from a service, balance on multiple nodes. I’ve been testing some configurations, but accessing a pod running on my second node doesn’t respond to my SSH request.
Moreover, I need to adress the multi-services question regarding source ports.
We could sum up the situation with this image: one external endpoint routing/load-balacing by source port and source IP, on pods running accross nodes. Should I use service, ingress, a solution like Traefik or Contour in addition to metallb? Or use something like HAProxy, whether within the cluster or not? I’m a bit lost.
Thanks,
EDIT: I read somewhere I could somehow process requests with DNAT in order to redirect to the right pod, responses would go through a gateway, avoiding SNAT use, hence source IP being preserved.
Cluster information:
Kubernetes version: 1.17
Cloud being used: bare-metal
Installation method:
Host OS: Debian 10
CNI and version: Calico