Well, that depends on where you are running and what infrastructure you have
available. This doesn’t seem to be a Kubernetes-specific issue - how would you
normally expose a VIP to these clients which is consistently routed to N
backends?
If you are on a cloud, they all have L4 LBs that do this.
If you are not, you need something to do that. You can set up F5 or IPVS or
something - that’s outside the domain of k8s.
Think about how the traffic and routing decisions are made. I don’t know
enough about your situation to fill in the blanks, but:
Client sends UDP traffic to an IP. How did they find that IP? How do they
know which port?
That IP is routed to a pod (your L7). If there are N > 1 such pods, who
decides which one gets used by this client, and how do they steer it there?
Once it is at your L7, you can make app-aware decisions and keep state, such
that responses and subsequent packets are handled.
Subsequent packets from the client are sent to that same IP. How do you ensure
they go the same L7 (which has state!) as before? Or how do you do
state-sharing between L7s?
There are too many decisions here to answer in the abstract. As an example,
here’s how it works (if I get your arch correctly) on Google Cloud:
You deploy your L7s as pods.
You create a Service with type=LoadBalancer (and session affinity on) for those
L7s, that gives you a public IP.
You put that IP into a DNS record somewhere.
Your clients send UDP to that IP.
Our L4 infrastructure (which only understands VMs) hashes on client IP and
sends the UDP traffic to a specific VM.
That VM hashes on client IP and service IP and sends the UDP traffic to a
specific L7 pod (and remembers that decision).
That’s your pod, you do whatever you need, and if there is a response it comes
from that pod.
The VM reverses the connection state and replies directly to your client (which
is blissfully unaware of any of this
Subsequent packets are sent to the same VM, which has connection state and
forwards to the same pod, which may have game state.
That all assumes you have enough info IN YOUR OWN PACKET to do the L7 routing.
If you need unique ports for each game, its a different story.
k8s doesn’t “handle it”. We have an API for “I need an external LB, someone
please make that happen”, and we have a handful of cloud-provider modules which
implement that request. If you are not in a cloud, you need to decide who is
answering that request and how.
When I mentioned the affinity API, what I meant is that whomever implements the
external LB has to support it, or it won’t work.