Kubernetes version:
Cloud being used: AKS
Installation method:
Host OS: Windows
CNI and version:
CRI and version:
Hi guys, need an advice. There’s a need to expose every pod individually to be externally reachable by some fqdn:port. It’s needed because pod is running stateful workload (media session).
What is the scalable way to make every pod reachable and addressable?
What if’s not necessarily HTTPS traffic that needs to be addressed to specific pod?
To expose a pod you need an IP and a port, both of which are either static or virtualized. If static is okay, you can use something like a node port service, but those port numbers are generally not human friendly.
You could do something like a pod host port if you’re willing to either lock a pod to a specific node, or to use dynamic DNS.
If you don’t want to do either of those, then your only real choice is a load balancer type service for each pod. This will provision a new IP address for each one. The selectors you use is up to you. You need a different one for every pod. This is approaching the design of StatefulSet.
Thanks, @thockin .
It is StatefulSet.
My concerns with one Service for each single pod - is scalability of resources behind it. It will require one Azure Load Balancer per each pod and we might have thousands of them.
Hostport solution, correct me if I’m wrong allows only exposing single pod on a node (or multiple but not single addressable), as it opens just one port of a node.
There’s no requirements for ports to be human readable 
@thockin , from IP perspective I have 2 IPs per pod: k8s private one and public IP.
But I need fqdn name matching SSL certificate that all pods have.
So if to try to expose every pod directly with its public IP - it will require to create DNS record for every single pod matching that certificate SN.
“Real” IP addresses are usually expensive and carefully managed resources.
Kubernetes automates allocation of IPs in a pretty abstract way, which works for many/most use cases. It may not be ideal for what you have in mind. This is, in large part, because MOST apps care about specific port numbers. If you run an HTTPS app and it’s not on port 443, your UX suffers.
If you don’t care about port numbers, you can “share” N IP addresses acorss M pods. For example, you can use 30K unique port numbers on one IP to front 30K pods. Now all you need is to set up the traffic path.
It’s possible, when setting up a Service, to ask for a specific IP. Some clouds allow you to share it across Services, as long as the ports are disjoint. But some also charge for each “route”, which might mean IP:port pair.
You could try having a small number of Services, and add ports blocks to them, but there are practical limits to how many ports you can actually list in a Service - YAML would be very cumbersome with 30K port definitions, and it may fall over elsewhere, but I have not tried it.
This may be a case where all of the existing automation misses the mark for what you want to do, and you may need to try to DIY.