Per availability zone service

General Discussions
1

Hi
we are using AWS and have several deployment in 3 zones. As traffic between zones we pay extra, we would like to make requests stays in the same zone as much as possible.

So the idea was create services per zone, filtering the app label AND the node availability zone where the pod is located. In 1.17 i can see that “Cloud Provider Labels” are official, is there any oficial way to push the node label to the pod? that looks like the easiest way… but i recall reading something about 1.18 and zoned services, but can’t find it… but i’m not totally sure.

So what is the best way to keep traffic in the same zone in kubernetees. 1.15 is what we have, but we do plan to upgrade to 1.18 when possible, so not limited to use more recent and correct ways.

thanks

Cluster information:

Kubernetes version: 1.15
Cloud being used: aws
Installation method: kops
Host OS: debian

2

The short answer is that this is not finished yet, so there isn’t a
“good” answer. You can cobble it together yourself by running
multiple services, but ick. It is something we know we need to
address, and we’ve had a number of discussions about how to implement
it, but it’s just not fully designed and implemented yet.

3

Hi

Thanks for the answer!

Having multiple services isn’t a big issue, a single service could allow fallback to other zones if ever needed, but i think we can live without it for now.

For multiple services path, does that requires also multiple deployments, one for each zone for all apps? or is there any way to push the node zone to the pod labels? If we had the zone label in the pods, we could refer in each service the pods in that zone only, like this:

  • kind: Service
    name: app1-a
    metadata:
    labels:
    run: app1
    zone: eu-west-1a
  • kind: Service
    name: app1-b
    metadata:
    labels:
    run: app1
    zone: eu-west-1b

But for this to work and keep only one deploytment, pods need to be populated with the zone label automatically. Any kubernete way to do this?

Thanks for the help

4

You’d need to let pods schedule and then copy the labels from the node
to the pod as a separate step. I’m hoping we can actually do that
automatically, but it’s not done yet.
https://github.com/kubernetes/enhancements/pull/1660 is a prereq
before we can consider it.

5

This kafka init script is loaded in the initContainer and can read the node zone label and push it to the initializing pod. I think this way we can have the already scheduled pods with their own zone info and use it later in the per zone service. Of course i need to allow that pod to alter the labels, but without a native way, this looks the best way to achieve this :slight_smile:

Thanks for the help , it was very useful!