We are using EKS and we find that when using preferredDuringSchedulingIgnoredDuringExecution doesn’t guarantee scheduling of pods evenly in zone/host. Our situation is that we have 3 zones and 3 EC2 instances in each zone (Min value of our ASG is 9). Though there would be a pod already on a node, scheduler would still schedule a second pod on it while there is another host that doesnt have any pods scheduled on it. Same with zones too, we would notice that only 2 zones get pods scheduled with 1 zone completely ignored.
We need to understand if preferredDuringSchedulingIgnoredDuringExecution guarantees even spread of pods across zones/hosts given the way we are using pod antiaffinity.
Cluster information:
Kubernetes version: 1.16
Cloud being used: (put bare-metal if not on a public cloud)
Installation method: EKS
Host OS: Amazon Linux
CNI and version:
CRI and version:
YAML Code snippet
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app: xyz
release: xyz
topologyKey: failure-domain.beta.kubernetes.io/zone
namespaces:
weight: 60
- podAffinityTerm:
labelSelector:
matchLabels:
app: xyz
release: xyz
topologyKey: kubernetes.io/hostname
weight: 40