Hi,
I’m trying to achieve service routing in which PODs on the same Node communicate with each other using endpoints on the same Node (and only using other endpoints in case when there is no “local” endpoint available).
There is trafficDistribution: PreferClose option present on Service (from 1.30) which seems to be exacly the thing I need, but I’m not sure how can I configure my cluster to serve the above purpose. I have prepared sample deployment to test this feature but without any success.
Cluster information:
Kubernetes version: 1.31.3
Cloud being used: bare-metal
Installation method: kubeadm
Host OS: ubuntu 22.04
CNI and version: weave 2.8.1
CRI and version: containerd 1.7.3
Test scenario:
- Create deployment with 2 receiver pods:
apiVersion: v1
kind: Namespace
metadata:
name: test
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/name: service-traffic-example
name: receiver
namespace: test
spec:
replicas: 2
selector:
matchLabels:
app: service-traffic-example
template:
metadata:
labels:
app: service-traffic-example
spec:
containers:
- image: gcr.io/google-samples/hello-app:2.0
name: receiver
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: svc-prefer-close
namespace: test
spec:
ports:
- name: http
protocol: TCP
port: 8080
targetPort: 8080
selector:
app: service-traffic-example
type: ClusterIP
trafficDistribution: PreferClose
- Create additional POD that will use exposed service:
kubectl run sender --image=nicolaka/netshoot -n test
- On my cluster with 2 worker nodes the above commands create 3 pods on 2 different nodes:
kubectl get pods -n test -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
receiver-7cfd89d78d-2sgd9 1/1 Running 0 23h 10.36.0.18 ait-42 <none> <none>
receiver-7cfd89d78d-4vct7 1/1 Running 0 23h 10.42.0.5 ait-03 <none> <none>
sender 1/1 Running 1 (21h ago) 23h 10.36.0.21 ait-42 <none> <none>
- Now, sender and receiver-7cfd89d78d-2sgd9 PODs are on the same node (ait-42) while receiver-7cfd89d78d-4vct7 is on a different one (ait-03).
- From within the sender POD I’m issuing CURL command to test loadbalancing of svc-prefer-close service with the following script:
#!/bin/bash
declare -A host_count
for i in $(seq 1 100); do
response=$(curl -s http://svc-prefer-close.test:8080)
hostname=$(echo "$response" | grep "Hostname:" | cut -d":" -f2 | xargs)
if [[ -n "$hostname" ]]; then
((host_count["$hostname"]++))
fi
done
for host in "${!host_count[@]}"; do
echo "$host: ${host_count[$host]}"
done
- The result shows that the traffic WAS NOT redirected to the receiver POD on the same node, but was distributed more or less evenly on the both PODS:
sender:~# ./script.sh
receiver-7cfd89d78d-2sgd9: 41
receiver-7cfd89d78d-4vct7: 59
So the big question is how should I configure service (or zones?) to actually prefer communication to pods on the same node?
Side remark: I know that there is also internalTrafficPolicy: Local service option but I can’t use it as I need my service to be able to route traffic outside of the node if there is no endpoint available on that node.