Service Topology and EndpointSlices complete usage

Cluster information:

Kubernetes version: 1.18.2
Cloud being used: bare-metal
Installation method: kubeadm
Host OS: ubuntu 18.04
CNI and version: calico
CRI and version:


hi.

I want to start cluster with Service Topology.

I read:
Service Topology - Kubernetes EndpointSlices - Kubernetes
and
Enabling EndpointSlices - Kubernetes

I enabled startup options for the service kube-proxy:
–feature-gates=EndpointSliceProxying=true

and created service:
dns.yaml:

 apiVersion: v1
 kind: Service
 metadata:
   name: kube-dns2
 spec:
   ports:
   - name: dns
     port: 53
     protocol: UDP
     targetPort: 53
   selector:
     k8s-app: kube-dns
   topologyKeys:
   - "kubernetes.io/hostname"

 # kubectl  get svc kube-dns2  -n  kube-system
NAME        TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
kube-dns2   ClusterIP   10.106.3.98   <none>        53/UDP    14h

Automatically created endpointslice:

# kubectl  describe  endpointslice kube-dns2-b8rp4  -n  kube-system   
Name:         kube-dns2-b8rp4
Namespace:    kube-system
Labels:       endpointslice.kubernetes.io/managed-by=endpointslice-controller.k8s.io
              kubernetes.io/service-name=kube-dns2
Annotations:  endpoints.kubernetes.io/last-change-trigger-time: 2020-04-21T23:34:13Z
AddressType:  IPv4
Ports:
  Name  Port  Protocol
  ----  ----  --------
  dns   53    UDP
Endpoints:
  - Addresses:  10.244.148.214
    Conditions:
      Ready:    true
    Hostname:   <unset>
    TargetRef:  Pod/coredns-6cfdfcf764-d4rjd
    Topology:   kubernetes.io/hostname=kuber-01
  - Addresses:  10.244.40.27
    Conditions:
      Ready:    true
    Hostname:   <unset>
    TargetRef:  Pod/coredns-6cfdfcf764-m5gf8
    Topology:   kubernetes.io/hostname=kuber-04
  - Addresses:  10.244.235.166
    Conditions:
      Ready:    true
    Hostname:   <unset>
    TargetRef:  Pod/coredns-6cfdfcf764-vwfgh
    Topology:   kubernetes.io/hostname=kuber-06
  - Addresses:  10.244.235.182
    Conditions:
      Ready:    true
    Hostname:   <unset>
    TargetRef:  Pod/coredns-6cfdfcf764-qrs5n
    Topology:   kubernetes.io/hostname=kuber-06

I use ipvs for kube-proxy
On the node in the ipvsadm -ln command listing

UDP  10.106.3.98:53 rr
  -> 10.244.40.27:53              Masq    1      0          0
  -> 10.244.40.54:53              Masq    1      0          0
  -> 10.244.97.70:53              Masq    1      0          0
  -> 10.244.97.75:53              Masq    1      0          0
  -> 10.244.97.76:53              Masq    1      0          0
  -> 10.244.97.78:53              Masq    1      0          0
  -> 10.244.106.1:53              Masq    1      0          0
  -> 10.244.106.2:53              Masq    1      0          0
  -> 10.244.106.24:53             Masq    1      0          0
  -> 10.244.106.50:53             Masq    1      0          0

How can I use the created service? Ipvs rules are created for proxying requests to all my pods, I want the request to be sent only to the local dns

I read that kube-proxy should switch from using endpoint to endpointslice when using the option --feature-gates=EndpointSliceProxying=true. but it does not work

I will be happy for any help or manual for using Service Topology or EndpointSlices .

Caution that the topology API is being reconsidered right now, so it may ultimately be different. It’s alpha for a reason.

I understand that this is an alpha version
I will be glad if you point out a mistake or help me understand the concept of how this should work

Did you enable the ServiceTopology gate?

no

in which component should i do this?
kube-proxy, kube-api, kube-sheduler?

thanks

In general, just enable the same set of gates in all components :slight_smile:

I enabled gates:
kube-apiserver: --feature-gates=ServiceTopology=true,EndpointSlice=true
kube-controller-manager: --feature-gates=ServiceTopology=true,EndpointSlice=true
kube-scheduler: --feature-gates=ServiceTopology=true,EndpointSlice=true
on all master server

kube-proxy: --feature-gates=EndpointSliceProxying=true,ServiceTopology=true,EndpointSlice=true on all server

But I still see ipvs rule:

UDP  10.106.3.98:53 rr
  -> 10.244.40.27:53              Masq    1      0          0
  -> 10.244.40.54:53              Masq    1      0          0
  -> 10.244.63.131:53             Masq    1      0          0
  -> 10.244.63.148:53             Masq    1      0          0
  -> 10.244.97.70:53              Masq    1      0          0
  -> 10.244.97.75:53              Masq    1      0          0
  -> 10.244.97.76:53              Masq    1      0          0
  -> 10.244.97.78:53              Masq    1      0          0
  -> 10.244.106.1:53              Masq    1      0          0

thanks

hmm … my proxy log:

docker logs c72778eaa26a
I0518 15:34:39.987860 1 node.go:136] Successfully retrieved node IP: хх.хх.хх.хх
I0518 15:34:39.987896 1 server_others.go:259] Using ipvs Proxier.
I0518 15:34:39.988118 1 server.go:583] Version: v1.18.2
I0518 15:34:39.988455 1 conntrack.go:100] Set sysctl ‘net/netfilter/nf_conntrack_max’ to 262144
I0518 15:34:39.988477 1 conntrack.go:52] Setting nf_conntrack_max to 262144
I0>518 15:34:39.988697 1 config.go:133] Starting endpoints config controller
I0518 15:34:39.988713 1 shared_informer.go:223] Waiting for caches to sync for endpoints config
I0518 15:34:39.988731 1 config.go:315] Starting service config controller
I0518 15:34:39.988735 1 shared_informer.go:223] Waiting for caches to sync for service config
I0518 15:34:40.288830 1 shared_informer.go:230] Caches are synced for service config
I0518 15:34:40.288831 1 shared_informer.go:230] Caches are synced for endpoints config

how can I understand that the kube-proxy really reads the configuration Endpoint Slices?
In the log I can not read it

Hey @zorruch, that’s a great question! I’m not sure what log level you’re running kube-proxy with, but it might be worth temporarily increasing it to debug this. Can you try adding --v=4 as a kube-proxy flag here? What version of kube-proxy are you running, is it also 1.18? With more verbose logging, you should see one of the following in your log output to indicate whether Endpoints or EndpointSlices are being used:

Calling handler.OnEndpointsSynced()
Calling handler.OnEndpointSlicesSynced()

thanks for your reply
I use an image k8s.gcr.io/kube-proxy:v1.18.2
and run with parameters:

 containers:
 - command:
   - /usr/local/bin/kube-proxy
   - --config=/var/lib/kube-proxy/config.conf
   - --hostname-override=$(NODE_NAME)
   - --feature-gates=EndpointSliceProxying=true,ServiceTopology=true,EndpointSlice=true
  - --v=4
   env:
   - name: NODE_NAME
     valueFrom:
       fieldRef:
         apiVersion: v1
         fieldPath: spec.nodeName

In the log I see:

docker logs ec146b67c267 2>&1 | grep OnEndpointsSynced I0522 15:42:39.826037 1 config.go:140] Calling handler.OnEndpointsSynced() docker logs ec146b67c267 2>&1 | grep OnEndpointSlicesSynced
$

kube-proxy does not switch to use EndpointSlices
Am I running it with invalid parameters?