Error accessing etcd on node of VIP in k8s high availability cluster

General Discussions
1

k8s: 1.21
docker: 20.10
keepalived: 3
haproxy: 3

I used kept + haproxy to create a highly available cluster, Etcd installed using kubedm stack. When everything was deployed, I checked the etcd log and found the following errors:

etcd-k8s-master3:

2021-11-26 04:14:01.012111 I | embed: rejected connection from "192.168.9.153:65036" (error "tls: \"192.168.9.153\" does not match any of DNSNames [\"k8s-master1\" \"localhost\"]", ServerName "", IPAddresses ["192.168.9.150" "127.0.0.1" "::1"], DNSNames ["k8s-master1" "localhost"])
2021-11-26 04:14:01.015309 I | embed: rejected connection from "192.168.9.153:65038" (error "tls: \"192.168.9.153\" does not match any of DNSNames [\"k8s-master1\" \"localhost\"]", ServerName "", IPAddresses ["192.168.9.150" "127.0.0.1" "::1"], DNSNames ["k8s-master1" "localhost"])

etcd-k8s-master1:

2021-11-26 05:07:52.579533 W | etcdserver: failed to reach the peerURL(https://192.168.9.152:2380) of member b9e8188b25446396 (Get https://192.168.9.152:2380/version: EOF)
2021-11-26 05:07:52.579546 W | etcdserver: cannot get the version of member b9e8188b25446396 (Get https://192.168.9.152:2380/version: EOF)
2021-11-26 05:07:52.717465 W | rafthttp: health check for peer b6eebc2dcbfecb89 could not connect: EOF
2021-11-26 05:07:52.717503 W | rafthttp: health check for peer b6eebc2dcbfecb89 could not connect: EOF
2021-11-26 05:07:55.882385 I | etcdserver/api/etcdhttp: /health OK (status code 200)

etcd status:

# kubectl get pods -A | grep etcd
kube-system   etcd-k8s-master1                          1/1     Running   44         68m
kube-system   etcd-k8s-master2                          1/1     Running   0          68m
kube-system   etcd-k8s-master3                          1/1     Running   0          67m

kubeadm init config:

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
clusterName: kubernetes
kubernetesVersion: v1.21.7
certificatesDir: /etc/kubernetes/pki
controllerManager: {}
controlPlaneEndpoint: "192.168.9.153:6445"
imageRepository: registry.aliyuncs.com/google_containers
apiServer:
  timeoutForControlPlane: 4m0s
  certSANs:
  - "k8s-master1"
  - "k8s-master2"
  - "k8s-master3"
  - "k8s-vip"
  - "192.168.9.150"
  - "192.168.9.151"
  - "192.168.9.152"
  - "192.168.9.153"
  - "127.0.0.1"
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.1.0.0/16
  podSubnet: 10.244.0.0/16
scheduler: {}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"

The error message “does not match any of dnsnames” is always displayed in the master node where the VIP is located

2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:50:56:9d:cb:cd brd ff:ff:ff:ff:ff:ff
    inet 192.168.9.150/16 brd 192.168.255.255 scope global noprefixroute ens192
       valid_lft forever preferred_lft forever
    inet 192.168.9.153/22 scope global ens192
       valid_lft forever preferred_lft forever
    inet6 fe80::a267:a0fa:7db3:d48c/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::2595:4723:8cba:b7a1/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::e300:31a3:fb10:fce4/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever

IP information:

master1(vip):

# ip route
default via 192.168.0.254 dev ens192 proto static metric 100 
10.10.0.0/16 dev docker0 proto kernel scope link src 10.10.0.1 linkdown 
10.244.135.192/26 via 192.168.9.152 dev tunl0 proto bird onlink 
blackhole 10.244.159.128/26 proto bird 
10.244.224.0/26 via 192.168.9.151 dev tunl0 proto bird onlink 
192.168.0.0/16 dev ens192 proto kernel scope link src 192.168.9.150 metric 100 
192.168.8.0/22 dev ens192 proto kernel scope link src 192.168.9.153

master2:

# ip route
default via 192.168.0.254 dev ens192 proto static metric 100 
10.10.0.0/16 dev docker0 proto kernel scope link src 10.10.0.1 linkdown 
10.244.135.192/26 via 192.168.9.152 dev tunl0 proto bird onlink 
10.244.159.128/26 via 192.168.9.150 dev tunl0 proto bird onlink 
blackhole 10.244.224.0/26 proto bird 
192.168.0.0/16 dev ens192 proto kernel scope link src 192.168.9.151 metric 100

master3:

# ip route
default via 192.168.0.254 dev ens192 proto static metric 100 
10.10.0.0/16 dev docker0 proto kernel scope link src 10.10.0.1 linkdown 
blackhole 10.244.135.192/26 proto bird 
10.244.159.128/26 via 192.168.9.150 dev tunl0 proto bird onlink 
10.244.224.0/26 via 192.168.9.151 dev tunl0 proto bird onlink 
192.168.0.0/16 dev ens192 proto kernel scope link src 192.168.9.152 metric 100