All components of kubernetes are repeatedly deleted and created

General Discussions
1

Hi all,
As shown below, the status of all components of k8s is between pending and running

912×269 178 KB

927×260 182 KB

And logs from journalctl -f -u kubelet indicates pods are outdated. Some of detail logs are listed below.

5月 21 17:42:14 k8s-master kubelet[1615]: E0521 17:42:14.168690 1615 kubelet.go:2167] Container runtime network not ready: NetworkReady=false reason:Ne
tworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
5月 21 17:42:15 k8s-master kubelet[1615]: W0521 17:42:15.733511 1615 kubelet.go:1622] Deleting mirror pod “etcd-k8s-master_kube-system(d10c0f7f-ba18-11
eb-84aa-0cc47adadbe0)” because it is outdated
5月 21 17:42:15 k8s-master kubelet[1615]: W0521 17:42:15.740525 1615 kubelet.go:1622] Deleting mirror pod “kube-apiserver-k8s-master_kube-system(d10c3d
2e-ba18-11eb-84aa-0cc47adadbe0)” because it is outdated
5月 21 17:42:15 k8s-master kubelet[1615]: W0521 17:42:15.749751 1615 kubelet.go:1622] Deleting mirror pod “kube-controller-manager-k8s-master_kube-syst
em(d0af2526-ba18-11eb-84aa-0cc47adadbe0)” because it is outdated
5月 21 17:42:15 k8s-master kubelet[1615]: W0521 17:42:15.756567 1615 kubelet.go:1622] Deleting mirror pod “kube-scheduler-k8s-master_kube-system(d0b161
f7-ba18-11eb-84aa-0cc47adadbe0)” because it is outdated
5月 21 17:42:16 k8s-master kubelet[1615]: W0521 17:42:16.437473 1615 kubelet.go:1622] Deleting mirror pod “etcd-k8s-master_kube-system(d10c0f7f-ba18-11
eb-84aa-0cc47adadbe0)” because it is outdated
5月 21 17:42:17 k8s-master kubelet[1615]: W0521 17:42:17.444155 1615 kubelet.go:1622] Deleting mirror pod “etcd-k8s-master_kube-system(d3a06510-ba18-11
eb-84aa-0cc47adadbe0)” because it is outdated
5月 21 17:42:19 k8s-master kubelet[1615]: W0521 17:42:19.169992 1615 cni.go:188] Unable to update cni config: No networks found in /etc/cni/net.d

Has anyone encountered this problem? What can I do for solving it?

Cluster information:

Kubernetes version: 1.12.9
Cloud being used: bare-metal
Installation method: kubeadm
Host OS: centos-7.9
CNI and version: contrail-k8s-cni
CRI and version: docker 18.06

2

I don’t see the container for your CNI there and there are log entries saying it’s not ready. On a fresh cluster, I don’t think pods can reach a ready state until the CNI is ready.

The error about deleting the mirror pod is probably irrelevant. That pod isn’t real and is just a abstraction from the kubelet process that runs on the node. Check the kubelet logs if you want to dig into that.

3

Is there a reason you are running version 1.12.9? If not might want to consider upgrading the latest release (current is 1.21) and support only goes back 3 minor versions (1.19).