All of our workers are in NotReady Status.
Checking kubelet in one of the worker shows
failed to ensure lease exists...
Error updating node status, will retry" err="error getting node
kubelet cannot access kube-apiserver
$ curl -k https://192.168.1.50:8443/healthz
curl: (7) Failed to connect to 192.168.1.50 port 8443 after 0 ms: Connection refused
Both of the kube-apiserver pod logs
Pod1
"Unable to authenticate the request" err="[invalid bearer token]
Pod2
E0328 07:45:21.122116 1 authentication.go:63] "Unable to authenticate the request" err="[x509: certificate has expired or is not yet valid: current time 2024-03-28T07:45:21Z is after 2024-03-15T04:18:44Z, verifying certificate SN=2931374492647536539, SKID=, AKID=65:18:F4:2D:E8:2B:25:B3:58:F4:F0:CD:AC:6E:EA:50:4C:0A:E7:90 failed: x509: certificate has expired or is not yet valid: current time 2024-03-28T07:45:21Z is after 2024-03-15T04:18:44Z]"
E0328 07:45:21.841516 1 authentication.go:63] "Unable to authenticate the request" err="[x509: certificate has expired or is not yet valid: current time 2024-03-28T07:45:21Z is after 2024-03-15T04:18:44Z, verifying certificate SN=3660521977606941286, SKID=, AKID=65:18:F4:2D:E8:2B:25:B3:58:F4:F0:CD:AC:6E:EA:50:4C:0A:E7:90 failed: x509: certificate has expired or is not yet valid: current time 2024-03-28T07:45:21Z is after 2024-03-15T04:18:44Z]"
E0328 07:45:21.856694 1 authentication.go:63] "Unable to authenticate the request" err="[invalid bearer token, Token has been invalidated]"
Pods in kube-system
Cluster information:
Kubernetes version:
Cloud being used: bare-metal
Installation method:
Host OS: Ubuntu 22.04 LTS
CNI and version: Calico v3.21
CRI and version: Containerd v1.5.9-0
You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.