How can I find which kubernetes certificate has expired?

Cluster information:

Kubernetes version: 1.21.0
Cloud being used: bare-metal
Installation method: kubeadm
Host OS: Arch Linux (linux 5.11.16, glibc 2.33)
CNI and version: 0.9.1
CRI and version: 1.21.0

Container runtime: containerd
Network plugin: weave

Problem:

Control plane is broken, seems to be a certificate issue.
Kubelet is starting but seems stuck during initialization phases. I think the relevant thing is that crictl logs kube-apiserver has a bunch of

1 authentication.go:63] “Unable to authenticate the request” err="[x509: certificate has expired or is not yet valid: current time 2021-06-02T13:18:50Z is after 2021-05-29T15:48:22Z

The thing is, kubeadm certs check-expiration seems happy, and I even manually checked a few yaml config files (base64 decoded certificates, and run them through openssl to check the date). Nevertheless, I asked kubeadm to renew all certificates and rebooted everything, to no effect.

Question:

Any idea how I can identify which certificate exactly has expired ?

Solved by a helpful dev here

Summary here

1 Like