Hi all, looks like after a snap update I’m having some issues with calico.
Listing the nodes, one of them is NotReady:
NAME STATUS ROLES AGE VERSION
nucworker2 NotReady 2y87d v1.28.12
nucworker3 Ready 427d v1.28.13
nucworker4 Ready 4d22h v1.28.13
First thing I noticed is the different snap version, but checking snap info on that node tells different:
installed: v1.28.13 (7150) 186MB classic
Checking on pods running on kube-system, this is what I find:
NAME READY STATUS RESTARTS AGE
csi-nfs-node-gstbf 3/3 Running 84 (19d ago) 427d
snapshot-controller-85f68864bb-znnw2 1/1 Running 2 (19d ago) 26d
dashboard-metrics-scraper-6b6f796c8d-mqrjz 1/1 Running 2 (19d ago) 26d
csi-nfs-node-s5bk9 3/3 Running 0 4d22h
kubernetes-dashboard-69469579cc-cgj7t 1/1 Running 0 4d18h
coredns-66bcf65bb8-qvb22 1/1 Running 2 (19d ago) 26d
calico-kube-controllers-7dcdf64c7d-9kr84 1/1 Running 0 36h
calico-node-nfg9k 1/1 Running 0 36h
calico-node-c2m5f 1/1 Running 0 36h
csi-nfs-node-7zfkq 3/3 Running 157 (19d ago) 2y87d
csi-nfs-controller-75d6c9589d-zrwmc 3/3 Running 0 36h
snapshot-controller-85f68864bb-9m6r8 1/1 Running 0 36h
hostpath-provisioner-76f65f69ff-bnj55 1/1 Running 0 36h
metrics-server-5f8f64cb86-hcl55 1/1 Running 0 36h
calico-node-cn8ph 0/1 Pending 0 33m
The last pod doesn’t start, checking the logs I find:
2024-08-27T09:55:21.936144518-03:00 stdout F 2024-08-27 12:55:21.892 [INFO][64] felix/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/nodes” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.936149339-03:00 stdout F 2024-08-27 12:55:21.892 [INFO][64] felix/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/kubernetesnetworkpolicies” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.936154262-03:00 stdout F 2024-08-27 12:55:21.892 [INFO][66] status-reporter/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/caliconodestatuses” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.936161863-03:00 stdout F 2024-08-27 12:55:21.892 [INFO][64] felix/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/kubernetesservice” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.936169686-03:00 stdout F 2024-08-27 12:55:21.892 [INFO][67] tunnel-ip-allocator/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/nodes/nucworker2” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.93617563-03:00 stdout F 2024-08-27 12:55:21.892 [INFO][67] tunnel-ip-allocator/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/ippools” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.936180784-03:00 stdout F 2024-08-27 12:55:21.897 [INFO][64] felix/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/kubernetesendpointslices” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.962235364-03:00 stdout F 2024-08-27 12:55:21.961 [INFO][64] felix/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/networksets” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:21.963265306-03:00 stdout F 2024-08-27 12:55:21.963 [INFO][64] felix/watchercache.go 181: Full resync is required ListRoot=“/calico/resources/v3/projectcalico.org/bgpconfigurations”
2024-08-27T09:55:22.158040214-03:00 stdout F 2024-08-27 12:55:22.156 [INFO][64] felix/watchercache.go 194: Failed to perform list of current data during resync ListRoot=“/calico/resources/v3/projectcalico.org/globalnetworkpolicies” error=connection is unauthorized: Unauthorized
2024-08-27T09:55:22.163546264-03:00 stdout F 2024-08-27 12:55:22.159 [INFO][64] felix/watchercache.go 181: Full resync is required ListRoot=“/calico/resources/v3/projectcalico.org/ippools”
Any idea what is missing? Thanks.