I have been following https://mswis.com/load-balancing-with-microk8s-kubernetes/ to try and get an end to end example of an app with load balancing running. I deployed everything using the manifests from the blog and everything looks good, however it does not work. So as I dig deeper, it doesn’t seem like metallb is the problem. I’ve deployed 2 replicas of helloworld using the manifests in the blog (this is a 4 node micrk8s cluster 1.27 on rpi). I’ve deployed everything in the blog and no obvious issues… but I get nothing from the metallb external ip address - or almost nothing, it has worked a couple of times… looked in all the logs, but not seeing anything stand out. So finally I try hitting the helloworld-service from the node that the pod is running on and it works about 50% of the time, it just hangs when it doesn’t work. So I am assuming that is a pretty big problem that needs to be tackled first… but I have no idea why something would work intermittently like this. The hello world-service is node port and this is the manifest:
- name: http
pretty simple - it all seems pretty simple, but now that I am down to the node port service being unreliable I don’t know how to further debug this at the moment. Any help on what to look at next would be appreciated.
Ok, I dig more digging overnight…
NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE
ingress 90s Normal nodeAssigned service/ingress announcing from node "uk8s04" with protocol "layer2"
kube-system 5m6s Warning Unhealthy pod/calico-node-t65b2 Readiness probe failed: calico/node is not ready: felix is not ready: readiness probe reporting 503
kube-system 96s Warning Unhealthy pod/calico-node-fmtmr Readiness probe failed: calico/node is not ready: felix is not ready: readiness probe reporting 503
kube-system 3m4s Warning Unhealthy pod/calico-node-f8mrh Readiness probe failed: calico/node is not ready: felix is not ready: readiness probe reporting 503
kube-system 97s Warning Unhealthy pod/calico-node-x295h Readiness probe failed: calico/node is not ready: felix is not ready: Get "http://localhost:9099/readiness": dial tcp 127.0.0.1:9099: connect: connection refused
looks like calico is my problem - and that makes sense, but I just installed via snap, so now I am really unsure of how to fix this - networking in microk8s is kind of a black box to me -
and finally the answer…
in Calico fails on 21.10 when running on raspberry pi · Issue #2680 · canonical/microk8s · GitHub
sudo apt install linux-modules-extra-raspi
I don’t believe I saw this in the install instructions anywhere… but I was just following a random blog - so maybe it is in the docs…
yes, my bad, it is in the docs… The blog I was following was probably pre ubuntu 21.10 I guess.