One of the nice features of using Cilium as a CNI is you don’t need MetalLB (or similar) anymore as a load balancer solution.
However, to make this work, it is required to disable kube-proxy in K8S, see L2 Announcements / L2 Aware LB (Beta) — Cilium 1.19.0-dev documentation
I could not find a way to make it work in Microk8s. Cilium works fine as a CNI, but the L2-announcement feature isn’t.
Adding “–init-only” to /var/snap/microk8s/current/args/kube-proxy (and restart microk8s) doesn’t help.
Of course I added a CiliumLoadBalancerIPPool and CiliumL2AnnouncementPolicy to K8S.
The service of type loadbalancer receives a correct external IP address.
The issue I’m facing is that the system does not respond to ARP requests from outside the cluster when ARP tries to figure out on which system the external IP address of the service is alive. So packets sent from an arbitrary system to the external IP address of the loadbalancer-service do not reach the Microk8s node.
Tried both the available Microk8s Cilium addon (v1.15.x) and the latest and greatest via “cilium install” (v1.18.x).
Also tried using Cilium as a CNI and MetalLB as a loadbalancer. That works flawlessly, but I was planning to get rid of MetalLB…