Addon: MetalLB

MetalLB Loadbalancer is a network LB implementation that tries to “just work” on bare metal clusters.

When you enable this add on you will be asked for an IP address pool that MetalLB will hand out IPs from:

microk8s enable metallb

Alternatively you can provide the IP address pool in the enable command:

microk8s enable metallb:10.64.140.43-10.64.140.49

:warning: Multiple comma separated ranges (eg microk8s enable as well as CIDR notation metallb:10.64.140.43-10.64.140.49,10.64.141.53-10.64.141.59,10.12.13.0/24) are supported on the latest and 1.19+ tracks.

Note that currently this add on does not work under Multipass on macOS, due to filtering that macOS applies to network traffic.

Setting up a MetalLB/Ingress service

For load balancing in a MicroK8s cluster, MetalLB can make use of Ingress to properly balance across the cluster ( make sure you have also enabled ingress in MicroK8s first, with microk8s enable ingress). To do this, it requires a service. A suitable ingress service is defined here:

apiVersion: v1
kind: Service
metadata:
  name: ingress
  namespace: ingress
spec:
  selector:
    name: nginx-ingress-microk8s
  type: LoadBalancer
  # loadBalancerIP is optional. MetalLB will automatically allocate an IP 
  # from its pool if not specified. You can also specify one manually.
  # loadBalancerIP: x.y.z.a
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443

You can save this file as ingress-service.yaml and then apply it with:

microk8s kubectl apply -f ingress-service.yaml

Now there is a load-balancer which listens on an arbitrary IP and directs traffic towards one of the listening ingress controllers.

(Thanks to jonathan gazeley for his contributions to this).

3 Likes

What’s the recommended approach to using metallb on production. I come from an EKS set up earlier, and had one AWS network loadbalancer pointing to main cluster IP.

Should I be using the public IP of one of the three nodes (since there is no master node anymore), as A record to point to the domain?

For example, in AWS, If I reboot the EC2 instance, the public IP of the machine is gone, I suppose I should be using a reserved public IP in one of the nodes machines (pseudo-master node)?

Using a node IP to access you cluster creates a single point of failure. I think you should try to integrate with the LB of the cloud you are on and if there is no cloud substrate use metallb with a floating IP. It would have been great if we had a few such use-cases as part of our docs or linked from the official docs.

Hi, probably what I am doing might advance the example you’re looking for but I may need to know more first.

My node is a Hetzner AX51.

It has just one public IP address. When MetalLB asks for an IP address range, is it looking for IPs to assign to containers, and then it will map to the public IP address?

Or am I not getting it?

I would add to the doc something along the lines of:

You can confirm addon is ready using:
sh -c "until microk8s.kubectl rollout status daemonset.apps/speaker -n metallb-system -w; do sleep 5; done"

Much like @faddat before, I’m using a single VM to run microk8s. When I enable metallb, what exactly should I use for range? The public IP?

Thanks

You would use a range of unused IPs (one IP per load balancer service you’ll be creating) in the internal subnet. You can then use NAT or whatever to reach them from the outside.

I mostly just use one IP, and set up nginx-ingress-microk8s as described at the top of this article, to which I then forward ports 80 & 443 from my public IP.

A load balancer’s job is to distribute traffic it receives to several nodes. The load balancer needs to know the IPs of its upstream nodes of course but why in the world would the nodes (in this case the Kubernetes cluster) need to know the IP of the load balancer?

Indeed why the heck do I even need a Kubernetes resource/service of type “LoadBalancer “? Is it just and advertisement to some IPAM that “Hey, I need a load balancer, please write me back the IP when you’re done!”

This and other questions at:

By AGuyWhoJustDoesntGetKubernetesLoadBalancers

Moving up the learning curve on on Kubernetes and I find that the issue of how to deal with on-prem clusters with no external load balancer is one of the stickiest learning curves. Much confusion.

What are the tradeoffs of a MetalLB installation versus using Ingress objects?

MetalLB and Ingress aren’t competitors. Using the instructions above, you can set up your ingress to listen on a MetalLB IP address. This virtual address is able to float around your cluster. You set up your firewall to send web traffic to that IP address, which will always be there, rather than sending it to a single node’s IP.

Without MetalLB, you’d send web traffic to the ingress directly on a node’s IP address, which could go down for maintenance or other reasons, causing an outage. With MetalLB, that virtual address just moves to a different node, and the ingress continues to be reachable.

I’m sure I have simplified some details since I’m still learning this too, but my explanation matches my experience so far. I hope it makes sense.