High Availability Host Numbers

General Discussions
1

Hi Folks,

How many control plan nodes and worker nodes are required for high availability? I have heard some of the following proposals:

3 Master 2 workers.

But would this avoid split brain scenarios? Is it sufficient to have an odd number of nodes at the control plane node level?

Also, when scaling, should the control plane nodes always scale in odd number increments? For example, from 3 to 5 to 7? Is there a similar requirement at the worker node level?

Thanks!
Daniel

2

K8s uses the RAFT consensus algorithm for quorum. In order to maintain quorum, you will need floor(n/2)+1 healthy master nodes.

Practicaly this means:

  • 1 master node: you will require 1 healthy master node for quorum, the loss of the master node will render the cluster headless.
  • 2 master nodes: you will require 2 healthy master nodes for quorum, the loss of either master node will render the cluster headless.
  • 3 master nodes: you will require 2 healty master nodes for quorum, the loss of one of the master nodes can be compensated.
  • 4 master nodes: you will require 3 healty master nodes for quorum, the loss of one on the master nodes can be compensated. A setup with 4 master nodes has no advantage over a 3 master nodes setup.
  • 5 master nodes: you will require 3 healthy master nodes for quorum, the loss of up to two master nodes can be compsensated.
  • 6 master nodes: you will require 4 healty master nodes for quorum, the loss of up to two master nodes can be compensated. No advantage compared to 5 master nodes.
  • 7 master nodes: you will require 4 healthy master nodes for quorum, the loss of up to three master nodes can be compsensated.

This is the reason why it is recommended to use an odd number of master nodes for the control plane. More then 7 master nodes will result in a overhead for determining cluster membership and quorum, it is not recommended. Depending on your needs, you typically end up with 3 or 5 master nodes.

1 Like
3

Thanks @meyay! This is great information!

4

Very useful and precise information about K8s RAFT quorom algorithm for control plane nodes!!