Hi, I’m looking to deploy a cluster consisting of Raft nodes on Azure. As part of the Raft protocol, these nodes keeps track of the current members of the cluster. I want to make my cluster more resilient, and was thinking of having Kubernetes manage changes to the cluster membership: since Kubernetes can monitor pod/node health anyway, it feels more intuitive to me to have it be responsible for pinging the Raft nodes if membership needs changing than adding this complexity to the Raft cluster itself. For example, when node auto repair triggers, some code would be run to trigger an appropriate cluster membership change in Raft.
I was wondering if there’s some standard way of doing this sort of thing in Kubernetes. I essentially want to run some code that is triggered at set intervals and whenever Kubernetes detects changes in the number of pods/nodes or their health. I was reading the documentation on controllers, and they seemed to do what I want, but I’d like to 1. make sure whether this is the appropriate abstraction for the job, and 2. whether there’s a preexisting controller that does something along these lines.
Any pointers would be appreciated; many thanks in advance!
Cluster information:
Cloud being used: Azure