I’ve been mulling over this idea on how to have any number of replica operators to handle queued work without a leader and I got something working today.
The example in that gist creates a CRD, a service account with role binding, a configmap with code, and a deployment with 3 replicas.
Once you deploy it, you can spam create some TaskOwner resources like so:
cat <<EOF | kubectl apply -f -
apiVersion: future.ideas.protosam/v1beta1
kind: TaskOwner
metadata:
name: mytaskname-$(openssl rand -hex 12)
EOF
Then see that they get claimed:
$ kubectl get task-owners
NAME OWNER AGE
mytaskname-0b47c71573cabd75b9e2e7f8 leaderless-operator-b6b6c8d54-pb68d 8s
mytaskname-21e021284a93c35bd59502c1 leaderless-operator-b6b6c8d54-48kdp 5s
mytaskname-26d19152a995f546a4e0dc0f leaderless-operator-b6b6c8d54-pb68d 14s
mytaskname-43dd3c1cf807befc2d748c1a leaderless-operator-b6b6c8d54-pb68d 10s
mytaskname-46ac838d5aaee4a35c8c1295 leaderless-operator-b6b6c8d54-pb68d 7s
mytaskname-522f19d9ca224518103f3a55 leaderless-operator-b6b6c8d54-pb68d 5s
mytaskname-58f3cbdcde1864967125206d leaderless-operator-b6b6c8d54-48kdp 6s
mytaskname-5b4ba1c8e863edbdac497e9d leaderless-operator-b6b6c8d54-pb68d 9s
mytaskname-6c5d56982b83bd5572053b7b leaderless-operator-b6b6c8d54-pb68d 15s
mytaskname-7b3a9bf11e5392b439e9ffbd leaderless-operator-b6b6c8d54-7r9zf 12s
mytaskname-8a8f771f1d5641effac9ec83 leaderless-operator-b6b6c8d54-7r9zf 11s
mytaskname-a46870f82f0f526307b6caa5 leaderless-operator-b6b6c8d54-pb68d 13s
mytaskname-d1a3fef9f9316f94fabcc3b1 leaderless-operator-b6b6c8d54-7r9zf 8s
mytaskname-f41e81af918426c282d9b39b leaderless-operator-b6b6c8d54-7r9zf 12s
Anyways, just wanted to share this as it makes scalable, leaderless, operators a possibility. As for reliability, I have not rigorously tested this yet to confirm my observations thus far. So if you happen to know of any caveats to this, the feedback would be great.