If I have multiple AWS EKS clusters and I have a mutating admission controller in place, then what is the general best practice for running the webhook server. Should I be running it in every EKS cluster or only a subset of them so that I can have cross-k8s-cluster injection ?
I like the idea of using latter since that would allow me to have lesser monitoring / operational overhead + since the webhook server would be load balanced across EKS clusters, I don’t have to worry about the webhook server going down in one EKS cluster (which can cause pod creation to fail).
Also - if I wish to achieve cross-EKS-cluster injection, running an ELB that points to worker nodes of multiple EKS clusters, which would actually be reaching out to nodePort type service on the worker node where the webhook server pod is running - the only way to go forward with this ?
Cluster information:
Kubernetes version: AWS-EKS 1.12
Cloud being used: AWS