Cluster information:
Kubernetes version: 1.21
Cloud being used: AWS, bare-metal
Installation method: TF
Host OS: os independent
After a full restart of K8S cluster we faced the issue where pods annotated so that vault-agent-injector injects secrets in them did not have their init container at all. After some research it seems to be related with the fact that the vault-agent-injector was not yet registered as a mutating webhook when these pods started. The only way to make them run was to scale the deployments RS to 0 and then to a positive value after ensuring that the vault-agent-injector was up and running.
Do you see any way to make sure that pods are not instantiated and/or that they are terminated if a list of mutations was not run?
We may have an admission controller checking for the presence of vault annotations and if so its init container so that we are sure that the mutation occurred?
Thanks for any help or idead on this.
Raph