I am building out production K8s clusters. One thing I have come across was the best practices around the cluster name (default: kubernetes) and domain (default: cluster.local).
My initial understanding of these was to change them to something meaningful like:
- clustername: k8s-clst01
- domain: k8s-domain01.local
This way, each cluster would have a unique name and if multiple cluster were going to be part of the same logical group, then they would share the domain name. If not, then increment domain name.
Situation to make me rethink:
I was recently building a new separate cluster (domain: k8s-domain02.local). I was asked if it was possible to move a statefulset from k8s-clst01.domain01.local to k8s-clst07.domain02.local.
I know about multi-cluster headless services and statefulset slices. Not enough to implement but aware they exist and basic concept. So not really an option here.
I immediately started to think how this would not work for certain applications (i.e. Cassandra). The Cassandra pods find each other via FQDN. In the destination K8s cluster, the domain would be different and this would break any possible migration I was thinking of (possible migrate using Kasten).
- Is the best practice to change the cluster name to a unique name?
- Is the best practice to change the domain name, keeping the basic concept linked clusters will share the domain name?
If the answer is ‘yes’:
a. How to handle migrating stateful apps between clusters?
Kubernetes version: v1.24.2
Cloud being used: bare-metal
Installation method: kubeadm
Host OS: Ubuntu 20.04
CNI and version: calico v3.24.0
CRI and version: containerd v1.5.9