I want to deploy a large-scale cluster with 5000 nodes. I use HAProxy+Keepalived for load balancing of API servers, deploying 6 API servers, 2 controller managers, and 2 schedulers. These three core services are all deployed using kubeadm, and ETCD has deployed two sets of high availability clusters, one for storing data and the other for storing events. However, the efficiency of executing VNet get pod-A is very low, and I also encountered performance bottlenecks such as etcd and even OOM when deploying business in the cluster. After checking the cluster metadata, it has exceeded 8GB. Therefore, I would like to ask what better deployment form can support running business on a 5000 node cluster.
I also want to know what deployment form Kubernetes officially claims to support up to 5000 nodes in a cluster.
Cluster information:
Kubernetes version: 1.28.15
Cloud being used: Virtual Machine Environment
Installation method: use kubeadm, etcd adopts a binary based high availability deployment and separates the storage of events and data data
Host OS: CentOS
CNI and version: calico, deploy by tigera operator and calico version is 3.27.3
CRI and version: containerd 1.7.14