I’m in the process of testing and capacity planning a K8 clusters on GKE. Currently, I have a single cluster with a 4 nodes configuration (32 vCPU’s, 200gb in total) and autoscaling set to on.
What I’ve found is that pods become unschedulable even when there’s more than enough capacity available - with only one host in the cluster provisioned with high cpu capacity at 68%. I’ve also limited the cluster not to expand beyond 4 nodes and it would seem the scheduler thinks it needs more node capacity and refuses to schedule the new requested containers.
Thinking I’m missing something important - like number of pods to specific host configuration (eg. n1-standard-8) or the resource monitor may be incorrectly reporting an incorrect resource status to the scheduler.
Can anyone share best practices with regard to pod capacity and host configuration (better to go with high memory or high cpu or combination of both) for auto-scaling clusters? I would like to maximum the number of allowed pods in the private network and allowable host resources given the GKE Quota limits.
Print screen can be found in this post - https://groups.google.com/forum/#!topic/gce-discussion/UC6__8AxPXE
I was running 153 pods, 76 disks, 90 services (combination of internal & public)