Failing to trigger aws autoscaling on a group with min of 0 nodes

General Discussions
1

Hi,

We have jupyterhub configured on kubernetes with Amazon AWS. In it, we have multiple profiles associated with different autoscaling groups that spawn different hardware configurations, and pods get allocated to those using taints. We autoscale when a new user requests a server and all nodes in that group are being used. We have no problem autoscaling for autoscaling groups with a min of 1 and above, but haven’t been able to trigger the autoscale up from 0 or down to 0.

Following the instructions in this link we have added the following tags to our autoscaling group:

Screen Shot 2020-10-29 at 11.14.56 AM
Screen Shot 2020-10-29 at 11.14.56 AM1569×358 35.8 KB

Given our taints for pods to be on this group:

tolerations:

  • key: “hub.jupyter.org/dedicated”
    operator: “Equal”
    value: “user”
    effect: “NoSchedule”
  • key: “GPUJobExclusivity”
    operator: “Equal”
    value: “true”
    effect: “NoSchedule”
  • key: “GPUtype”
    operator: “Equal”
    value: “TeslaV100”
    effect: “NoSchedule”
  • key: “numberOfGPUs”
    operator: “Equal”
    value: “8”
    effect: “NoSchedule”

Cluster information:

Kubernetes was set up following this link
Kubernetes version: v1.14.1
Cloud being used: AWS
Installation method:
Host OS: Cent OS

also:
k8s.gcr.io/cluster-autoscaler:v1.13.1

Any idea what we could be missing?

Thanks!