HPA(HorizontalPodAutoscale) is used to scale pods when pod get high resource utilization. i was worry about many application may try to scale up at the same time and exhaust all the cluster resources. So i want to disable hpa when the cluster resource utilization is high and scale up the pod manually in this case. Is it any way to achieve?
Sounds an interesting scenario, but there are still some specifics to clarify. I don’t think there is a simple answer at the moment, nor do I think there will be one. You will have to characterize your application and your cluster before attempting to come up with a solution. For example,
- Is the application critical? You may want to scale out your cluster if the answer is yes. When your critical applications are impacted due to resource exhaustion, you will add resources to the cluster rather than restraining your application.
- Are the HPA applications equally important? When you have more than one beasts in the same zoo, the first thing you will consider is to make sure they don’t fight with each other. You may want to impose namespace level resource constraints as a starting point.
- Scaling applications based on resource utilization … is that the right thing to do? Maybe and maybe not. There are apps that don’t behave well when resource utilization is not high, and there are apps that fail to respond to added resources. You may want to examine your HPA settings and see if resource utilization is the dominant factor.
- There are many scenarios where people worry about things that never happen. There are many scenarios people do things because they are doable, not because they worth your time and energy. In your case, is it a realistic scenario that these “many” HPA apps will scale up “at the same time”? Further investigations into the situation may help to design a “good-enough” solution.