Common max replica on multiple HPA scaled deployments

General Discussions
1

I have deployments consuming different RabbitMQ queues. Each deployment has a different HPA tied to the queue metrics with different scaling formulas.
All of this works well.

The pods write into a DB and I would like to have the total number of pods for all deployments limited to a number to avoid overloading the DB.

I didn’t find a way to do this.
I tried tweaking the metric formula to include a max based on the pod count of the other deployments but it’s not very reliable. The lag of the metrics causes scaling up and down. I’m pretty sure it’s not the correct way.
Any other idea?

2

yes, trying to coordinate HPA scaling through metric manipulation isn’t the ideal approach.

you can try creating a custom controller that:

  • try monitoring all your deployments

  • Calculates the total pods across all deployments

  • try adjusting individual HPA maxReplicas

3

Hi,

@RenardDesMers , one of the options for limiting total number of pods of all deployments would be setting ResourceQuota. Put all your consuming app deployments in a single namespace and set and a hard limit of pods number:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: simple-quota
  namespace: my-namespace
spec:
  hard:
	pods: "10"  # Maximum of 10 pods allowed

It might be a bit dirty, but should do the job.

HTH