Unresponsiveness when hpa scaling

General Discussions
1

Hi,

I am having some issues when upscaling happens due to a load test.
Exactly when the pods are being added, the applications stop responding for some seconds…
image

When the “unresponsiveness valleys” in the above graphic happens exactly when the pods are added. I have done repeated tests over this week and there is a direct correlation.

We tried the same exact test from inside the kubernetes cluster, connecting directly to the service, and there was no unresponsiveness.

Any hint or idea of what could be happening?

Cluster information:

Kubernetes version: 1.19.6
Cloud being used: Azure
Installation method:
Host OS:
CNI and version:
CRI and version:

You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.

2

Maybe your new pods start receiving traffic before they can handle it?
Can you make sure to add readiness probes to your pods?

1 Like
3

Thanks rata for the quick response :slight_smile:
That does not seem to be the case; the pods are starting properly and the liveness and readiness probes are in place.
Also,

  • There are no pods restarting due to a failing liveness check.
  • Once the startup of a pod is done, the pods stay ready and that does not change,

I did check the pods state during the different tests with kubectl get pods -w and kubectl describe pod <podname> to see its events.