kubernetes readiness probe unexpectedly calls readiness endpoint 6-7 times

General Discussions
1

I have configured liveness and readiness probes in my deployment yaml file.

livenessProbe:
        httpGet:
          path: /api/health
          port: http
        initialDelaySeconds: 5
        periodSeconds: 20
        timeoutSeconds: 1

      readinessProbe:
        httpGet:
          path: /api/health/ready
          port: http
         periodSeconds: 120
         timeoutSeconds: 5
         successThreshold: 2

After deployment, somehow the readiness probe is calling the /health/ready endpoint 6 times simultaneously every 120 seconds. Ideally, it should call only once if the call is successful. Even if the call is unsuccessful, the next call should happen after the first call fails. Here kubernetes triggers 6 requests to /health/ready endpoint at the same time.

Furthermore, the probe makes requests using internal IP, like, calling http://10.0.0.1/api/health/ready as well as using a fully formed external url going through FQDN like https://my-example-api.com/api/health/ready

Help me understand the kubernetes readiness functionality. Why the readiness probes 6 times? And why it probes through FQDN?

1 Like
2

You mean “6 times per replica”? (Obvs if there are 6 replicas in the deployment, each will get called.)

3

No replica, only 1 pod and 1 deployment.

4

OK.

Is it maybe that something else is making the calls? like a load balancer? That might explain the different URL. Can you tell where they’re coming from? If there’s a k8s Service fronting it, there may be some masquerading happening on the node, hiding the true caller.