NGINX Ingress adds 1500ms to response time

General Discussions
1

Hey, looking to get some feedback/assistance, as I’m new to Kube and having a tough time getting my application deployed properly. For context, I’m running a PHP (laravel) app, and on a conventional web server I get response times of 100ms and below, and now I’m getting 2s+

Cluster information:

Kubernetes version: 1.25.0
Cloud being used: Hetzner Cloud
Installation method: Manual
Host OS: Ubuntu 20.04
PHP: 8.1.9
Ingress Controller: Nginx

I have a deployment of 2 replicas, combined by a Cluster IP service (8080/TCP), and an ingress that maps to the service (all requests to beta.mydomain.com go to api-service).

When I run a curl command from my master node to the api’s pod, using the private IP address, I get a response time of 500ms (poor image performance, working on it, but 500ms is what I expect currently). However when I run a curl command outside the cluster using the domain, it adds an additional 1500ms~ to the response time.

Here is how I setup the nginx controller:

helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx -f ingress-values.yaml --namespace ingress-nginx --create-namespace

ingress-values.yaml:

controller:
  replicaCount: 1
  service:
    enabled: true
    type: LoadBalancer
    annotations:
      load-balancer.hetzner.cloud/location: hel1
  resources:
    limits:
      cpu: "800m"
      memory: "2096Mi"
    requests:
      cpu: "800m"
      memory: "2096Mi"
  ingressClassResource:
    name: nginx
    enabled: true
    default: true
    controllerValue: "k8s.io/ingress-nginx"
  ingressClassByName: true

And here is my api ingress yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-production
  name: uc-api-ingress
spec:
  rules:
    - host: beta.mydomain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: uc-api-service
                port:
                  number: 8080
  tls:
    - hosts:
        - beta.mydomain.com
      secretName: beta.mydomain.com
``` If you need any additional information, just ask. I've been struggling to figure this out, and really would appreciate any help.

Thanks!
2

if you can share a bit more information:

  • what is the BODY size for this request?
  • how many concurrent requests are you doing or is it just a single request?
  • Do you have redirects in the HTTP request ?

there are several places I would look into in addition that are no HTTP request related:

  • what is the cpu consumption of the ingress Pods? is it throttling? are there non-voluntary context switches? (you can user RTM metrics tools such Sosivio as an examine to check this)
  • if your Pod throttling?

further deeper issues might be differences in MTU between the client performing the request and the ingress Pod and routing loops, but I wouldn’t check that until you check the above mentioned issues.