HPA for PHP FPM and Nginx or Pod with both Containers?

General Discussions
1

Learning about Horizontal Pod Autoscalers. Still rather confused on how to set one up for my PHP App.

Current Setup

Currently have a setup with these deployments/pods behind an ingress nginx resource:

  • php fpm
  • php worker
  • nginx
  • mysql
  • redis
  • workspace

NB The database services may be replaced by managed database services so that would leave me with 4 pods.

Horizontal Pod Autoscaler Setup

Now to run an HPA I could add this as a resource part of the php deployment (based on DO article)

. . .
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: php
    spec:
      containers:
      - image: myphpimage
        imagePullPolicy: Always
        name: nginx
        resources:
          limits:
            cpu: 300m
          requests:
            cpu: 100m
            memory: 250Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
. . .

to the php fpm app pod and have it replicate when need be and trigger Cluster Autoscaling in return. But All Ingress data will hit Nginx pod first, not PHP FPM.

Ingress Nginx Setup

From Ingress all goes to the Nginx Service which accesses the Nginx web server:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-resource
  namespace: smt-local
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
    # host can be set in /etc/host or with laravel valet `valet link`
  - host: smart48k8.local
    http:
      paths:
        # https://kubernetes.io/docs/concepts/services-networking/ingress/#path-types 
        # `/` is all paths using Prefix
      - pathType: Prefix
        path: /
        backend:
          service:
            # refers to our app service exposed to the cluster
            name: nginx-service
            port:
              number: 80

So it feels like I need to take care of Nginx scaling as well at least…

Options

Now, to have HPA deal with Nginx and PHP FPM I could either:

  1. Add two HPA - one for php and one for nginx
  2. merge the Nginx and PHP FPM deployments as two containers in one pod and HPA them

Questions

  1. I could do choice one and scale both pods, but I wonder how that could work… Perhaps I get 5 Nginx pods and 10 php pods… would ingress nginx and all still play nice?
  2. I could go for the second option, but then I wonder, wil my nginx service still work well? Is this possible?
  3. PHP App Examples - Are there HPA PHP App example setups with horizontal pod autoscaling you can recommend?
2

@jasperf These are interesting questions. I have not implemented HPA yet for our Nginx/PHP-FPM applications, but I will be looking at this over next few months.

Based on my previous research, it seemed that different forums were suggesting keeping Nginx and PHP-FPM together (as 2 containers in a pod) and then scaling those pods. I think the main argument for keeping things like this is that the Nginx container can be running with very few resources, and so the additional overhead is very small for the benefit of reduction in complexity. (ie. There is no need to scale two deployments, only one.) We currently run combined pods like this, without the HPA.

That said we do not run additional pods for our application, since we use the cloud-hosted database offerings (eg. RDS in AWS) instead of hosting the DB in Kubernetes. Additionally, we do not use a Redis or similar.

I am not sure how your other application components would need to be scaled, but I suppose you would just use Prometheus metrics as well.

1 Like
3

@domhdp I have since decided to merge both the Nginx Pod and the PHP FPM Pod. I did this as I would be able to use Horizontal Pod Autoscaling on one Pod and yes Nginx does not use many resources.

Another reason for keeping them together in one pod is that we will use LE (Let’s Encrypt) SSL and other SSL providers (Xolphin) and the Nginx container needs to be tightly coupled to the PHP FPM container. We cannot have Nginx scale and PHP FPM not as each Laravel PHP application container will queue the creation, updating and removals of Let’s Encrypt to the SSL certificates at Nginx.

We are however looking at queuing and dealing with SSL Certificates in the Nginx Ingress using Cert Manager. This so the Ingress can deal with certificates. We do however need to see how we can have our Laravel PHP application container enqueue certificates with the Ingress Nginx Pod and how we can make sure Cert Manager sends the requests for websites with SSL to the proper Nginx/PHP FPM Pod. This all in an automated fashion. We also looked into Traefik, but we kind of got stuck at ways to automate the adding of multiple sites with SSL certificates using a PHP enqueue system.