Ingress-nginx for simple httpd deployment doesn't work, what am I missing?

General Discussions
1

Cluster information:

Kubernetes version:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:48:33Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:42:41Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}

Cloud being used: bare-metal/vsphere
Installation method: kubeadm
Host OS: Ubuntu 20.04
CNI and version: Weave
CRI and version: Docker

I’m using MetalLB.

My problem and info

I’ve been trying to get ingress to work with a simple httpd:latest deployment. I’m trying to get it to respond on http://httpdtest.example.org. But it always times out.

I have httpdtest.example.org pointed at my ingress-nginx service’s external-ip via my laptops /etc/hosts file.

I once had my instance of longhorn’s UI responding properly (though it isn’t anymore), so I’m 85% certain I have ingress-nginx configured properly. (I configured ingress per their instructions here.)

Which leave me wondering what I’m missing in my httpdtest manifests.

deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: httpd
  name: httpd-deployment
  namespace: httpdtest
spec:
  replicas: 3
  selector:
    matchLabels:
      app: httpd
  template:
    metadata:
      labels:
        app: httpd
    spec:
      containers:
        - image: httpd:latest
          imagePullPolicy: IfNotPresent
          name: httpd
          ports:
            - containerPort: 80
              protocol: TCP
          resources:
            requests:
              cpu: "1.0"
              memory: "1G"
            limits:
              cpu: "1.0"
              memory: "1G"

service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: httpdtest-service
  namespace: httpdtest
  labels:
    app: httpd
spec:
  ports:
    - port: 80
      name: tcp-80
      protocol: TCP
      targetPort: 80
  selector:
    app: httpd
  type: ClusterIP

ingress.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpdtest-ingress
  namespace: httpdtest
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: httpdtest.example.org
    http:
      paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: httpdtest-service
              port:
                number: 80

Everything appears to be running:

$ kubectl -n httpdtest get all    
NAME                                    READY   STATUS    RESTARTS   AGE
pod/httpd-deployment-8475bd6c66-45xdf   1/1     Running   0          27m
pod/httpd-deployment-8475bd6c66-cmdp2   1/1     Running   0          27m
pod/httpd-deployment-8475bd6c66-r5xm6   1/1     Running   0          27m

NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/httpd-service       ClusterIP   172.20.209.13   <none>        80/TCP    33m
service/httpdtest-service   ClusterIP   172.20.100.56   <none>        80/TCP    26m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/httpd-deployment   3/3     3            3           27m

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/httpd-deployment-8475bd6c66   3         3         3       27m

Is there any other info that would help shed light on this?

Anyone see something I’m missing?

Thanks in advance!

2

Per some advice in Slack, my ingress.yaml now looks like:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpdtest-ingress
  namespace: httpdtest
  annotations:
    kubernetes.io/ingress.class: nginx
  labels:
    app: httpd
spec:
  ingressClassName: nginx
  rules:
  - host: httpdtest.lanecc.edu
    http:
      paths:
        - path: ""
          pathType: ImplementationSpecific
          backend:
            service:
              name: httpdtest-service
              port:
                number: 80

Unfortunately those changes don’t appear to have changed anything.

I did see the ingress-nginx controller logs say that config was reloaded in response to my changes.

I also thought it might be useful to mention that I configured ingress-nginx via:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

And then I manually set the spec.loadBalancerIP value via kubectl -n ingress-nginx edit service/ingress-nginx-controller

Also, can someone confirm that the fact the ingress-nginx IngressClass is in a different namespace doesn’t matter? Or do I need an IngressClass per namespace?

3

To add one more wrinkle, if I run:

kubectl delete -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

And then, once it’s done, run:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

http://httpdtest.example.org/ works. But then, just a little bit later, it stops working.

I don’t see any errors in the logs of the ingress-nginx-controller pod.

4

Apparently my attempts to keep things simple while learning about Kubernetes by avoiding Helm backfired on me.

I decided to just try the Helm install from the ingress-nginx docs and everything seems to work just fine now. Both my little httpd test deployment and my longhorn-ui config that’s behind httpd auth.

And they’re still working after waiting 15 minutes or so, when previously they’d have stopped working by then.

For the record, I did this:

$ kubectl delete -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

Then:

helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace
1 Like