Nginx bouncing me back and forth although I have only one back-end (TLS related question)

General Discussions
1

I started to play more and more with Kuberentes these days I see how easy it is to get your hands dirty ;-))) I try to deploy my first K8s application and decided to go with dockuwiki because it is quite simple.

I already have weave-net CNI, Nginx and MetalLB (the last two are deployed via helm). Everything works as expected except the SSL configuration (so far).

I have installed the charts without modifying the values.

root@sofx1022k8m3155.home.lan:~# helm list -A
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
ingress-nginx           ingress-nginx   1               2024-02-29 10:48:36.80626882 +0000 UTC  deployed        ingress-nginx-4.10.0    1.10.0
metallb-loadbalancer    metallb-system  1               2024-02-29 10:37:28.811058798 +0000 UTC deployed        metallb-4.14.2          0.14.3
root@sofx1022k8m3155.home.lan:~#

Here is the manifest I use to deploy my app:

root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# cat sofk1022wiki31520.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: sofk1022wiki31520
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sofk1022wiki31520
  namespace: sofk1022wiki31520
spec:
  selector:
    matchLabels:
      application: sofk1022wiki31520
  replicas: 1
  template:
    metadata:
      labels:
        application: sofk1022wiki31520
    spec:
      volumes:
        - name: sofk1022wiki31520
          hostPath:
            path: /mnt/nfs_share/sofk1022wiki31520/config
            type: Directory
      containers:
      - name: sofk1022wiki31520
        image: lscr.io/linuxserver/dokuwiki:latest
        ports:
        - containerPort: 80
        volumeMounts:
         - name: sofk1022wiki31520
           mountPath: /config
        env:
        - name: PUID
          value: "1001"
        - name: PGID
          value: "1001"
        - name: TZ
          value: "Europe/Sofia"
---
apiVersion: v1
kind: Service
metadata:
  name: sofk1022wiki31520
  namespace: sofk1022wiki31520
  labels:
    application: sofk1022wiki31520
spec:
  ports:
  - port: 80
    protocol: TCP
  selector:
    application: sofk1022wiki31520
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sofk1022wiki31520
  namespace: sofk1022wiki31520
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - w.d3v.ma3x.org
    secretName: sofk1022wiki31520
  rules:
  - host: w.d3v.ma3x.org
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: sofk1022wiki31520
            port:
              number: 80
---
apiVersion: v1
kind: Secret
metadata:
  name: sofk1022wiki31520
  namespace: sofk1022wiki31520
type: kubernetes.io/tls
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVwakNDQW82Z0F3SUJBZ0lVUFRzeTJ2OVR3TmFGM2d3UElpRzhwa2RhZDZ3d0RRWUpLb1pJaHZjTkFRRUwKQlFBd1F6RUxNQWtHQTFVRUJoTUNRa2N4RGpBTUJnTlZCQWdNQlZOdlptbGhNUTR3REFZRFZRUUhEQVZUYjJacApZVEVVTUJJR0ExVUVDZ3dMU0c5dFpTQk1ZVzRnUTBFd0hoY05Nakl3TmpBNE1EZzFOekl5V2hjTk16SXdOakExCk1EZzFOekl5V2pCV01Rc3dDUVlEVlFRR0V3SkNSekVPTUF3R0ExVUVDQXdGVTI5bWFXRXhEakFNQmdOVkJBY00KQlZOdlptbGhNUkV3RHdZRFZRUUtEQWhJYjIxbElFeGhiakVVTUJJR0ExVUVBd3dMU0c5dFpTQk1ZVzRnUTBFdwpnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFEdjJrbmNFYTIvbHR3SU1tekJkYXZ0CktIZ2lFd1VTUWV0TWNpRFJKNUozbHFPRTdjRkJTQWlkUGFzcmFPUTE2S2pTU1ZqWjB4REs0cTNGOU9BdDU3VWEKNGg4WldiQVRDQmZyckVsOGYvQTl3VERmNUhrTVZWUFE2U0V1VHhqR0VUVDhsVzV3K3BIVk5TU2puK0poUUw4RwpTNElsZDRRUEI3VUhvcEFLbDlGYy92UmY0YjBxbVU5eWVteGtnbHBOVmhpRk1aOVg2aU9EbTUrS3hoQ0ZJYzNoClVBMkxKQTJKN05SZk0wYWZYQWJYeGRQbWdPTDNqcldzcDBIMVJtczVWUDgyM0hXSkF0TlR1NkdKbUVTZWFFaXUKNFJlcnBlMERNUkNpUkwzY3lzQU5kdlEzVk10WGlsTkRKTGRnUHFVSmZxbFcyZHJORmZXVmtwU0h6U21MQmlYeApBZ01CQUFHamZ6QjlNQWtHQTFVZEV3UUNNQUF3SFFZRFZSME9CQllFRkVZcnpEV3hNbUNXVmNJbUV3UEJIaTAwClJjckVNQXNHQTFVZER3UUVBd0lGb0RBZEJnTlZIU1VFRmpBVUJnZ3JCZ0VGQlFjREFnWUlLd1lCQlFVSEF3RXcKSlFZRFZSMFJCQjR3SEljRWZ3QUFBWUlJYUc5dFpTNXNZVzZDQ2lvdWFHOXRaUzVzWVc0d0RRWUpLb1pJaHZjTgpBUUVMQlFBRGdnSUJBRjJKWHJKelVUWmU1VnVhNHROSzFrcm5PUmtFVFRsKy9yM1Qrc3B1WHU1NlNvUktyOStMCnpkQVlBQkkxZyt2K0MvN1h3emcybWY2cEp0ZVZVVWNxdkFnV0RyYU1SSTYrK0xEb20vSkw4ZzRsbVNWMDNTdGIKRHdRSEpLWlJlWTQ4UnJDc3N3ZURhNVhjQnhMYTFFZElZakNlMnVtNXlwdG8zWVg5QkVaVENoTk8xVzVxVWdScwpuTytyb2R3SWFyTkVYT0Nqa1RwTHJhUmp0aXBKRHEwckd5ZThwbVVzMlpZMzZZMlZKbHg4cG4wMk1YT2RTMDZsCkpMdWw1dFA2TnhjRi9PcW9rYmdERzFGUjY5SDgyTUhxKytlT1pxTVh0VUxFLzROaWwxa2RkeWdJNC9ONVhwc3YKUE03cE9qMWh2M0h3S0lvVzdBWDM4NUxpZ3ZCcTB0ZG4vQ01jNzdjYVlvSTQwTmlsRkVRanh4UTZXNDVlaW1VYwo1c2tHQXlqOFVpcFdZcE9BMzNiR3RzM0NxNElEU1FGMDU3MTEvWC9xQ1RIdEhLUkJNVTk3UGMxTWpKTS9kNWFDCjN5L0Jka25zOEI2M2NpL2h1MG9Wbm9tWlRSaXpueVlJa0pXbTFhOGRrWjNnVlhxT3BUem5jWk1Vd0JjVHprQVcKNEFxdWE3Z3hCaDMvazFtblRMZ0x3SXovaFFMWHUySDhtbUxTSXR4aW0vQmhXREF5U2RIU3pDTm1zcFQxR3Z3Nwp2UDRQUVFZSlB2MWpXQThMWWsrNlI5aEV0bGs4Y2ZBVGVKNXNKdzlFK3UxaXNMOVEzb1ZZeW9rTk80K3RzWUdqCkw4YmdNMVVmTS9HcEgrT0h0SlNLVTlycmFRSTlpd215TXZwbmRZLzVJRDJNWkhVblNmR1o5OFdGCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NBUUVBNzlwSjNCR3R2NWJjQ0RKc3dYV3I3U2g0SWhNRkVrSHJUSElnMFNlU2Q1YWpoTzNCClFVZ0luVDJySzJqa05laW8wa2xZMmRNUXl1S3R4ZlRnTGVlMUd1SWZHVm13RXdnWDY2eEpmSC93UGNFdzMrUjUKREZWVDBPa2hMazhZeGhFMC9KVnVjUHFSMVRVa281L2lZVUMvQmt1Q0pYZUVEd2UxQjZLUUNwZlJYUDcwWCtHOQpLcGxQY25wc1pJSmFUVllZaFRHZlYrb2pnNXVmaXNZUWhTSE40VkFOaXlRTmllelVYek5HbjF3RzE4WFQ1b0RpCjk0NjFyS2RCOVVack9WVC9OdHgxaVFMVFU3dWhpWmhFbm1oSXJ1RVhxNlh0QXpFUW9rUzkzTXJBRFhiME4xVEwKVjRwVFF5UzNZRDZsQ1g2cFZ0bmF6UlgxbFpLVWg4MHBpd1lsOFFJREFRQUJBb0lCQVFDbTZMclpRQnd1NVNaOQpOY25rM3FRanh6UENVb1lHN0lSM2tEd3lFRjJtaHRlSDI0UE9qL2dlcEJtSGZkakJnd1JvUWM1bzFxNHE0WStYCjRBNVpJNU11QUFrd05PK0lidXRtZjJBc0dVUHZWWTlUWHNTMXhuTXJQL0J0TDNISDg1Vng1L01RLysyWDl3Rm0KTkE3dktvVkVOaXQ5TXl4YzRUb2k5QWRVdDgxajdsYndLSVY0RlZiVURNWmViUEE5NFkrV0NuV0FkRWJPT1U4cQppY0ViRW5WYnZUZys2N1h6ZGdRNjVwRHA1TWZFblVXTFl5aFdKOVlzWElFdHJDaERjWFE5dkE4eW1WQ0VHNlltCktWYmRZZnJkSk9ja3lXVC9GSlVyRWdGMG9rcEZnMm5oY1dHb3RJOWlFR1RER1RGT2VGU1lvQjB5R2pZeTV1VkMKeDFHYUZtL1pBb0dCQVBvamlpMXNnbm4zL2dIVkU3eW1iUU1VOHBBb1ZMZkNUQ05HNDliZ01nM09FOUROL3h6SQpaMXp3aWE1UHpJUjNXRjIyS1RyU3ZlUkxHSXN5a3d4aGNMUmY0akIwWTQzRjdBR0ZVTk5YSTdxb2hGcXIwWXk0CmhlL2hGRGczdGxnNElSeGtIZ0tjbVN1RDliK0VTLzJPRGNEQ3IybjBWRHMyS0VhU1NKT3FwNTNmQW9HQkFQVjUKREJreWttQ2YwUS82Tmt5cmc3WHA0dDU5c3hYeWpMZjY4MWtFbWN3TmhJbGU2M1NWVDZ1aGV4V1FOSHhvVTBUbQpnRXh5NkNyWVBjQVQzMmJMUTNDQ2NJOHBRUFpKUlpINUt1anpKMFpDRnN1RVhpT05tcEU4WDFLZ3Ewa3ljMUQzClpzV0h1WjNuT0FBL2o4cGpOMUpDczU2YTVzdnA0NTVBNDlkSk1SWXZBb0dCQU1jS1FxaDBnTHAzS0VyaTkycVUKNzRlVlFwS05WU1JIMm5menI5YkszV2w0d3ZIVENSNHBKWDZoOUp5YzZQdTdUN2g1L0FMUHNNU1pNUFdLajlJNQpNd3FFaGFMUzZtamt4akJMbWc0aGQ4L095a3F3RC9XdTJ2MFBHQnpIMERRUXJlQStsbGF6cjNvZDdnMndReHVHCnp0REUyZm93cmxuVGRXYjZmZi9xdXFmM0FvR0JBSy9yNkhmR2ZwRXcrQ3R6THFjVDVIamkvZXhtT1lwcEllQlkKd1QzeXhpWXZLZVVwV2FWYnZKVk1UV0hPejA2NWxiT3pYZzlyNm1lS1FmYitSd1RzZDVDU2F3aXVrQmZ4dXlJNgphRDQ2eE5MZFdtalFuSWJ1WE8vcFRJZHpEazZQRVlYZFNKSVlDemJYRnBOVytqNFpqUFF2U1h0Sm1FNWJlUDFaCmsvQWVURWVSQW9HQUtNaVJ3Z2pPMTFpUjRGb3JQS1J2eTI4QStlNmZ5L3FqUExVUVZoUmxRazFoKzdJVjBRMUEKUXlSeWw2ZDlwK0s2MExLdE40MnYrS1p2SmV6cHk4T2lQUjFwbUI5aVRnZzdpL2F0OFdjcnRFdDNEdG90V2NadQpJWWFzK2VmNEY3VDJudmVRRmFkaU9TYndkb0tycmJZQStDNXZ6Ynd1aDZickpSOExhS3gwUjhBPQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=

root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520#

I have pointed that hostname to the external IP of the load balancer (MetalLB):

root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# kubectl get ingress -A
NAMESPACE           NAME                CLASS   HOSTS                     ADDRESS         PORTS     AGE
sofk1022wiki31520   sofk1022wiki31520   nginx   w.d3v.ma3x.org            192.168.15.16   80, 443   9d
test-blue           ingress-blue        nginx   blue.nginx.example.com    192.168.15.16   80        13d
test-green          ingress-green       nginx   green.nginx.example.com   192.168.15.16   80        13d
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# kubectl get service -A
NAMESPACE           NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                      AGE
default             kubernetes                           ClusterIP      10.96.0.1        <none>          443/TCP                      20d
ingress-nginx       ingress-nginx-controller             LoadBalancer   10.104.153.204   192.168.15.16   80:30647/TCP,443:31436/TCP   19d
ingress-nginx       ingress-nginx-controller-admission   ClusterIP      10.105.18.242    <none>          443/TCP                      19d
kube-system         kube-dns                             ClusterIP      10.96.0.10       <none>          53/UDP,53/TCP,9153/TCP       20d
metallb-system      metallb-webhook-service              ClusterIP      10.110.24.87     <none>          443/TCP                      19d
sofk1022wiki31520   sofk1022wiki31520                    ClusterIP      10.98.8.33       <none>          80/TCP                       10d
test-blue           nginx-deploy-blue                    ClusterIP      10.103.242.69    <none>          80/TCP                       13d
test-green          nginx-deploy-green                   ClusterIP      10.110.218.40    <none>          80/TCP                       13d
test                nginx                                LoadBalancer   10.97.58.213     192.168.15.15   80:30830/TCP                 19d
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520#

Here is the DNS record:

root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# host w.d3v.ma3x.org
w.d3v.ma3x.org has address 192.168.15.16
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520#

The issue is that, although I have TLS settings enabled, I get “Kubernetes Ingress Fake Certificate”. Which means that my TLS block for that particular ingress is simply ignored.

Here is how the certificate looks like and what I expect to see:

root@muttley.home.lan:~/self-signed-ssl/2022/tls# openssl x509 -in my-service.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            3d:3b:32:da:ff:53:c0:d6:85:de:0c:0f:22:21:bc:a6:47:5a:77:ac
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = BG, ST = Sofia, L = Sofia, O = Home Lan CA
        Validity
            Not Before: Jun  8 08:57:22 2022 GMT
            Not After : Jun  5 08:57:22 2032 GMT
        Subject: C = BG, ST = Sofia, L = Sofia, O = Home Lan, CN = Home Lan CA

....snip.....
                46:2B:CC:35:B1:32:60:96:55:C2:26:13:03:C1:1E:2D:34:45:CA:C4
            X509v3 Key Usage:
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Client Authentication, TLS Web Server Authentication
            X509v3 Subject Alternative Name:
                IP Address:127.0.0.1, DNS:home.lan, DNS:*.home.lan

Unfortunately I see the default Nginx certificate:

root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# curl -kv -I https://w.d3v.ma3x.org
*   Trying 192.168.15.16:443...
* Connected to w.d3v.ma3x.org (192.168.15.16) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: O=Acme Co; CN=Kubernetes Ingress Controller Fake Certificate
*  start date: Mar  9 14:11:36 2024 GMT
*  expire date: Mar  9 14:11:36 2025 GMT
*  issuer: O=Acme Co; CN=Kubernetes Ingress Controller Fake Certificate
*  SSL certificate verify result: self-signed certificate (18), continuing anyway.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x559fcf098eb0)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> HEAD / HTTP/2
> Host: w.d3v.ma3x.org
> user-agent: curl/7.81.0
> accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200
HTTP/2 200
< date: Tue, 19 Mar 2024 14:30:36 GMT
date: Tue, 19 Mar 2024 14:30:36 GMT
< content-type: text/html; charset=utf-8
content-type: text/html; charset=utf-8
< x-powered-by: PHP/8.3.4
x-powered-by: PHP/8.3.4
< vary: Cookie
vary: Cookie
< set-cookie: DokuWiki=vuc5aufhsf5fviost2pjqd5402; path=/; secure; HttpOnly; SameSite=Lax
set-cookie: DokuWiki=vuc5aufhsf5fviost2pjqd5402; path=/; secure; HttpOnly; SameSite=Lax
< expires: Thu, 19 Nov 1981 08:52:00 GMT
expires: Thu, 19 Nov 1981 08:52:00 GMT
< cache-control: no-store, no-cache, must-revalidate
cache-control: no-store, no-cache, must-revalidate
< pragma: no-cache
pragma: no-cache
< set-cookie: DW68700bfd16c2027de7de74a5a8202a6f=deleted; expires=Thu, 01 Jan 1970 00:00:01 GMT; Max-Age=0; path=/; secure; HttpOnly; SameSite=Lax
set-cookie: DW68700bfd16c2027de7de74a5a8202a6f=deleted; expires=Thu, 01 Jan 1970 00:00:01 GMT; Max-Age=0; path=/; secure; HttpOnly; SameSite=Lax
< strict-transport-security: max-age=31536000; includeSubDomains
strict-transport-security: max-age=31536000; includeSubDomains

<
* Connection #0 to host w.d3v.ma3x.org left intact
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520#

Additional confusion comes by the fact that when I hit refresh a few times, my browser start complaining about the exception I need to make, and this process continues over and over again.

It works for next few minutes, then ask about confirmation again.
More info I have collected:

root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# kubectl -n sofk1022wiki31520 describe pod sofk1022wiki31520-6db744c87d-rbwwp
Name:             sofk1022wiki31520-6db744c87d-rbwwp
Namespace:        sofk1022wiki31520
Priority:         0
Service Account:  default
Node:             sofx1022k8w3156.home.lan/192.168.15.6
Start Time:       Tue, 19 Mar 2024 07:08:59 +0000
Labels:           application=sofk1022wiki31520
                  pod-template-hash=6db744c87d
Annotations:      <none>
Status:           Running
IP:               10.40.0.4
IPs:
  IP:           10.40.0.4
Controlled By:  ReplicaSet/sofk1022wiki31520-6db744c87d
Containers:
  sofk1022wiki31520:
    Container ID:   containerd://5d089bb417e13ed2454ea4767ed642c3930cdea9e9d13a5524faaffbcd277107
    Image:          lscr.io/linuxserver/dokuwiki:latest
    Image ID:       lscr.io/linuxserver/dokuwiki@sha256:ab15dfd4e0489457915f20d753aafa0929b3feeeb5b739b54446fcacd12e7bc4
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 19 Mar 2024 07:09:08 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      PUID:  1001
      PGID:  1001
      TZ:    Europe/Sofia
    Mounts:
      /config from sofk1022wiki31520 (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-g52dx (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  sofk1022wiki31520:
    Type:          HostPath (bare host directory volume)
    Path:          /mnt/nfs_share/sofk1022wiki31520/config
    HostPathType:  Directory
  kube-api-access-g52dx:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# kubectl -n sofk1022wiki31520 get service
NAME                TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
sofk1022wiki31520   ClusterIP   10.98.8.33   <none>        80/TCP    10d
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# kubectl -n sofk1022wiki31520 describe service sofk1022wiki31520
Name:              sofk1022wiki31520
Namespace:         sofk1022wiki31520
Labels:            application=sofk1022wiki31520
Annotations:       <none>
Selector:          application=sofk1022wiki31520
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.98.8.33
IPs:               10.98.8.33
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.40.0.4:80
Session Affinity:  None
Events:            <none>
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520#

Please help me to solve the issue.

Thanks

When I go back in time I remembered that I have SSL connection for that ingress even I haven’t configured such. For instance on the beginning I tried simple ingress only for port 80, but I was immediately redirected to the same page by using the default certificate.

I was advised to disable redirect to SSL as mentioned here: TLS/HTTPS - Ingress-Nginx Controller

And I did id only for that ingress. Nothing has changed.

root@sofx1022k8m3155.home.lan:~# kubectl -n sofk1022wiki31520 describe ingress sofk1022wiki31520
Name:             sofk1022wiki31520
Labels:           <none>
Namespace:        sofk1022wiki31520
Address:          192.168.15.16
Ingress Class:    nginx
Default backend:  <default>
TLS:
  sofk1022wiki31520 terminates w.d3v.ma3x.org
Rules:
  Host            Path  Backends
  ----            ----  --------
  w.d3v.ma3x.org
                  /   sofk1022wiki31520:80 (10.40.0.4:80)
Annotations:      nginx.ingress.kubernetes.io/ssl-redirect: false
Events:           <none>
root@sofx1022k8m3155.home.lan:~#

Cluster information:

Kubernetes version: v1.28.7
Cloud being used: bare-metal
Installation method: apt based packages
Host OS: Ubuntu 22.04.4 LTS (amd64)
CNI and version: weaveworks/weave-kube:latest
CRI and version: containerd (1.6.28-2, amd64)

2

Hi,
Your certificate should include FQDN in its SAN list. Otherwise ingress will return default certificate.

3

I see, do this means that Nginx doesn’t support wildcard certificates? In my case “*.home.lan” is a wildcard.

4

It does support, but your certificate does not include w.d3v.ma3x.org fqdn configured in ingress resource

5

Aaa, I see. Sorry about that, my fault.

Will try to issue a new certificate and check how it behaves.

What about the bouncing? I mean although I’m being redirected to the correct application, in my case w.d3v.ma3x.org and the default SSL certificate is picked up, after few minutes of inactivity when I refresh the page the Firefox complains again for that the SSL certificate is not recognized and I have to make an exception for it, but the certificate is the same.

6

I assume you could write down certificate thumbprint and once you get a bounce, compare it to the previous thumbprint record. Unless there is certificate replacement, this sounds more like a browser thing, rather than a k8s thing.

7

Understand.

Ok, let me first test with new certificate with correct FQDN entry then if the issue still continue will dig into it.

Many thanks for your help.

8

I’m able to connect to my service with the correct SSL certificate.

root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520# openssl s_client -showcerts -servername w.d3v.ma3x.org -connect w.d3v.ma3x.org:443 </dev/null
CONNECTED(00000003)
depth=0 C = BG, ST = Sofia, L = Sofia, O = Ma3x, OU = IT Dept, CN = w.d3v.ma3x.org
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = BG, ST = Sofia, L = Sofia, O = Ma3x, OU = IT Dept, CN = w.d3v.ma3x.org
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 C = BG, ST = Sofia, L = Sofia, O = Ma3x, OU = IT Dept, CN = w.d3v.ma3x.org
verify return:1
---
Certificate chain
 0 s:C = BG, ST = Sofia, L = Sofia, O = Ma3x, OU = IT Dept, CN = w.d3v.ma3x.org
   i:CN = ma3x.org, C = BG, L = Sofia
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 23 12:53:35 2024 GMT; NotAfter: Mar 23 12:53:35 2025 GMT
-----BEGIN CERTIFICATE-----
MIIDmjCCAoKgAwIBAgIUIHfJGNWhOfhc/PL9JgQNAtdBV7wwDQYJKoZIhvcNAQEL
BQAwMDERMA8GA1UEAwwIbWEzeC5vcmcxCzAJBgNVBAYTAkJHMQ4wDAYDVQQHDAVT
b2ZpYTAeFw0yNDAzMjMxMjUzMzVaFw0yNTAzMjMxMjUzMzVaMGcxCzAJBgNVBAYT
AkJHMQ4wDAYDVQQIDAVTb2ZpYTEOMAwGA1UEBwwFU29maWExDTALBgNVBAoMBE1h
M3gxEDAOBgNVBAsMB0lUIERlcHQxFzAVBgNVBAMMDncuZDN2Lm1hM3gub3JnMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4QSNDzTH67QqPHscLoLKt/HQ
E6/UMqMMn+J3EIJjti34mjlxNU752HXng0BejOcM2WG55HPGFWhNWyyuV3j39692
guXdpl4aRuYWBO5eWLXengmp+DCeLNpapXFan4riH5WXOFHXIlfImv0MTSDcaPKv
B9phOxgozlkvT4XnfHewL3IDxX7rmZBugxyqqXyQ0HVQPqMX+asuOAl0Jqu1zD/+
I4zHTThwE45kx5jtzJD52Y3ab+eLvFM6J/EoVF7CRP6lXBNSBGZGzshG5JfC31zq
ozaNK07zw4eBopy7xghTPRVCAX7FC6hC3YrHc6fXVFCxjasixvTySM9j1BI2mwID
AQABo3UwczAfBgNVHSMEGDAWgBRmLnvzdAZIZ+f3jkb+E0HUj2cwfTAJBgNVHRME
AjAAMAsGA1UdDwQEAwIE8DAZBgNVHREEEjAQgg53LmQzdi5tYTN4Lm9yZzAdBgNV
HQ4EFgQU9t3BFw/92hbZ9on2Uj7CtjEPDLYwDQYJKoZIhvcNAQELBQADggEBACVg
3vdeTLRXNKLSWb+GTXgaF9hgRIGimJ+U8kahB08/M/bbxJWc5OSsg8TI60n0SsUA
+1AQ8Rr/KUXOafRyo6SEJXNl9TMmfydGVHVhki00dUK1bVI54Hnc9e0W/Sm9KSrC
D3X+DyX7FSguzdDEr0cx78VCTlm8gKchaJbLKZqx9Ib+aI82YAaOUWh8U0BgSfUH
MzrgpMxQtfPUeAQlRAcKGAHmDpASlgyFunQUw1FxFnFbOCZAjZReezIKsLU6JoFq
6cIrQZcGPGLZN5ttSw307qtT3sDSfykWyKTyjlYNP9vKWbPcp/PM00t2tgkcOl/q
c4gj/0k82Bheta7yEK8=
-----END CERTIFICATE-----
---
Server certificate
subject=C = BG, ST = Sofia, L = Sofia, O = Ma3x, OU = IT Dept, CN = w.d3v.ma3x.org
issuer=CN = ma3x.org, C = BG, L = Sofia
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1486 bytes and written 396 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 21 (unable to verify the first certificate)
---
DONE
root@sofx1022k8m3155.home.lan:/mnt/nfs_share/sofk1022wiki31520#

I confirm the issue with the bouncing SSL certificate is gone. The correct certificate is taken in place.