HTTPS 308 permanent redirection k8s ingress nginx

Asking for help? Comment out what you need so we can get more information to help you!

Cluster information:

Kubernetes version: 1.19
Cloud being used: AWS
Installation method:
Host OS:
CNI and version:
CRI and version:

I have created an EKS cluster following the examples from AWS EKS, I have deployed the nginx ingress controller on top from kubernetes/nginx, Created an ingress resource which points to back end k8s services, However, when I am trying to query via curl -kv https://dev01.cricket.com/demo/hello it returns Permanent Redirect 308

Here is my service-nlb.yaml

— (service-nlb.yaml)
kind: Service
apiVersion: v1
metadata:
name: ingress-nginx
namespace: default
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-internal: “0.0.0.0/0”
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-2:xxxxx:certificate/x-037xxxd-4a1e-x-x"
service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
spec:
ports:
- name: https
port: 443
targetPort: 80

Not sure what I am doing missing to get 308 error on https

ingress.yaml

kind: Ingress
metadata:
  labels:
    app: jg
    env: dev
  name: jg-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    **ingress.kubernetes.io/ssl-redirect: "false"**
spec:
  rules:
    - host: dev01.cricket.com
      http:
        paths:
          - backend:
              serviceName: jg-service
              servicePort: 8080
            path: /demo
    - host: dev02.cricket.com
      http:
        paths:
          - backend:
              serviceName: sad-service
              servicePort: 8080
            path: /sad
  tls:
   - hosts:
        - dev01.cricket.com
     secretName: my-tls-secret```
when I am doing curl -kv dev01.cricket.com it returns 308 permanent redirect

curl -kv https://dev01.cricket.com/demo/hello
*   Trying 10.41.168.92...
* TCP_NODELAY set
* Connected to dev01.cricket.com (10.41.168.92) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=dev01.cricket.com; OU=management:idms.group.140787; O=Cricket Inc.; ST=California; C=US
*  start date: Apr 19 08:37:06 2021 GMT
*  expire date: May 19 08:37:05 2023 GMT
*  issuer: CN=Cricket Corporate Server CA 1; OU=Certification Authority; O=Cricket Inc.; C=US
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7ff9eb80d200)
> GET /demo/hello HTTP/2
> Host: dev01.cricket.com
> User-Agent: curl/7.64.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 308
< server: Cricket
< date: Tue, 04 May 2021 16:38:38 GMT
< content-type: text/html
< content-length: 171
< location: https://dev01.cricket.com/demo/hello
< strict-transport-security: max-age=31536000; includeSubdomains
< x-frame-options: SAMEORIGIN
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
<
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>
* Connection #0 to host dev01.cricket.com left intact
* Closing connection 0
I am not sure why the TLS termination is not happening and the permanent redirect error is happening if I do the same curl on http it returns 301

curl -kv http://dev01.cricket.com/demo/hello
*   Trying 10.41.162.164...
* TCP_NODELAY set
* Connected to dev01.cricket.com (10.41.162.164) port 80 (#0)
> GET /demo/hello HTTP/1.1
> Host: dev01.cricket.com
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: Cricket
< Date: Tue, 04 May 2021 19:17:26 GMT
< Content-Type: text/html
< Content-Length: 162
< Connection: keep-alive
< Location: https://dev01.cricket.com/demo/hello
<
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>Cricket</center>
</body>
</html>
* Connection #0 to host dev01.cricket.com left intact
* Closing connection 0
backend-service.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jgdeployment
  labels:
    app: jg
spec:
  replicas: 3
  selector:
    matchLabels:
      app: jg
  template:
    metadata:
      labels:
        app: jg
    spec:
      containers:
      - name: jg
        image: docker.cricket.com/jga:1.0.0
        ports:
        - containerPort: 8080
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /demo/hello
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 10
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /demo/hello
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: jg-service
  namespace: "default"
spec:
  type: ClusterIP
  ports:
    - port: 8080
      targetPort: 8080
  selector:
    app: jg
Not sure what I am doing missing to get 308 error on https.