Hello All,
I am running an ingress on GKE cluster and have issue with backend being UNHEALTHY. In short, ingress is connected to a svc, svc has endpoints and the endpoints has liveness probe configured with success. Not sure what the problem is. Any help would be great. Have captured the outputs as below
kk@jumphost:~$ kubectl get ingress jenkins-ingress
NAME HOSTS ADDRESS PORTS AGE
jenkins-ingress * 34.107.221.217 80 13h
kk@jumphost:~$ kubectl describe ingress jenkins-ingress
Name: jenkins-ingress
Namespace: default
Address: 34.107.221.217
Default backend: jenkins-service:8080 (10.4.2.15:8080)
Rules:
Host Path Backends
---- ---- --------
* * jenkins-service:8080 (10.4.2.15:8080)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"jenkins-ingress","namespace":"default"},"spec":{"backend":{"serviceName":"jenkins-service","servicePort":8080}}}
ingress.kubernetes.io/backends: {"k8s-be-31308--007ae74e899741f4":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-jenkins-ingress--007ae74e899741f4
ingress.kubernetes.io/target-proxy: k8s-tp-default-jenkins-ingress--007ae74e899741f4
ingress.kubernetes.io/url-map: k8s-um-default-jenkins-ingress--007ae74e899741f4
Events: <none>
2) Looking at the service, it has endpoints as below
kk@jumphost:~$ kubectl get svc jenkins-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins-service NodePort 10.8.9.248 <none> 8080:31308/TCP 6d13h
kk@jumphost:~$ kubectl describe svc jenkins-service
Name: jenkins-service
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"jenkins-service","namespace":"default"},"spec":{"ports":[{"port":...
Selector: app=jenkins
Type: NodePort
IP: 10.8.9.248
Port: <unset> 8080/TCP
TargetPort: 8080/TCP
NodePort: <unset> 31308/TCP
Endpoints: 10.4.2.15:8080
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
3) The endpoint has liveness probe configured and it is passing
kk@jumphost:~$ kubectl describe pod jenkins-deployment-b5664c765-jsz47
Name: jenkins-deployment-b5664c765-jsz47
Namespace: default
Priority: 0
Node: gke-cluster-1-default-pool-5a375c66-jq71/10.128.0.4
Start Time: Tue, 17 Mar 2020 12:14:59 +0000
Labels: app=jenkins
pod-template-hash=b5664c765
Annotations: kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container jenkins
Status: Running
IP: 10.4.2.15
IPs: <none>
Controlled By: ReplicaSet/jenkins-deployment-b5664c765
Containers:
jenkins:
Container ID: docker://9f31b3cdec6df6dbfa2606e8231ab1e828618529c770fb381805bbb09e982a7e
Image: jenkins:2.60.3
Image ID: docker-pullable://jenkins@sha256:eeb4850eb65f2d92500e421b430ed1ec58a7ac909e91f518926e02473904f668
Port: 8080/TCP
Host Port: 0/TCP
State: Running
Started: Tue, 17 Mar 2020 12:15:02 +0000
Ready: True
Restart Count: 0
Requests:
cpu: 100m
Liveness: http-get http://:8080/login delay=0s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/test-pd from task-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-jlcsw (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
task-volume:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: my-disk-claim-1
ReadOnly: false
default-token-jlcsw:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-jlcsw
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>