Hey y’all! I’m having some issues with GKE’s HTTP(s) ingress. I set it up with GKE Autopilot and a simple container-native load balancing setup, and the initial config seems to work just fine. After deploying, however, something happens and the Ingress returns a 502 error for about 5-6 minutes. The pods are online during this time, and the service does seem functional (the pods pass service health checks). It seems like there’s a large delay between when the pods get created and when they actually get added to the ingress as a backend:
Not entirely sure if that’s it, but I’ve included all the config that I’m using to make this happen. It’s probably a dumb mistake that I’m missing somewhere. Does anyone know what’s going on with this?
Cluster information:
Kubernetes version: 1.18.12-gke.1210
Cloud being used: Google Cloud with GKE Autopilot
YAML
Config for the deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: foundation-deployment
spec:
strategy:
# Added after reading this GitHub issue: https://github.com/kubernetes/ingress-gce/issues/34#issuecomment-398831429
type: RollingUpdate
rollingUpdate:
maxUnavailable: 0
selector:
matchLabels:
app: foundation-web
template:
metadata:
labels:
app: foundation-web
spec:
serviceAccountName: # Connects to our database account
containers:
# Run Cloud SQL proxy so we can safely connect to Postgres on localhost.
- name: cloud-sql-proxy
image: gcr.io/cloudsql-docker/gce-proxy:1.17
resources:
requests:
cpu: "250m"
memory: 100Mi
limits:
cpu: "500m"
memory: 100Mi
command:
# Connects to Cloud SQL Proxy
securityContext:
runAsNonRoot: true
- name: foundation-web
image: # Pulls latest version of image from GCR
imagePullPolicy: Always
env:
# Env-specific config
resources:
requests:
memory: "500Mi"
cpu: "2"
limits:
memory: "1000Mi"
cpu: "2"
livenessProbe:
httpGet:
path: /healthz
port: 4000
initialDelaySeconds: 5
periodSeconds: 5
readinessProbe:
httpGet:
path: /healthz
port: 4000
initialDelaySeconds: 5
periodSeconds: 5
ports:
- containerPort: 4000
Config for the service:
apiVersion: v1
kind: Service
metadata:
name: foundation-web-service
annotations:
cloud.google.com/backend-config: '{"ports": {"4000": "foundation-ingress-config"}}'
cloud.google.com/neg: '{"ingress": true}'
spec:
type: NodePort
selector:
app: foundation-web
ports:
- port: 4000
targetPort: 4000
Config for the BackendConfig:
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: foundation-ingress-config
spec:
timeoutSec: 40
connectionDraining:
drainingTimeoutSec: 60
Config for the Ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: foundation-web-ingress
labels:
name: foundation-web-ingress
spec:
backend:
serviceName: foundation-web-service
servicePort: 4000
You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.
