Aws-load-balancer-controller annotations not working

General Discussions
,
1

Cluster information:

Kubernetes version: 1.19
Cloud being used: AWS EKS
Installation method: Terraform

I’m trying to automatically start an ALB in my EKS cluster by using the aws-load-balancer-controller

This is what the logs of my deployment look like:

$ kubectl logs -n kube-system deployment.apps/aws-load-balancer-controller
Found 2 pods, using pod/aws-load-balancer-controller-6d9894cf4-57rlq
{"level":"info","ts":1621805437.8682935,"msg":"version","GitVersion":"v2.2.0","GitCommit":"68c417a7ea37ff153f053d9ffef1cc5c70d7e211","BuildDate":"2021-05-14T21:49:05+0000"}
{"level":"info","ts":1621805441.0391128,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":1621805441.042128,"logger":"setup","msg":"adding health check for controller"}
{"level":"info","ts":1621805441.042287,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/mutate-v1-pod"}
{"level":"info","ts":1621805441.0423834,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/mutate-elbv2-k8s-aws-v1beta1-targetgroupbinding"}
{"level":"info","ts":1621805441.04245,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-elbv2-k8s-aws-v1beta1-targetgroupbinding"}
{"level":"info","ts":1621805441.0425208,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-networking-v1beta1-ingress"}
{"level":"info","ts":1621805441.0435557,"logger":"setup","msg":"starting podInfo repo"}
{"level":"info","ts":1621805443.0431697,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
I0523 21:30:43.043172       1 leaderelection.go:242] attempting to acquire leader lease  kube-system/aws-load-balancer-controller-leader...
I0523 21:30:43.061542       1 leaderelection.go:252] successfully acquired lease kube-system/aws-load-balancer-controller-leader
{"level":"info","ts":1621805443.1433892,"logger":"controller-runtime.webhook.webhooks","msg":"starting webhook server"}
{"level":"info","ts":1621805443.1435783,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.1437511,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.143842,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.1441207,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"channel source: 0xc000090500"}
{"level":"info","ts":1621805443.1442277,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"channel source: 0xc000090550"}
{"level":"info","ts":1621805443.1447785,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.1448064,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.1448874,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.1447225,"logger":"controller","msg":"Starting EventSource","controller":"service","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.145254,"logger":"controller","msg":"Starting Controller","controller":"service"}
{"level":"info","ts":1621805443.1451356,"logger":"controller-runtime.certwatcher","msg":"Updated current TLS certificate"}
{"level":"info","ts":1621805443.1455188,"logger":"controller-runtime.webhook","msg":"serving webhook server","host":"","port":9443}
{"level":"info","ts":1621805443.145757,"logger":"controller-runtime.certwatcher","msg":"Starting certificate watcher"}
{"level":"info","ts":1621805443.2442284,"logger":"controller","msg":"Starting EventSource","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.2452686,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"channel source: 0xc0000905a0"}
{"level":"info","ts":1621805443.2455094,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.2455108,"logger":"controller","msg":"Starting workers","controller":"service","worker count":3}
{"level":"info","ts":1621805443.3447967,"logger":"controller","msg":"Starting Controller","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding"}
{"level":"info","ts":1621805443.3450336,"logger":"controller","msg":"Starting workers","reconcilerGroup":"elbv2.k8s.aws","reconcilerKind":"TargetGroupBinding","controller":"targetGroupBinding","worker count":3}
{"level":"info","ts":1621805443.345854,"logger":"controller","msg":"Starting EventSource","controller":"ingress","source":"kind source: /, Kind="}
{"level":"info","ts":1621805443.345981,"logger":"controller","msg":"Starting Controller","controller":"ingress"}
{"level":"info","ts":1621805443.346044,"logger":"controller","msg":"Starting workers","controller":"ingress","worker count":3}

And the annotations in my kustomize ingress config (I’m patching the Kubeflow 1.3 config):

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: istio-ingress
  namespace: istio-system
  annotations:
    # create AWS Application LoadBalancer
    kubernetes.io/ingress.class: alb
    # external type
    alb.ingress.kubernetes.io/scheme: internet-facing
    # AWS Certificate Manager certificate's ARN
    alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:us-east-2:1111111:certificate/111-222-333"
    # open ports 80 and 443 
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    # ExternalDNS settings: https://rtfm.co.ua/en/kubernetes-update-aws-route53-dns-from-an-ingress/
    external-dns.alpha.kubernetes.io/hostname: kubeflow.my-domain.com
    # redirect all HTTP to HTTPS
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'

But so far nothing happens, no alb is being created.

$ kubectl get services -n istio-system
NAME                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                      AGE
authservice             ClusterIP   172.20.17.19     <none>        8080/TCP                                                                     39m
cluster-local-gateway   ClusterIP   172.20.0.189     <none>        15020/TCP,80/TCP                                                             39m
istio-ingressgateway    NodePort    172.20.81.219    <none>        15021:31033/TCP,80:31875/TCP,443:31517/TCP,31400:32737/TCP,15443:32018/TCP   39m
istiod                  ClusterIP   172.20.186.197   <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP                                        39m

(The NodePort is supposed to be replaced by an ALB here I believe).
I’m out of idea on how to debug this. Any help would be greatly appreciated, Thank you.

2

I am experiencing the same thing. Did you find a solution? Thanks.

3

No, I ended up using this repo with everything configured.