How to make Traefik compatible with Microk8s

General Discussions microk8s
1

I have a working setup on Minikube with Traefik as ingress controller. I tried to use that setup on Microk8s but Traefik is not able to work and although I can see the Traefik dashboard and it says that everything is working but every time I try to use the ingress urls I face timeout but if I use the endpoint IP of that service (which I can see in the traefik dashboard) I am able to access to that service but not fully. I can have access to IP/service1 but I can’t have access to any of its sub urls, IP/service1/sub-service1 not working.

I also tried microk8s.enable ingress but it created an nginx ingress for me and then I disabled it because I want to use traefik.

Do I need to change my configuration so it becomes compatible with Microk8s? If yes how?

I have to mention that I have two ingress files:

  • traefik-ui.yaml : which contains both the service and ingress for my traefik. I use this service+ingress to access the traefik dashboard and as I mentioned it works
  • wws-ingress.yaml : is my main ingress which enables the communication with my components inside kubernetes and this is the part that doesn’t work.

My yaml files:

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik-ui.minikube
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

traefik-ui.yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: wws
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.frontend.rule.type: PathPrefixStrip
    traefik.frontend.passHostHeader: "true"
    traefik.backend.loadbalancer.sticky: "true"
    #traefik.ingress.kubernetes.io/rule-type: ReplacePathRegex
    traefik.wss.protocol: http
    traefik.wss.protocol: https
spec:
  rules:
  - host: streambridge.local
    http:
      paths:
      - path: /streambridge
        backend:
          serviceName: streambridge
          servicePort: 9999
      - path: /dashboard
        backend:
          serviceName: dashboard
          servicePort: 9009
      - path: /gateway
        backend:
          serviceName: gateway
          servicePort: 8080
      - path: /rdb
        backend:
          serviceName: rethinkdb
          servicePort: 8085

Minikube commands (this works without a problem):

kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-ds.yaml

kubectl apply -f traefik-ui.yaml
kubectl apply -f wws-ingress.yaml

And in Microk8s I tried:

microk8s.kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-rbac.yaml
microk8s.kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-ds.yaml
microk8s.kubectl apply -f traefik-ui.yaml
microk8s.kubectl apply -f wws-ingress.yaml
1 Like
2

Hi @AliV

I think the problem is what you put as the host in the ingress rules. You should use something that resolves to the ingress controller. Find out what the ClusterIP of ingress is and use something like host: streambridge."ingress_cluster_ip".xip.io.

Here is an example. Let’s first install MicroK8s and install traefik

sudo snap install microk8s --classic
microk8s.kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-rbac.yaml
microk8s.kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-ds.yaml

Let’s create an nginx service and enable the dashboard. We will be adding ingress rules for these two services:

microk8s.kubectl create deployment nginx --image=nginx -n kube-system
microk8s.kubectl expose deployment nginx --port 80 -n kube-system
microk8s.enable dashboard

After some time my deployment looks likes this:

> microk8s.kubectl get all --all-namespaces     
NAMESPACE     NAME                                                  READY   STATUS    RESTARTS   AGE
kube-system   pod/heapster-v1.5.2-6b5d7b57f9-8hrkh                  4/4     Running   0          2m12s
kube-system   pod/kubernetes-dashboard-6fd7f9c494-jsshc             1/1     Running   0          6m35s
kube-system   pod/monitoring-influxdb-grafana-v4-78777c64c8-r9kmj   2/2     Running   0          6m35s
kube-system   pod/nginx-65f88748fd-2wngk                            1/1     Running   0          6m37s
kube-system   pod/traefik-ingress-controller-vlwn5                  1/1     Running   0          7m22s

NAMESPACE     NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
default       service/kubernetes                ClusterIP   10.152.183.1     <none>        443/TCP             7m42s
kube-system   service/heapster                  ClusterIP   10.152.183.107   <none>        80/TCP              6m35s
kube-system   service/kubernetes-dashboard      ClusterIP   10.152.183.3     <none>        443/TCP             6m35s
kube-system   service/monitoring-grafana        ClusterIP   10.152.183.219   <none>        80/TCP              6m35s
kube-system   service/monitoring-influxdb       ClusterIP   10.152.183.237   <none>        8083/TCP,8086/TCP   6m35s
kube-system   service/nginx                     ClusterIP   10.152.183.10    <none>        80/TCP              6m36s
kube-system   service/traefik-ingress-service   ClusterIP   10.152.183.25    <none>        80/TCP,8080/TCP     7m37s

NAMESPACE     NAME                                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system   daemonset.apps/traefik-ingress-controller   1         1         1       1            1           <none>          7m37s

NAMESPACE     NAME                                             READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/heapster-v1.5.2                  1/1     1            1           6m34s
kube-system   deployment.apps/kubernetes-dashboard             1/1     1            1           6m35s
kube-system   deployment.apps/monitoring-influxdb-grafana-v4   1/1     1            1           6m35s
kube-system   deployment.apps/nginx                            1/1     1            1           6m37s

NAMESPACE     NAME                                                        DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/heapster-v1.5.2-5c5498f57c                  0         0         0       6m34s
kube-system   replicaset.apps/heapster-v1.5.2-6b5d7b57f9                  1         1         1       2m12s
kube-system   replicaset.apps/heapster-v1.5.2-89b48dff                    0         0         0       2m33s
kube-system   replicaset.apps/kubernetes-dashboard-6fd7f9c494             1         1         1       6m35s
kube-system   replicaset.apps/monitoring-influxdb-grafana-v4-78777c64c8   1         1         1       6m35s
kube-system   replicaset.apps/nginx-65f88748fd                            1         1         1       6m37s

Note that the ClusterIP we got for the traefik-ingress-service is 10.152.183.25. That claster IP is random, I will get back to that.

Let’s create ingress rules for nginx and grafana:

> cat ./traefik-microk8s-all-clusterip.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example
  namespace: kube-system
spec:
  rules:
  - host: example.10.152.183.25.xip.io
    http:
      paths:
      - path: /dashboard
        backend:
          serviceName: monitoring-grafana
          servicePort: 80
      - path: /
        backend:
          serviceName: nginx
          servicePort: 80


> microk8s.kubectl apply -f ./traefik-microk8s-all-clusterip.yaml

Now nginx is at http://example.10.152.183.25.xip.io and grafana at http://example.10.152.183.25.xip.io/dashboard

Having a random ClusterIP of the ingress sevice may not be convenient. What you can do for this is to either pin the ClusterIP as we do here https://github.com/ubuntu/microk8s/blob/master/microk8s-resources/actions/dns.yaml#L28 for the dns service, or expose the ingress service with type NodePort so it is available on 127.0.0.1 on a port you select.

To expose the ingress service using NodePort you would need to apply something similar to this:

kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  type: NodePort
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
      nodePort: 32100
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-admin-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 8080
      name: admin

Note that now in your ingress rules you should use something like host: example.127.0.0.1.xip.io and the services of the above exmaple will be at http://example.127.0.0.1.xip.io:32100 and at http://example.127.0.0.1.xip.io:32100/dashboard

1 Like
3

Thank you for your answer but it is not working.
But before I describe what I did I want to talk about the host. In Minikube I gave host: streambridge.local as host and then I added the ip address of Minikube to my /etc/hosts and then everything were working and I was able to use all parts of my application and I strongly believe that it is not the problem because it is working perfectly on Minikube.

I followed your instruction and again I was not able to connect and use my services either with the dynamic cluster IP of traefik-ingress-service or NodePort and I was just able to access the first level of my services if I used the back-end IP of that service from ingress.

Then I removed everything and I followed your code step by step to just see if I can run traefik on Microk8s but even just following your step is not working and I just get Gateway Timeout.

I guess the problem is not the rules and configuration of ingress because if that was the problem it shouldn’t work on Minikube either but it is working there. Maybe the problem is traefik is not compatible with Microk8s? Or it needs to be configured in a specific way because as I said even your answer is not working for me.

4

I wonder if the pod to pod connectivity is fine. Do you get any warnings with microk8s.inspect?

5

I tested your configuration on a different machine and it was working there and I found out that something is wrong with my machine and after spending a good amount of time on this with the help of two of my colleagues and trying everything we found out that the problem is iptable in my machine and we solved it as described here: https://github.com/ubuntu/microk8s/issues/72 .
Thank you for your helps.

6

@AliV I am also want achieve traefik ingress controller with microk8 instead of inbuilt one. I have mac machine and installed microk8 in vagrant based on ubuntu 18.04 (through vagrant setup)

For configuring traefik I used as following

microk8s.kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-rbac.yaml

microk8s.kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-ds.yaml

my Vagrantfile

Vagrant.configure("2") do |config|
   config.vm.box = "bento/ubuntu-18.04"
  config.vm.network "private_network", ip: "192.168.50.4"
end

When i am tried to access 192.168.50.4 in my host machine (mac) with the default ingress controller responding. But the traefik setup is not.

Do i need do anything on iptables level ? or anything will solve this

7

@kjackal hey i am deploying traefik as ingress controller to deploy custom application. i had disabled ingress that comes defualt in microk8s.
my microk8s is running on a machine at cloud. and i want to access traefik dashboard or any application running in pod from my machine(laptop).
how i’ll be able to do this thing?