How to craft selector that matches the kube-system namespace

General Discussions
1

How do I create a selector to select the kubernetes-dashboard, which is in the ‘kube-system’ namespace? I can’t seem to get that to work in a selector.

Cluster information:

Kubernetes version: 1.41.1
Cloud being used: bare-metal (two ubuntu 18.04 LXC containers ontop of Gentoo Linux).
Installation method: kubeadm
Host OS: Ubuntu 18.04 (as far as k8s is concerned it is Ubuntu, but kernel is from Gentoo host).
CNI and version:
CRI and version:

ostara $ kubectl get pods -l k8s-app=kubernetes-dashboard
No resources found.

ostara $ kubectl get pods -l k8s-app=kubernetes-dashboard -A
NAMESPACE     NAME                                    READY   STATUS    RESTARTS   AGE
kube-system   kubernetes-dashboard-758bc877fb-jnl8l   1/1     Running   1          7h5m

ostara $ kubectl get pods -l k8s-app=kubernetes-dashboard,namespace=kube-system
No resources found.

Field Selectors - Kubernetes suggests “metadata.namespace=kube-system”, but that does not work either:

ostara $ kubectl get pods -l k8s-app=kubernetes-dashboard,metadata.namespace=kube-system
No resources found.

I cannot leverage the “-A” (–all-namespaces) in the selector as far as I can tell.

I adapted the metallb layer2 tutorial (https://metallb.universe.tf/tutorial/layer2/). The following applies with no error, but does not produce the desired result:

apiVersion: v1
kind: Service
metadata:
  name: my-dashboard
  annotations:
    metallb.universe.tf/address-pool: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard
  type: LoadBalancer

.

ostara $ kubectl get services -A
NAMESPACE     NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                  AGE
default       kubernetes             ClusterIP      10.96.0.1        <none>          443/TCP                  7h8m
default       my-dashboard           LoadBalancer   10.96.0.233      192.168.0.192   443:30508/TCP            48m
kube-system   heapster               ClusterIP      10.97.236.218    <none>          80/TCP                   7h2m
kube-system   kube-dns               ClusterIP      10.96.0.10       <none>          53/UDP,53/TCP,9153/TCP   7h8m
kube-system   kubernetes-dashboard   ClusterIP      10.99.96.164     <none>          443/TCP                  7h2m
kube-system   metrics-server         ClusterIP      10.105.32.241    <none>          443/TCP                  7h2m
kube-system   monitoring-influxdb    ClusterIP      10.110.244.182   <none>          8086/TCP                 7h2m

ostara $ sudo tcpdump -i br0 -n -s 0 "host 192.168.0.192"
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
23:13:52.918826 ARP, Request who-has 192.168.0.192 tell 192.168.0.2, length 28
23:13:53.977187 ARP, Request who-has 192.168.0.192 tell 192.168.0.2, length 28
23:13:55.007060 ARP, Request who-has 192.168.0.192 tell 192.168.0.2, length 28
23:13:56.047155 ARP, Request who-has 192.168.0.192 tell 192.168.0.2, length 28
23:13:57.087273 ARP, Request who-has 192.168.0.192 tell 192.168.0.2, length 28
23:13:58.127230 ARP, Request who-has 192.168.0.192 tell 192.168.0.2, length 28

Ok. So the kube-system dashboard is untouched, but I have a new service, and it has an “EXTERNAL-IP” that is external to the flannel network (yeah?). However, metallb is not responding to ARP requests broadcast for 192.168.0.192 (as the tutorial suggests that it should).

I suspect that the “selector” in the above YAML is at fault, but I’m not sure how to construct a selector to match a deployment in the non-default namespace.

ostara $ kubectl get pods,nodes -A -o wide
NAMESPACE        NAME                                        READY   STATUS    RESTARTS   AGE     IP             NODE           NOMINATED NODE   READINESS GATES
kube-system      pod/coredns-fb8b8dccf-7fvf9                 1/1     Running   0          7h37m   10.244.0.2     k8s-master-1   <none>           <none>
kube-system      pod/coredns-fb8b8dccf-xr55z                 1/1     Running   0          7h37m   10.244.0.3     k8s-master-1   <none>           <none>
kube-system      pod/etcd-k8s-master-1                       1/1     Running   0          7h37m   192.168.0.70   k8s-master-1   <none>           <none>
kube-system      pod/heapster-f66f76688-ptcr4                1/1     Running   0          7h31m   10.244.1.2     k8s-worker-1   <none>           <none>
kube-system      pod/kube-apiserver-k8s-master-1             1/1     Running   0          7h36m   192.168.0.70   k8s-master-1   <none>           <none>
kube-system      pod/kube-controller-manager-k8s-master-1    1/1     Running   2          7h36m   192.168.0.70   k8s-master-1   <none>           <none>
kube-system      pod/kube-flannel-ds-amd64-28kmp             1/1     Running   0          7h33m   192.168.0.71   k8s-worker-1   <none>           <none>
kube-system      pod/kube-flannel-ds-amd64-67k4m             1/1     Running   0          7h37m   192.168.0.70   k8s-master-1   <none>           <none>
kube-system      pod/kube-proxy-99mxs                        1/1     Running   0          7h37m   192.168.0.70   k8s-master-1   <none>           <none>
kube-system      pod/kube-proxy-f6xrs                        1/1     Running   0          7h33m   192.168.0.71   k8s-worker-1   <none>           <none>
kube-system      pod/kube-scheduler-k8s-master-1             1/1     Running   2          7h36m   192.168.0.70   k8s-master-1   <none>           <none>
kube-system      pod/kubernetes-dashboard-758bc877fb-jnl8l   1/1     Running   1          7h31m   10.244.0.4     k8s-master-1   <none>           <none>
kube-system      pod/metrics-server-556cc675cf-5mc5g         1/1     Running   0          7h31m   10.244.1.5     k8s-worker-1   <none>           <none>
kube-system      pod/monitoring-influxdb-866db5f944-2cm2t    1/1     Running   0          7h33m   10.244.1.4     k8s-worker-1   <none>           <none>
metallb-system   pod/controller-cd8657667-t24cd              1/1     Running   0          57m     10.244.1.7     k8s-worker-1   <none>           <none>
metallb-system   pod/speaker-g6twp                           1/1     Running   0          57m     192.168.0.71   k8s-worker-1   <none>           <none>

NAMESPACE   NAME                STATUS   ROLES    AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
            node/k8s-master-1   Ready    master   7h38m   v1.14.1   192.168.0.70   <none>        Ubuntu 18.04.2 LTS   4.19.27-gentoo-r1   docker://18.9.2
            node/k8s-worker-1   Ready    <none>   7h33m   v1.14.1   192.168.0.71   <none>        Ubuntu 18.04.2 LTS   4.19.27-gentoo-r1   docker://18.9.2

.

ostara ~ # lxc list
+--------------+---------+------------------------+------------------------------------------+------------+-----------+
|     NAME     |  STATE  |          IPV4          |                   IPV6                   |    TYPE    | SNAPSHOTS |
+--------------+---------+------------------------+------------------------------------------+------------+-----------+
| foo          | STOPPED |                        |                                          | PERSISTENT |           |
+--------------+---------+------------------------+------------------------------------------+------------+-----------+
| k8s-master-1 | RUNNING | 192.168.0.70 (eth0)    | fd8b:6647:f36a::d34 (eth0)               | PERSISTENT |           |
|              |         | 172.17.0.70 (docker0)  | fd8b:6647:f36a:0:8000:ff:fe00:100 (eth0) |            |           |
|              |         | 10.244.0.1 (cni0)      |                                          |            |           |
|              |         | 10.244.0.0 (flannel.1) |                                          |            |           |
+--------------+---------+------------------------+------------------------------------------+------------+-----------+
| k8s-worker-1 | RUNNING | 192.168.0.71 (eth0)    | fd8b:6647:f36a::94a (eth0)               | PERSISTENT |           |
|              |         | 172.17.0.71 (docker0)  | fd8b:6647:f36a:0:8000:ff:fe00:101 (eth0) |            |           |
|              |         | 10.244.1.1 (cni0)      |                                          |            |           |
|              |         | 10.244.1.0 (flannel.1) |                                          |            |           |
+--------------+---------+------------------------+------------------------------------------+------------+-----------+
2

Have you tried using the --namespace or -n selector?

kubectl get pods -l k8s-app=kubernetes-dashboard -n kube-system

3

Yes, and I showed that in my post. However, “-n kube-system” is not a selector. What syntax would I use to express that in YAML under the “/spec/selector” yaml path?

4

I’m pretty sure the -l label itself is restricted to the namespace you are currently in as it is only search the labels. Did you try using the --field-selector metadata.namespace=kube-system

kubectl get pods -l k8s-app=kubernetes-dashboard --field-selector metadata.namespace=kube-system

If the yaml has the namespace in the metadata it should get picked up. If that doesn’t work using the --namespace flag should work, not sure why it didn’t for you.

1 Like