Cilium Network Policy

General Discussions
, ,
1

Hi all, I have a microk8s one node cluster set up. When enabling Cilium network policies I cannot access the whoami webpage; however, I can access it fine without the network policy or if I uncomment the fromEntities and toEntities “all” parts. Can anyone help me with what I am doing wrong please?

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert -v -f ../Docker Compose/whoami.yml --out ./whoami.yml
    kompose.version: 1.26.1 (a9d05d509)
  creationTimestamp: null
  labels:
    io.kompose.service: whoami-deployment
  name: whoami
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: whoami-deployment
  strategy: {}
  template:
    metadata:
      annotations:
        kompose.cmd: kompose convert -v -f ../Docker Compose/whoami.yml --out ./whoami.yml
        kompose.version: 1.26.1 (a9d05d509)
      creationTimestamp: null
      labels:
        io.kompose.service: whoami-deployment
        io.kompose.network/web-internal: "true"
    spec:
      containers:
        - image: containous/whoami:v1.5.0
          imagePullPolicy: IfNotPresent
          name: whoami
          ports:
            - containerPort: 80
          resources:
            limits:
              memory: "128Mi"
              cpu: "0.2"
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
      restartPolicy: Always
      affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: whoami
                    operator: In
                    values:
                      - "true"
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert -v -f ../Docker Compose/whoami.yml --out ./whoami.yml
    kompose.version: 1.26.1 (a9d05d509)
  creationTimestamp: null
  labels:
    io.kompose.service: whoami-service
    io.kompose.network/web-internal: "true"
  name: whoami-service
spec:
  ports:
    - name: "8007"
      port: 8007
      targetPort: 80
  selector:
    io.kompose.service: whoami-deployment
---
apiVersion: v1
kind: Service
metadata:
  name: whoami-service-np
  labels:
    io.kompose.network/web-internal: "true"
spec:
  type: NodePort
  selector:
    io.kompose.service: whoami-deployment
  ports:
#  # By default and for convenience, the `targetPort` is set to the same value as the `port` field.
    - port: 80
      targetPort: 80
#      # Optional field
#      # By default and for convenience, the Kubernetes control plane will allocate a port from a range (default: 30000-32767)
      nodePort: 30999
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami-ingress
  labels:
    io.kompose.network/web-internal: "true"
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`whoami-kube.here`)
      middlewares:
        - name: whoami-basicauth-middleware
      kind: Rule
      services:
        - name: whoami-service
          port: 8007
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: whoami-basicauth-middleware
  labels:
    io.kompose.network/web-internal: "true"
spec:
  basicAuth:
    secret: whoami-basicauth
---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  creationTimestamp: null
  name: web-internal
specs:
  - endpointSelector:
      matchLabels:
        "io.kompose.network/web-internal": "true"
    egress:
    - toEndpoints:
      - matchLabels:
            "io.kompose.network/web-internal": "true"
            "app.kubernetes.io/name": "traefik"
            "kubernetes.io/metadata.name": "kube-system"
    - toCIDR:
      - "192.168.1.0/24"
#    - toEntities:
#      - all
    ingress:
    - fromEndpoints:
      - matchLabels:
            "io.kompose.network/web-internal": "true"
            "app.kubernetes.io/name": "traefik"
            "kubernetes.io/metadata.name": "kube-system"
#    - fromEntities:
#      - all
    - fromCIDR:
      - "192.168.1.0/24"
2

So, I’ve narrowed down the issue. It appears fromCIDR and toCIDR is not functioning how it should be and allowing traffic from outside the cluster. I know this because if I uncomment the “fromEntities - world” and “toEntities - world” parts, I can access whoami via the nodeport services. Could I possibly get some help please? Why isn’t fromCIDR and toCIDR working?

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  creationTimestamp: null
  name: web-internal
specs:
  - endpointSelector:
      matchLabels:
        "io.kompose.network/web-internal": "true"
    egress:
    - toEndpoints:
      - matchLabels:
            "io.kompose.network/web-internal": "true"
            "app.kubernetes.io/name": "traefik"
            "kubernetes.io/metadata.name": "kube-system"
            k8s:io.kubernetes.pod.namespace: kube-system
            "io.kompose.service": "whoami-deployment"
    - toCIDR:
      - 192.168.1.0/24

#    - toEntities:
#      - host
#      - world
#      - all
    ingress:
    - fromEndpoints:
      - matchLabels:
            "io.kompose.network/web-internal": "true"
            "app.kubernetes.io/name": "traefik"
            "kubernetes.io/metadata.name": "kube-system"
            k8s:io.kubernetes.pod.namespace: kube-system
            "io.kompose.service": "whoami-deployment"
#    - fromEntities:
#      - host
#      - world
#      - all
    - fromCIDR:
      - 192.168.1.0/24
3

Resolved this issue. I found out it was to do with the source IP address not being kept once it enters the cluster and had to add “externalTrafficPolicy: Local” to the nodeport server spec. Its a bit similar to putting a port in host mode on docker swarm.