Routing externally sourced packets to a container receiving on IP 0.0.0.0

General Discussions
1

I have an embedded system with a single node Kubernetes cluster on Linux.
Within this cluster I have a Pod whose main container is receiving UDP packets on 0.0.0.0:9101. These packets are coming from another node which is using the IP address of the cluster host node as the destination address in these packets, i.e. dest=10.0.100.1:9101 (where 10.0.100.1 is the IP of the node hosting the cluster). [The Pod’s ClusterIP is 10.42.0.14].

Miraculously, these packets successfully make it to the container even though the cluster has no K8s Service exposing port 9101 to the outside world. Indeed, the cluster has no K8s Service using port 9101.

I understand that using ‘0.0.0.0:9101’ means “accept any packet addressed to port 9101 irrespective of the interface it arrived on”. But in the context of a Kubernetes Pod, I would have expected “any interface” to be restricted to just the interfaces within the Pod’s networking namespace (for this Pod ‘eth0’ and ‘lo’). I would NOT expect it to include the physical interfaces of the hosting node. But this seems to be the case. Otherwise I fail to see how packets addressed to cluster host node make it to the container without the intervention of a K8s Service. Is my supposition correct?

I’m not comfortable with the idea that a container is reachable from the outside world without the intervention of a Service. Seems like a potential security hole.

Furthermore, what happens if two Pods on the same node both have containers receiving on 0.0.0.0:9101? Are inbound packets destined for port 9101 (a) duplicated to both pods, (b) sent exclusively to one arbitrarily chosen Pod or (c ) randomly divided between the two pods? Any of the above options lead to undesirable results.

Consider a Deployment having 10 replicas on a 5 node cluster. This means that at least one node must be running more than one Pod in that Deployment, so it’s not a far-fetched scenario. If the nodes have enough computing power to support it, it’s entirely feasible

Cluster information:

Kubernetes version: K3s v1.19.3+k3s2
Cloud being used: bare-metal
Host OS: CentOS 7-9.2009.1
CNI and version: Weave 2.7.0
CRI and version: containerd.

2

I would have expected “any interface” to be restricted to just the interfaces within the Pod’s networking namespace

That’s correct. There’s something unusual going on here - the pod is using the hostNetwork or has a hostPort or something else is routing it.

I’d start with netstat in the root netns and see if the pod is actually listening. Then look to routing tables or iptables or other things that could be happening.