Hybrid Kubernetes/HPC Cluster

Hi everyone,

Kubernetes version:1.17
Cloud being used: bare-metal
Installation method: yum over hybrid VM/bare-metal
Host OS: Centos 7.6

I have a networking question for our kubernetes deployments. We have an existing SLURM HPC cluster with physical nodes and VM master controllers. We also have a two-node Kubernetes cluster (both VMs). We would like to share the physical nodes between the two modes of Kubernetes and SLURM orchestration as needed. On our HPC cluster, we use a public subnet for routing traffic to fileservers and outside and a private VLAN network switch/connection for PXE boot node provisioning only.

My question is the additional networking requirements for kubernetes cluster on physical nodes, do we need to present a separate private VLAN for the Weave Net pods networking toolkit?
Or are the Weave Net ip ranges reside on any physical server regardless of the networking switch VLAN configurations?
From my tests with VM, it seems like weavenet doesn’t really care about the actual networking VLAN presented to it. rather it creates its own based virtual network layer depending on the availability of the IP space. But I would like to get any confirmation on this from the community.

I hope this description is not too confusing… I would really appreciate any help on this.

Thanks,
Arash

weave-net is an overlay network (vxlan), it should not need a separate vlan configuration. You will want to take into consideration the MTU overhead with your settings though. - More info here: https://www.weave.works/docs/net/latest/tasks/manage/fastdp/#packet-size-mtu

Thanks a lot for your response. Network bandwidth varies between nodes but we can adjust MTU as needed. the vxlan part that you mentioned make things much easier for the networking team so they won’t have to dedicate separate vlan configs.