I am working on setting up a kubernetes cluster for my business to host several internal applications. Our company has a set of servers on premises and some servers on the cloud (AWS). We are hoping it might be possible to assemble a cluster with both local, bare-metal nodes, and nodes on the cloud. I haven’t seen any information on setting up a cluster in this way and I’m not sure if it’s possible, but if anyone has information regarding this or can point me in the right direction, I would be grateful.
This is very much an anti-pattern. Kubernetes is really meant to have nodes be on the same network and “near” each other. People have done stretched networks like that before, but it takes serious forethought and planning.
I’ve had an interest in something along these lines. The use case I had in mind was a cluster that normally uses your on-prem hardware, but in case a spike in workloads gets thrown at it (say, serving a website that spikes in traffic, or a CI system and a bunch of compile/test jobs arrive), it can decide to scale up temporarily by requesting instances from a cloud provider, and assign some of the workload to those new instances, then later scale back down by removing those cloud instances when the spike subsides.
I’ve heard a term for this: “cloudbursting”. And, I’ve seen a talk or two that mentioned virtual kubelet as a potential tool for doing something like this.