Wiping all clean, let’s start from the the beginning.
I’m looking for the cheapest, non-cloud (no AWS,GCP, etc) setup, to have a 3 cp 3 w node cluster.
I don’t understand volumes.
When I order 3 × 4 vcpu 6GB RAM 100GB nvm instances for the CP, they don’t need persistent volumes.
But for workers I do need PV and PVC.
I don’t understand, what is the least stressful, least maintenance required way to use the 3 worker nodes’ NVM storage.
The cluster will never be expanded, only possibly migrated eventually.
I don’t have the money to have a dedicated external NAS.
Then, load balancing, I dont understand that either. MetalLB and nginx ingress confuse me.
Can I simply use DNS-LB?
Which IPs would I add there, all 6 or only the worker nodes?
Each worker has 32TB outgoing, I dont want to order a separate instance just for LB, it would lower the total amount of outgoing available. I want to make use of what little resources I have available.
Yes, I read the documentation, but I’d like a human to explain it in a hands on way, please, because I still don’t exactly know what I need to order.
I’d use microk8s and the cluster would be a live experiment with real world workload.
I’ve set up countless local k8s installations, but they’re all local only, no letsencrypt, no public IPs.
I don’t want to order 6 instances only to find out I’m missing external storage and a static load balancer or something else too perhaps?
Budget is very tight, I’ll be effectively saving on food to afford that experiment.