We’re curious to hear opinions on running independent workloads on a single cluster vs opting for separate cluster per application (or logical group of applications).
What sort of thing have you seen, or are you opting for? What factors did you consider when making that decision? Any opinions - strong or otherwise?
This a topic that has been discussed a lot, I think. There are several threads in the kubernetes users Google group that are worth reading, IMO.
In my experience, it makes sense to have different clusters when you have some hard isolations requirements (there are some limitations today, that are being addressed step by step) or when you want to upgrade kubernetes and not impact all on the cluster or things like that.
For example, we use different clusters for staging/production. One reason is that: don’t impact production on kubernetes upgrades. We have more reasons (they run on different accounts, etc.).
If you would have “weird” requirements for isolation, you might be better with different clusters.
I’m also interested in this topic. You mentioned there were many conversations about it already - do you think you could link to one or two to help us out? It’s not exactly an easy thing to google for