As We know that, the design, philosophy, architecture etc of the Kubernetes is inspired or influenced by Borg and Omega at Google.
Over the course of this time and ongoing journey, there might have been some events where components in kubernetes were suggested, added, altered or even removed in order to progress or build the kubernetes system as it is now.
As I understand, these things are overlooked while talking about the kubernetes eco-system. I believe, to comprehensively understand a fascinating complex system, It is essential to go through the journey with kubernetes by understanding these events.
I guess my first question would be what features/design choices do you have in mind? Maybe we can get a good conversation going by focusing on something specific?
Controller Manager and Scheduler are the components that coordinate through the API Server. These are bundled into separate servers called the Controller Manager and the Scheduler. The choice to break these out was so they couldn’t cheat. If the core parts of the system had to talk to the API Server like every other component it would help ensure that we were building an extensible system from the start. The fact that there are just two of these is an accident of history. They could conceivably be combined into one big binary or broken out into a dozen+ separate servers. Core Kubernetes: Jazz Improv over Orchestration
Correct me If I understand the message from above paragraph correctly, we decided to do a trade-off by separating scheduler from Controller Manager to achieve extensibility.
What does cheat mean in this context? How could it have been a bad decision to keep them as a single binary? and Are there other components which could be separated from Controller Manager?
The idea was to make sure that components in the system used APIs that were treated just like end user APIs. This helps developers be more thoughtful about architecture and the interaction between components. It also makes the system fundamentally more extensible.
This really somewhat predates/mirrors the thinking around microservices. You have a bunch of smaller services that communicate over well factored, versioned, and well specified interfaces.
The issue with putting these things in the same binary is that it is too easy to use “private” APIs that aren’t exposed over the network. That is easier than doing a network API and it simplifies versioning but it leads to tight coupling and a monolith.
So while it is possible to keep things loosely coupled through public APIs and be in the same binary, the temptation to do a quick and dirty fix or implementation would be huge. By running these things as a separate binary we make the path of least resistance for developers on the project to do the “right thing”.