I’m creating a Platform where users (no technical knowledge) can quickly spin up my software in a pre-configured way using my api / web panel.
I’ve run into the problem of how I manage scale and how to restrict my users properly.
Initially I tried using a database as my source-of-truth. Basically I write changes to the configuration of my software (triggered by end users) to a db and then run a reconciler.
This is problematic because I also need to have fast updates. So now I need to duplicate my logic everywhere to attempt a fast-update and if it fails, fallback to the reconciler
This kinda sucks so I considered making k8s my source of truth. This approach immediately has 2 downsides. When the k8s api is down I have downtime, too and customer requests are hitting kubernetes 1 to 1. I’m worried that this will scale very badly.
Also apart from these issues I would have to implement 4-6 crud endpoints for every single resource my users can create/configure. This is very tedious and repetitive code.
Is there some pre-made solution for this that I can’t find? How am I supposed to solve this?
Should I do a 1 cluster per customer architecture? I don’t really like this approach because then I lose most monitoring and it’s a giant hit on performance.