We are in the process of setting up an on-premise kubernetes cluster to run kafka which will be sourcing data from a few databases. We plan to run kafka with 4 brokers. We then have kafka consumers that will process and transform these messages. Some messages need to be processed in less than a min, some less than 5 mins and some every hour.
This system is expected to grow over time.
I know I may not be sharing much in terms of details… does this use case fit the traditional k8s deployment or would k3s be a better fit ?