Use central monitoring for multiple k8s cluster or each have their own

Is it better to monitor multiple k8s clusters by kube Prometheus stack separately or in single kube Prometheus stack that deployed in one cluster also monitors the others? As there is complexity in aggregating all monitoring stacks in a single Prometheus and If you centralize monitoring by deploying a single Kube Prometheus Stack in one cluster while scraping other clusters, you will lose out-of-the-box dashboards, alerts, and recording rules for those external clusters, I think it would be better to each cluster have its own monitoring stack. please help me with your advice

1 Like

There are 3 clusters in my company 2 of them has production roles and the other one contains services like artifactory, GitLab, and jira Providing these to other clusters, for us is necessary to monitor both production clusters and their services which send their metrics to Prometheus pushgateway, for now, kube Prometheus stack has been deployed in one of the production clusters and monitors services inside itself and k8s components but if I want to centralize monitoring stack there will be too much complexity.