Streaming Kubernetes events to Kafka

Hello everyone,

Do you know of any native or non-native mechanism for Kubernetes that enables streaming resource events (pod/service/configmap added/updated/deleted) to Kafka or a webhook?

My major concern with Kubernetes watches is the limited scalability of event processing. I need to consume events e.g., for all pods deployed in the cluster, and with watches, as far as I experienced, one can only have a single thread receiving the events - the one that initiated the watch. It seems to be a bottleneck for large workloads.

At best, I expect events to be sent to an event broker and then consumed from the queue by a group of processing workers.

Looking for something similar to Webhook backend in Kubernetes Audit feature (Auditing - Kubernetes) but for resource events not resource requests/responses.

Thanks

1 Like