Which would be the best approach to implement a log and metrics management solution in a Kubernetes environment?
My doubt would be in the log collecting tool before outputting data to Elasticsearch and then visualise it in Kibana.
In terms of log collectors I saw many articles favouring Fluentd and/or Fluentbit instead of Logstash. However even with Fluentd or Fluentbit I saw three different approaches, such as:
-
using Fluent Bit daemonset on every node to collect container and application logs from the node file system and transfer the logs to Fluentd. Fluentd receives, filters, and then transfer logs to Elasticsearch
-
using Fluent Bit daemonset on every node to collect container and application logs from the node file system and transfer the logs directly to Elasticsearch
-
using Fluentd deployed as cluster level and capturing and parsing stdout and stderr logs to send them directly to Elasticsearch
Thanks!