Fluentd /elastic search default conf: changes and storage


Using the microk8s.enable fluentd you get directly to a working setup, but -if I understood the elsticsearch conf- the storage is emptyDir. I’m still beginner but that seems dangerous. After a reboot of the node I lost the initial sertup I made. I guess and think it’s due to that volume. Is that correct? What’s the standard way to deploy it, can I set the elasticsearch data on a glusterfs volume? should it stay in a local storage?

What’s the std what to enable the plugin but force it to use a custom manifest?

Docs on the plugin completely skip there hints…
Any hints/ link for reading is appreciated


Managing elasticsearch for large volume of logs requires more work than what the addon currently do.
The addon is taken directly from upstream kubernetes here (kubernetes/cluster/addons/fluentd-elasticsearch at master · kubernetes/kubernetes · GitHub)

If this is something that needs to be production grade considering the volume of data it needs then you may want to checkout docs from elasticsearch. I have to admin that i have not deployed elasticsearch in kubernetes other than for dev.
Local path storage is a good way to start but make sure you use pod anti affinity so that no 2 elasticsearch will run on the same node. Coz ES is pretty resource intensive.

As an alternative, loki is a good one. Getting started with Grafana and Loki in under 4 minutes | Grafana Labs
This is under the assumption that you will be using it to search for logs and not some analytics stuffs.