Hi,
Using the microk8s.enable fluentd you get directly to a working setup, but -if I understood the elsticsearch conf- the storage is emptyDir. I’m still beginner but that seems dangerous. After a reboot of the node I lost the initial sertup I made. I guess and think it’s due to that volume. Is that correct? What’s the standard way to deploy it, can I set the elasticsearch data on a glusterfs volume? should it stay in a local storage?
What’s the std what to enable the plugin but force it to use a custom manifest?
Docs on the plugin completely skip there hints…
Any hints/ link for reading is appreciated
TIA
sandro
*
Managing elasticsearch for large volume of logs requires more work than what the addon currently do.
The addon is taken directly from upstream kubernetes here (kubernetes/cluster/addons/fluentd-elasticsearch at master · kubernetes/kubernetes · GitHub)
If this is something that needs to be production grade considering the volume of data it needs then you may want to checkout docs from elasticsearch. I have to admin that i have not deployed elasticsearch in kubernetes other than for dev.
Local path storage is a good way to start but make sure you use pod anti affinity so that no 2 elasticsearch will run on the same node. Coz ES is pretty resource intensive.
As an alternative, loki is a good one. Getting started with Grafana and Loki in under 4 minutes | Grafana Labs
This is under the assumption that you will be using it to search for logs and not some analytics stuffs.