How to deploy logstash with persistent volume on kubernetes?

General Discussions
1

Using GKE to deploy logstash by statefulset kind with pvc. Also need to install an output plugin.

When don’t use while true; do sleep 1000; done; in container’s command args, it can’t deploy with pvc successfully.

The pod will cause CrashLoopBackOff error.

  Normal   Created                 13s (x2 over 14s)  kubelet                  Created container logstash
  Normal   Started                 13s (x2 over 13s)  kubelet                  Started container logstash
  Warning  BackOff                 11s (x2 over 12s)  kubelet                  Back-off restarting failed container

From here I found it can try to add sleep. So the statefulset with pvc can deploy successfully.

But when check its logs will find:

/bin/sh: bin/logstash-plugin: No such file or directory
/bin/sh: bin/logstash: No such file or directory

How to do it in a good way to start container to install an logstash output plugin?

The whole manifest file:

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: logstash-to-gcs
  namespace: logging
spec:
  serviceName: "logstash"
  selector:
    matchLabels:
      app: logstash
  updateStrategy:
    type: RollingUpdate
  replicas: 3
  template:
    metadata:
      labels:
        app: logstash
    spec:
      containers:
      - name: logstash
        image: docker.elastic.co/logstash/logstash:7.10.0
        resources:
          limits:
            memory: 2Gi
        ports:
          - containerPort: 5044
        volumeMounts:
          - name: config-volume
            mountPath: /usr/share/logstash/config
          - name: logstash-pipeline-volume
            mountPath: /usr/share/logstash/pipeline
          - name: logstash-data
            mountPath: /usr/share/logstash
        command: ["/bin/sh","-c"]
        args:
          - bin/logstash-plugin install logstash-output-google_cloud_storage;
            bin/logstash -f /usr/share/logstash/pipeline/logstash.conf;
            while true; do sleep 1000; done;
      volumes:
        - name: config-volume
          configMap:
            name: logstash-configmap
            items:
              - key: logstash.yml
                path: logstash.yml
        - name: logstash-pipeline-volume
          configMap:
            name: logstash-configmap
            items:
              - key: logstash.conf
                path: logstash.conf
  volumeClaimTemplates:
  - metadata:
      name: logstash-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
2

it looks like the error is saying it can’t find the binaries. can you confirm the path in the container to the logstash and logstash-plugin files. Maybe you need to put /bin/logstash instead of bin/logstash.

3

I added an initContainers to run initialization first, it works.