How to get logs with index in opensearch use with opentelemetry api version opentelemetry.io/v1beta1?

General Discussions
1

So i try to get Logs and Index in my opensearch use with opentelemetry sidecar. the opentelemetry api version is opentelemetry.io/v1beta1.

So the index is not create in my opensearch dashboard

this code:

      apiVersion: apps/v1
      kind: StatefulSet
      metadata:
        name: my-app
        annotations:
          sidecar.opentelemetry.io/inject: "true"  # ✅ Ensures sidecar injection
      spec:
        replicas: 2
        selector:
          matchLabels:
            app: my-app
        serviceName: "my-app"
        template:
          metadata:
            labels:
              app: my-app
            annotations:
              sidecar.opentelemetry.io/inject: "true"
              sidecar.opentelemetry.io/config: "otel-sidecar-config"
          spec:
            nodeSelector:
              kubernetes.io/hostname: minikube
            containers:
              - name: nginx
                image: nginx:latest
                ports:
                  - containerPort: 80
                volumeMounts:
                  - name: nginx-logs
                    mountPath: /var/log/nginx

            volumes:
              - name: opentelemetry-config
                configMap:
                  name: otel-sidecar-config

        volumeClaimTemplates:
          - metadata:
              name: nginx-logs
            spec:
              accessModes: ["ReadWriteOnce"]
              resources:
                requests:
                  storage: 1Gi

      ---

      apiVersion: opentelemetry.io/v1beta1
      kind: OpenTelemetryCollector
      metadata:
        name: otel-sidecar-config
      spec:
        mode: sidecar
        config:
          receivers:
            filelog:
              include:
                - /var/log/nginx/test1.log
                - /var/log/nginx/test2.log
              start_at: beginning  
              operators:      
                - type: json_parser
                  timestamp:
                    parse_from: attributes.@t
                    layout: '%Y-%m-%dT%H:%M:%S.%LZ'
                  severity:
                    parse_from: attributes.@l
          processors: {}

          exporters:
            otlp:
              endpoint: https://opensearch-cluster-master.open.svc.cluster.local:9200        
              headers:
                Authorization: "Basic YWRtaW46YWRtaW4="  # admin:admin base64-encoded
              tls:
                insecure_skip_verify: true
              retry_on_failure:
                enabled: true 
              logs_index: newcust

          service:
            pipelines:
              logs:
                receivers: [filelog]
                processors: []
                exporters: [otlphttp]

so i try to see this code and i have two pods my-app-0 , my-app-1 so there are error is :

dev_2@Tadhak-dev-02:~/Backup/Opensearch$ minikube kubectl – logs my-app-1 -n open -c otc-container
Error: failed to get config: cannot unmarshal the configuration: decoding failed due to the following error(s):

error decoding ‘exporters’: error reading configuration for “otlp”: decoding failed due to the following error(s):

‘’ has invalid keys: logs_index
2025/03/26 09:13:13 collector server run finished with error: failed to get config: cannot unmarshal the configuration: decoding failed due to the following error(s):

error decoding ‘exporters’: error reading configuration for “otlp”: decoding failed due to the following error(s):

‘’ has invalid keys: logs_index

there are not support this “logs_index” so how to create my “newcust” index in my opensearch with all logs.

Kubernetes version:
Client Version: v1.31.0
Kustomize Version: v5.4.2
Server Version: v1.31.0

You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.

2

logs_index is a property of the OpenSearch exporter. To index data in OpenSearch, you either need to use this exporter instead of otlp or you keep using the generic otlp exporter but send your data to Data Prepper and let this tool index the data into OpenSearch.