generated from amazon-archives/__template_Custom
-
Notifications
You must be signed in to change notification settings - Fork 310
Closed
Labels
bugSomething isn't workingSomething isn't working
Milestone
Description
Describe the bug
I tried to upgrade DataPrepper from 2.13 to 2.14 while keeping the same configuration but there was an exception preventing the pipelines from starting.
The issue involves the otel_logs_source i.e.:
2026-03-03T13:37:56,806 [otel-logs-pipeline-sink-worker-10-thread-1] ERROR org.opensearch.dataprepper.core.pipeline.common.PipelineThreadPoolExecutor - Pipeline [otel-logs-pipeline] process worker encountered a fatal exception, cannot proceed further
java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "String.replace(java.lang.CharSequence, java.lang.CharSequence)" because the return value of "org.opensearch.dataprepper.plugins.source.otellogs.OTelLogsSourceConfig.getHttpPath()" is null
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.opensearch.dataprepper.core.pipeline.common.PipelineThreadPoolExecutor.afterExecute(PipelineThreadPoolExecutor.java:75)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.NullPointerException: Cannot invoke "String.replace(java.lang.CharSequence, java.lang.CharSequence)" because the return value of "org.opensearch.dataprepper.plugins.source.otellogs.OTelLogsSourceConfig.getHttpPath()" is null
at org.opensearch.dataprepper.plugins.source.otellogs.OTelLogsSource.configureHttpService(OTelLogsSource.java:180)
at org.opensearch.dataprepper.plugins.source.otellogs.OTelLogsSource.createServer(OTelLogsSource.java:174)
at org.opensearch.dataprepper.plugins.source.otellogs.OTelLogsSource.start(OTelLogsSource.java:118)
at org.opensearch.dataprepper.core.pipeline.Pipeline.startSourceAndProcessors(Pipeline.java:251)
at org.opensearch.dataprepper.core.pipeline.Pipeline.lambda$execute$5(Pipeline.java:302)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
... 2 more
To Reproduce
DataPrepper and Otel Collector are deployed both as containers in kubernetes
The same configuration template works if I use the following image (which matches 2.13.0 DataPrepper version)
image: opensearchproject/data-prepper@sha256:57c53434d040ee9fb4cf7224d75371ca2390cb36d29c9324025f8220a89eca8f
Steps to reproduce the behavior:
- apply the following yaml files
apiVersion: v1
kind: ServiceAccount
metadata:
name: opentelemetrysa
namespace: default
---
apiVersion: v1
kind: ConfigMap
metadata:
name: dataprepper-pipeline-file
namespace: default
data:
pipelines.yaml: |
# Minimal reproduction with file sink for persistent logs
otel-logs-pipeline:
source:
otel_logs_source:
port: 21892
ssl: false
# Bug: Missing http_path causes NullPointerException
sink:
- stdout:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: dataprepper-file-sink
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: dataprepper-file-sink
template:
metadata:
labels:
app: dataprepper-file-sink
spec:
securityContext:
runAsUser: 1000
runAsNonRoot: True
seccompProfile:
type: RuntimeDefault
serviceAccountName: opentelemetrysa
containers:
- name: dataprepper
image: opensearchproject/data-prepper:2.14.0
resources: {}
securityContext:
readOnlyRootFilesystem: false
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
ports:
- containerPort: 21890
- containerPort: 21891
- containerPort: 21892
- containerPort: 4900
volumeMounts:
- name: pipeline-config
mountPath: /usr/share/data-prepper/pipelines
- name: logs-output
mountPath: /tmp/dataprepper-logs
volumes:
- name: pipeline-config
configMap:
name: dataprepper-pipeline-file
- name: logs-output
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: dataprepper-file-sink
namespace: default
spec:
selector:
app: dataprepper-file-sink
ports:
- name: otlp-traces
port: 21890
targetPort: 21890
- name: otlp-metrics
port: 21891
targetPort: 21891
- name: otlp-logs
port: 21892
targetPort: 21892
- name: health
port: 4900
targetPort: 4900
- check the logs
Expected behavior
The pipeline should work as earlier
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working
Type
Projects
Status
Done