Skip to content

Bug fix: Prevent exemplar drop when filtered attributes exceed 128-rune limit#7883

Open
NesterovYehor wants to merge 8 commits intoopen-telemetry:mainfrom
NesterovYehor:fix/prometheus-exporter-exemplar-limit
Open

Bug fix: Prevent exemplar drop when filtered attributes exceed 128-rune limit#7883
NesterovYehor wants to merge 8 commits intoopen-telemetry:mainfrom
NesterovYehor:fix/prometheus-exporter-exemplar-limit

Conversation

@NesterovYehor
Copy link
Contributor

@NesterovYehor NesterovYehor commented Feb 8, 2026

I’ve added a test case for #6718 to show what happens when filtered attributes are too long.

Currently, the test fails because the Prometheus client_golang has a 128-rune limit for exemplars. When we move filtered attributes into the exemplar and they exceed this limit, the whole thing gets dropped, including the trace_id and span_id. It even causes a parsing error in the output:

--- FAIL: TestExemplars (0.00s)
    --- FAIL: TestExemplars/exemplar_overflow_does_not_drop_exemplar (0.00s)
        exporter_test.go:1405:
            Error Trace:    /Users/yehornesterov/dev/Go/opentelemetry-go/exporters/prometheus/exporter_test.go:1405
            Error:          Expected value not to be nil.
            Test:           TestExemplars/exemplar_overflow_does_not_drop_exemplar
FAIL
FAIL    go.opentelemetry.io/otel/exporters/prometheus    0.373s

To fix this, I can add a check that ensures we always fit the trace_id and span_id first. Then, we only add the user's filtered attributes if there is enough space left in the 128-rune budget. If an attribute doesn't fit, we just skip it so we don't lose the Trace ID.

@dashpole
Copy link
Contributor

Thanks for adding the test. You will need to add the fix as well for us to merge the PR.

@NesterovYehor
Copy link
Contributor Author

okey, I started by adding the test to clearly capture the failure case and expected behavior. I’ll add the fix next so we can get this merged.

@dmathieu dmathieu marked this pull request as draft February 11, 2026 08:12
@codecov
Copy link

codecov bot commented Feb 12, 2026

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.7%. Comparing base (24567b3) to head (ab3eec2).

Files with missing lines Patch % Lines
exporters/prometheus/exporter.go 89.4% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #7883     +/-   ##
=======================================
- Coverage   81.7%   81.7%   -0.1%     
=======================================
  Files        304     304             
  Lines      23283   23296     +13     
=======================================
+ Hits       19034   19043      +9     
- Misses      3862    3865      +3     
- Partials     387     388      +1     
Files with missing lines Coverage Δ
exporters/prometheus/exporter.go 90.6% <89.4%> (-0.2%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@NesterovYehor NesterovYehor changed the title test: add reproduction for Prometheus exemplar 128-rune limit breach Bug fix: Prevent exemplar drop when filtered attributes exceed 128-rune limit Feb 12, 2026
@NesterovYehor NesterovYehor marked this pull request as ready for review February 12, 2026 12:34
@NesterovYehor NesterovYehor marked this pull request as draft February 12, 2026 14:13
@NesterovYehor NesterovYehor force-pushed the fix/prometheus-exporter-exemplar-limit branch from 92005e2 to cfec6ba Compare February 12, 2026 14:31
@NesterovYehor NesterovYehor marked this pull request as ready for review February 12, 2026 14:43
@dashpole
Copy link
Contributor

My only concern with this is that we (Prometheus) are planning to relax the exemplar limit in OM 2.0: prometheus/docs#2835. I'm not sure if we would be able to remove this code without causing problems for people still using OM 1.0.

@bwplotka do you think we should contribute this fix to the prometheus client instead so that it can have different exemplar behavior for OM 1.0 vs 2.0?

@NesterovYehor NesterovYehor force-pushed the fix/prometheus-exporter-exemplar-limit branch from ab3eec2 to b2e1eef Compare February 14, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants