Skip to content

feat(observability): traces end-to-end (gateway + UI panel)#5381

Draft
Ma77Ball wants to merge 91 commits into
apache:mainfrom
Ma77Ball:obs/pr7/traces
Draft

feat(observability): traces end-to-end (gateway + UI panel)#5381
Ma77Ball wants to merge 91 commits into
apache:mainfrom
Ma77Ball:obs/pr7/traces

Conversation

@Ma77Ball

@Ma77Ball Ma77Ball commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Distributed-trace retrieval through the gateway plus a trace-tree UI panel.
Backend:

  • Trace query builder and parseTraces.
  • TracesResource exposing trace retrieval by id, with id validation and scope enforcement at the gateway.
    Frontend:
  • Traces panel rendering spans as a tree built from parent-span relationships, with pivot-from-log so a log row can open its trace directly. Span fields are bound with text interpolation only.

Any related issues, documentation, or discussions?

Closes: #5373
Part of #4070. Stacked on #5380.

How was this PR tested?

  • Backend specs for the trace query builder and parser; sbt scalafmtCheckAll passes.
  • Frontend traces-panel specs, including the span-tree builder; prettier-eslint and eslint pass.
  • Compile and the full test suites run in this PR's CI.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Opus 4.8 in compliance with ASF

Ma77Ball and others added 7 commits June 5, 2026 04:49
…, SDK bootstrap (default-off)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s panel

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nt scope, health, routing)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ca/eBPF profiling

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tracing primitives

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… trace tree panel

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…UI metrics panel (ECharts)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added engine dependencies Pull requests that update a dependency file frontend Changes related to the frontend GUI docs Changes related to documentations dev common labels Jun 5, 2026
@codecov-commenter

codecov-commenter commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 63.37262% with 808 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.23%. Comparing base (7e9cabf) to head (f2f9e93).
⚠️ Report is 24 commits behind head on main.

Files with missing lines Patch % Lines
...observability/gateway/ObservabilityResources.scala 0.00% 224 Missing ⚠️
...nt/user/list-item/card-item/card-item.component.ts 47.23% 76 Missing and 10 partials ⚠️
...observability/logs-panel/logs-panel.component.html 56.92% 55 Missing and 1 partial ⚠️
...ala/org/apache/texera/observability/OtelInit.scala 63.30% 43 Missing and 8 partials ⚠️
...apache/texera/web/observability/gateway/dtos.scala 75.39% 40 Missing and 7 partials ⚠️
...texera/web/observability/gateway/AuditLogger.scala 0.00% 41 Missing ⚠️
.../user/list-item/card-item/card-item.component.html 56.52% 38 Missing and 2 partials ⚠️
...xera/web/observability/gateway/ScopeResolver.scala 19.44% 27 Missing and 2 partials ⚠️
...era/web/observability/gateway/GatewayContext.scala 0.00% 19 Missing ⚠️
...rvability/traces-panel/traces-panel.component.html 44.11% 19 Missing ⚠️
... and 29 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5381      +/-   ##
============================================
+ Coverage     52.87%   53.23%   +0.35%     
- Complexity     2509     2665     +156     
============================================
  Files          1073     1120      +47     
  Lines         41382    43894    +2512     
  Branches       4444     4864     +420     
============================================
+ Hits          21882    23368    +1486     
- Misses        18214    19133     +919     
- Partials       1286     1393     +107     
Flag Coverage Δ *Carryforward flag
access-control-service 70.91% <100.00%> (+6.30%) ⬆️
agent-service 34.36% <ø> (ø) Carriedforward from fed8677
amber 53.14% <60.10%> (-0.47%) ⬇️
computing-unit-managing-service 1.65% <ø> (ø)
config-service 56.71% <ø> (ø)
file-service 57.06% <ø> (ø)
frontend 48.76% <68.14%> (+1.45%) ⬆️
pyamber 90.73% <100.00%> (+0.01%) ⬆️ Carriedforward from fed8677
python 90.73% <ø> (-0.03%) ⬇️ Carriedforward from fed8677
workflow-compiling-service 58.69% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 4 better · 🔴 2 worse · ⚪ 9 noise (<±5%) · 0 without baseline

Compared against main b8859b4 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🟢 bs=10 sw=10 sl=64 395 0.241 25,389/33,397/33,397 us 🟢 -39.7% / 🔴 +6.7%
bs=100 sw=10 sl=64 813 0.496 121,606/150,711/150,711 us ⚪ within ±5% / 🔴 -8.9%
🔴 bs=1000 sw=10 sl=64 921 0.562 1,081,066/1,180,232/1,180,232 us 🔴 +6.1% / 🔴 +15.4%
Baseline details

Latest main b8859b4 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 395 tuples/sec 365 tuples/sec 410.82 tuples/sec +8.2% -3.9%
bs=10 sw=10 sl=64 MB/s 0.241 MB/s 0.223 MB/s 0.251 MB/s +8.1% -3.9%
bs=10 sw=10 sl=64 p50 25,389 us 25,990 us 23,785 us -2.3% +6.7%
bs=10 sw=10 sl=64 p95 33,397 us 55,369 us 34,980 us -39.7% -4.5%
bs=10 sw=10 sl=64 p99 33,397 us 55,369 us 34,980 us -39.7% -4.5%
bs=100 sw=10 sl=64 throughput 813 tuples/sec 824 tuples/sec 891.94 tuples/sec -1.3% -8.9%
bs=100 sw=10 sl=64 MB/s 0.496 MB/s 0.503 MB/s 0.544 MB/s -1.4% -8.9%
bs=100 sw=10 sl=64 p50 121,606 us 121,100 us 112,277 us +0.4% +8.3%
bs=100 sw=10 sl=64 p95 150,711 us 151,756 us 139,802 us -0.7% +7.8%
bs=100 sw=10 sl=64 p99 150,711 us 151,756 us 139,802 us -0.7% +7.8%
bs=1000 sw=10 sl=64 throughput 921 tuples/sec 932 tuples/sec 1,041 tuples/sec -1.2% -11.5%
bs=1000 sw=10 sl=64 MB/s 0.562 MB/s 0.569 MB/s 0.635 MB/s -1.2% -11.6%
bs=1000 sw=10 sl=64 p50 1,081,066 us 1,071,888 us 972,714 us +0.9% +11.1%
bs=1000 sw=10 sl=64 p95 1,180,232 us 1,111,871 us 1,023,057 us +6.1% +15.4%
bs=1000 sw=10 sl=64 p99 1,180,232 us 1,111,871 us 1,023,057 us +6.1% +15.4%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,505.74,200,128000,395,0.241,25388.97,33397.08,33397.08
1,100,10,64,20,2459.70,2000,1280000,813,0.496,121606.05,150710.68,150710.68
2,1000,10,64,20,21710.01,20000,12800000,921,0.562,1081065.92,1180232.09,1180232.09

Ma77Ball added 19 commits June 14, 2026 03:42
  ResponseParsersSpec (fresh-build compile failure)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common dependencies Pull requests that update a dependency file dev docs Changes related to documentations engine frontend Changes related to the frontend GUI platform Non-amber Scala service paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Observability] Distributed traces: gateway endpoint and dashboard panel

2 participants