Skip to content

perf(debugger): sample probes in breakpoint conditions#8967

Merged
watson merged 9 commits into
masterfrom
watson/DEBUG-5778/refactor-sampling
Jun 26, 2026
Merged

perf(debugger): sample probes in breakpoint conditions#8967
watson merged 9 commits into
masterfrom
watson/DEBUG-5778/refactor-sampling

Conversation

@watson

@watson watson commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Moves Node.js debugger probe sampling into generated breakpoint conditions so unsampled probe hits can avoid pausing the debuggee process. The debugger bootstrap installs a runtime sampler in the debuggee process and shares a buffer with the devtools worker. Breakpoint conditions make the sampling decision inline and record sampled probe indexes into that buffer; the worker then only processes the probes the condition path sampled.

Motivation

Probe sampling was previously decided after V8 had already paused the application and control had moved to the devtools worker. That meant probes dropped by per-probe sampling or the global snapshot sampling limit could still add pause overhead. Moving the decision into the breakpoint condition makes the common unsampled case cheaper.

Additional Notes

Added focused tests for probe_sampler, breakpoint condition rebuilding, sampler state cleanup on probe removal, pause handling with sampled probe indexes, overflow handling, and stale/inconsistent sampled probe state.

Known limitations

Breakpoint conditions read the sampler from the realm-local globalThis, so a probe whose code runs in a separate V8 realm (e.g. a vm.runInContext script) can't reach it and silently won't fire, rather than crash.

In practice the only affected user is one who runs application code in a vm context with an absolute-path filename and places a probe on a line inside it:

const vm = require('node:vm')

const context = vm.createContext({})
vm.runInContext(
  'function handler () { /* a probe on this line no longer fires */ }',
  context,
  { filename: '/app/handler.js' } // absolute path, otherwise the probe never matched at all
)

This only narrows an already-degraded, undocumented path for Dynamic Instrumentation: even before this change such probes produced correlation-less snapshots, because global.require doesn't exist in the vm realm. Test Optimization's failing-test replay is unaffected, as it runs in a separate worker that sets unconditional breakpoints.

watson commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator Author

@github-actions github-actions Bot added semver-patch debugger Dynamic Instrumentation & Live Debugger labels Jun 17, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 6.36 MB
Deduped: 7.43 MB
No deduping: 7.43 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@datadog-official

datadog-official Bot commented Jun 17, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d869c02 | Docs | Datadog PR Page | Give us feedback!

@codecov

codecov Bot commented Jun 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 22.22222% with 42 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.54%. Comparing base (2f4b07f) to head (d869c02).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
packages/dd-trace/src/debugger/probe_sampler.js 9.09% 40 Missing ⚠️
packages/dd-trace/src/debugger/index.js 33.33% 2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (22.22%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #8967       +/-   ##
===========================================
- Coverage   92.92%   68.54%   -24.39%     
===========================================
  Files         886      757      -129     
  Lines       50777    43983     -6794     
  Branches    10002     8844     -1158     
===========================================
- Hits        47183    30146    -17037     
- Misses       3594    13837    +10243     
Flag Coverage Δ
aiguard-integration-active 41.69% <22.22%> (-0.06%) ⬇️
aiguard-integration-latest 41.57% <22.22%> (-0.19%) ⬇️
aiguard-integration-maintenance ?
aiguard-macos 34.55% <22.22%> (-0.12%) ⬇️
aiguard-ubuntu 34.79% <22.22%> (-0.02%) ⬇️
aiguard-windows ?
apm-capabilities-tracing-macos ?
apm-capabilities-tracing-ubuntu-active ?
apm-capabilities-tracing-ubuntu-latest ?
apm-capabilities-tracing-ubuntu-maintenance ?
apm-capabilities-tracing-ubuntu-oldest ?
apm-capabilities-tracing-windows ?
apm-integrations-aerospike-18-gte.5.2.0 33.07% <22.22%> (-0.12%) ⬇️
apm-integrations-aerospike-20-gte.5.5.0 ?
apm-integrations-aerospike-22-gte.5.12.1 33.10% <22.22%> (-0.12%) ⬇️
apm-integrations-aerospike-22-gte.6.0.0 33.10% <22.22%> (-0.12%) ⬇️
apm-integrations-aerospike-eol- 42.67% <ø> (ø)
apm-integrations-child-process 34.00% <22.22%> (-0.12%) ⬇️
apm-integrations-confluentinc-kafka-javascript-18 ?
apm-integrations-confluentinc-kafka-javascript-20 ?
apm-integrations-confluentinc-kafka-javascript-22 ?
apm-integrations-confluentinc-kafka-javascript-24 39.89% <22.22%> (-0.14%) ⬇️
apm-integrations-couchbase-eol ?
apm-integrations-couchbase-oldest 33.09% <22.22%> (-0.28%) ⬇️
apm-integrations-dns 32.94% <22.22%> (-0.12%) ⬇️
apm-integrations-elasticsearch 34.13% <22.22%> (-0.11%) ⬇️
apm-integrations-http-latest ?
apm-integrations-http-maintenance ?
apm-integrations-http-oldest 41.92% <22.22%> (-0.13%) ⬇️
apm-integrations-http2 ?
apm-integrations-kafkajs-latest ?
apm-integrations-kafkajs-oldest ?
apm-integrations-net 33.62% <22.22%> (-0.12%) ⬇️
apm-integrations-next-11.1.4 36.54% <ø> (ø)
apm-integrations-next-12.3.7 36.54% <ø> (ø)
apm-integrations-next-13.0.0 29.10% <22.22%> (-0.10%) ⬇️
apm-integrations-next-13.2.0 ?
apm-integrations-next-13.5.11 ?
apm-integrations-next-14.0.0 ?
apm-integrations-next-14.2.35 ?
apm-integrations-next-14.2.7 ?
apm-integrations-next-15.0.0 ?
apm-integrations-next-15.4.0 ?
apm-integrations-next-latest ?
apm-integrations-oracledb ?
apm-integrations-prisma-18-gte.6.16.0.and.lt.7.0.0 ?
apm-integrations-prisma-latest-all ?
apm-integrations-restify 35.60% <22.22%> (-0.12%) ⬇️
apm-integrations-sharedb 32.54% <22.22%> (-0.11%) ⬇️
apm-integrations-tedious ?
appsec-fastify ?
appsec-graphql ?
appsec-integration-active 36.09% <22.22%> (-0.03%) ⬇️
appsec-integration-latest 36.09% <22.22%> (-0.03%) ⬇️
appsec-integration-maintenance 36.15% <22.22%> (-0.03%) ⬇️
appsec-integration-oldest 36.15% <22.22%> (-0.03%) ⬇️
appsec-kafka 40.23% <22.22%> (-0.12%) ⬇️
appsec-ldapjs 39.76% <22.22%> (-0.12%) ⬇️
appsec-lodash 39.68% <22.22%> (-0.12%) ⬇️
appsec-mongodb-core 43.84% <22.22%> (-0.12%) ⬇️
appsec-mongoose ?
appsec-mysql ?
appsec-next-latest-11.1.4 27.41% <22.22%> (-0.08%) ⬇️
appsec-next-latest-12.3.7 27.54% <ø> (ø)
appsec-next-latest-13.0.0 29.18% <22.22%> (?)
appsec-next-latest-13.2.0 29.21% <22.22%> (-0.10%) ⬇️
appsec-next-latest-13.5.11 ?
appsec-next-latest-14.0.0 29.23% <22.22%> (-0.10%) ⬇️
appsec-next-latest-14.2.35 ?
appsec-next-latest-14.2.6 ?
appsec-next-latest-14.2.7 29.23% <22.22%> (-0.10%) ⬇️
appsec-next-latest-15.0.0 ?
appsec-next-latest-latest 29.27% <22.22%> (-0.10%) ⬇️
appsec-next-oldest-11.1.4 27.45% <22.22%> (-0.09%) ⬇️
appsec-next-oldest-12.3.7 29.24% <22.22%> (-0.10%) ⬇️
appsec-next-oldest-13.0.0 29.24% <22.22%> (-0.10%) ⬇️
appsec-next-oldest-13.2.0 ?
appsec-next-oldest-13.5.11 ?
appsec-next-oldest-14.0.0 ?
appsec-next-oldest-14.2.35 ?
appsec-next-oldest-14.2.6 ?
appsec-next-oldest-14.2.7 ?
appsec-next-oldest-15.0.0 ?
appsec-next-oldest-latest 27.74% <ø> (ø)
appsec-node-serialize 38.97% <22.22%> (-0.12%) ⬇️
appsec-passport 42.59% <22.22%> (-0.13%) ⬇️
appsec-postgres ?
appsec-sourcing 38.45% <22.22%> (-0.11%) ⬇️
appsec-stripe 40.49% <22.22%> (-0.12%) ⬇️
appsec-template 39.26% <22.22%> (-0.12%) ⬇️
appsec-ubuntu ?
appsec-windows ?
debugger-ubuntu-active ?
debugger-ubuntu-latest ?
debugger-ubuntu-maintenance ?
debugger-ubuntu-oldest ?
instrumentations-instrumentation-ai 45.76% <ø> (ø)
instrumentations-instrumentation-aws-sdk 45.33% <ø> (ø)
instrumentations-instrumentation-bluebird 27.63% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-body-parser 35.63% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-child_process 33.31% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-connect 48.74% <ø> (ø)
instrumentations-instrumentation-cookie-parser 29.53% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-couchbase-18 46.47% <ø> (ø)
instrumentations-instrumentation-couchbase-eol ?
instrumentations-instrumentation-crypto 27.59% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-express 29.73% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-express-mongo-sanitize 29.63% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-express-multi-version 42.09% <ø> (ø)
instrumentations-instrumentation-express-session 35.45% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-fastify 48.67% <ø> (ø)
instrumentations-instrumentation-fetch 45.50% <ø> (ø)
instrumentations-instrumentation-fs 27.31% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-generic-pool 27.52% <ø> (ø)
instrumentations-instrumentation-hono 28.83% <22.22%> (-0.09%) ⬇️
instrumentations-instrumentation-http 37.88% <22.22%> (-0.13%) ⬇️
instrumentations-instrumentation-http-client-options 37.49% <22.22%> (-0.13%) ⬇️
instrumentations-instrumentation-kafkajs 49.40% <ø> (ø)
instrumentations-instrumentation-knex ?
instrumentations-instrumentation-koa 46.34% <ø> (ø)
instrumentations-instrumentation-light-my-request 35.27% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-mongoose 28.72% <22.22%> (-0.09%) ⬇️
instrumentations-instrumentation-multer 35.30% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-mysql2 33.42% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-openai-lifecycle 46.44% <ø> (ø)
instrumentations-instrumentation-otel-sdk-trace 25.51% <22.22%> (-0.02%) ⬇️
instrumentations-instrumentation-passport 39.23% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-passport-http 38.93% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-passport-local 39.37% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-pg 33.15% <22.22%> (-0.12%) ⬇️
instrumentations-instrumentation-promise 27.57% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-promise-js 27.57% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-q 27.60% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-restify 47.99% <ø> (ø)
instrumentations-instrumentation-router 43.90% <ø> (ø)
instrumentations-instrumentation-stripe 28.12% <22.22%> (-0.09%) ⬇️
instrumentations-instrumentation-url 27.43% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-when 27.59% <22.22%> (-0.10%) ⬇️
instrumentations-instrumentation-zlib 27.48% <22.22%> (-0.10%) ⬇️
instrumentations-integration-esbuild-0.16.12-active ?
instrumentations-integration-esbuild-0.16.12-latest ?
instrumentations-integration-esbuild-0.16.12-oldest ?
instrumentations-integration-esbuild-latest-active ?
instrumentations-integration-esbuild-latest-latest ?
instrumentations-integration-esbuild-latest-maintenance ?
llmobs-ai ?
llmobs-anthropic ?
llmobs-bedrock 36.03% <22.22%> (-0.11%) ⬇️
llmobs-google-genai 35.93% <22.22%> (-0.11%) ⬇️
llmobs-langchain ?
llmobs-openai-latest ?
llmobs-openai-oldest ?
llmobs-sdk-active 43.30% <22.22%> (-0.15%) ⬇️
llmobs-sdk-latest 43.30% <22.22%> (-0.15%) ⬇️
llmobs-sdk-maintenance 43.40% <22.22%> (-0.15%) ⬇️
llmobs-sdk-oldest ?
llmobs-vertex-ai 35.60% <22.22%> (-0.12%) ⬇️
master-coverage 68.54% <22.22%> (?)
openfeature-macos 37.46% <22.22%> (-0.06%) ⬇️
openfeature-ubuntu ?
openfeature-unit-active 50.13% <ø> (ø)
openfeature-unit-latest 50.13% <ø> (ø)
openfeature-unit-maintenance 50.49% <ø> (ø)
openfeature-unit-oldest 50.49% <ø> (ø)
openfeature-windows ?
platform-core 46.76% <ø> (ø)
platform-esbuild 18.70% <22.22%> (+0.01%) ⬆️
platform-instrumentations-misc 29.57% <22.22%> (-0.03%) ⬇️
platform-integration-active ?
platform-integration-latest ?
platform-integration-maintenance ?
platform-shimmer 47.74% <ø> (ø)
platform-unit-guardrails 45.60% <ø> (ø)
platform-webpack 18.35% <22.22%> (+0.01%) ⬆️
plugins-aws-durable-execution-sdk-js ?
plugins-axios ?
plugins-azure-cosmos ?
plugins-azure-event-hubs ?
plugins-azure-service-bus ?
plugins-bullmq ?
plugins-cassandra 33.57% <22.22%> (-0.27%) ⬇️
plugins-cookie 40.97% <ø> (ø)
plugins-cookie-parser 40.80% <ø> (ø)
plugins-crypto 42.99% <ø> (ø)
plugins-dd-trace-api 33.16% <22.22%> (-0.12%) ⬇️
plugins-express-mongo-sanitize 40.85% <ø> (ø)
plugins-express-session 40.71% <ø> (ø)
plugins-fastify ?
plugins-fetch 34.50% <22.22%> (-0.11%) ⬇️
plugins-fs 33.58% <22.22%> (-0.12%) ⬇️
plugins-generic-pool 40.28% <ø> (ø)
plugins-google-cloud-pubsub ?
plugins-grpc 36.44% <22.22%> (-0.12%) ⬇️
plugins-handlebars 40.85% <ø> (ø)
plugins-hapi 35.73% <22.22%> (-0.12%) ⬇️
plugins-hono 35.84% <22.22%> (-0.27%) ⬇️
plugins-jest ?
plugins-knex ?
plugins-langgraph 32.29% <22.22%> (-0.11%) ⬇️
plugins-ldapjs 39.27% <ø> (ø)
plugins-light-my-request 40.41% <ø> (ø)
plugins-limitd-client ?
plugins-lodash 40.46% <ø> (ø)
plugins-memcached ?
plugins-microgateway-core 34.68% <22.22%> (-0.12%) ⬇️
plugins-modelcontextprotocol-sdk 32.25% <22.22%> (-0.11%) ⬇️
plugins-moleculer ?
plugins-mongodb ?
plugins-mongodb-core ?
plugins-mongoose ?
plugins-multer 40.77% <ø> (ø)
plugins-mysql 34.53% <22.22%> (-0.12%) ⬇️
plugins-mysql2 ?
plugins-nats 36.22% <22.22%> (-0.13%) ⬇️
plugins-node-serialize 40.70% <ø> (ø)
plugins-opensearch 33.66% <22.22%> (-0.17%) ⬇️
plugins-passport-http 40.58% <ø> (ø)
plugins-pino ?
plugins-postgres ?
plugins-process 42.99% <ø> (ø)
plugins-pug 40.97% <ø> (ø)
plugins-redis 34.24% <22.22%> (-0.13%) ⬇️
plugins-router 37.96% <22.22%> (-0.34%) ⬇️
plugins-sequelize 40.23% <ø> (ø)
plugins-test-and-upstream-amqp10 ?
plugins-test-and-upstream-amqplib ?
plugins-test-and-upstream-apollo 34.76% <22.22%> (-0.11%) ⬇️
plugins-test-and-upstream-avsc 33.64% <22.22%> (-0.12%) ⬇️
plugins-test-and-upstream-bunyan 29.05% <22.22%> (-0.10%) ⬇️
plugins-test-and-upstream-connect 36.23% <22.22%> (-0.12%) ⬇️
plugins-test-and-upstream-koa 35.79% <22.22%> (-0.12%) ⬇️
plugins-test-and-upstream-protobufjs 33.87% <22.22%> (-0.12%) ⬇️
plugins-test-and-upstream-rhea 39.19% <22.22%> (+0.07%) ⬆️
plugins-undici 34.93% <22.22%> (-0.11%) ⬇️
plugins-url ?
plugins-valkey 33.71% <22.22%> (+0.03%) ⬆️
plugins-vm 42.99% <ø> (ø)
plugins-winston 29.71% <22.22%> (-0.10%) ⬇️
plugins-ws 36.98% <22.22%> (-0.12%) ⬇️
profiling-macos 43.01% <22.22%> (-0.13%) ⬇️
profiling-ubuntu ?
serverless-aws-sdk-latest-bedrockruntime ?
serverless-aws-sdk-latest-client ?
serverless-aws-sdk-latest-eventbridge ?
serverless-aws-sdk-latest-kinesis ?
serverless-aws-sdk-latest-lambda ?
serverless-aws-sdk-latest-s3 ?
serverless-aws-sdk-latest-sns ?
serverless-aws-sdk-latest-sqs ?
serverless-aws-sdk-latest-util ?
serverless-aws-sdk-oldest-aws-sdk ?
serverless-aws-sdk-oldest-bedrockruntime ?
serverless-aws-sdk-oldest-client ?
serverless-aws-sdk-oldest-dynamodb ?
serverless-aws-sdk-oldest-eventbridge ?
serverless-aws-sdk-oldest-kinesis ?
serverless-aws-sdk-oldest-s3 ?
serverless-aws-sdk-oldest-serverless-peer-service ?
serverless-aws-sdk-oldest-sns ?
serverless-aws-sdk-oldest-sqs ?
serverless-aws-sdk-oldest-stepfunctions ?
serverless-aws-sdk-oldest-util ?
serverless-azure-functions-eventhubs ?
serverless-lambda 34.29% <22.22%> (-0.14%) ⬇️
test-optimization-cucumber-latest-latest ?
test-optimization-cucumber-oldest-7.0.0 ?
test-optimization-cypress-latest-12.0.0-commonJS ?
test-optimization-cypress-latest-12.0.0-esm ?
test-optimization-cypress-latest-14.5.4-commonJS ?
test-optimization-cypress-latest-14.5.4-esm ?
test-optimization-cypress-latest-latest-commonJS ?
test-optimization-cypress-latest-latest-esm ?
test-optimization-cypress-oldest-12.0.0-commonJS ?
test-optimization-cypress-oldest-12.0.0-esm ?
test-optimization-cypress-oldest-14.5.4-commonJS ?
test-optimization-cypress-oldest-14.5.4-esm ?
test-optimization-jest-latest-latest ?
test-optimization-jest-latest-oldest ?
test-optimization-jest-oldest-latest ?
test-optimization-jest-oldest-oldest ?
test-optimization-mocha-latest-latest ?
test-optimization-mocha-latest-oldest ?
test-optimization-mocha-oldest-latest ?
test-optimization-playwright-latest-latest-playwright-active-test-span ?
test-optimization-playwright-latest-latest-playwright-atr ?
test-optimization-playwright-latest-latest-playwright-efd ?
test-optimization-playwright-latest-latest-playwright-final-status ?
test-optimization-playwright-latest-latest-playwright-impacted-tests ?
test-optimization-playwright-latest-latest-playwright-reporting ?
test-optimization-playwright-latest-latest-playwright-test-management ?
test-optimization-playwright-latest-oldest-playwright-active-test-span ?
test-optimization-playwright-latest-oldest-playwright-atr ?
test-optimization-playwright-latest-oldest-playwright-efd ?
test-optimization-playwright-latest-oldest-playwright-final-status ?
test-optimization-playwright-latest-oldest-playwright-impacted-tests ?
test-optimization-playwright-latest-oldest-playwright-reporting ?
test-optimization-playwright-latest-oldest-playwright-test-management ?
test-optimization-playwright-oldest-latest-playwright-active-test-span ?
test-optimization-playwright-oldest-latest-playwright-atr ?
test-optimization-playwright-oldest-latest-playwright-efd ?
test-optimization-playwright-oldest-latest-playwright-final-status ?
test-optimization-playwright-oldest-latest-playwright-impacted-tests ?
test-optimization-playwright-oldest-latest-playwright-reporting ?
test-optimization-playwright-oldest-latest-playwright-test-management ?
test-optimization-playwright-oldest-oldest-playwright-active-test-span ?
test-optimization-playwright-oldest-oldest-playwright-atr ?
test-optimization-playwright-oldest-oldest-playwright-efd ?
test-optimization-playwright-oldest-oldest-playwright-final-status ?
test-optimization-playwright-oldest-oldest-playwright-impacted-tests ?
test-optimization-playwright-oldest-oldest-playwright-reporting ?
test-optimization-playwright-oldest-oldest-playwright-test-management ?
test-optimization-selenium-latest 45.47% <22.22%> (+<0.01%) ⬆️
test-optimization-selenium-oldest ?
test-optimization-testopt-active ?
test-optimization-testopt-latest ?
test-optimization-testopt-maintenance ?
test-optimization-testopt-oldest ?
test-optimization-vitest-latest ?
test-optimization-vitest-oldest 42.33% <22.22%> (-5.57%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@watson
watson changed the base branch from master to graphite-base/8967 June 17, 2026 19:53
@watson
watson force-pushed the graphite-base/8967 branch from 4776de7 to eb0fd43 Compare June 17, 2026 19:53
@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch from 7f047cd to bc04cb3 Compare June 17, 2026 19:53
@watson
watson changed the base branch from graphite-base/8967 to watson/remove-dead-code June 17, 2026 19:53
@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch 3 times, most recently from 9aa4315 to 63aa918 Compare June 18, 2026 05:30
@pr-commenter

pr-commenter Bot commented Jun 18, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-06-26 16:56:26

Comparing candidate commit d869c02 in PR branch watson/DEBUG-5778/refactor-sampling with baseline commit 2f4b07f in branch master.

📊 Benchmarking dashboard

Found 1 performance improvements and 0 performance regressions! Performance is the same for 2222 metrics, 63 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:startup-with-tracer-everything-24

  • 🟩 max_rss_usage [-7.141MB; -3.692MB] or [-3.870%; -2.001%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

  • unstable execution_time [-182.176ms; +167.299ms] or [-6.944%; +6.377%]

scenario:appsec-appsec-enabled-26

  • unstable execution_time [-215.495ms; +204.944ms] or [-8.519%; +8.101%]

scenario:appsec-appsec-enabled-with-attacks-26

  • unstable execution_time [-160.159ms; +153.625ms] or [-5.555%; +5.329%]

scenario:appsec-control-20

  • unstable execution_time [-153.754ms; +169.375ms] or [-8.725%; +9.612%]

scenario:appsec-control-24

  • unstable execution_time [-106.690ms; +98.835ms] or [-8.811%; +8.162%]

scenario:appsec-control-26

  • unstable execution_time [-120.476ms; +104.079ms] or [-9.922%; +8.571%]

scenario:debugger-line-probe-with-snapshot-default-20

  • unstable cpu_user_time [-83.838s; -17.579s] or [-130.460%; -27.355%]
  • unstable execution_time [-107.961s; -22.635s] or [-135.642%; -28.438%]
  • unstable instructions [-291.5G instructions; -61.1G instructions] or [-98.467%; -20.646%]
  • unstable max_rss_usage [-72.203MB; -14.998MB] or [-34.387%; -7.143%]
  • unstable throughput [+349.982op/s; +1671.723op/s] or [+28.413%; +135.718%]

scenario:debugger-line-probe-with-snapshot-default-24

  • unstable cpu_user_time [-77.785s; -16.195s] or [-140.222%; -29.195%]
  • unstable execution_time [-103.682s; -21.648s] or [-144.266%; -30.122%]
  • unstable instructions [-209.9G instructions; -41.0G instructions] or [-107.959%; -21.067%]
  • unstable max_rss_usage [-184.744MB; -38.029MB] or [-68.108%; -14.020%]
  • unstable throughput [+385.386op/s; +3028.938op/s] or [+19.813%; +155.717%]

scenario:debugger-line-probe-with-snapshot-default-26

  • unstable cpu_user_time [-65.634s; -13.761s] or [-135.372%; -28.383%]
  • unstable execution_time [-86.091s; -18.050s] or [-139.866%; -29.324%]
  • unstable instructions [-202.9G instructions; -42.5G instructions] or [-103.956%; -21.775%]
  • unstable max_rss_usage [-187.602MB; -39.184MB] or [-69.477%; -14.511%]
  • unstable throughput [+536.294op/s; +2565.537op/s] or [+29.263%; +139.989%]

scenario:debugger-line-probe-with-snapshot-minimal-20

  • unstable cpu_user_time [-26.849s; -5.626s] or [-90.691%; -19.005%]
  • unstable execution_time [-35.790s; -7.496s] or [-100.147%; -20.976%]
  • unstable instructions [-28.8G instructions; -5.8G instructions] or [-21.243%; -4.256%]
  • unstable max_rss_usage [-50.363MB; -10.317MB] or [-25.740%; -5.273%]
  • unstable throughput [+296.103op/s; +1420.299op/s] or [+20.904%; +100.271%]

scenario:debugger-line-probe-with-snapshot-minimal-24

  • unstable cpu_user_time [-27.776s; -5.824s] or [-115.083%; -24.130%]
  • unstable execution_time [-36.678s; -7.689s] or [-121.437%; -25.457%]
  • unstable instructions [-48.6G instructions; -10.2G instructions] or [-54.655%; -11.459%]
  • unstable max_rss_usage [-126.023MB; -26.349MB] or [-54.292%; -11.351%]
  • unstable throughput [+589.244op/s; +2810.663op/s] or [+25.492%; +121.597%]

scenario:debugger-line-probe-with-snapshot-minimal-26

  • unstable cpu_user_time [-24.676s; -5.169s] or [-104.464%; -21.883%]
  • unstable execution_time [-31.216s; -6.536s] or [-110.461%; -23.130%]
  • unstable instructions [-48.9G instructions; -10.1G instructions] or [-48.209%; -9.932%]
  • unstable max_rss_usage [-130.535MB; -26.463MB] or [-55.264%; -11.203%]
  • unstable throughput [+474.573op/s; +2266.869op/s] or [+23.143%; +110.547%]

scenario:debugger-line-probe-without-snapshot-20

  • unstable cpu_user_time [-22.317s; -4.676s] or [-83.175%; -17.427%]
  • unstable execution_time [-30.817s; -6.454s] or [-94.230%; -19.735%]
  • unstable instructions [-16.0G instructions; -3.0G instructions] or [-12.545%; -2.349%]
  • unstable max_rss_usage [-49.862MB; -10.352MB] or [-25.403%; -5.274%]
  • unstable throughput [+285.623op/s; +1369.357op/s] or [+19.678%; +94.341%]

scenario:debugger-line-probe-without-snapshot-24

  • unstable cpu_user_time [-23.273s; -4.500s] or [-104.196%; -20.146%]
  • unstable execution_time [-31.804s; -6.390s] or [-112.634%; -22.628%]
  • unstable instructions [-45.0G instructions; +4.0G instructions] or [-50.266%; +4.420%]
  • unstable max_rss_usage [-131.479MB; -26.503MB] or [-55.015%; -11.090%]
  • unstable throughput [+287.970op/s; +2711.994op/s] or [+13.230%; +124.591%]

scenario:debugger-line-probe-without-snapshot-26

  • unstable cpu_user_time [-19.488s; -3.046s] or [-85.783%; -13.409%]
  • unstable execution_time [-25.825s; -4.616s] or [-94.252%; -16.846%]
  • unstable instructions [-46.7G instructions; +20.5G instructions] or [-42.495%; +18.666%]
  • unstable max_rss_usage [-137.483MB; -27.248MB] or [-55.580%; -11.015%]
  • unstable throughput [+16.487op/s; +2062.167op/s] or [+0.921%; +115.138%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-403.503ms; +316.821ms] or [-8.152%; +6.400%]
  • unstable execution_time [-403.462ms; +313.308ms] or [-8.030%; +6.235%]
  • unstable throughput [-107814.247op/s; +135411.943op/s] or [-6.447%; +8.097%]

scenario:plugin-couchbase-upsert-20

  • unstable cpu_user_time [-382.970ms; +589.175ms] or [-8.581%; +13.202%]
  • unstable execution_time [-382.435ms; +590.057ms] or [-8.549%; +13.191%]
  • unstable throughput [-2402612.061op/s; +1586708.163op/s] or [-8.505%; +5.617%]

scenario:plugin-graphql-long-with-depth-and-collapse-off-24

  • unstable cpu_user_time [-135.349ms; +346.159ms] or [-2.917%; +7.461%]
  • unstable execution_time [-140.741ms; +370.809ms] or [-2.850%; +7.509%]
  • unstable max_rss_usage [-22.923MB; +41.714MB] or [-4.074%; +7.413%]
  • unstable throughput [-3.232op/s; +1.332op/s] or [-7.509%; +3.094%]

scenario:plugin-graphql-long-with-depth-off-20

  • unstable max_rss_usage [-5.555MB; +14.217MB] or [-3.926%; +10.048%]

scenario:test-optimization-large-suite-20

  • unstable max_rss_usage [-3.703MB; +10.146MB] or [-4.474%; +12.259%]

@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch from 63aa918 to 08a01ab Compare June 18, 2026 12:03
@watson
watson force-pushed the watson/remove-dead-code branch from eb0fd43 to 5793b97 Compare June 18, 2026 12:03
@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch 2 times, most recently from 5b3deae to fa6cdfa Compare June 18, 2026 13:49
@watson
watson force-pushed the watson/remove-dead-code branch from 5793b97 to e2e691d Compare June 18, 2026 13:49
watson added a commit that referenced this pull request Jun 18, 2026
The probe sampling refactor in #8967 moved sampled probe tracking into the
sampler buffer, but left behind the old snapshot probe index bookkeeping.

Remove the dead resizeable ArrayBuffer path and Node.js version gate from the
paused debugger handler.
Base automatically changed from watson/remove-dead-code to master June 18, 2026 15:45
@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch from fa6cdfa to 6894af5 Compare June 18, 2026 15:45
@watson
watson marked this pull request as ready for review June 18, 2026 15:46
@watson
watson requested review from a team as code owners June 18, 2026 15:46

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6894af5eed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/dd-trace/src/debugger/devtools_client/index.js
Comment thread packages/dd-trace/src/debugger/devtools_client/probe_sampler.js Outdated
Comment thread packages/dd-trace/src/debugger/devtools_client/probe_sampler.js Outdated
tlhunter pushed a commit that referenced this pull request Jun 18, 2026
The probe sampling refactor in #8967 moved sampled probe tracking into the
sampler buffer, but left behind the old snapshot probe index bookkeeping.

Remove the dead resizeable ArrayBuffer path and Node.js version gate from the
paused debugger handler.
Comment thread packages/dd-trace/src/debugger/devtools_client/probe_sampler.js Outdated
@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch 2 times, most recently from 1f351f5 to 75ba72e Compare June 19, 2026 11:57
grantseltzer
grantseltzer previously approved these changes Jun 19, 2026
BridgeAR
BridgeAR previously approved these changes Jun 23, 2026

@BridgeAR BridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this is looking really great!

I have a few small nits, nothing blocking. The only question is about cross realm probes.

The cross realm aspect we are loosing is something I am uncertain about. Should we just keep the old approach when cross realm breakpoints are used? We can detect these, if I am not mistaken and could handle them accordingly.

Comment thread packages/dd-trace/src/debugger/devtools_client/breakpoints.js Outdated
Comment thread packages/dd-trace/src/debugger/devtools_client/probe_sampler.js Outdated
Comment thread packages/dd-trace/src/debugger/probe_sampler.js
Comment thread packages/dd-trace/src/debugger/probe_sampler.js Outdated
@watson
watson dismissed stale reviews from BridgeAR and grantseltzer via 3f928f1 June 25, 2026 09:30
@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch from 75ba72e to 3f928f1 Compare June 25, 2026 09:30
@watson
watson requested a review from BridgeAR June 25, 2026 09:34
BridgeAR
BridgeAR previously approved these changes Jun 25, 2026

@BridgeAR BridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, while I still believe it would be nice to keep the old logic as fallback for cross realm probes

tylfin
tylfin previously approved these changes Jun 26, 2026
watson added 9 commits June 26, 2026 18:34
Move probe sampling into generated breakpoint conditions so unsampled hits can avoid pausing
the application. The devtools worker now reads sampled probe indexes from a shared buffer
instead of evaluating conditions and sampling after every pause.

Install a runtime sampler in the debuggee, track probe indexes in worker state, and clear
sampler state when probes are removed. This keeps per-probe sampling, user conditions, and
the global snapshot sampling limit in the condition path.

Add focused coverage for sampler expressions, breakpoint condition rebuilding, pause
handling, overflow handling, and stale sampled probe indexes.
@watson
watson dismissed stale reviews from tylfin and BridgeAR via d869c02 June 26, 2026 16:34
@watson
watson force-pushed the watson/DEBUG-5778/refactor-sampling branch from 3f928f1 to d869c02 Compare June 26, 2026 16:34
@watson
watson enabled auto-merge (squash) June 26, 2026 16:38
@watson
watson merged commit c4a68af into master Jun 26, 2026
804 of 805 checks passed
@watson
watson deleted the watson/DEBUG-5778/refactor-sampling branch June 26, 2026 17:03
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jun 26, 2026
pabloerhard pushed a commit that referenced this pull request Jun 26, 2026
The probe sampling refactor in #8967 moved sampled probe tracking into the
sampler buffer, but left behind the old snapshot probe index bookkeeping.

Remove the dead resizeable ArrayBuffer path and Node.js version gate from the
paused debugger handler.
pabloerhard pushed a commit that referenced this pull request Jun 26, 2026
Move probe sampling into generated breakpoint conditions so unsampled hits can avoid pausing
the application. The devtools worker now reads sampled probe indexes from a shared buffer
instead of evaluating conditions and sampling after every pause.

Install a runtime sampler in the debuggee from the debugger bootstrap, hand the shared buffer
to the worker, and uninstall it on cleanup so the sampler survives debugger stop/start. Track
probe indexes in worker state and clear per-probe sampler state when a probe is removed. This
keeps per-probe sampling, user conditions, and the global snapshot sampling limit in the
condition path.

Rework the sirun debugger benchmark to exercise the new condition path: add a second
breakpoint target, make the startup-guard share and operation counts configurable, drop the
install-only variant, and honor per-variant iteration counts in the overview collector.

Add focused coverage for sampler expressions, breakpoint condition rebuilding, pause handling,
overflow handling, stop/start, and stale sampled probe indexes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debugger Dynamic Instrumentation & Live Debugger semver-patch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants