Skip to content

Conversation

@samikshya-db
Copy link
Collaborator

Summary

This stacked PR builds on #320 and adds statement execution hooks to complete end-to-end telemetry collection.

Stack: Part 3 of 3


Changes

Exported Methods for Driver Integration

telemetry/interceptor.go

  • ✅ Exported BeforeExecute() - starts metric tracking for a statement
  • ✅ Exported AfterExecute() - records metric with timing and error info
  • ✅ Exported AddTag() - adds tags to current metric context
  • ✅ Exported CompleteStatement() - marks statement complete and flushes

Statement Execution Hooks

connection.go

  • ✅ Added hooks to QueryContext():

    • Calls BeforeExecute() with statement ID from operation handle GUID
    • Uses defer to call AfterExecute() and CompleteStatement()
  • ✅ Added hooks to ExecContext():

    • Calls BeforeExecute() with statement ID
    • Proper error handling (includes stagingErr)
    • Uses defer to call AfterExecute() and CompleteStatement()

Documentation

telemetry/DESIGN.md

  • ✅ Updated Phase 6 to mark as completed
  • ✅ Added statement execution hooks to Phase 7 checklist

Integration Flow

Connection.QueryContext()
    ↓
BeforeExecute(statementID) → creates metricContext with startTime
    ↓
[Statement Execution]
    ↓
AfterExecute(err) → records metric with latency and error
    ↓
CompleteStatement(statementID, failed) → flushes aggregated metrics

Testing

All tests passing

  • ✅ 99 telemetry tests (2.018s)
  • ✅ All driver tests (58.576s)
  • ✅ No breaking changes
  • ✅ Telemetry properly disabled when not configured

End-to-End Telemetry

With this PR, the telemetry system is fully functional end-to-end:

  1. Collection - Metrics collected from QueryContext/ExecContext
  2. Aggregation - Statement-level aggregation with batching
  3. Circuit Breaker - Protection against failing endpoints
  4. Export - HTTP POST with retry and exponential backoff
  5. Feature Flags - Server-side control with 5-level priority
  6. Resource Management - Per-host clients with reference counting

Related Issues


Checklist

  • Export interceptor methods for driver use
  • Add hooks to QueryContext
  • Add hooks to ExecContext
  • Update DESIGN.md checklist
  • All tests passing
  • No breaking changes

This commit completes the telemetry implementation by adding hooks to
QueryContext and ExecContext methods to collect actual metrics.

Changes:
- Export BeforeExecute(), AfterExecute(), CompleteStatement() methods
  in telemetry.Interceptor for use by driver package
- Add telemetry hooks to connection.QueryContext():
  - Call BeforeExecute() with statement ID from operation handle GUID
  - Use defer to call AfterExecute() and CompleteStatement()
- Add telemetry hooks to connection.ExecContext():
  - Call BeforeExecute() with statement ID from operation handle GUID
  - Use defer to call AfterExecute() and CompleteStatement()
  - Handle both err and stagingErr for proper error reporting
- Update DESIGN.md:
  - Mark Phase 6 as completed (all checklist items)
  - Add statement execution hooks to Phase 7 checklist

Testing:
- All 99 telemetry tests passing
- All driver tests passing (58.576s)
- No breaking changes to existing functionality

This enables end-to-end telemetry collection from statement execution
through aggregation and export to the Databricks telemetry service.

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants