Phoenix-native Tuning Engines observability example?

[cerebrixos]

The previous PR was closed because it only linked out to Tuning Engines docs and did not show a direct Phoenix integration. That feedback makes sense.

Would a Phoenix-specific cookbook/example be a better fit if it demonstrates the actual observability path instead of a provider listing?

Proposed shape:

• use Phoenix/OpenInference instrumentation for an OpenAI-compatible client
• set the model client base URL to https://api.tuningengines.com/v1
• keep the API key in an environment variable such as TE_INFERENCE_KEY
• attach safe correlation metadata such as run_id, request_id, tenant app/workflow name, and model alias
• show Phoenix traces/spans for the model call while Tuning Engines handles gateway policy, routing, approvals, cost, and token analytics

The goal would be to show Phoenix as the observability/evaluation surface and Tuning Engines as the governed inference endpoint. No secrets, no vendor-specific backend internals, and no generic “please list us” page.

If that direction is acceptable, I can open a replacement PR focused on a Phoenix-native example rather than restoring the closed listing PR.

[rehan243]

oh interesting — the thing people usually miss here is that just forwarding traces/spans isn't enough if you want full observability across gateway events and model calls. We've tried something similar with LangChain and FastAPI endpoints hooked into Phoenix/OpenInference, and you gotta be careful about what metadata you propagate (especially stuff like tenant or workflow info).

For your shape, using the client base URL override to hit https://api.tuningengines.com/v1 makes sense, but the magic happens if you can instrument both outbound requests and responses, so you capture gateway routing decisions (policy, cost, etc) alongside LLM metrics. We've used something like this:

from phoenix.trace import instrument_openai_client

client = instrument_openai_client(
    base_url="https://api.tuningengines.com/v1",
    api_key=os.getenv("TE_INFERENCE_KEY"),
    metadata={
        "run_id": run_id,
        "request_id": req_id,
        "tenant_app": app_name,
        "model_alias": alias,
    }
)

Not gonna lie, correlating spans across multiple services gets messy when you start mixing vendor-managed gateways and your own infra. Question: how are you planning to surface gateway-side analytics (approvals, cost, routing) in Phoenix traces? Are you thinking custom span attributes, or something like a linked trace context? Would love to know how deep you wanna go on the end-to-end path.

[cerebrixos]

Great point. I’d separate this into two modes:

1. If the model call routes through Tuning Engines, then gateway-side analytics are first-class: policy decisions, approvals, usage, cost, latency, routing metadata, MCP/tool/skill/agent events, and runtime state references are recorded against the same run_id / request_id.

2. If the app is using another vendor-managed gateway, Tuning Engines cannot see that gateway’s internal decisions unless the app or gateway emits OTLP/shared metadata. In that case the integration is correlation, not control.

So for Phoenix, I’d use Phoenix/OpenInference for the app-side span tree and propagate stable IDs:

• te.run_id
• te.request_id
• optional te.trace_url
• runtime/model/resource metadata

Tuning Engines can then ingest its own runtime traces through /api/v1/traces, and also supports OTLP ingestion through /api/v1/otel/v1/traces when enabled for the tenant. The trace detail joins model usage, cost, latency, policy decisions, approvals, and state references such as LangGraph checkpoints, Temporal workflow IDs, MCP resources, vector namespaces, or memory references.

So the answer is: custom span attributes are useful for Phoenix display, but shared IDs / linked trace context are the durable join mechanism. Full gateway analytics are available when Tuning Engines is the gateway; otherwise we correlate external spans with the Tuning Engines governance ledger.

[cerebrixos]

@rehan243 Do let us know any additional information needed to approve the merge. Thanks so much in advance!

[cerebrixos]

Hello Team: Just a quick nudge requesting you to review and help on this :)