[RFC] RL-Insight Trajectory Collection via Gateway Instrumentation

[tardis-key]

Feature proposal

https://github.com/verl-project/rl-insight

Instrument GatewaySession.run_generation to collect per-turn input/output records (prompt messages + decoded assistant response + metadata), and expose a get_summary() interface so rl-insight can pull structured session data for custom visualization and offline debugging.

Motivation and use case

The Gateway already sees the full generation context — run_generation receives prompt messages and returns a decoded assistant_msg. But this data is discarded during trajectory materialization: only token IDs, masks, and logprobs survive into Trajectory. This makes debugging token-blind (can't inspect what the model actually said without offline decode) and blocks external observability tools like rl-insight from building response-aware dashboards.

Related area

uni_agent/gateway/gateway.py (_handle_chat_completions → session.run_generation), uni_agent/gateway/session/session.py (GatewaySession, TrajectoryBuffer), uni_agent/framework/framework.py (session lifecycle).

Known gap

1. TrajectoryBuffer / Trajectory carry only token IDs — no raw response text. The summary interface holds raw text in parallel; adding response_text to Trajectory itself is a longer-term follow-up.
2. Consider providing a summary/info aggregation interface, serving both uni-agent's own debugging needs and external tools like rl-insight that can directly fetch data for visualization.

@zackcxb @wuxibin89 @mengchengTang

[MarcelShi96]

Hi, I’d like to work on this RFC.

I checked the current main branch. The core gap described here still exists.

The Gateway has evolved since this issue was opened: it now supports separate
OpenAI/Anthropic paths, concurrent multi-chain generation, and last-assistant
rollback. These behaviors affect the summary schema and ordering semantics.

For an initial PR, I propose:

1. Keep Trajectory and the existing training/TQ interfaces unchanged.
2. Add a versioned, provider-neutral SessionSummary / GenerationRecord
   representation for per-generation inputs, decoded outputs, and agreed metadata.
3. Expose get_summary() through GatewaySession and the required Actor/Manager
   boundary.
4. Preserve the existing finalize_session() -> list[Trajectory] API. If summaries
   must remain available during finalization, add a backward-compatible atomic
   finalize-with-summary interface.
5. Keep the internal structured summary separate from any sanitized/truncated
   telemetry representation sent to RL-Insight.

Before implementing, could maintainers please confirm:

1. Is the pull-based get_summary() interface in this issue still the target, or
   should it align with the push-based wrapper proposed in
   [RFC] Uni-Agent Trajectory Observability in RL-Insight rl-insight#131, or support both?
2. Should each record contain the pre-parser decoded text, the post-parser
   structured assistant_msg, or both?
3. Should rolled-back generations remain in the summary as audit history, or
   should only the surviving branch be exposed?
4. Does get_summary() only need to support active sessions, or must the summary
   also be returned atomically at finalization?
5. Should collection be always enabled, configurable, or enabled only when an
   observability consumer is configured? What redaction and size limits are
   expected?
6. Should inputs use Uni-Agent’s canonical normalized messages, or preserve
   provider-specific OpenAI/Anthropic wire payloads?
7. Which metadata fields are required in the first version—for example chain and
   generation IDs, timing, finish reason, token counts, sampling parameters,
   rollback state, tool names, and Framework run/sample/session identity?

Once these points are confirmed, I can share a concrete v1 JSON schema before
starting the implementation.

[tardis-key]

@MarcelShi96 Welcome! Let's figure out what you can work on together

Regrading what to collect -- we currently have the collection interface define and sample in verl-project/rl-insight#132. For now, just prepare the trajectory data according to the requirements there. You can take a look at the RFCs on rl-insight, where our discussions are documented. As for the data preparation work on the uni-agent side, you may need further clarification from uni-agent maintainers.

[yyDing1]

Hi @MarcelShi96, thanks for your interest. Here are my thoughts:

First, we may collect runtime information in GatewaySession, where we can directly observe normalized input messages, decoded assistant responses, chain selection, rollback events, token usage, and generation timing.

Then, expose this information through get_summary() and retrieve the summary when the session is finalized.

Enrich the summary in the Framework. The Framework layer should add information unavailable to the Session, such as global step, runner identity (task config, train / val), and final reward.

Finally, push the sanitized summary through the RL-Insight wrapper.

cc @zackcxb