Skip to content

Support server-side conversation history on responses API#697

Merged
jaredoconnell merged 1 commit intovllm-project:mainfrom
jaredoconnell:feat/responses-server-history
Apr 17, 2026
Merged

Support server-side conversation history on responses API#697
jaredoconnell merged 1 commit intovllm-project:mainfrom
jaredoconnell:feat/responses-server-history

Conversation

@jaredoconnell
Copy link
Copy Markdown
Collaborator

Summary

Add support for server-side conversation history via previous_response_id on the /v1/responses endpoint. This allows multi-turn benchmarks to offload conversation state to the server instead of replaying the full history in every request, which is critical for benchmarking servers that use response storage (e.g. vLLM with VLLM_ENABLE_RESPONSES_API_STORE=1).

Details

  • Add server_history option to OpenAIHTTPBackend, validated to only work with /v1/responses
  • Update ResponsesRequestHandler.format() to set previous_response_id from the prior response instead of replaying history when server_history is enabled.
  • Add unit tests for validation and request formatting with server_history
  • Document server-side history usage and requirements in docs/guides/multiturn.md

Test Plan

  • Run pytest tests/unit/backends/openai/test_http.py -k server_history
  • Run pytest tests/unit/backends/openai/test_request_handlers.py -k server_history
  • (Integration) Against a vLLM server with VLLM_ENABLE_RESPONSES_API_STORE=1, run a multiturn benchmark with --request-format /v1/responses --backend-kwargs '{"server_history": true}' and confirm turn 2+ requests use previous_response_id

  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

@jaredoconnell jaredoconnell requested a review from sjmonson April 16, 2026 16:02
Copy link
Copy Markdown
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Assisted-by: Cursor AI
Signed-off-by: Jared O'Connell <[email protected]>
@jaredoconnell jaredoconnell force-pushed the feat/responses-server-history branch from ac0d1e3 to 7b6164c Compare April 17, 2026 23:19
@jaredoconnell jaredoconnell merged commit 4255e0a into vllm-project:main Apr 17, 2026
11 checks passed
@jaredoconnell jaredoconnell deleted the feat/responses-server-history branch April 17, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants