Skip to content

vLLM backend passes deprecated raw prompts to InputProcessor #8924

Description

@DimensionSTP

Description

The Triton vLLM backend passes raw string or dictionary prompts directly to AsyncLLM.generate. With the vLLM version included in Triton 26.07, this emits the following warning for ordinary generation requests:

Passing raw prompts to InputProcessor is deprecated and will be removed in v0.18. You should instead pass the outputs of Renderer.render_cmpl() or Renderer.render_chat().

The backend should render completion prompts before calling AsyncLLM.generate, so it follows the current vLLM API and avoids relying on the deprecated raw-prompt path.

Reproduction

  1. Run nvcr.io/nvidia/tritonserver:26.07-vllm-python-py3 with a vLLM model.
  2. Submit a normal generation request through the Triton vLLM backend.
  3. Inspect the Triton server log.
  4. The raw-prompt deprecation warning is emitted once per request.

Expected behavior

Use the engine renderer to convert the existing text or multimodal completion prompt into an EngineInput, then pass that rendered input to AsyncLLM.generate without changing the Triton request or response contract.

Validation

  • Environment: Triton 26.07, vLLM 0.24.0.
  • Baseline and rendered-input paths returned the same deterministic text output.
  • Baseline emitted the warning; the rendered-input path emitted no warning.
  • Qwen3-0.6B targeted vLLM backend L0 tests passed: 7 tests, 0 errors, 0 failures.
  • Qwen3.5-9B text and multimodal generation smoke tests completed successfully with the rendered-input path.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions