[Bug]: OpenAI chat completions response missing finish_reason

[lirc571]

Prerequisites

• I have searched existing issues and discussions to avoid duplicates
• I am using the latest version (or have tested against main/nightly)

Description

When calling a model with OpenAI chat completions API, the response is missing {finish_reason: stop} even if it is set in upstream, causing client agents to enter an infinite loop.

Steps to reproduce

Call a model with chat completions API.

Expected behavior

finish_reason should be forwarded

Actual behavior

finish_reason is not forwarded

Affected area(s)

Transports (HTTP)

Version

v1.4.9

Environment

Linux

Relevant logs/output

Regression?

No response

Severity

Medium (some functionality impaired)

[Pratham-Mishra04]

Hey @lirc571, are you using any SDK to call Bifrost? And is this happening with a specific provider or model?

[lirc571]

@Pratham-Mishra04 I was using a vllm server as an openai provider on bifrost. Connected opencode to it and it keeps going even if there is no tool call.

I then attached another proxy in front of bifrost and found out the response did not have finish_reason.

I'm having trouble with Anthropic API as well (opencode always stops the loop after a tool call). I'll do some more tests and create another issue for it.

[Pratham-Mishra04]

Hey @lirc571 we always send back the finish reason if the downstream provider includes it. Maybe the vLLM server isn’t returning it? Also, is this happening on streaming or non-streaming calls? I’d appreciate your help debugging this, as I’m having trouble reproducing it.

[lirc571]

Hi @Pratham-Mishra04 , below is my setup (new-api refers to this project):

+-----------------+                              
|                 |                              
|                 |                              
|     vLLM        |                              
|                 |                              
|                 |                              
+-----------------+          +------------------+
        ^       ^            |                  |
        |       +------------+                  |
        |                    |      bifrost     |
+-------+---------+          |                  |
|                 |<---------+                  |
|                 |          +------------------+
|     new-api     |                              
|                 |                              
|                 |                              
+-----------------+                              

With the following request:

{
  "stream": true,
  "model": "gpt-oss-120b-tools",
  "messages": [
    {"role": "user", "content": "Say 'Hello' to me"}
  ]
}

Below are the responses I get from calling the 3 services respectively (for bifrost, one with newapi as the provider and the other with vllm as the provider) logged with Bruno:

• vllm
• newapi
• bifrost-vllm
• bifrost-newapi

From the responses above, we can see

• Both the origin vllm server and new-api responses include finish_reason in all chunks (last one being "stop", the others null)
• For bifrost proxying vllm, only the last chunk has "finish_reason":"stop", the null ones are missing (okay since it's not standardized)
• For bifrost proxying new-api, finish_reason is not included in any chunk (causing error to downstream clients)

Ideally, bifrost should keep the structure and content of the upstream response as much as possible.

[Pratham-Mishra04]

Got it, thanks for for detailed response. I'll get back to you on this