openai compatible chat completions endpoints #818
Description
Version
latest 9/18/25
Which installation method(s) does this occur on?
Source
Describe the bug.
When integrating a UI with a NAT agent on the openai compatible agent endpoint (openai_api_v1_path ), streaming response are not correctly shown in the UI because the stream output format does not follow that of the spec close enough. The content field is populated, but delta is null. The delta field should be populated and the content field should be null for a streaming response.
Also the configuration needed to enable post_openai_api_compatible_endpoint is not intuitive in that it need both openai_api_v1_path and openai_api_v1_path defined to different points.
The expected response was created from a hack in this branch. https://github.com/gfreeman-nvidia/AIQToolkit/tree/stream_openai
With the current response, Open WebUI shows waiting for response. With the expected response, the answer is shown to the user in Open WebUI.
Minimum reproducible example
Configuration:
front_end:
_type: fastapi
workflow:
method: POST
openai_api_v1_path: /v1/openai/deployments/agent/chat/completions # the actual openai api endpoint
openai_api_path: /openai/deployments/agent/chat/completions # needs to be defined as something to allow above endpoint to work
description: Executes E2E Agent demo
step_adaptor:
mode: custom
custom_event_types: [] # this prevents streaming intermediate steps
request:
curl -X POST http://localhost:18740/v1/openai/deployments/agent/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer sk-1234567890" -d '{
"model": "agent",
"messages": [
{
"role": "user",
"content": "tell me a joke"
}
],
"stream": true
}'
expected response:
data: {"id":"6e0bdc92-8641-4cae-88f0-33da439ae317","choices":[{"message":null,"delta":{"content":"Why did the scarecrow win an award? Because he was outstanding in his field!","role":"assistant"},"finish_reason":"stop","index":0}],"created":1758206823,"model":"","object":"chat.completion.chunk","system_fingerprint":null,"service_tier":null,"usage":null}
current response:
data: {"id":"7df4aa1b-5fdf-4a0d-bc57-481ee7def86d","choices":[{"message":{"content":"Why did the scarecrow win an award? Because he was outstanding in his field!","role":null},"delta":null,"finish_reason":"stop","index":0}],"created":1758206879,"model":"","object":"chat.completion.chunk","system_fingerprint":null,"service_tier":null,"usage":null}Relevant log output
Click here to see error details
[Paste the error here, it will be hidden by default]
Other/Misc.
No response
Code of Conduct
- I agree to follow the NeMo Agent toolkit Code of Conduct
- I have searched the open bugs and have found no duplicates for this bug report