feat(api-server): surface reasoning_content on /v1/chat/completions - #74692
Open
adaofeliz wants to merge 1 commit into
Open
feat(api-server): surface reasoning_content on /v1/chat/completions#74692adaofeliz wants to merge 1 commit into
adaofeliz wants to merge 1 commit into
Conversation
/v1/runs and /v1/responses already relay reasoning.available previews as structured events, but /v1/chat/completions never wired a callback to receive them, so reasoning/thinking text was silently dropped for that endpoint — even though it's the one most OpenAI-SDK clients use. Add a reasoning.available-filtered tool_progress_callback in both the streaming and non-streaming paths: - Streaming: emits delta.reasoning_content chunks (same field name DeepSeek/Moonshot/OpenRouter use), separate from delta.content and the existing hermes.tool.progress tool-lifecycle events. - Non-streaming: accumulates previews into message.reasoning_content on the final response. Tool-call events are unaffected — the filter only forwards reasoning.available, so tool_start_callback/tool_complete_callback still own the tool-lifecycle channel exactly as before.
teknium1
reviewed
Jul 30, 2026
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for tracing the missing chat-completions callback path. The endpoint gap is real, but this implementation captures only the fallback preview channel rather than provider reasoning deltas.
Problems
gateway/platforms/api_server.pyproposed lines 3932 and 3964 wiretool_progress_callbackonly. Providerreasoning_contentis dispatched viaagent/chat_completion_helpers.py:3210-3214→AIAgent._fire_reasoning_delta(), which invokes onlyreasoning_callbackinrun_agent.py:5831-5842. The API adapter does not currently thread areasoning_callbackthrough_create_agent(gateway/platforms/api_server.py:2348-2355,2642-2659), so actual provider thinking deltas remain dropped.- The added tests manually emit
reasoning.availablethroughtool_progress_callback; they do not cover thereasoning_callbacktransport that suppliesreasoning_content.
Suggested changes
- Thread
reasoning_callbackthrough_run_agentand_create_agent, then use it for the SSE chunks and non-streaming accumulation. - Test that callback directly and retain a combined lifecycle assertion so tool events remain single-emitted.
Automated hermes-sweeper review.
| agent_task = asyncio.ensure_future(self._run_agent( | ||
| user_message=user_message, | ||
| conversation_history=history, | ||
| ephemeral_system_prompt=system_prompt, |
Contributor
There was a problem hiding this comment.
This callback only receives reasoning.available previews. Provider reasoning deltas flow through AIAgent._fire_reasoning_delta() → reasoning_callback (run_agent.py:5831-5842), so please thread and use reasoning_callback here as well; otherwise standard reasoning_content streaming remains absent.
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
/v1/runsand/v1/responsesalready relayreasoning.availablepreviews as structured events, but/v1/chat/completionsnever wired a callback to receive them — so reasoning/thinking text is silently dropped for that endpoint, even though it's the one most OpenAI-SDK clients (Open WebUI, plainopenaiSDK, etc.) actually use.Fix
Add a
reasoning.available-filteredtool_progress_callbackto both the streaming and non-streaming paths of_handle_chat_completions:delta.reasoning_contentchunks — the same field name DeepSeek/Moonshot/OpenRouter already use for thinking traces — as a separate SSEdata:line, distinct fromdelta.contentand the existingevent: hermes.tool.progresstool-lifecycle events.message.reasoning_contenton the final JSON response.Tool-call events are unaffected: the new callback only forwards
reasoning.availableand ignores every othertool_progress_callbackevent type, sotool_start_callback/tool_complete_callbackstill own the tool-lifecycle channel exactly as before (no duplicate emits, per the prior comment this PR updates).Testing
test_stream_includes_reasoning_contentandtest_non_streaming_includes_reasoning_contenttotests/gateway/test_api_server.py::TestChatCompletionsEndpoint, mirroring the existingtest_stream_includes_tool_progresspattern.python3 -m pytest tests/gateway/test_api_server.py -q→ 87 passed (same 7 pre-existing failures as unmodifiedmain, unrelated to this change — missingdotenv/network fixtures in a fresh checkout).ruff checkclean on both changed files./v1/runs, which shares_create_agent/_run_agent) that reasoning surfaces without any tool-call leakage.Note
On an
Idempotency-Keycache hit (non-streaming),reasoning_contentwill be absent even if the original computation had reasoning — the idempotency cache only stores(result, usage), not the side-channel reasoning previews. This mirrors how streaming chunks also aren't replayed on cache hit, so it's a pre-existing limitation of that cache's shape rather than something this PR needs to solve.