feat(api_server): stream model reasoning as reasoning.delta on /v1/runs/events - #61259
feat(api_server): stream model reasoning as reasoning.delta on /v1/runs/events#61259studying-end wants to merge 1 commit into
Conversation
…ns/events The HTTP API Server (/v1/runs) never wired reasoning_callback, so the model's real reasoning (DeepSeek reasoning_content, codex reasoning, Bedrock thinking, inline <think>) was captured internally but dropped on this codepath. Wire reasoning_callback through _create_agent and emit a reasoning.delta SSE event from the /v1/runs handler (mirrors what tui_gateway already does). Clients now receive the real reasoning stream ahead of the answer. Changes: gateway/platforms/api_server.py (_create_agent accepts/forwards reasoning_callback; /v1/runs _run_and_close adds _reasoning_cb pushing reasoning.delta, payload field 'delta', same shape as message.delta); tests/gateway/test_api_server_runs.py (TestRunEvents::test_events_stream_emits_reasoning_delta).
Duplicate of #15169 (earliest open PR for this fix). Both wire |
|
The implementation addresses a verified current-main gap: Problems
Suggested changes
Automated hermes-sweeper review. |
What does this PR do?
The HTTP API Server (
/v1/runs) never wiredreasoning_callback, so the model's real reasoning — DeepSeekreasoning_content, codexresponse.reasoning.*.delta, Bedrock thinking, inline<think>— was captured internally (on_reasoning_delta→_fire_reasoning_delta) but dropped on the API Server codepath. As a result/v1/runs/{id}/eventsonly ever streamed the answer (message.delta), and the one-shotreasoning.availableevent carried an answer-derived snippet (first 500 chars ofassistant_message.content), not real reasoning.The CLI/TUI already display real reasoning (they set
reasoning_callback). This PR brings the API Server to parity: wirereasoning_callbackthrough_create_agentand emit areasoning.deltaSSE event from the/v1/runshandler (mirrors the existing pattern intui_gateway/server.py). Clients now receive the real reasoning stream ahead of the answer:Backward compatible:
reasoning.availableis unchanged; clients that don't handlereasoning.deltaare unaffected. Only/v1/runsis changed —/v1/chat/completionsand/v1/responsesare untouched.Related Issue
N/A (no existing issue; happy to open one if preferred).
Type of Change
Changes Made
gateway/platforms/api_server.py—_create_agentnow accepts and forwardsreasoning_callback; the/v1/runshandler (_run_and_close) wires a_reasoning_cbthat pushes areasoning.deltaSSE event (payload fielddelta, same shape asmessage.delta).tests/gateway/test_api_server_runs.py— addedTestRunEvents::test_events_stream_emits_reasoning_delta: assertsreasoning_callbackis wired through_create_agentand that its output reaches the SSE stream asreasoning.delta.Additive only: +60 lines across 2 files, no change to existing events.
How to Test
uv run --extra dev pytest tests/gateway/test_api_server_runs.py -q→ all pass, including the newtest_events_stream_emits_reasoning_delta.reasoning_effort != none),POST /v1/runs, thenGET /v1/runs/{id}/events→ confirmreasoning.deltaevents arrive ahead ofmessage.deltaand contain real reasoning (distinct from the final answer).Checklist
Code
feat(api_server): stream model reasoning as reasoning.delta on /v1/runs/eventspytest tests/ -qand all tests pass — ran the affected filetests/gateway/test_api_server_runs.py(24 passed); the fulltests/suite was not run on this basetest_events_stream_emits_reasoning_deltaDocumentation & Housekeeping
reasoning.deltaevent mirrorstui_gateway's existing one)cli-config.yaml.exampleif I added/changed config keys — or N/A (no new config keys)CONTRIBUTING.mdorAGENTS.md— or N/AScreenshots / Logs
/v1/runs/{id}/eventswith a reasoning model — before this PR onlymessage.delta+reasoning.availablewere emitted; after,reasoning.deltastreams first: