fix(api-server): emit approval events on legacy chat-completions SSE stream - #51878
fix(api-server): emit approval events on legacy chat-completions SSE stream#51878liuhao1024 wants to merge 1 commit into
Conversation
…stream
The /v1/chat/completions streaming path did not register an approval
notify callback, so approval-required tool runs blocked silently with
no event reaching the WebUI. Mirror the runs-API approval wiring:
- Add approval_notify_callback parameter to _run_agent
- Define _approval_notify in _handle_chat_completions that redacts
credentials and pushes __approval__ tagged tuples to the SSE queue
- Register the session key in _run_approval_sessions so
POST /v1/runs/{completion_id}/approval can resolve pending approvals
- Emit both event: approval.request and hermes.approval.request on
the SSE stream for backward compatibility
- Clean up _run_approval_sessions when the agent task completes
Fixes NousResearch#51871
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing the missing chat-completions approval bridge; current main still starts this stream without approval wiring at gateway/platforms/api_server.py:2311-2322, so the premise is valid.
Problems
gateway/platforms/api_server.py:2022reusesgateway_session_key/session_idas the approval key. That is unsafe for concurrent requests:tools/approval.py:1459-1485has one callback and pending queue per key. Current main deliberately changed runs to use a unique run ID in66325a770;tests/gateway/test_api_server_runs.py:361-420covers the cross-run approval isolation this would regress.- The status created at
gateway/platforms/api_server.py:2024-2030is never made terminal:_on_agent_doneonly removes approval state at:2057-2060. Current cleanup retains nonterminal statuses (gateway/platforms/api_server.py:4772-4779), so completed streams remainrunningindefinitely.
Suggested changes
- Key chat-completion approvals by
completion_id, and add the concurrent shared-session isolation regression. - Set a terminal status (or remove the synthetic status) when the task ends, then assert lifecycle cleanup in the integration test.
Automated hermes-sweeper review.
|
|
||
| # Register the approval session so POST /v1/runs/{completion_id}/approval | ||
| # can resolve pending approvals on the legacy chat-completions path. | ||
| approval_session_key = gateway_session_key or session_id or completion_id |
There was a problem hiding this comment.
Use completion_id as the approval key. tools.approval owns one callback and queue per key, so a shared session or memory key lets concurrent streams overwrite each other's callback and resolve the wrong dangerous command. Current main fixed this exact class for runs in 66325a770 by using the unique run ID.
| # chat-completions path so stale entries don't accumulate. | ||
| def _on_agent_done(_fut): | ||
| _stream_q.put(None) | ||
| self._run_approval_sessions.pop(completion_id, None) |
There was a problem hiding this comment.
This removes approval state but leaves the status created at lines 2024-2030 as running. Current status cleanup only expires terminal states, so set a terminal status here (or remove the synthetic status) to avoid retaining completed chat streams forever.
What does this PR do?
Wire approval-request events into the legacy
/v1/chat/completionsSSE stream so WebUI (and compatible clients) receiveapproval.request/hermes.approval.requestevents when a guarded tool blocks for approval.Related Issue
Fixes #51871
Type of Change
Changes Made
gateway/platforms/api_server.py: Addapproval_notify_callbackparameter to_run_agent; define_approval_notifyclosure in_handle_chat_completions(streaming path) that redacts credentials and pushes__approval__tagged tuples; register approval session key in_run_approval_sessionssoPOST /v1/runs/{completion_id}/approvalresolves pending approvals; emit bothevent: approval.requestandevent: hermes.approval.requestin the SSE writer; clean up_run_approval_sessionson agent task completion.tests/gateway/test_chat_completions_approval.py: 11 regression tests covering callback queue behavior, redaction pass-through, SSE dual-event emission, approval session lifecycle, and_run_agentregistration/unregistration.How to Test
pytest tests/gateway/test_chat_completions_approval.py tests/gateway/test_approval_prompt_redaction.py -q— all 19 tests should pass.pytest tests/gateway/test_approve_deny_commands.py tests/tools/test_approval.py -q— all 254 existing approval tests should still pass.rm -rfguard), send a streaming/v1/chat/completionsrequest, verify the SSE stream emitsevent: approval.requestwithrun_id,choices, and redactedcommand.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/gateway/test_approval_prompt_redaction.py tests/gateway/test_approve_deny_commands.py tests/tools/test_approval.py tests/gateway/test_chat_completions_approval.py -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/ACode Intelligence
gateway/platforms/api_server.py(_handle_chat_completions,_run_agent,_write_sse_chat_completion,_handle_run_approval)_run_agent, new tagged-tuple type in SSE queue, no changes to existing runs-API approval path_approval_notifyin the/v1/runspath (line ~3965); reuses_redact_approval_commandfor credential redaction; uses_run_approval_sessionsfor resolution lookup