Skip to content

fix(gateway): stream synthesized recovery finals - #31478

Closed
BROCCOLO1D wants to merge 7 commits into
NousResearch:mainfrom
BROCCOLO1D:fix/31449-recovery-stream-deltas-v2
Closed

fix(gateway): stream synthesized recovery finals#31478
BROCCOLO1D wants to merge 7 commits into
NousResearch:mainfrom
BROCCOLO1D:fix/31449-recovery-stream-deltas-v2

Conversation

@BROCCOLO1D

Copy link
Copy Markdown
Contributor

Summary

Why

  • partial_stream_recovery and fallback_prior_turn_content assign final_response from local recovery text and then break without another model delta.
  • SSE/Open WebUI-style clients can otherwise receive only a finish chunk and render a blank bubble even though Hermes has a final answer.

Changes

  • agent/conversation_loop.py: adds a small helper for synthesized final deltas and uses it for guardrail halt, partial stream recovery, and prior-turn fallback finals.
  • tests/run_agent/test_run_agent.py: asserts partial-stream recovery and prior-turn fallback text is emitted to stream consumers.

Validation

  • python -m ruff check agent/conversation_loop.py tests/run_agent/test_run_agent.py
  • python -m pytest tests/run_agent/test_tool_call_guardrail_runtime.py tests/run_agent/test_run_agent.py::TestRunConversation::test_partial_stream_recovery_on_empty_stub tests/run_agent/test_run_agent.py::TestRunConversation::test_prior_turn_fallback_emits_stream_delta -o 'addopts=' -q

Scope

  • In scope: final text synthesized by recovery/guardrail paths that would otherwise bypass stream deltas.
  • Out of scope: changing retry policy, recovery selection, or the broader SSE protocol.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery labels May 24, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This is a resubmission of the closed #31475 (identical diff). Competes with #31477 for fixing #31449.

Both PRs address the same gap: recovery-path final text not streamed to SSE/TUI clients. This PR covers 3 sites (guardrail halt + partial_stream_recovery + fallback_prior_turn_content) using a helper function. #31477 covers only the 2 recovery sites but includes a None sentinel flush.

@BROCCOLO1D

BROCCOLO1D commented May 24, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up pushed in 1140d26 to align the helper with the stream-close behavior called out in the duplicate-scope comparison: _emit_synthesized_final_delta now emits the final text and the None close sentinel in independent guarded calls, so SSE/TUI writers still receive the close sentinel if final text delivery raises. Added regression coverage for the close sentinel and the callback-error path.

Validation:

  • python -m pytest tests/run_agent/test_run_agent.py -k 'partial_stream_recovery_on_empty_stub or prior_turn_fallback_emits_stream_delta or synthesized_final_delta_closes_stream_after_callback_error' -o 'addopts=' -q

@BROCCOLO1D

Copy link
Copy Markdown
Contributor Author

Closing this as stale: there has been no activity or maintainer follow-up for over a week, and I don't want to leave inactive work open. Happy to reopen/rework if this becomes useful.

@BROCCOLO1D BROCCOLO1D closed this Jun 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants