Skip to content

fix: persist and emit agent response during verification stop loop - #62676

Closed
yingliang-zhang wants to merge 3 commits into
NousResearch:mainfrom
yingliang-zhang:fix/verification-stop-emit-response
Closed

fix: persist and emit agent response during verification stop loop#62676
yingliang-zhang wants to merge 3 commits into
NousResearch:mainfrom
yingliang-zhang:fix/verification-stop-emit-response

Conversation

@yingliang-zhang

@yingliang-zhang yingliang-zhang commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Problem

When verify-on-stop or pre_verify triggers, the agent produces a complete response (e.g., a detailed summary with tables and analysis), but that response is suppressed from both the durable transcript (state.db) and the UI. The user only sees the terse post-verification reply — never the full answer.

This is the bug behind repeated user reports of "系统消息冲走了回复" (system messages flushed the response). PR #62657 attempted to fix this by removing the synthetic flag from final_msg, but only addressed DB persistence — the response was still not emitted to the UI.

Root Cause (Three Layers)

Layer 1: DB persistence (fixed by #62657)

final_msg["_verification_stop_synthetic"] = True_is_ephemeral_scaffolding() returns True → _flush_messages_to_session_db() skips it → not persisted

Layer 2: UI emission (fixed by initial #62676)

No call to _emit_interim_assistant_message()not displayed

Layer 3: Streaming buffer overwrite (fixed by this update)

Even with _emit_interim_assistant_message() called, the Desktop gateway streams the response via stream_delta_callback. _interim_content_was_streamed() detects already_streamed=True, so the callback only calls on_segment_break() (paragraph break) instead of on_commentary() (standalone message bubble). The streamed text lives in the streaming buffer and gets overwritten by subsequent verification messages.

Fix

run_agent.py_emit_interim_assistant_message() gains force_display parameter:

  • When True, already_streamed is forced to False
  • The gateway callback calls on_commentary(text) → standalone permanent message bubble
  • The full answer survives the verification loop

agent/conversation_loop.py — both verify-on-stop and pre_verify paths:

  1. Remove synthetic flag from final_msg → persists to DB
  2. Call _emit_interim_assistant_message(final_msg, force_display=True) → standalone UI bubble
  3. Call _flush_messages_to_session_db() → persist before verification loop
  4. Change nudge from role="user" to role="system" → internal instruction, not durable user message

Tests

File Changes
test_verification_continuation_budget.py Role sequence updated; final_msg no longer synthetic; new test asserts already_streamed=False (force_display)
test_verification_stop_caching.py Assistant response now persists (not dropped); nudge still dropped

Results: 5/5 pass (test_verification_continuation_budget.py), 50/50 pass (test_verification_stop.py)

Supersedes #62657

PR #62657 only fixed DB persistence. This PR is a complete fix: DB persistence + UI emission + streaming buffer survival.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing both the persistence and UI paths; the suppression is real on the checked-out main (agent/conversation_loop.py:5153-5176, 5210-5227).

Problems

  • The new role="system" nudges at agent/conversation_loop.py:5199 and its pre_verify counterpart are not repaired away before the next request. repair_message_sequence only repairs assistant/tool/user shapes (agent/agent_runtime_helpers.py:408-550), while the request builder forwards every live message and prepends the stable system prompt (agent/conversation_loop.py:787-851). This creates a mid-history synthetic system turn.
  • Flushing the blocked answer before the synthetic nudge means the nudge is omitted by persistence (run_agent.py:1834-1848) but both assistant reports survive. On the next request, consecutive assistants are merged (agent/agent_runtime_helpers.py:429-467), so the attempted and verified reports do not remain distinct. The new callback test does not cover this persisted/resume path.

Suggested changes

  • Keep a protocol-valid continuation mechanism rather than inserting a system turn mid-history.
  • Separate UI preview from durable transcript state, or add an end-to-end persistence/resume test for both verification paths.

Automated hermes-sweeper review.

Comment thread agent/conversation_loop.py Outdated
Comment thread agent/conversation_loop.py
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Jul 11, 2026
@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 11, 2026
@yingliang-zhang
yingliang-zhang force-pushed the fix/verification-stop-emit-response branch from f705205 to c0d5998 Compare July 18, 2026 01:19
When verify-on-stop or pre_verify triggers, the agent's complete response
was flagged _verification_stop_synthetic and suppressed from both the
durable transcript (DB) and the UI. The user only saw the terse
post-verification reply, never the full answer — even after PR NousResearch#62657
which only fixed DB persistence without emitting to the UI.

This fix:
1. Removes the synthetic flag from final_msg in both verify-on-stop and
   pre_verify paths, so the response persists to state.db
2. Calls _emit_interim_assistant_message(final_msg) so the response
   reaches the UI immediately, before verification runs
3. Calls _flush_messages_to_session_db() to persist before the
   verification loop continues
4. Changes the nudge from role="user" to role="system" — it's an
   internal instruction, not a user message. This avoids creating a
   durable user→assistant pair and the resulting assistant→assistant
   adjacency in the persisted transcript is repaired at API-call time
   by repair_message_sequence (Pass 0: consecutive-assistant merge)

Tests:
- Updated test_verification_continuation_budget.py: role sequence now
  [user, assistant, system, assistant]; final_msg no longer has synthetic
  flag; only the nudge does
- Updated test_verification_stop_caching.py: assistant response now
  persists (not dropped); nudge still dropped
- New test_verify_on_stop_emits_interim_response_to_ui: verifies the
  full response reaches the interim_assistant_callback
- 70 tests passed, 1 pre-existing failure deselected

Closes NousResearch#62657 (supersedes — this is a complete fix)
When the Desktop gateway streams the response, _emit_interim_assistant_message
detects already_streamed=True and only calls on_segment_break() (paragraph
break) instead of on_commentary() (standalone message bubble). The streamed
text lives in the streaming buffer and gets overwritten by subsequent
verification messages — the user never sees the full answer.

Fix: add force_display parameter to _emit_interim_assistant_message. When
True, already_streamed is forced to False, so the gateway calls
on_commentary() to insert a permanent message bubble before verification
runs.

Both verify-on-stop and pre_verify paths pass force_display=True.
@yingliang-zhang
yingliang-zhang force-pushed the fix/verification-stop-emit-response branch from c0d5998 to 88e2a31 Compare July 19, 2026 02:27
yingliang-zhang added a commit to yingliang-zhang/hermes-agent that referenced this pull request Aug 16, 2026
Add force_display: bool = False parameter to _emit_interim_assistant_message.
When True:
- Bypass _interim_text_was_delivered dedup gate (under stress with repeated
  verification candidates, near-identical interim texts were suppressed as
  'already delivered', but the streaming buffer may have been flushed
  between first delivery and the dedup check)
- Force already_streamed=False so messaging gateway calls on_commentary()
  (standalone permanent bubble) instead of on_segment_break() (paragraph
  separator that gets overwritten by subsequent messages)

Wire force_display=True at verify_on_stop and pre_verify call sites in
conversation_loop.py where the response MUST survive as a permanent UI
element.

Supersedes the closed NousResearch#62676 which introduced the same parameter but was
not merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants