fix: persist agent response during verification stop loop - #62657
Closed
yingliang-zhang wants to merge 1 commit into
Closed
fix: persist agent response during verification stop loop#62657yingliang-zhang wants to merge 1 commit into
yingliang-zhang wants to merge 1 commit into
Conversation
When verify_on_stop triggers, the agent's complete response was flagged _verification_stop_synthetic and suppressed from the durable transcript (run_agent.py _EPHEMERAL_SCAFFOLDING_FLAGS). The user only saw the brief post-verification response, never the full answer — their complete reply appeared to be 'washed away' by the verification nudge. Fix: remove the synthetic flag from the agent's response message (final_msg) so it persists normally. The nudge message keeps the flag (internal instruction, not for display). The resulting assistant→assistant gap in the persisted transcript (nudge is stripped) is repaired at API-call time by repair_message_sequence (Pass 0: consecutive-assistant merge). The _verification_stop_synthetic flag stays registered in _EPHEMERAL_SCAFFOLDING_FLAGS so the nudge is still filtered from both persistence sinks (SQLite flush + JSON snapshot).
alt-glitch
added
type/bug
Contributor
Author
|
Superseded by #62676 which provides a complete fix: DB persistence + UI emission. The previous fix only addressed persistence — the user still could not see the response. |
Contributor
Author
|
CI failure in The failing assertion This PR only changes:
|
yingliang-zhang
added a commit
to yingliang-zhang/hermes-agent
that referenced
this pull request
Jul 19, 2026
When verify-on-stop or pre_verify triggers, the agent's complete response was flagged _verification_stop_synthetic and suppressed from both the durable transcript (DB) and the UI. The user only saw the terse post-verification reply, never the full answer — even after PR NousResearch#62657 which only fixed DB persistence without emitting to the UI. This fix: 1. Removes the synthetic flag from final_msg in both verify-on-stop and pre_verify paths, so the response persists to state.db 2. Calls _emit_interim_assistant_message(final_msg) so the response reaches the UI immediately, before verification runs 3. Calls _flush_messages_to_session_db() to persist before the verification loop continues 4. Changes the nudge from role="user" to role="system" — it's an internal instruction, not a user message. This avoids creating a durable user→assistant pair and the resulting assistant→assistant adjacency in the persisted transcript is repaired at API-call time by repair_message_sequence (Pass 0: consecutive-assistant merge) Tests: - Updated test_verification_continuation_budget.py: role sequence now [user, assistant, system, assistant]; final_msg no longer has synthetic flag; only the nudge does - Updated test_verification_stop_caching.py: assistant response now persists (not dropped); nudge still dropped - New test_verify_on_stop_emits_interim_response_to_ui: verifies the full response reaches the interim_assistant_callback - 70 tests passed, 1 pre-existing failure deselected Closes NousResearch#62657 (supersedes — this is a complete fix)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When
verify_on_stoptriggers (agent edited code, then tried to end the turn), the agent's complete response was silently suppressed from the durable transcript. The user only saw the brief post-verification response — their full reply appeared to be "washed away" by the verification nudge.Root Cause
In
agent/conversation_loop.py, when the verification stop loop triggers:final_msg) is flagged_verification_stop_synthetic = True_EPHEMERAL_SCAFFOLDING_FLAGS(run_agent.py:233)_is_ephemeral_scaffolding()filters the message from both persistence sinks (SQLite flush + JSON snapshot)final_responseis set toNone(for finalizer logic)Result: the user never sees the full response. Only the brief post-verification summary is persisted.
Fix
Remove the
_verification_stop_syntheticflag from the agent's response (final_msg). The response persists normally as a durable assistant message. The nudge message keeps the flag (internal instruction, not for display).The resulting assistant→assistant gap in the persisted transcript (nudge is stripped by the ephemeral filter) is repaired at API-call time by
repair_message_sequence(Pass 0: consecutive-assistant merge,agent_runtime_helpers.py:408).Changes
agent/conversation_loop.py_verification_stop_syntheticflag fromfinal_msg; update commenttests/run_agent/test_verification_continuation_budget.py_is_ephemeral_scaffolding()Verification
The 2 pre-existing failures in
tests/agent/test_verification_stop_caching.pyare unrelated (MINIMUM_CONTEXT_LENGTH model validation, fails on origin/main too).Relationship to #62598
Complementary but independent:
[System:]marker role persistence (role=user→role=system)Different code paths, no file conflicts.