feat(agent): false-stop detection — nudge past premature stop after tool calls - #78113
Open
yingliang-zhang wants to merge 1 commit into
Open
feat(agent): false-stop detection — nudge past premature stop after tool calls#78113yingliang-zhang wants to merge 1 commit into
yingliang-zhang wants to merge 1 commit into
Conversation
yingliang-zhang
force-pushed
the
feat/false-stop-detection
branch
from
August 12, 2026 08:53
5fe882f to
2ad5f50
Compare
…ool calls When an LLM produces finish_reason=stop with text indicating intent to continue (a colon-preamble that lost its tool_calls, or a narrated continuation) right after a tool round, the conversation silently ends without executing the intended tool call (NousResearch#42503). Extract the inlined detection logic into a dedicated agent/false_stop.py module following the verification_stop.py / kanban_stop.py pattern: - build_false_stop_nudge(): pure function returning nudge text or None - false_stop_detection_enabled(): config + env gate, mirrors verify_on_stop_enabled() with surface-aware "auto" default - Module-level marker constants (24 CJK + 13 English intent markers, done/wait/question exclusion guards) — previously rebuilt per iteration Two detection signatures: - Signature A: short colon-preamble (<120 chars) after a tool round - Signature B: narrated continuation (≤200 chars, CJK ending, intent markers, no done/wait/question markers) after a tool round Both gated by _was_in_tool_round (last 8 messages). Bounded to 2 nudges per turn, resets on tool round and genuine completion. Synthetic nudge flagged _false_stop_synthetic in _EPHEMERAL_SCAFFOLDING_FLAGS for transcript hygiene. Complementary to intent_ack_continuation (turn-start, not post-tool) and NousResearch#57610 (progress-placeholder text). Config: agent.false_stop_detection: "auto" (on for CLI/TUI/desktop/ codex/local, off for messaging platforms). Env: HERMES_FALSE_STOP_DETECTION. Tests: 39 tests in 5 categories (gating, Signature A, Signature B, budget/reset, transcript hygiene). No regressions on verification_stop, verification_stop_caching, or verification_continuation_budget.
yingliang-zhang
force-pushed
the
feat/false-stop-detection
branch
from
August 16, 2026 01:55
2ad5f50 to
bee0183
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When an LLM produces
finish_reason=stopwith text indicating intent to continue (e.g., a colon-preamble that lost itstool_calls, or a narrated continuation), the conversation silently ends without executing the intended tool call (#42503).This is observed across model families — K3, GLM, DeepSeek, Qwen, and others. The model writes a preamble like
接下来读取文件:or继续把剩余页面读完。withfinish_reason=stopand notool_calls, right after a tool round completed.Fix
Add a bounded false-stop detection mechanism that nudges the model to issue the actual tool call:
<120chars) ending with a colon (:or:), produced after a tool round. Likely the start of a tool-call narration that lost itstool_calls.≤200chars) ending with CJK sentence punctuation, containing at least one intent marker, and NOT containing done/wait/question markers, produced after a tool round.Gated by
agent.false_stop_detection: "auto"(on for interactive surfaces — CLI, TUI, desktop, codex, local — off for messaging platforms). 2-attempt budget, resets on tool round and genuine completion. Synthetic nudge is stripped from durable transcript and trajectory via_false_stop_syntheticephemeral flag.Complementary to existing features
intent_ack_continuation(already upstream): fires at turn START (no prior tool calls) when model "announces an action but calls no tool." False-stop detection is the POST-tool-round version — it fires after a tool round completed (the_was_in_tool_roundgate checks last 8 messages for assistant withtool_calls)._dropped_toolcall_nudge(already upstream): handlesfinish_reason=tool_callswith emptytool_callsarray. Signature A handlesfinish_reason=stopwith a text preamble — adjacent but distinct.Implementation
Extracted to a dedicated
agent/false_stop.pymodule following theagent/verification_stop.py(273 lines) andagent/kanban_stop.py(108 lines) pattern:build_false_stop_nudge(content, messages, attempts)— pure function returning nudge text orNonefalse_stop_detection_enabled(config)— config + env gate, mirrorsverify_on_stop_enabled()Marker lists are production-hardened from real deployments: 24 CJK + 13 English intent markers, with done/wait/question exclusion guards. Signature B's ending set is CJK-only (
。!?;…) to avoid over-triggering on English completions ending in..Test plan
scripts/run_tests.sh tests/agent/test_false_stop.py— all pass (35 tests across 5 categories: gating, Signature A, Signature B, budget/reset, transcript hygiene)tests/agent/test_verification_stop.py,tests/agent/test_verification_stop_caching.py,tests/run_agent/test_verification_continuation_budget.pypy_compileon all changed files