fix(agent): optimize dflash stall recovery telemetry

[OmarB97]

What changed

• Enables the existing tool-use enforcement prompt for the dflash model alias.
• Adds bounded stall-retry telemetry with per-turn summaries and local NDJSON logging.
• Adds a retry-only continuation nudge so the fallback lane is told to emit the promised tool call instead of repeating the preamble.
• Keeps truly empty responses on the existing empty-response recovery path.
• Makes low-information terminal redirects recoverable by default, while preserving hard halt behavior when hard_stop_enabled is explicitly enabled.

Why

The dflash retry path now recovers most no-tool-call stalls, but live agent sessions still showed avoidable turn churn. A recent trace also showed low-information terminal guardrails ending an interactive turn too aggressively. This patch reduces the initial stall frequency for the local alias, makes retry outcomes measurable, and lets guardrail redirects steer the model without prematurely finalizing the turn.

How to Review

Review agent/stall_retry.py for telemetry and retry nudge behavior, agent/conversation_loop.py for result summaries, agent/prompt_builder.py for the dflash prompt-guidance match, and agent/tool_guardrails.py for the hard-stop gate on low-information redirects.

Evidence

• scripts/run_tests.sh tests/run_agent/test_run_agent.py tests/run_agent/test_tool_call_guardrail_runtime.py tests/agent/test_stall_retry.py tests/agent/test_tool_guardrails.py tests/agent/test_prompt_builder.py -- -q passed: 529 tests, 0 failed.
• git diff --check passed with rc=0.
• python -m py_compile agent/stall_retry.py agent/conversation_loop.py agent/prompt_builder.py agent/tool_guardrails.py hermes_cli/config.py passed with rc=0.

Verification

scripts/run_tests.sh tests/run_agent/test_run_agent.py tests/run_agent/test_tool_call_guardrail_runtime.py tests/agent/test_stall_retry.py tests/agent/test_tool_guardrails.py tests/agent/test_prompt_builder.py -- -q
git diff --check
python -m py_compile agent/stall_retry.py agent/conversation_loop.py agent/prompt_builder.py agent/tool_guardrails.py hermes_cli/config.py

Risks / Gaps

• Telemetry writes bounded local JSONL and omits raw request messages; low risk, no follow-up needed.
• This is stacked on the open dflash retry PR series until those upstream reviews land; already tracked by the upstream PR chain.

Collaborators

Participants: Omar, Codex.

Process: Derived from live Hermes/dflash MeshBoard traces and validated with focused regression coverage plus the broader main-loop test file.

Task context: MeshBoard task hermes-dflash-stall-optimization-pass.

Related work: Fork mirror PR OmarB97#37.

[OmarB97]

Review result: approve (non-blocking suggestions)

Reviewed PR #35694 (fix/agent: optimize dflash stall recovery telemetry). This is a substantial, well-structured change. All findings are non-blocking; intended for follow-up.

Key strengths

• Default-off design — no behavior change unless HERMES_STALL_RETRY_MODEL is set. Safe to merge.
• Comprehensive stall detection — regex-based action preamble detection, trailing-colon heuristic, incomplete-ending detection, completion-phrase exclusion. Good balance of precision vs. recall.
• Low-information tool loop guardrails — sophisticated detection of repeated empty/unchanged tool results with graduated response (warn → redirect → halt).
• Good test coverage — new test_stall_retry.py (298 lines), expanded test_tool_guardrails.py (190+ additions), plus runtime provider, profile override, compaction, and interrupt-recovery tests.
• Clean MeshBoard launcher integration — HERMES_SKIP_PROFILE_OVERRIDE, HERMES_LLM_BASE_URL tap routing, minimal footprint.

Suggestions (all non-blocking)

1. conversation_loop.py: The three stall-retry early-return paths (limit_exhausted, failed_no_tool_call, exception) share ~40 lines of identical boilerplate per path. A _stall_retry_fail(failure_subclass, error_msg) helper would reduce ~120 lines to ~15.

2. runtime_provider.py: The OAuth provider exclusion list (qwen-oauth, xai-oauth, google-gemini-cli, minimax-oauth) will drift as new OAuth providers are added. Consider adding a config-level marker or at minimum a comment pointing to the canonical OAuth provider list.

3. stall_retry.py _retry_messages_with_nudge: Always appends the stalled assistant content as a new message. If api_messages already contains the original assistant turn (depends on call site), the retry model sees a duplicate. Harmless but not ideal for prompt hygiene.

4. tool_guardrails.py _clear_low_information_state: When called with no args, clears ALL tools' low-information state — not just the current tool. This means a single successful mutating call resets every tool's redirection counter. Intentional, but naming is ambiguous (suggests per-tool scope).

5. tool_guardrails.py _has_mutating_shell_signal: Classifies python commands as mutating. python --version and python -c are read-only. Consider excluding the -c / --version / -m pip list patterns from the mutating classification.

6. tool_guardrails.py _tool_reported_loop_block: Coupled to specific error keys (already_searched, already_read). If the file tool's error format changes, this path silently stops firing. Consider a more general key or a unit test that exercises the coupling.

7. cli.py interrupt handling: New interrupt_requested_without_message flag closes a race window where _interrupt_requested is set but the queue has no message. The break + fallback result path handles this correctly. Clean fix.

[OmarB97]

Rebased and refreshed this upstream mirror at head 147369fbf81cdc501ff858e49b378840c8bd5b24.

Current state:

• GitHub reports the PR CLEAN / MERGEABLE.
• Required checks are green: tests 1-6, e2e, typechecks, lint, Nix, Docker amd64/arm64, attribution, common ancestor, and supply-chain checks.
• Local focused verification passed before push: py_compile, git diff --check, and scripts/run_tests.sh ... with 395 passed.
• meshctl pr merge --dry-run accepted the PR: base is 3 commits behind, but no changed-file overlaps newer main.

I attempted the actual merge through meshctl pr merge, but GitHub rejected it because OmarB97 does not have MergePullRequest permission on NousResearch/hermes-agent. This PR is ready for an upstream maintainer to merge.

[OmarB97]

Closing as a redundant intermediate of a cumulative stack. These PRs form a stack where each is a strict superset of the previous: #35632 ⊂ #35673 ⊂ #35694 ⊂ #35698 ⊂ #35702. Both ends have now been refreshed onto current upstream/main — #35632 (the clean base) and #35702 (the tip, which carries the full feature set of the entire stack). This intermediate layer is therefore a strict subset of #35702 and would create a mutually-exclusive / duplicate merge against the already-refreshed base and tip. Closing in favor of #35632 + #35702, which together deliver everything here with no redundancy.