fix(agent): gate verify-on-stop nudge off for messaging surfaces - #52412
Merged
OutThisLife merged 1 commit intoJun 26, 2026
Merged
Conversation
The verify-on-stop guard (PRs NousResearch#52296, NousResearch#52297) defaulted ON for every session, so on gateway messaging surfaces (Telegram, Discord, etc.) the model complied with the nudge by writing a hermes-verify temp script and emitting an ad-hoc verification summary, which the gateway delivered to the end user as chat noise. Resolve a surface-aware default instead. The DEFAULT_CONFIG value becomes the sentinel "auto", which verify_on_stop_enabled() resolves to ON for interactive coding surfaces (CLI, TUI, desktop) and programmatic callers, and OFF for conversational messaging surfaces. The surface is read from HERMES_SESSION_PLATFORM (what the gateway actually binds), with HERMES_SESSION_SOURCE and HERMES_PLATFORM as fallbacks, matching the sibling resolution in skill_commands.py and prompt_builder.py. An explicit HERMES_VERIFY_ON_STOP env var or a boolean agent.verify_on_stop config still overrides in either direction. The passive evidence ledger and the call site are untouched.
tonydwb
approved these changes
Jun 25, 2026
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
Makes the verify-on-stop guard surface-aware so it stops leaking internal verification machinery into user-facing chat replies on messaging surfaces (Telegram, Discord, WhatsApp, etc.).
Looks Good
- Non-messaging session surfaces frozenset defines known non-messaging surfaces (cli, tui, desktop, gateway, etc.)
- Checks both HERMES_SESSION_PLATFORM and HERMES_SESSION_SOURCE with fallback to HERMES_PLATFORM
- Default-deny design: unrecognized identities treated as messaging (OFF) so new platforms never leak verification receipts before the set is updated
- Config default is auto which resolves to ON for interactive surfaces, OFF for messaging
- Precedence: explicit env var, then config value, then surface-aware default
- 4 files, 199 additions, well-scoped
Reviewed by Hermes Agent (cron)
1 task
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…-messaging-surface-leak fix(agent): gate verify-on-stop nudge off for messaging surfaces
habarmc1223-sudo
pushed a commit
to habarmc1223-sudo/hermes-agent-fluxmem
that referenced
this pull request
Jul 8, 2026
…-messaging-surface-leak fix(agent): gate verify-on-stop nudge off for messaging surfaces
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…-messaging-surface-leak fix(agent): gate verify-on-stop nudge off for messaging surfaces
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…-messaging-surface-leak fix(agent): gate verify-on-stop nudge off for messaging surfaces
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…-messaging-surface-leak fix(agent): gate verify-on-stop nudge off for messaging surfaces
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Makes the verify-on-stop guard (added in #52296 / #52297) surface-aware so it stops leaking internal verification machinery into user-facing chat replies on messaging surfaces.
On a gateway messaging session (Telegram, Discord, etc.), after an agent edits local files in a workspace with no canonical test command, the harness injects the edit -> verify nudge, which tells the model to write a
/tmp/hermes-verify-*.pyscript, run it, and "summarize it explicitly as ad-hoc verification." The model complies, and that ad-hoc verification receipt becomes the final turn response the gateway delivers to the human user as chat noise:The guard is valuable in a coding CLI session; on a chat surface it is internal machinery the user should never see.
This PR resolves a surface-aware default instead of a hardcoded one.
agent.verify_on_stopnow defaults to the sentinel"auto", whichverify_on_stop_enabled()resolves to:The surface is read from
HERMES_SESSION_PLATFORM(what the gateway actually binds), withHERMES_SESSION_SOURCEandHERMES_PLATFORMas fallbacks, mirroring the sibling platform resolution inagent/skill_commands.pyandagent/prompt_builder.py. An explicitHERMES_VERIFY_ON_STOPenv var or a booleanagent.verify_on_stopconfig still overrides in either direction. The passive verification ledger and the call site are untouched.Related Issue
Fixes #52411
Type of Change
Changes Made
agent/verification_stop.py:verify_on_stop_enabled()resolves a surface-aware default when no explicit env/config is set; honors an explicit boolean config or recognized string tokens; new_session_is_messaging_surface()readsHERMES_SESSION_PLATFORM+HERMES_SESSION_SOURCE+HERMES_PLATFORMand classifies against_NON_MESSAGING_SESSION_SURFACES(the local/programmatic surfaces, mirroringapps/desktop/src/lib/session-source.tsLOCAL_SESSION_SOURCE_IDS).hermes_cli/config.py:DEFAULT_CONFIGagent.verify_on_stopchanges fromTrueto the sentinel"auto"so the surface-aware default is actually reachable throughload_config().cli-config.yaml.example: documentsagent.verify_on_stopand the"auto"default.tests/agent/test_verification_stop.py: surface-aware coverage including messaging-platform OFF, interactive/programmatic ON (CLI, TUI, desktop, codex, api_server, webhook), env/config override both directions, the"auto"sentinel, and an end-to-end test that drivesverify_on_stop_enabled()through the realload_config()to prove the sentinel reaches the surface branch.How to Test
pytest tests/agent/test_verification_stop.py -q(32 passing).HERMES_VERIFY_ON_STOP=1forces ON on a messaging surface;agent.verify_on_stop: falseforces OFF on a coding surface.Checklist
Code
fix(agent):)pytest tests/ -qon the affected suites and all tests pass (verification + full config suite: 213 passing)Documentation & Housekeeping
cli-config.yaml.example, docstrings)cli-config.yaml.examplesince I changed a config key defaultScreenshots / Logs