fix(agent): detect SOUL.md drift on session restore via the shared identity resolver - #72253
fix(agent): detect SOUL.md drift on session restore via the shared identity resolver#72253Sora-bluesky wants to merge 1 commit into
Conversation
829eb1e to
feac335
Compare
|
Thanks for the focused SOUL.md fix. The current-main premise is real: Problems
Suggested changes
Automated hermes-sweeper review. |
|
The header is fixed. The body's first line now reads |
6f8132a to
4980347
Compare
|
Rebased onto current main (30fcf95) as a single commit (4980347b, was 3 commits ending at 6f8132a6). Upstream had landed the Bot Chat capability-epoch block in |
4980347 to
b975a31
Compare
…ia the shared identity resolver Editing SOUL.md never reached a continuing session: the restore path reuses the persisted system prompt verbatim, and _stored_prompt_matches_runtime only rejects Model/Provider/cwd/Platform drift, not identity content drift (NousResearch#68563). The identity resolution (SOUL.md or the hardcoded default, with the context-length-dependent truncation) is extracted from build_system_prompt_parts into resolve_identity_block(), and stored_identity_is_stale() asks that same resolver what identity a fresh build would use. The comparison is anchored on the identity block plus HERMES_AGENT_HELP_GUIDANCE (identity is slot NousResearch#1, so the pair must be the stored prompt's prefix); a stored prompt without that anchor is reported as not stale — the resolver can only assert staleness when it can locate and compare slot NousResearch#1. Rebuilding on session restore would break the AGENTS.md byte-stability contract (the one allowed rebuild point is context compression), so the drift check is applied at the compaction keep-prompt gate: a compaction that persists its rebuilt prompt refuses to reuse the cached bytes when the stored identity block no longer matches SOUL.md. Restore path I/O is unchanged (never reads SOUL.md, never rewrites stored prompt bytes), covered by regression tests. The resolver reports provenance and checkability: SOUL.md that exists but cannot be read is NOT treated as stale (rebuilding would persist a default-identity downgrade), and a missing HERMES_HOME reports checkable=False so a transient mount gap cannot demote a custom identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
b975a31 to
3805153
Compare
What changed
This PR no longer rebuilds the system prompt when it detects SOUL.md drift during session restore.
agent/conversation_loop.pyis back to themainbehavior (the three restore-path hunks from the original commit are reverted, net diff zero).Identity drift detection moved to the one place
AGENTS.md:19-23already allows the system prompt to change: the Hermes-native compaction keep-prompt gate inagent/conversation_compression.py. That gate used to keep the cached prompt whenever built-in memory state matched. It now also checksstored_identity_is_stale()and falls through to a full rebuild when the identity block no longer matches what's on disk. The rebuilt prompt gets persisted the same way any other compaction-time prompt already does:update_system_promptwrites it directly when compaction runs in place, andpublish_compression_childcarries it along when compaction rotates to a new session id.One related fix:
resolve_identity_block()inagent/system_prompt.pytreated a missing HERMES_HOME directory the same as a HERMES_HOME that exists but has no SOUL.md in it, both landing oncheckable=Trueand a default identity. Those are different failure modes. If HERMES_HOME itself isn't there, we can't say anything meaningful about identity, so that case now reportscheckable=Falseand the staleness check fails open (no rebuild) rather than confidently concluding "identity is default."Why not fix it at restore time
The original approach rebuilt the prompt inside
_restore_or_build_system_prompt, which runs wheneveragent._cached_system_promptis unset (agent/turn_context.py:613-615).AGENTS.md:19-23states the system prompt is never rebuilt mid-conversation except for the compression exception, because doing so busts prompt caching and multiplies request cost. Restore isn't that exception; compaction is.Limitation: sessions running the app-server runtime
Sessions using
api_mode == "codex_app_server"take a different compaction path (_compress_context_via_codex_app_server), and that path has no system-prompt write-back at all, onmainor in this PR. Adding a staleness check there wouldn't do anything, because Hermes never transports a system prompt to a CAS session in the first place, new or existing: the app-server turn path skipsactive_system_promptentirely (agent/conversation_loop.py:1249-1256),CodexAppServerSession.__init__(agent/transports/codex_app_server_session.py:274-311) takes no prompt argument, and the per-turn call (agent/codex_runtime.py:698,run_turn(user_input=...)) sends only the user's message. This PR leaves that path untouched. If you're running against the app-server runtime, editing SOUL.md has no effect on the session through Hermes at all; prompt delivery for that runtime is a separate gap, out of scope here.About issue #68563
The issue asks for SOUL.md edits to reach a running session right away. What this PR delivers is narrower: on the default (non-app-server) configuration, an edit reaches the session at the next Hermes-native compaction, not immediately, and it's persisted to the database when it does. Immediate reflection is a different feature: AGENTS.md:19-23 disallows rebuilding the system prompt mid-conversation outside the compression exception, so restore was never a place this could live. Getting from "eventually, at compaction" to "on demand" needs a command a user invokes on purpose, so the cost of rebuilding is one they choose to pay rather than one that fires on every turn regardless of the input file.
Two things could be split out as follow-ups, out of scope here:
/reload-soul) as a real answer to "I want this applied now."Neither is part of this PR. #68563 itself stays open for the maintainers to judge.
Testing
TestPromptStabilityInvariantand two new restore-path pins confirm identity drift never touches the cached prompt or callsupdate_system_promptduring restore.TestIdentityStalenessRebuild's scenarios (edited, truncated, and deleted SOUL.md) now assert verbatim reuse on restore instead of rebuild, matching the behavior above.TestSessionStartHookGuardmoved out of this file; it belongs to a separate PR fixing theon_session_startdouble-fire bug, which is a pre-existing issue unrelated to SOUL.md identity.