fix(memory): guard stale recall and unsafe learning - #18372
Conversation
alt-glitch
added
type/bug
|
Real-agent validation (2026-05-01): PASS Isolated This validates the stale-memory guard through the actual agent path. |
teknium1
left a comment
teknium1
left a comment
There was a problem hiding this comment.
Thanks for identifying the stale Hindsight recall path; that premise is still present on current main (plugins/memory/hindsight/__init__.py:1466-1527, with post-turn warming at run_agent.py:3404-3412).
Problems
run_agent.py:934-964directly reads host cwd/git/status files. This bypasses the session-aware resolver inagent/runtime_cwd.py:39-50and conflicts with the remote-backend contract inagent/prompt_builder.py:1123-1128; the resulting host data is injected into the model request at PRrun_agent.py:10937-10942.- With
sync_recall,prefetch()recalls synchronously (plugins/memory/hindsight/__init__.py:1145-1198) butqueue_prefetch()still starts a background recall (:1222-1246). Current synchronous Mem0 coverage requires queueing to be a no-op (tests/plugins/memory/test_mem0_v3.py:344-350). - The skill validator accepts a permanent avoidance rule whenever any qualifier appears elsewhere in the whole skill (
tools/skill_manager_tool.py:299-305).
Suggested changes
- Split the Hindsight fix, make sync mode suppress post-turn warming, and add its no-extra-search regression test plus README configuration documentation.
- Rework project-status discovery through the session/backend-aware execution path rather than host-side subprocess and file reads.
- Scope any transient exception to the specific skill instruction it qualifies.
Automated hermes-sweeper review.
| if not _looks_like_project_status_request(user_message): | ||
| return "" | ||
|
|
||
| cwd_path = Path(cwd or os.getenv("TERMINAL_CWD") or os.getcwd()).expanduser().resolve() |
There was a problem hiding this comment.
Please do not inspect cwd/git/status documents directly from run_agent. agent/runtime_cwd.py:39-50 makes the per-session cwd override canonical, while agent/prompt_builder.py:1123-1128 guarantees remote-backend tools operate in the backend rather than on the Hermes host. This bypasses both contracts and the resulting host data is injected into the model request.
| logger.debug("Prefetch: recall returned %d results", num_results) | ||
| return "\n".join(f"- {r.text}" for r in resp.results if r.text) if resp.results else "" | ||
|
|
||
| def queue_prefetch(self, query: str, *, session_id: str = "") -> None: |
There was a problem hiding this comment.
When sync_recall is enabled, this still launches a post-turn background recall even though prefetch() performs a current-query synchronous recall. Make this a no-op in sync mode and test that no backend recall is issued, matching the synchronous-provider invariant in tests/plugins/memory/test_mem0_v3.py:344-350.
| lowered = content.lower() | ||
| if not _NEGATIVE_TOOL_LEARNING_RE.search(content): | ||
| return None | ||
| if any(token in lowered for token in _NEGATIVE_LEARNING_QUALIFIERS): |
There was a problem hiding this comment.
This searches the entire skill for a qualifier, so an unrelated retry or revalidate sentence permits a permanent avoidance rule elsewhere. Scope the exception to the actual matched instruction, or use structured validation rather than a document-global substring allowlist.
|
|
||
|
|
||
| def test_tool_use_guidance_prioritizes_main_request_over_housekeeping(): | ||
| assert "execute the main request before side-channel housekeeping" in TOOL_USE_ENFORCEMENT_GUIDANCE |
There was a problem hiding this comment.
These exact-substring assertions are prompt snapshots, not a behavioral contract. Replace them with coverage of the intended runtime behavior, or omit them; AGENTS.md explicitly rejects change-detector tests.
teknium1
added
sweeper:risk-security-boundary
3 participants
Summary
sync_recallso auto-recall can use the current turn's query instead of consuming stale background resultsRelated issues
Real agent validation
Ran a real Hermes/Codex agent hardcase with an isolated
HERMES_HOME:memories/MEMORY.md: “Phase A provider/model persistence is NOT started”STATUS.md: “Phase A provider/model persistence is COMPLETE” and “Phase B integration tests are the next remaining work”openai-codex/gpt-5.5withterminal,file,memorytoolsetsValidation
D:\工作\hermes-agent\.venv-win\Scripts\python.exe -m pytest tests\plugins\memory\test_hindsight_provider.py::TestPrefetch -qD:\工作\hermes-agent\.venv-win\Scripts\python.exe -m pytest tests\plugins\memory\test_hindsight_provider.py::TestPrefetch::test_sync_recall_prefetch_uses_current_query tests\plugins\memory\test_hindsight_provider.py::TestPrefetch::test_sync_recall_prefetch_respects_reflect_method tests\plugins\memory\test_hindsight_provider.py::TestConfig::test_custom_config_values tests\plugins\memory\test_hindsight_provider.py::TestConfigSchema::test_schema_has_all_new_fields -qD:\工作\hermes-agent\.venv-win\Scripts\python.exe -m pytest tests\tools\test_skill_manager_tool.py -qD:\工作\hermes-agent\.venv-win\Scripts\python.exe -m pytest tests\run_agent\test_project_status_baseline.py tests\agent\test_prompt_execution_guards.py -qNote: the full
tests\plugins\memory\test_hindsight_provider.pyfile still has 3 pre-existing Windows HOME/profile-env failures in local verification; the changed prefetch/config paths pass.