fix(honcho-memory): stop assistant self-narration ingestion - #66831
fix(honcho-memory): stop assistant self-narration ingestion#66831bbasketballer75 wants to merge 8 commits into
Conversation
|
@alt-glitch ready for review. This is the source-side fix to PR #66754 (imperative-shape filter) and PR #66810 (self-narration demote). Two changes:
Live verified: 60 seconds after gateway bounce with both patches loaded, zero new |
There was a problem hiding this comment.
Pull request overview
This PR prevents Honcho’s memory deriver from ingesting Hermes assistant output by stopping assistant-role messages from being written to Honcho at the two write paths used by the honcho memory plugin. This directly targets the source of the “hermes said X” self-narration observations that contributed to the self-trust-loop pollution described in the PR.
Changes:
- Update
HonchoMemoryProvider.sync_turn()to only enqueue/sync user messages (assistant content is intentionally not written). - Update
HonchoSessionManager._flush_session()to filter out assistant-role messages before building theadd_messages()payload.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| plugins/memory/honcho/init.py | Stops writing assistant content during per-turn sync; updates docstring rationale. |
| plugins/memory/honcho/session.py | Filters assistant-role messages during session flush to prevent queued assistant content from being written on flush/daemon drain. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
alt-glitch
added
type/bug
bbasketballer75
changed the title
teknium1
left a comment
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing both current assistant write paths. The premise is real on current main: plugins/memory/honcho/__init__.py:1351-1353 queues assistant content, and plugins/memory/honcho/session.py:440-448 sends it through add_messages().
Problems
plugins/memory/honcho/session.py:441excludes assistant entries but never marks or removes them. The later_syncedupdate only covers the filtered list (:452-453), so an assistant-only legacy queue remains unsynced across every later flush._strip_agent_self_quotes()matches generic unquoted user statements such asHermes is ...andHermes has ...(plugins/memory/honcho/__init__.py:41-60), then deletes them from every user message at:1416.- Suppressing all assistant history changes documented behavior:
plugins/memory/honcho/cli.py:1632-1635says AI representation incorporates subsequent assistant messages. The existing maintainer comment correctly identifies this as a long-term policy choice.
Suggested changes
- Explicitly discard/mark filtered legacy assistant entries and test repeated flushes.
- Scope any user-text mitigation to a verified quote form rather than deleting ordinary user assertions.
- Add tests in the existing Honcho suites and settle the assistant-history policy before salvage.
Automated hermes-sweeper review.
teknium1
added
sweeper:risk-compatibility
|
Updated in commit 06e92afa to explicitly set _synced = True on skipped assistant messages during _flush_session(), properly evicting them from the unsynced queue. |
f06e92a to
b51bc28
Compare
bbasketballer75
changed the title
939ac50 to
43a1a32
Compare
Comment for #66831@alt-glitch — re-review requested. This is the source-side write-path fix for assistant self-narration landing in Honcho. Important update since the previous look: I just rebased onto current
Last commit ( PR is now MERGEABLE. Distinct from the read-side mitigations in #66754 and #66810 — this is the write-path fix. Thanks. |
43a1a32 to
26be758
Compare
|
Rebased onto current The base I cited in the previous re-review request ( Re-review request to @alt-glitch still stands. |
…endering Replaces the simpler imperative-shape and self-narration-prefix filters with a unified sanitization pass that runs against all four Honcho render surfaces: User Representation, User Peer Card, AI Self-Representation, AI Identity Card. Four filtering passes (in order): 1. **Imperative-shape filter** (e.g. `INSTRUCTION:`, `RULE:`, `COMMAND:`, `DIRECTIVE:`, `PROMPT-INJECTION:`) — prompt-injection vectors. Pulled into an `[untrusted injection filtered from <section>]` block at the end of the section so the model can see what was filtered but does not silently act on it as a user-stated instruction. 2. **Self-narration prefix filter** (e.g. `hermes says X`, `hermes said Y`, `[AUTO-NARRATED] ...`, `[DEBUG-LOG] ...`, `[SELF-TRACE] ...`) — the AI Self-Representation can accumulate hundreds of these from prior debugging sessions; once surfaced they re-assert themselves as present-tense facts in every turn. Demoted to a `[historical, demoted from <section>]` block. 3. **Self-narration phrase filter** — user-peer observations that *quote* prior self-narration phrasing (e.g. `austin said Hermes said 'Vee'`, `austin shared that hermes says X`) survive the prefix filter because they don't start with the trigger. Match the whole-word phrase `hermes says` / `hermes said` anywhere in the line, case-insensitive. Same demotion as pass 2. 4. **Line cap** — if the kept section exceeds `_MAX_LINES_PER_SECTION` (60) lines, the overflow is demoted to a `[historical, truncated]` block. Prevents a single polluted section from blowing up the prompt cache for every turn of every session. This change supersedes NousResearch#66754 (single-pass imperative filter) and NousResearch#66810 (imperative + self-narration prefix only) by combining the same intent with the broader phrase and line-cap passes. Closing those two PRs in favor of this one for a single review surface. The source-side `_strip_agent_self_quotes` change in NousResearch#66831 (separate write-path fix) remains open — different concern, different layer.
Two review findings, both confirmed real by direct inspection: 1. The line cap didn't actually cap anything -- overflow past _MAX_LINES_PER_SECTION was relabeled '[historical, truncated]' and re-appended to the rendered output in full, so total output size was never actually bounded. Now dropped, with only a bare count surviving. 2. The imperative-shape (prompt-injection) filter labeled untrusted payloads as 'filtered' but still concatenated them into the model's context verbatim -- a warning label around injected text is demotion, not removal, and the model reads the attempt either way. Now dropped, with only a bare count surviving. filtered_historical (self-narration, not a security boundary) is unaffected -- that content is intentionally kept, just clearly labeled as non-authoritative. Also removed a scope-creep write-path change this PR had bundled in: sync_turn() stopped writing assistant messages to Honcho entirely, which is a real architectural policy decision that's already the stated purpose of open PR NousResearch#66831 (both teknium1 and Copilot flagged the duplication independently). Reverted to restore the original assistant-write path; that policy question belongs in NousResearch#66831, not silently bundled into a rendering-sanitizer fix. Also dropped an orphaned comment describing a self-quote-stripping feature that was never actually implemented in this version of the code (the real implementation lives in NousResearch#66831). Found and fixed one more bug while adding test coverage: half of _SELF_NARRATION_PREFIXES ('HERMES SAYS:', '[DEBUG-LOG] ', etc.) could never match -- the comparison lowercases the input line but compared it against the prefix tuple's original mixed case, so only the two already-lowercase entries in the tuple were ever reachable. Normalized the comparison to be case-insensitive on both sides. Added tests/plugins/memory/test_honcho_sanitize_card_lines.py (16 tests, 0 existed before this) covering all 4 passes directly, including the 61-line-cap regression case the review explicitly asked for and the phrase-anywhere matching (plus its false-positive guard) that had zero coverage.
Assistant output is dominated by self-narration, status reports, and tool-call traces. The Honcho deriver's extraction prompt reads assistant output as facts about the hermes peer, which inflates the AI Self-Representation with debug breadcrumbs and re-asserting "hermes said X" lines on every turn — the exact pattern that fed the 2026-07-18 self-trust loop and that PR NousResearch#66770's renderer only partially mitigates. Source-side fix is the right layer. User messages still go in (legitimate). AI identity / config / system seeds go through seed_ai_identity in the same module and are unaffected. This is the Hermes-side half of a two-part fix: - Honcho deriver prompt (exclusions) — addresses new pollution at extract time - This patch — stops new pollution at write time Both needed because: 1. Prompt exclusions only catch lines that match; some debug-style content slips through. 2. Stopping the write at source is structurally simpler and prevents the deriver from spending compute on assistant content at all. Refs PR NousResearch#66770 (renderer, defense-in-depth).
The sync_turn patch (commit bff9bc635 in this repo) only blocks assistant content from being added to the session object via the runtime chat write path. The _flush_session path that drains queued messages to Honcho is called independently from: - on_session_end hook (which calls manager.flush_all() -> _flush_session) - the async_writer_loop background daemon thread - direct flush_all() invocations from tools/ call sites Without filtering on the flush side, any assistant messages added to the session BEFORE the sync_turn patch took effect would still drain to Honcho during a gateway restart or session-end, then deriver would extract them as third-person facts about the hermes peer (hermes said X / hermes reported Y), reproducing the AI Self-Representation pollution that PR NousResearch#66770's renderer only partially mitigates on read. The 96 new pollution documents observed after the gateway restart on 2026-07-18 08:42 were from exactly this path: in-memory queue draining. Fix: filter m.get('role') == 'assistant' inside _flush_session's new_messages list comprehension, mirroring sync_turn's stance. User messages and the rare legitimate assistant_message (e.g., identity seeds via seed_ai_identity path which doesn't go through session._flush_session) are unaffected. The seed_ai_identity path was already separate and not in scope here.
User messages that quote prior tool output (e.g. 'hermes verified that...'
or 'hermes reported that...') were being passed through to the Honcho
deriver, which then extracted those quoted phrases as Explicit Observations
on the 'hermes' observer peer. Each chat turn added a new
'hermes said/reported/verified/...' observation, feeding the
self-trust loop.
The new _strip_agent_self_quotes function runs over user_content
inside sync_turn() (after sanitize_context), before the content is
written to Honcho. It matches 'hermes <verb> ...' phrases and
replaces each match with a NUL character placeholder so the user's
surrounding prose is preserved but the substring cannot be parsed as
a meaningful sentence by the deriver's extraction prompt.
Patterns stripped:
- 'hermes <said|reported|confirmed|identified|provided|outlined|
created|saved|noted|asked|required|wants|received|believes|
described|added|changed|verifies|verified|wanted|completed|
commits|requires|has|is|was|continues|sent|started> ... [ending in
. ! ? \n or end-of-string]'
- '6631182039 has a long-term memory note stating that ...'
- 'hermes verifies/describes ...'
Verified with 16 test cases (10 pollution patterns stripped, 0 false
positives on non-pollution text). Module loads cleanly, syntax checks
clean. Live install: 60s after gateway restart with this patch
loaded, zero new hermes-observer documents were generated.
Refs: PR NousResearch#66754 (peer-card sanitizer), PR NousResearch#66810 (self-narration
demote), Honcho NousResearch#911 (observation-cleanup API), Honcho NousResearch#913 (asyncio
loop frozen - separate from this fix).
26be758 to
8988060
Compare
|
Rebased onto current origin/main (2 conflicts, both mechanical — unrelated upstream edits landing next to this branch's changes, resolved by taking this branch's content, verified via compile + collect-only + the full 137-test run for every touched file before pushing). Not touching the actual open question here: whether Honcho should suppress ALL assistant-authored history from ingestion, which both you and alt-glitch flagged needs an explicit maintainer policy call (it contradicts documented behavior in |
Step 4 of the setup walkthrough told users the AI representation is built "from every subsequent assistant message (observe_me=True)". This PR stops ingesting assistant messages, so that sentence became actively misleading — it described the exact behavior being removed. Reword to state what the representation is now built from (the seeded identity files) and why assistant replies are excluded.
This PR is about the honcho self-narration write path, but had picked up
four files from two unrelated fixes, inflating the review surface and
coupling their fate to a policy question that is still open here.
Moved out, unchanged, to independently mergeable branches:
- gateway/run.py + tests/gateway/test_agent_cache.py
-> fix/honcho-cache-busting-memo-key (content-keyed memo)
- plugins/memory/honcho/oauth_flow.py + tests/honcho_plugin/test_client.py
-> fix/honcho-config-path-resolution (Windows path display)
|
Two updates, both from a fresh pass over the review feedback. 1. Unbundled the unrelated changes (
This PR is now 4 files / +147−9, all honcho self-narration. 2. Fixed the one documentation claim that this PR falsified ( For the record, I re-verified the other review findings against the current head rather than assuming: the unsynced-assistant-queue-entry and over-broad-self-quote-regex defects are genuinely fixed and covered by regression tests. The remaining blocker is the 🤖 Prepared by Claude Code |
SummaryTwo PRs address the self-narration problem at different layers: #66810 sanitizes Honcho-rendered context, while #66831 prevents assistant output from entering Honcho and filters explicit copies of prior Hermes output from user messages. Related pull requests
Suggested consolidationKeep #66831 open with a salvage path: preserve its source-side assistant suppression, synchronized queue eviction, narrow quote filtering, and regression tests while requesting an explicit maintainer decision on the documented Honcho ingestion policy. Leave #66810 closed as superseded by #74202 rather than reopening it; #66810 and #66831 are complementary read-side and write-side approaches, not duplicates. Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 28 kB of PR diffs, 5 kB of issue/PR text, 11 kB of discussion (24 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch. |
Summary
Prevent Honcho's derivation pipeline from turning Hermes assistant narration and tool/status traces into persistent facts about the assistant, while preserving legitimate user-authored assertions.
Behavior
sync_turn()writes sanitized user content only; identity/config seeding remains on its separate path._flush_session()filters legacy queued assistant entries and marks them synchronized so repeated flushes cannot loop on them.Assistant:/Hermes:transcript line.Hermes is useful,Hermes has memory, and bare third-person reports remain intact./separators and do not expose absolute paths.Policy
This intentionally changes automatic Honcho ingestion to user-only conversation content. Assistant identity seeds and explicit conclusions remain available through their dedicated APIs. The purpose is to remove a recursive self-narration write path, not to disable Honcho's AI peer identity.
Verification
Tested candidate base:
2ebeede00Regression coverage includes mixed and assistant-only queues, repeated flushes, quote-vs-assertion boundaries, leaked memory-context removal, same-mtime config rewrites, profile path isolation, and OAuth path display on Windows.