feat(memory): add auto_archive opt-in to escape oversize drift (#26045)

[veawho]

Problem

Drift signal #2 () blocks all mutations with no in-tool recovery path. The operator must hand-edit (which itself causes further drift because manual edits don't go through the tool) or bump in and restart the gateway.

Real-world hit on 2026-06-09: a single 4,937-char entry (1.7x the 2,200-char limit) blocked every memory add for the rest of the session. The fix at the time required raising to 12,000 in config + gateway restart + manual file rewrite.

Solution

Add an opt-in parameter to :

• When drift signal Support passing morph snapshot id #2 fires AND : move the oversize entry to a sidecar file ( or ) with an envelope header, then re-serialize the primary file without it. Original snapshot is preserved as audit trail.
• Drift signal Terminal tool #1 (round-trip mismatch / structural corruption) is NOT auto-fixed — that's structural corruption that needs human eyes. only repairs the oversize case.

Backward compatibility

• **Default is ** — every existing call site behaves identically.
• The drift guard continues to refuse mutations by default.
• 7 new tests in ; full 76-test suite passes.

Relationship to other PRs

• memory(action=replace) silently clobbers external writes to MEMORY.md (data-loss race with patch tool / shell appends / concurrent sessions) #26045 (CLOSED): original silent-clobber bug — fixed via commit 6855d17.
• fix(memory): refuse mutations when on-disk file changed since last read (#26045) #30993 (): adds SHA-256 on-disk-change guard to //. Complements this PR — defends against the replace path (concurrent writer race); this PR defends against the add path (oversize entry).
• fix(memory): skip drift guard for add (append-only) action #42880 (): proposes to skip the drift guard on add. We disagree with that direction because it re-opens memory(action=replace) silently clobbers external writes to MEMORY.md (data-loss race with patch tool / shell appends / concurrent sessions) #26045; our PR keeps the guard on but adds an opt-in escape hatch.

Test plan

pytest tests/tools/test_memory_tool.py -v
# 76 passed in 0.50s (69 existing + 7 new)

Files changed

• tools/memory_tool.py: +107/-4 — new method, accepts , top-level and plumb the flag, registry handler forwards it.
• tests/tools/test_memory_tool.py: +110 — new class with 7 tests.

Refs #26045

[veawho]

Cross-reference: real-world reproduction from issue #26045

I'm filing this comment to make the link to #26045 explicit and to share the concrete reproduction that motivated this PR.

In a Hermes session on 2026-06-09, a single memory entry grew to 4,937 chars (1.7x the 2,200-char ) — likely the result of a manual edit / tool operation that swallowed the leading separator between v17 and v18. From that moment on:

1. (signal Support passing morph snapshot id #2) fired on every subsequent — silent block, no way out from within the tool.
2. The only recovery path was to manually rewrite MEMORY.md (which itself risks more drift), bump to 12,000 in , restart the gateway, and retry — a 4-step ordeal that takes the agent out of its workflow.

This PR adds the missing escape hatch: automatically moves the oversize entry to and lets the add succeed on retry. Default behaviour is unchanged (), so the drift guard still does its job for the typical case.

Why this complements (rather than competes with) the existing fix surface

• fix(memory): refuse mutations when on-disk file changed since last read (#26045) #30993 () defends the replace path with a SHA-256 on-disk-change guard. Strictly additive — defends against concurrent-writer races on replace/remove.
• fix(memory): skip drift guard for add (append-only) action #42880 () proposes to skip the drift guard on . This re-opens the original data-loss class from memory(action=replace) silently clobbers external writes to MEMORY.md (data-loss race with patch tool / shell appends / concurrent sessions) #26045 and should not be merged as-is — but it does highlight that the current guard is too strict for legitimate user intent.
• feat(memory): add auto_archive opt-in to escape oversize drift (#26045) #42899 (this PR) keeps the guard on, but adds an opt-in escape hatch. The escape only fires when (a) the user explicitly asks for it AND (b) drift is caused by signal Support passing morph snapshot id #2 (oversize), not signal Terminal tool #1 (structural corruption). Signal Terminal tool #1 still requires human eyes.

Notes from the #26045 discussion thread

@jrhouston-trilogy in #26045 recommended a split-file architecture and an explicit side store for content that doesn't fit the tool's mental model. This PR's is a pragmatic implementation of that side-store idea — we don't split the file (UX cost too high), but we do route oversize content to a clearly-marked sibling that the tool never touches, so it survives any future drift events.

Worth noting: the rotation in already captures the pre-drift-detected state (not post-write like the issue thread suggested for #26045's replace case), so this PR preserves both the and the new archive as an audit trail.

Reproducer for this PR specifically

Before this PR:

# MEMORY.md contains a single 4500-char entry
result = store.add("memory", "new entry")
# => {"success": False, "error": "Refusing to write MEMORY.md: file on disk has content that wouldn't round-trip...", "drift_backup": "/path/MEMORY.md.bak.1234"}

After this PR (with opt-in):

result = store.add("memory", "new entry", auto_archive=True)
# => {"success": True, "message": "Auto-archived oversize entries to /path/MEMORY-archive-20260609-225500.md. Primary file now has 0 entries (0/2200 chars). Original file preserved at /path/MEMORY.md.bak.1234. Retry your add."}
# Then the next add (without auto_archive) succeeds normally.

Test plan (already passing locally)

pytest tests/tools/test_memory_tool.py -v
# 76 passed in 0.50s (69 existing + 7 new)

cc anyone who's been watching the #26045 thread — happy to iterate on the API shape if there's a preferred direction.