feat: silent stop hook — direct save instead of blocking MCP calls

[jphein]

Summary

• Stop hook saves diary checkpoints directly via Python API instead of blocking Claude with MCP tool call instructions
• Single-line ANSI-colored terminal notification (✦ MemPalace Checkpoint saved — N messages archived) replaces 3-4 expanded MCP tool blocks
• Desktop toast via notify-send for users not watching the terminal
• Precompact hook still blocks (rare, warrants thorough AI-driven save)

Fixes #554

Changes

• hooks_cli.py: Add _extract_recent_messages(), _save_diary_direct(), _notify() helpers; hook_stop saves silently and never blocks
• tests/test_hooks_cli.py: Updated stop hook tests, added 3 new tests for _extract_recent_messages
• No new dependencies (notify-send fails silently on non-Linux)

Test plan

• pytest tests/ -v — 576 passed
• ruff check — clean
• Verified terminal notification renders correctly
• Verified desktop toast appears via notify-send
• Precompact hook still blocks as expected

🤖 Generated with Claude Code

[web3guru888]

This is a substantial quality-of-life improvement for agent-heavy workflows — and the architectural split between stop (silent) vs precompact (blocking) is the right model.

What works well:

The _extract_recent_messages() implementation is solid. Filtering <command-message> and <system-reminder> tags avoids capturing Claude's internal scaffolding as diary content. The 200-char truncation prevents oversized entries from single verbose messages. The errors="replace" on file open is good defensive practice.

The _notify() approach — ANSI terminal line + notify-send silent fail — is exactly right. It degrades gracefully on non-Linux (the OSError: [Errno 2] No such file or directory: 'notify-send' case is caught). One minor note: on macOS, users might prefer osascript as a fallback for the desktop toast, though that's a follow-up concern not a blocker.

Concern worth flagging:

The _save_diary_direct() content is minimal — it's a pipe-delimited string of recent messages truncated to 80 chars each. This is a deliberately lightweight checkpoint, not a full session summary. That's fine for the stop hook (precompact still blocks for thoroughness), but the entry format might be surprising to users who expect their diary entries to look like prose. Might be worth a one-liner in CLAUDE.md explaining the checkpoint format.

_ingest_transcript() launching a background mempalace mine ... --mode convos subprocess is a nice addition — it means the palace gets the full conversation even if the explicit diary save is lightweight. The non-blocking Popen is correct here.

The change to PRECOMPACT_BLOCK_REASON text (more explicit enumerated instructions) is also an improvement — the numbered list format tends to produce more reliable tool-call sequencing than prose.

We use a similar pattern in our integration — a lightweight checkpoint on every cycle, with thoroughness reserved for context boundary events. Glad to see this upstream. LGTM.

[Copilot]

Pull request overview

This PR shifts the stop hook to perform a non-blocking checkpoint save (via direct Python calls) and adds supporting improvements across mining/search/MCP tooling and export.

Changes:

• Stop hook now saves a diary checkpoint directly (no blocking MCP tool-call instructions) and emits a concise notification.
• Mining/search improvements: room detection and chunking updates, optional similarity filtering in programmatic search, and parallel file preparation with batched upserts.
• New/expanded capabilities: markdown exporter + CLI command, additional MCP drawer CRUD/list tools, and broader test coverage.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
mempalace/hooks_cli.py	Implements direct diary checkpoint saving, notifications, and transcript ingest; stop hook no longer blocks.
tests/test_hooks_cli.py	Updates stop hook expectations and adds tests for extracting recent transcript messages.
mempalace/mcp_server.py	Adds metadata pagination/cache, search filtering, and drawer get/list/update tools + schemas.
tests/test_mcp_server.py	Adds coverage for new drawer read/list/update MCP tools.
mempalace/searcher.py	Introduces shared build_where_filter() and adds distance-threshold filtering to search_memories().
mempalace/layers.py	Reuses build_where_filter() and caps L1 scanning volume.
mempalace/palace.py	Adds epsilon mtime comparison and bulk mined-metadata prefetch helper.
mempalace/miner.py	Improves room detection and chunking; adds parallel file prep path with batched upserts and a workers option.
tests/test_miner.py	Adds unit tests for detect_room() and chunk_text() behavior and constants.
mempalace/convo_miner.py	Batches drawer writes per file to reduce ChromaDB call overhead.
mempalace/config.py	Adds configurable chunk sizing/overlap/min size defaults.
mempalace/exporter.py	New exporter to write palace contents as a browsable markdown tree.
tests/test_exporter.py	Adds tests for exporter structure and markdown content.
mempalace/cli.py	Adds export command and plumbs --workers through mine.
tests/test_cli.py	Updates mine CLI tests to include workers argument.
mempalace/knowledge_graph.py	Changes default query direction to include both incoming and outgoing relationships.
mempalace/entity_detector.py	Expands STOPWORDS to reduce false-positive entity detection for common technical terms.
CLAUDE.md	Adds repo-specific dev notes and documents fork deltas.
.claude-plugin/plugin.json	Bumps plugin version to 3.1.0.
.codex-plugin/plugin.json	Bumps plugin version to 3.1.0.
.claude-plugin/hooks/mempal-stop-hook.sh	Improves Python resolution (env override/venv/system) for hook execution.
.claude-plugin/hooks/mempal-precompact-hook.sh	Improves Python resolution (env override/venv/system) for hook execution.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This comment implies the transcript will “land in palace” before compaction, but _ingest_transcript() spawns an async subprocess and returns immediately. Either run the ingest synchronously here (if that guarantee is important) or adjust the comment/behavior to make it clear it’s only best-effort/background.

[Copilot]

bulk_check_mined() paginates over the entire collection, which can be very expensive when the palace contains many wings/projects. Since mine() uses this only to skip re-mining for a single project, consider allowing an optional where-filter (e.g., wing) so callers can restrict the metadata scan to the relevant subset and avoid O(total_palace_drawers) work per mine run.

[Copilot]

The PR title/description focuses on the silent stop hook change, but this PR also introduces additional user-facing features/behavior (e.g., new export command, exporter module/tests, miner parallelism/workers, new MCP drawer tools). Please either update the PR description to cover these changes (and their motivation/risk) or split them into separate PRs to keep review scope manageable.

[Copilot]

This testing note says “573 expected”, but the PR description/test plan indicates 576 tests passing and the suite has grown (new exporter/miner/MCP tests). Please update the expected test count here to match current reality so it remains useful during development.

[Copilot]

This section says the stop hook “saves to palace via MCP tools”, but the stop hook now writes the diary checkpoint directly via Python (calling tool_diary_write without blocking) and only the precompact hook still instructs MCP tool usage. Please update this description so it matches the current hook behavior.

[Copilot]

hook_stop() updates the last-save marker before these direct-save calls run. If _save_diary_direct() fails (or transcript ingest fails), the session may still be marked as “saved” and the next checkpoint will be skipped until the next interval. Consider having _save_diary_direct() return success/failure and only advancing the last-save point on success (or track last_attempt separately).

[jphein]

Superseded by consolidated PR — all changes plus review feedback incorporated into a single clean PR from main.