feat: progressive disclosure + <private> tag filter

[zackchiutw]

Summary

Two privacy-minded additions to the MCP search and write paths:

1. <private>...</private> tag handling on write

tool_add_drawer and tool_diary_write now strip <private>…</private> blocks (case-insensitive, multiline, non-greedy) before sanitize + embed. Entries whose entire content sits inside a private tag are refused, so nothing leaks through metadata, drawer IDs, or the embedding itself.

2. Progressive disclosure on read

tool_search returns a ~30-char summary + drawer_id per hit by default. Callers that need the raw text opt in via full=true, or fetch on demand via tool_get_drawer (which now accepts a single ID or an array). Stored content stays verbatim — only the preview is truncated — so this respects the "Verbatim always" design principle in CLAUDE.md. The two-step flow also keeps agent context small when search is exploratory.

Why

Private tags let users mark thoughts they want remembered locally but never fed back to the model — useful for rough reasoning, personal notes, and conversation drafts. Progressive disclosure is the read-side complement: long drawer text no longer floods the agent response when it only needed to know which memory matched.

What changes

File	Change
mempalace/privacy.py (new)	redact_private(text) -> (cleaned, is_fully_private) and summarize_for_search(text, n_chars)
mempalace/mcp_server.py	tool_search gains full=False default + schema entry; tool_get_drawer accepts str or list[str]; tool_add_drawer / tool_diary_write strip + reject fully-private content
mempalace/searcher.py	Hits now include drawer_id pulled from chroma ids so the two-step fetch round-trips reliably
tests/test_privacy.py (new)	17 unit tests covering redaction edge cases + summary truncation
tests/test_mcp_server.py	New TestProgressiveDisclosure class (20+ integration tests)
mempalace/instructions/search.md	Documents the default summary response, full=true opt-out, and the recommended search → get_drawer flow
mempalace/instructions/help.md	Adds a Privacy section + updates the write-side tool list

Compatibility notes

• Default response shape change. tool_search no longer returns full drawer text by default. Callers that depended on full text can pass full=true to restore the old behavior. LLM callers typically benefit from the new default (smaller responses, clear drawer_id for follow-up).
• No stored data is altered or re-embedded. Redaction only affects new writes.
• tool_get_drawer(id=...) is backwards compatible (single ID still works); new array form is additive.

Example — private tag

tool_add_drawer(
    wing="brainstorm", room="ideas",
    content="Public idea: build a MCP logger.\n<private>Skeptical — might be over-engineering.</private>"
)
# Stores and embeds only:  "Public idea: build a MCP logger."

Example — progressive disclosure

# 1. broad search returns compact previews
results = tool_search(query="auth rework", limit=3)
#  [{"drawer_id": "abc123", "text": "Switched to JWT after the session-leak incident...", ...}, ...]

# 2. fetch only the hits you actually want verbatim
tool_get_drawer(id=["abc123", "def456"])

Test plan

• pytest tests/test_privacy.py tests/test_searcher.py tests/test_mcp_server.py — 112 passed
• ruff check — clean
• ruff format --check on modified files — clean
• <private> stripping is case-insensitive (<PRIVATE>, <Private> all work)
• Fully-private entry rejected with success=False + reason field
• tool_get_drawer accepts both id="abc" and id=["abc", "def"]
• tool_search(full=true) returns the pre-existing full-text response shape

Notes for reviewers

• This PR is independent from feat: add Weibull decay and rerank pipeline for search results #1032 (rerank pipeline). They both touch mempalace/mcp_server.py:tool_search schema but at different call sites; a trivial merge conflict is expected if both land.
• The summary character budget (~30 chars backing up to the last word boundary) is hard-coded in privacy.py. Happy to make that configurable if reviewers prefer.

🤖 Generated with Claude Code

[igorls]

Hi, thanks for the contribution.

This PR has merge conflicts with develop, and the branch has not been updated in over 7 days, which puts it before our most recent release. The conflicts are likely against work that landed in that release.

Could you rebase onto develop so we can take another look?

If this change is no longer relevant, feel free to close the PR.

(This message is part of a periodic backlog pass, sent to all open PRs that match this state.)

[igorls]

Thanks for this contribution, and apologies for the slow turnaround.

develop has moved a fair way since this was opened and the branch no longer merges cleanly. If you're still interested in landing it, could you rebase onto current develop? Once it merges cleanly and CI is green I'll get it reviewed for the 3.8.0 cycle.

If you'd rather not pick it back up, no problem at all — just say so and I'll close it out, and thanks either way for taking the time to send it.