merge: upstream/develop into fork main (27 commits, last sync 2026-05-22)

[jphein]

Summary

Periodic upstream sync — 27 commits from `MemPalace/mempalace:develop` since the previous sync via #105 earlier today.

Notable upstream changes pulled in

• Two of JP's own upstream PRs MERGED: #1142 (docs-releasing) and #1494 (docs-recovery-runbook). Now visible in our `main`.
• Tunnel fixes: #1467 (tunnel-file config-aware path resolution) and #1468 (create_tunnel validates source/target rooms exist) — both architectural correctness fixes in `palace_graph`.
• Hooks shallow-path guard: #1585 plus its gemini-review follow-up (`749d433`) — `hooks_cli` parents[3] access now guarded against transcript paths that don't have 3 ancestor levels.
• Security-warning visibility fix: #1257 — scam-callout dangling link folded back into the alert.
• README install-instructions polish: #1237 pipx, #1395 Claude Code retention, #1436 codex discoverability.

Conflict resolutions (3 files)

• `CHANGELOG.md` — fork's two `Unreleased` headers (AGE-integration plan + postgres-cutover) kept; upstream's `Unreleased` tunnel-fix block re-headed to `Unreleased — 2026-05-22 — Upstream tunnels fixes (cherry-picked from MemPalace/develop)`. Both narratives preserved chronologically.
• `README.md` — fork's title block and three fork-specific sections (`## The thesis`, the `## What this fork ships` axis-organized inventory, and `## Sources`) kept. Upstream's pipx/pip install paragraphs and Claude-Code retention checklist note dropped from where they were colliding — they're already covered by the !IMPORTANT callout at top + the existing `## Quickstart` further down. Upstream's CAUTION scam-alert block kept and rewired to include both upstream and fork repo URLs as official sources.
• `mempalace/convo_miner.py` — single comment-wording conflict; kept the fork's `(Upstream feat: configurable chunk_size, chunk_overlap, min_chunk_size MemPalace/mempalace#1024 review fix.)` annotation that references the upstream PR by number.

Test plan

• `ruff check .` + `ruff format --check .` clean
• `pytest tests/test_convo_miner.py tests/test_convo_miner_unit.py` — 62 passed
• Full suite verification by CI

🤖 Generated with Claude Code

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates a significant set of upstream changes into the fork, bringing various bug fixes, architectural improvements, and new documentation. The primary focus is on enhancing the stability and correctness of the tunneling system, improving the robustness of CLI hooks in diverse environments, and providing clearer guidance for users, particularly regarding Claude Code integration and data recovery. The merge also includes updates to the changelog and README to reflect these changes and address security advisories.

Highlights

• Upstream Sync & Conflict Resolution: Synchronized the fork with 27 commits from the upstream MemPalace/mempalace:develop branch, resolving conflicts in CHANGELOG.md, README.md, and mempalace/convo_miner.py to preserve both fork-specific and upstream changes.
• Tunneling System Improvements: Implemented fixes for explicit tunnels, ensuring their storage path respects MempalaceConfig.palace_path and adding validation to create_tunnel to confirm source and target rooms exist in the Chroma index, preventing tunnels to phantom rooms.
• Hooks Shallow-Path Guard: Added a guard to hooks_cli to prevent IndexError when accessing parents in shallow filesystem paths, improving robustness in environments like Docker containers.
• Documentation & User Guidance: Introduced new documentation for releasing procedures and a comprehensive recovery guide for chromadb segment corruption. Updated README.md with a critical security warning and added a dedicated guide for Claude Code retention.
• MCP Integration Enhancements: Expanded MCP setup guidance in the CLI and documentation to explicitly include Codex CLI, alongside Claude Code, for broader tool integration.
• Concurrency Control for Convo Mining: Refactored mine_convos to use a per-palace lock, preventing concurrent mining operations from conflicting, while allowing dry-runs to bypass the lock.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[gemini-code-assist]

Code Review

This pull request introduces several critical fixes and enhancements to the MemPalace system, primarily focusing on tunnel management and session retention. Key changes include relocating the tunnels.json file to follow the configured palace_path (fixing a bug where tunnels were stored in a hardcoded home directory), adding validation to ensure explicit tunnel endpoints exist in the Chroma index, and implementing a per-palace lock for conversation mining to prevent concurrent write issues. Additionally, the PR adds extensive documentation for Claude Code session retention and recovery procedures for corrupted ChromaDB metadata. Feedback highlights an opportunity to optimize endpoint validation by combining database queries and suggests a more defensive approach to handling corrupted tunnel files to prevent accidental data loss during overwrites.

[gemini-code-assist]

The logic to find the venv root by indexing parents[3] assumes a specific directory depth (e.g., <venv>/lib/pythonX.Y/site-packages/mempalace/). While the added len(parents) > 3 guard prevents crashes on shallow paths, the hardcoded index 3 might still point to the wrong directory (like .../lib/) in some Python environments. Consider using a more robust way to find the bin directory relative to the package root, or verify if parents[4] is more appropriate for standard lib/pythonX.Y/site-packages layouts.

[gemini-code-assist]

In _load_tunnels, if the JSON file is corrupt, the function logs a warning and returns an empty list. In the context of create_tunnel, this list is then mutated and saved, which will effectively overwrite the corrupt file and cause data loss of all previous tunnels. While this matches the previous silent behavior, a more defensive approach would be to raise an error or move the corrupt file to a backup location before starting fresh.

[gemini-code-assist]

Validating room existence for explicit tunnels is a great correctness improvement. However, calling _check_room_exists twice (once for source, once for target) results in two separate queries to ChromaDB. While acceptable for low-frequency operations, these could be combined into a single query using a $or filter on the metadata to reduce overhead.

Suggested change

	if not _check_room_exists(source_wing, source_room, col):
	raise ValueError(f"Source room '{source_room}' does not exist in wing '{source_wing}'")
	if not _check_room_exists(target_wing, target_room, col):
	raise ValueError(f"Target room '{target_room}' does not exist in wing '{target_wing}'")
	if kind == "explicit":
	col = _get_collection(config)
	# Check both rooms in a single query for efficiency
	if col is not None:
	try:
	query = {"$or": [
	{"$and": [{"wing": source_wing}, {"room": source_room}]},
	{"$and": [{"wing": target_wing}, {"room": target_room}]}
	]}
	results = col.get(where=query, limit=2, include=[])
	found = {(r["wing"], r["room"]) for r in results.get("metadatas", [])}
	if (source_wing, source_room) not in found and not _check_room_exists(source_wing, source_room, col):
	raise ValueError(f"Source room '{source_room}' does not exist in wing '{source_wing}'")
	if (target_wing, target_room) not in found and not _check_room_exists(target_wing, target_room, col):
	raise ValueError(f"Target room '{target_room}' does not exist in wing '{target_wing}'")
	except Exception:
	pass