chore: sync upstream/develop through da5a48c (post-v3.5.0, 213 commits) - #369
Conversation
Adds .cs, .csproj, .sln, .razor, and .cshtml so C#/.NET projects are indexed by the project miner. .razor/.cshtml are analogous to the already-supported .jsx/.tsx. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated READABLE_EXTENSIONS in miner.py to include ".swift", ".kt", and ".kts". - Added tests in test_miner.py to ensure scanning includes Swift and Kotlin files.
Add _try_pi_jsonl parser for Pi agent session files stored at
~/.config/pi/agent/sessions/{encoded-cwd}/{timestamp}_{uuid}.jsonl.
Uses type "message" entries with role "user"/"assistant". Skips
toolResult messages, model_change, thinking_level_change, and other
operational events. Requires session header (type "session" with
"version" key) to avoid false positives.
Format documented at github.com/badlogic/pi-mono session.md and
verified via Context7. Sample data provided by tunnckoCore in #59.
Refs: #59
Adds _try_gemini_json parser to normalize.py for three layouts:
1. Gemini API contents format (~/.gemini/sessions/*.json):
{"contents": [{"role": "user", "parts": [{"text": "..."}]}, ...]}
2. Messages-wrapper variant:
{"messages": [{"role": "user", ...}, {"role": "model", ...}]}
3. Flat top-level list with role="model".
This complements the existing _try_gemini_jsonl parser (which handles
~/.gemini/tmp/<hash>/chats/session-*.jsonl with session_metadata
sentinel) — JSONL covers Gemini CLI runtime sessions, JSON covers
exported / Studio-saved transcripts.
## Review feedback addressed (PR #204)
bgauryy review:
- #1 Parser-precedence bug: _try_gemini_json runs *before*
_try_claude_ai_json so the {"messages":[..., role=model, ...]}
layout is no longer silently claimed by the Claude parser. The
Gemini parser's has_model_role guard prevents false-positives
against Claude / ChatGPT data.
- #2 Layout 2a coverage: TestGeminiJson.test_messages_wrapper_format
+ test_messages_wrapper_does_not_get_claimed_by_claude pin the
fix in place.
- #3 Test conflicts with current main: rebased onto develop;
tests restructured into TestGeminiJson class.
- #4 tempfile/os.unlink → pytest tmp_path everywhere.
- #5 elif not text → else (the elif branch was dead).
- #6 Module docstring updated to mention Google AI Studio.
Tests: 9 new cases in TestGeminiJson covering all three layouts,
multi-part text joining, non-text part skipping, has_model_role
disambiguation, dispatch-chain regression for review #1.
Add _try_continue_json() normalizer for Continue.dev AI assistant sessions (~/.continue/sessions/*.json). Parses history array with role/content pairs, handles tool calls, system messages, and metadata. Closes #59 (partial — adds Continue.dev format support) Includes comprehensive test coverage for valid sessions, edge cases, malformed input, and unicode content.
Adds first-class Cursor IDE integration alongside the existing Claude
Code and Codex hook flows, so Cursor users get the same automatic
diary saves, pre-compaction transcript capture, and session-start
memory recall — without changing any default behaviour for existing
users.
What's included
---------------
Cursor hook scripts (hooks/cursor/):
- mempal_save_hook_cursor.sh — Stop event, counter +
loop_count guard, pending-save marker consumption, background
mempalace mine, followup_message emission.
- mempal_precompact_hook_cursor.sh — synchronous mine before
compaction, drops a pending_save marker, returns user_message.
- mempal_wake_hook_cursor.sh — sessionStart event,
wing-scoped recall guidance via additional_context.
- lib/common.sh — shared parsing + state helpers
(bash 3.2 safe, no heredoc-in-subshell traps).
- install.sh — idempotent installer with
--scope, --variant, --dry-run, --uninstall. Recognises existing
entries by basename so re-installs across paths work.
- STDIN_SHAPE.md, README.md — payload schemas + quick
reference.
Cursor plugin (.cursor-plugin/ + repo-root components):
- plugin.json, marketplace.json, README.md.
- skills/mempalace/SKILL.md — model-invocable skill mirroring the
Claude plugin's skill surface.
- commands/mempalace-{help,init,mine,search,status}.md — slash
commands for marketplace-published installs (filename = slug).
- mcp.json — auto-registers the mempalace MCP
server, wrapped under the documented mcpServers key.
Examples + docs:
- examples/cursor/hooks.json, hooks.minimal.json + README.
- website/guide/cursor-hooks.md + sidebar entry.
- README.md and CHANGELOG.md updates.
Tests (129 new, all green):
- tests/test_cursor_hooks_shell.py — 75 behavioural tests for
the three hook scripts: kill switches, input parsing, counter
logic, loop prevention, pending markers, wing inference, logging.
- tests/test_cursor_hooks_install.py — 19 contract tests for the
installer: dry-run, idempotent merge, basename-matched uninstall,
refusal to overwrite malformed JSON.
- tests/test_cursor_plugin_manifest.py — 35 contract tests for the
plugin: manifest validity, version sync with mempalace.version,
mcp.json shape, skill/command frontmatter, default-discovery
layout invariants.
Design notes
------------
- Local-first and zero-API by default; hooks never call external
services. Same privacy model as the existing Claude Code hooks.
- Fail-open: hook scripts deliberately do not use set -e so a broken
hook can never block the user's conversation.
- Cursor preCompact cannot block + return a followup, so we
synchronously mine the transcript and drop a pending_save marker
that the next stop hook consumes — guarantees verbatim capture
before context window compression.
- Cursor's default plugin discovery requires real commands/, skills/,
and mcp.json at the plugin root (verified against the cached
cloudflare plugin); .cursor-plugin/{commands,skills} are convenience
symlinks back to those canonical locations.
- bash 3.2 compatibility throughout: avoids heredoc-in-command-
substitution parser bugs; uses python -c for JSON parsing;
basename-matched entry recognition in install.sh.
- All changes are additive. No existing files are removed, no
existing hooks change behaviour, and no new runtime dependencies
are introduced.
Co-authored-by: Cursor <cursoragent@cursor.com>
Five fixes from the Gemini Code Assist review on MemPalace#1632 — three real bugs, two cleanups, all consistent with the bash-3.2-compatibility contract documented in the original commit. Bug fixes (high) ---------------- 1. hooks/cursor/lib/common.sh — config.json kill-switch check used a `python3 - <<'PYEOF' ... PYEOF` heredoc inside a `$(...)` command substitution. The heredoc body contains parens which trips the macOS bash 3.2.57 parser bug. Replaced with a `python -c '...'` call passing the config path as argv[1]. Matches the pattern already used in mempal_parse_stdin in the same file. 2. hooks/cursor/install.sh — a relative `--install-dir` was written verbatim into hooks.json. Cursor invokes hook commands from its own working directory (typically the project root), so a relative command path would silently fail to launch the hook. Now resolved to an absolute path against `$PWD` before being baked in. 3. hooks/cursor/mempal_save_hook_cursor.sh — `MEMPAL_SAVE_INTERVAL=0` would crash bash on `$((NEXT % 0))` (division by zero). Extended the existing sanitiser case to coerce 0 to the default interval alongside empty / non-numeric values. Cleanups (medium) ----------------- 4. hooks/cursor/install.sh — the EMPTY_CHECK_PY temp file is now inlined as `python -c '...'`. Removes a small leak window (tmpfile would linger if the script were interrupted between mktemp and rm -f) and shortens the script. 5. hooks/cursor/install.sh — `mktemp -t prefix` has subtly different semantics on BSD (macOS) vs GNU mktemp. Switched to the portable absolute-template form `mktemp "${TMPDIR:-/tmp}/...XXXXXX"` which behaves identically on both. Regression tests ---------------- - tests/test_cursor_hooks_shell.py test_save_interval_zero_is_coerced_to_default — guards fix #3. - tests/test_cursor_hooks_install.py — new TestInstallDirAbsolutePath class: test_relative_install_dir_is_absolutized_in_hooks_json — guards fix #2 against regression. test_absolute_install_dir_is_preserved_verbatim — guards that the relative-to-absolute resolution does not mangle paths that were already absolute. Verification ------------ - bash -n on all three edited scripts: clean. - uv run pytest tests/test_cursor_hooks_*.py tests/test_cursor_plugin_manifest.py: 132 passed (was 129; +3 regression tests). - uv run pytest tests/ --ignore=tests/benchmarks: 2399 passed, 3 skipped (pre-existing). - uv run ruff check . / ruff format --check .: clean. Co-authored-by: Cursor <cursoragent@cursor.com>
Resolves the maintainer review on the Cursor IDE support PR. Cursor-only
scope; cross-IDE items (wing-naming convention, shared-file merge order)
are coordinated on the separate Antigravity branch.
followup_message default (the one "decide before merge" item):
- Keep the stop-hook followup ON by default. Cursor's transcript format
is undocumented and mempalace/normalize.py has no Cursor parser, so the
background `mempalace mine --mode convos` is best-effort only and does
not yet yield clean verbatim drawers. The followup is therefore the
load-bearing verbatim-capture path; defaulting it off would leave a
default Cursor install capturing nothing.
- Add an opt-out (MEMPAL_CURSOR_SILENT=1, or MEMPAL_VERBOSE=false) for
users who want the Claude-style "zero tokens in chat" behaviour. The
hook still mines and keeps its counters/markers when silenced.
- Correct the misleading "background mine captures it" comments in the
save and precompact hooks; update hooks/cursor/README.md and the guide.
Hygiene fixes:
- Drop the hardcoded "version" field from .cursor-plugin/plugin.json and
marketplace.json (mempalace/version.py is the single source of truth);
tests now assert the field stays absent.
- Remove the committed .cursor-plugin/{commands,skills} symlinks (they
break on Windows clones with core.symlinks=false and were redundant
with the real repo-root components that `source: "."` already serves);
add a guard test that no symlinks exist under .cursor-plugin/.
- Document the preCompact synchronous-mine timeout tradeoff and that an
incremental/append-only mine is recoverable if killed (no corruption).
- Add a Cursor-namespaced, daily-throttled TTL sweep (MEMPAL_STATE_TTL_DAYS,
default 30) to lib/common.sh that GCs stale cursor_*.count/.pending only,
after the kill-switch check; shared logs and antigravity_* are untouched.
Verification: full suite green (2424 passed, 3 skipped), ruff check +
format clean, bash -n clean on all cursor scripts. +30 Cursor tests
(followup opt-out, state GC, TTL validation, no-symlink/version guards).
Co-authored-by: Cursor <cursoragent@cursor.com>
Fixes lint CI: ruff format --check flagged blank-line and long-dict wrapping in the Continue.dev parser tests.
The mempalace_diary_write tool declared a top-level anyOf in its input schema to require either entry or content. Anthropic's Messages API rejects any tool schema with a top-level anyOf/oneOf/allOf and returns a 400 for the entire tools array, so every MCP session failed to start. The entry/content constraint is already enforced at dispatch: content is remapped to entry before the handler runs, and a missing value returns -32602. Removing the combinator restores compatibility without weakening validation. Closes MemPalace#1711
The openclaw skill was last updated when mempalace exposed 19 MCP tools. Since then 13 more agent-facing tools have landed; this PR documents the 8 that openclaw should expose so agents can call them natively instead of falling back to `npx mcporter call ...`: Search & Browse: - mempalace_list_drawers (paginated drawer listing) - mempalace_get_drawer (fetch a single drawer by id) Palace Graph: - mempalace_create_tunnel (explicit cross-wing link) - mempalace_list_tunnels (enumerate explicit tunnels) - mempalace_delete_tunnel (remove an explicit tunnel) - mempalace_follow_tunnels (walk explicit tunnels from a room) Write / Session: - mempalace_update_drawer (mutate content or relocate a drawer) - mempalace_memories_filed_away (ack the silent auto-save hook) The 3 admin-only tools (mempalace_sync, mempalace_hook_settings, mempalace_reconnect) are intentionally left out — they're host/admin operations, not agent-facing memory operations. The Hermes MemoryProvider plugin landing in MemPalace#1684 makes the same call. Version bumped 3.3.0 -> 3.4.0 (additive tool surface, no breaking changes to existing tool docs).
- Fix mempalace_find_tunnels params: (required) -> optional. The MCP handler defaults both wing_a and wing_b to None (mempalace/mcp_server.py:1277), so the prior docs were factually wrong. Caught by gemini-code-assist on PR MemPalace#1719. - Clarify implicit-vs-explicit tunnel distinction with consistent casing and a brief in-line definition (implicit = discovered from drawer content overlap; explicit = user/agent-declared link). Suggested by copilot-pull-request-reviewer. - Split the mempalace_memories_filed_away one-liner into a short description plus 'Returns' and 'When to call' sub-bullets for readability. Suggested by copilot-pull-request-reviewer.
Ports the OpenClaw "search before answering" protocol to the Cursor and Claude plugin surfaces so the agent reads the palace before answering about past work, people, projects, or prior decisions instead of guessing from model memory. - integrations/shared/recall-protocol.md: single source of truth for the recall protocol, referenced by the skill and the rule so they cannot drift. - skills/mempalace-recall/SKILL.md: recall-only skill (the mempalace skill keeps setup/mine/status); cross-linked from the ops skill. - rules/mempalace-recall.mdc: plugin recall rule, alwaysApply: false so it only fires on recall-relevant turns and never adds MCP latency to greenfield work. - examples/cursor/rules/: opt-in copies for non-plugin users, including an aggressive alwaysApply: true variant documented with its latency tradeoff. - .claude-plugin/skills/mempalace-recall/SKILL.md: Claude plugin parity. - tests: assert the recall skill and rules/ discovery layout; the shipped rule must be alwaysApply: false. - docs: .cursor-plugin/README.md and the cursor-hooks guide now describe the three layers of recall (hook + skill + rule). The Antigravity plugin mirror lands as a follow-up on the antigravity branch, where .antigravity-plugin/ exists. Co-authored-by: Cursor <cursoragent@cursor.com>
…emPalace#1747) A clean `mempalace repair --yes` (legacy path) finished without _vacuum_and_rebuild_fts5: the bulk delete_collection + re-upsert cycle leaves the FTS5 inverted index inconsistent, so the next repair aborts at the sqlite integrity preflight. rebuild_index() got this cleanup when MemPalace#1517 was fixed; cmd_repair never did. Extract the shared epilogue _post_rebuild_cleanup() (close chroma handles, then VACUUM + rebuild FTS5) and call it from both full-rebuild paths so they cannot drift apart again. Cleanup runs on the legacy success path only; failure/restore paths are unchanged. Closes MemPalace#1747 Co-Authored-By: nord- <3777600+nord-@users.noreply.github.com>
Mirrors the portable fake-client arms of test_pgvector_backend.py
against a real PostgreSQL+pgvector server and adds live-only arms the
in-memory fake cannot exercise: real <=> operator ground truth, JSONB
pushdown vs local-fallback equivalence, cross-namespace isolation on
real tables, 8-connection concurrent writers, and the advisory-lock
serialization of run_maintenance('reindex') under a 2-connection race.
Gated on MEMPALACE_PGVECTOR_LIVE_DSN (same pattern as the qdrant live
gate); skips cleanly when unset. First run: 15/15 pass on PostgreSQL
16.10 + pgvector 0.8.2 (+AGE 1.6.0 in the same server), psycopg 3.3.4.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…actly-one-ran asserts
- Stub _write_marker on the 8 concurrent writer backends: upsert()
rewrites the marker on every call with a plain open('w'), so backends
sharing one local_path race on the same file (sharing violations on
Windows) — a test-design artifact, not the contract under test
- Guard the fixture's created list with a lock for the threaded tests
- Assert exact distance-ordered ids in the query/filter arms
- Reindex race: exactly one 'ran' (index absent beforehand, so the
advisory-lock winner must build)
Re-run live after changes: 15/15 pass (PG 16.10, pgvector 0.8.2).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…MemPalace#1770) One session.run over a repair-scale batch (5000 docs) allocates attention buffers far beyond available RAM and the kernel kills the process. Mirror chromadb's ONNXMiniLM_L6_V2 and embed in sub-batches of 32; per-chunk padding also stops one long doc inflating the whole batch. Co-Authored-By: mojie5 <262519016+mojie5@users.noreply.github.com>
…1770) Two threads sharing a cold EmbeddinggemmaONNX via _EF_CACHE could each build a full model session, and two factory callers could each keep a private instance. The load is now double-check locked with the session published last, and the factory cache has an atomic check-then-construct behind a lock-free fast path. __call__ wraps a bare string, returns [] for None and empty input before the lazy download, and its annotation matches the accepted types.
…MemPalace#1778) Pre-3.4 the hallway store was hardcoded at ~/.mempalace/hallways.json regardless of the configured palace_path, so two palaces on one host silently shared one file. Mining into palace-A leaked records into palace-B's hallway code paths. Apply the 3.3.6 tunnel-file migration pattern: * MempalaceConfig.hallway_file resolves to <dirname(palace_path)>/hallways.json * hallways._get_hallway_file(config) reads through MempalaceConfig * hallways._legacy_hallway_file() exposes the pre-migration hardcoded path for one-time orphan detection; _load_hallways logs a one-line warning when the legacy file exists but the configured one doesn't, matching palace_graph._load_tunnels behavior. No auto-migration — silent merging risks clobbering newer data. Atomic-write + 0600 semantics unchanged. Module-level _HALLWAY_FILE constant kept and honored when monkey-patched directly, so the three existing test sites that patch it (test_hallways.py, test_hallways_pagination.py, test_mcp_server.py) keep working without modification. New coverage in tests/test_hallways_palace_scoped.py mirrors the analogous tunnel tests: resolver default + custom palace_path + env-var redirect, orphaned-legacy warning + no-warning when paths match, and an end-to-end multi-palace isolation regression guard. Closes MemPalace#1778
…g tests to resolver Replaces the back-compat shim in _load_hallways/_save_hallways (which honored direct monkey-patches of the _HALLWAY_FILE module constant) with a clean single-source-of-truth resolver, matching the palace_graph tunnel-file migration in 3.3.6. The three existing test sites (tests/test_hallways.py, tests/test_hallways_pagination.py, tests/test_mcp_server.py) now monkey-patch _get_hallway_file and _legacy_hallway_file directly, exactly mirroring the helper in tests/test_palace_graph_tunnels.py. Production code now has one branch through the path resolution instead of two. No behavior change. 269/269 hallway + tunnel + mcp-server tests pass on Python 3.11 and 3.12, ruff clean.
Two catches on tests/test_hallways_palace_scoped.py TestMultiPalaceIsolation.test_save_then_load_under_different_palace_returns_empty: 1. Stale comment referencing the removed _HALLWAY_FILE back-compat shim (deleted in the prior fixup commit). Removed. 2. _legacy_hallway_file was not monkey-patched, so the test isolation gap let _load_hallways check the host's real ~/.mempalace/hallways.json when evaluating the legacy-warning branch. Now patched to a tmp_path sibling, matching the helper pattern used in test_palace_graph_tunnels.
…ace#1840 follow-up) (MemPalace#1892) * fix(pgvector): skip document column for metadata-only fetches (MemPalace#1840 follow-up) Closes the explicit "separate follow-up to keep this low-risk" callout in PR MemPalace#1840's description. For remote pgvector deployments (TLS over WAN), `mempalace_status` and every other metadata-only consumer was transferring the full `document` column over the wire even when nothing read it. A single scroll over a 177K-drawer palace on a 175 ms-RTT link moved ~150 MB of document text plus ~50 MB of metadata; this PR drops that to ~50 MB. scroll_rows / _scroll gain `with_document: bool = True`. When False, SELECT projects NULL::text instead of the document column. Positional _row parser unchanged (record[1] stays the document slot, just receives NULL). Existing callers default to True and see byte-for-byte identical behavior. PgVectorCollection.get_all_metadata override: where=None path goes single-scroll with with_document=False. Filtered path falls back to base to keep _matches_where running on array/object metadata values (same correctness contract as MemPalace#1840's filtered-path decision). Tests: - Update _FakePgVectorClient.scroll_rows to accept with_document; mirror the NULL-becomes-empty-string semantics when False - Update 5 existing scroll_calls assertions to include with_document=True (unchanged intent) - test_pgvector_get_all_metadata_skips_document_column: assert exactly one scroll call with with_document=False - test_pgvector_get_all_metadata_filtered_falls_back_to_base: assert filtered path preserves with_document=True Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA * fix(pgvector): extend with_document=False fast path to filtered get_all_metadata Per gemini-code-assist review feedback on MemPalace#1892: _matches_where only reads metadata, so the where=None vs where=set conditional fall-back was unnecessary. The filtered path can use the same single-scroll with_document=False fast path and apply the post-filter locally on metadata dicts — extending the wire-byte win to every get_all_metadata caller, not just unfiltered ones. Mirrors the pushdown + local _matches_where pattern already used by _rows in the same file: pushdown when _requires_local_filter is False, post-filter in Python otherwise. Same correctness contract as MemPalace#1840's filtered get path. Renames test_pgvector_get_all_metadata_filtered_falls_back_to_base to test_pgvector_get_all_metadata_filtered_uses_fast_path and asserts the new behavior (with_document=False + pushdown forwards the equality filter to SQL). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…1890) * feat(convo): preserve authored timestamp from transcripts Conversation drawers only carried `filed_at` (ingest time), so a bulk re-mine collapsed every drawer to a single instant and the chronological signal was lost — even though each Claude Code / Codex JSONL line already carries an ISO-8601 `timestamp`. The recency-window fallback and any date-aware consumer then saw ingest order, not when content was written. - convo_miner: derive `authored_at` (per-file max line `timestamp`) and store it as drawer metadata; falls back to `filed_at` when absent - searcher: surface `authored_at` in search results, and break exact hybrid-score ties toward the more recently authored drawer (ISO strings sort chronologically; missing dates sort oldest) — benchmark-neutral as it only reorders exact ties - tests: cover `_extract_authored_at` (latest wins, skips/tolerates lines without timestamps, non-jsonl/missing -> None) and the tie-break Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(search): surface authored_at in CLI + backfill for existing data Completes the authored_at work so the field is visible end-to-end and existing palaces can adopt it without re-mining. - layers: CLI `search` output shows an `authored:` date line per result (peer of the existing date; markdown drawers fall back to filed_at) - scripts/backfill_authored_at.py: in-place migration that stamps authored_at on convos drawers from their source transcripts — metadata only (no re-embedding), idempotent, dry-run by default - docs/authored-at.md: documents created_at (ingest) vs authored_at (written) and both backfill paths (in-place / drop-and-recreate) - tests: backfill integration tests over an ephemeral ChromaDB collection Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(search): address review — non-string timestamp guard + top-level authored_at tiebreak Two correctness fixes from the PR review: - _extract_authored_at: only compare when the parsed `timestamp` is a str. A non-string timestamp on a malformed/foreign JSONL line previously raised TypeError outside the try and could crash the mine. - _hybrid_rank: the tie-break read `authored_at` only from nested `metadata`, but the search_memories path (MCP / Claude Code) carries it at the top level of each hit — so the tie-break silently no-op'd there. Read both shapes. - tests: non-string timestamp cases, and a top-level-shape tie-break test (which fails before this fix). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style: apply ruff format to authored_at changes CI ruff format --check flagged 4 files; ruff check already passed. Formatting only — no behavior change. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
…ansport can write (MemPalace#1859) * fix(palace): process-wide mine_palace_lock re-entrancy for threaded HTTP transport The MCP HTTP transport (ThreadingHTTPServer) acquires the long-lived writer-lease on one thread (_acquire_mcp_writer_lock) but dispatches each write request on a different worker thread. The lock re-entrancy guard was thread-local, so write handlers (add_drawer/update_drawer) failed to see the process-held lease, re-acquired the flock, and self-conflicted with "palace ... is held by PID <self>". Reads worked (no lock); writes over the HTTP transport were impossible. Make the re-entrancy record process-wide (pid-tagged, guarded by a threading.Lock) so a write from any thread of the process that already holds the lease passes through. Safe: flock is per-process and HTTP writes are serialized by _HTTP_REQUEST_LOCK. Preserves fork-safety, same-thread nesting (miner.mine -> ChromaCollection.upsert), and cross-process protection (MineAlreadyRunning still raised between processes). Add cross-thread same-process regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(palace): reset lock guard on fork to avoid inherited-locked deadlock Address review (PR MemPalace#1859): `_palace_lock_guard` is a threading.Lock, so a child forked while another thread held it would inherit it locked (the holder thread is gone in the child) and deadlock on the next acquire. Register an os.register_at_fork(after_in_child=...) handler that replaces the guard with a fresh unlocked lock and clears state; the child must reacquire the flock anyway. Guarded by hasattr(os, "register_at_fork") for Windows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
…) (MemPalace#1891) * feat(mcp): add since/before date filter to list_drawers (MemPalace#1128) mempalace_list_drawers previously filtered only by wing/room. This adds optional since/before ISO date bounds on filed_at: since is inclusive, before is exclusive. The filter runs in Python after the rows are fetched. ChromaDB 1.5.7 rejects string operands for $gte/$lt and filed_at is stored as an ISO string, so a server-side where comparison is not available; the tool already collapses and paginates the full result set in Python. Drawers whose filed_at is missing or unparseable are excluded while a bound is active, and inverted bounds (since >= before) return a clear error. * test: close chromadb clients between tests to fix Windows handle leak (MemPalace#1128) chromadb 1.5.7 caches one System per palace path and only frees the SQLite/HNSW file handles on client.close(); the collection fixture and the per-test MCP cache reset only dereferenced the client, so handles leaked across the session. Harmless on POSIX (rmtree unlinks open files), but on Windows the handles stay locked and accumulate until an HNSW segment write in a later test's setup fails, which surfaced here as TestDeleteBySource::test_commit_purges_matching_closets asserting 0 == 2. Close the client in the collection fixture and in _reset_mcp_cache so the handles are released between tests. * test: release backend chromadb clients between tests (MemPalace#1128) palace.get_collection() caches one PersistentClient per palace_path on the process-wide backend singleton and never closes it; sweep, repair and several CLI tests reach the store through it. chromadb frees the rust-side SQLite/HNSW file handles only on client.close(), so the handles leak across the whole session: a 30-palace probe shows ~200 open file descriptors into the palace tree, dropping to 0 once the clients are closed. On POSIX the open handles are harmless (rmtree unlinks open files), but on Windows they stay locked and accumulate until a later test's HNSW segment write fails ("Failed to apply logs to the hnsw segment writer"), e.g. test_sweeper.py::TestSweeperTandem::test_sweep_recovers_untaken_message_at_cursor_timestamp. Drain the cached clients in the autouse _reset_mcp_cache teardown via close_palace(), which closes each PersistentClient (releasing its handles) without marking the backend closed so it stays reusable. Complements the collection-fixture and _client_cache close() added earlier.
…Palace#1868) * feat: add metadata facet support for qdrant * added benchmark * updated benchmark * chore: remove tracking for local scratch benchmark * feat: add metadata facet support for qdrant -clean * Update mempalace/mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update mempalace/backends/qdrant.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update mempalace/backends/qdrant.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update tests/test_qdrant_backend.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update tests/test_qdrant_backend.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update tests/test_qdrant_backend.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update tests/test_mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * /fix always working tool_status() fallback fixed * /fix fallback added to tool_list_rooms * Update mempalace/mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update mempalace/mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update tests/test_qdrant_backend.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * /fix rebuilt the room populating logic * /add added temporary files for atomic transactions * Update mempalace/mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update mempalace/mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update tests/test_mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * /fix ai slop * /fix added default facet limit * Update tests/test_qdrant_backend.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * /fix added max workers pool * Update mempalace/mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update mempalace/mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update mempalace/backends/qdrant.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * /fix added clear() * Update tests/test_mcp_server.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fix(qdrant): validate facet filter before existence check; fix taxonomy test - facet_counts now validates the where filter and rejects local-only filters before the _remote_exists() short-circuit, so an unsupported filter raises UnsupportedCapabilityError even on an unmaterialized collection (matches get()/lexical_search() ordering). - test_tool_get_taxonomy_uses_metadata_facets compared concurrent room facet calls via set(), but a call() with a dict kwarg is unhashable; compare order-independently via membership instead. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
…emPalace#1895) * feat(graph): auto-populate the associative graph from mined sessions Conversation mining never set the `entities` drawer metadata that hallways consume, so mined sessions produced an empty associative graph (and starved the entity-navigation / tunnel-recommendation features built on top of it). Add a no-LLM structural entity extractor and wire it into the convos mine: - entities: structural-only extractor (author-quoted code spans, URLs, file paths, qualified identifiers, CamelCase / snake_case symbols). No wordlists, no NLP models, precision-biased so prose doesn't pollute the graph. - convo_miner: set `entities` per chunk, and compute hallways after a convos mine (mirroring the project-file path). Hallways run before the FTS5 validation, which opens a direct sqlite connection that can invalidate the live Chroma collection handle on some Chroma builds. - cli: `mempalace hallways` lists the associative graph (CLI parity with the list_hallways MCP tool). - tests: extractor precision/ranking, entities metadata at mine time, CLI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(graph): address review — semicolon safety, leading-underscore snake, negative limit - entities `_clean`: strip `;` out of tokens so a URL query string or backtick span can't split the `;`-joined entities metadata field - entities `_SNAKE`: optional leading/trailing `_?` so `_extract_authored_at` and similar are matched in plain text (previously only caught via backticks) - cli `hallways`: clamp `--limit` with max(0, ...) so a negative value shows nothing instead of slicing from the end - tests for all three Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
… (MemPalace#1897) Documents running MemPalace as a central memory service for a team: HTTP MCP transport (--transport http with bearer-token auth), a networked backend (Qdrant via REST, no extra dep; or pgvector), and optional GPU embedding. Covers the security model (non-loopback token requirement, Host/Origin DNS-rebinding guard, TLS-in-front), client connection, and operating notes. Adds the page to the guide sidebar. Addresses MemPalace#1877.
…lace#1900) * feat(serve): turnkey secure remote MCP server (MemPalace#1877) Add `mempalace serve`: a secure-by-default wrapper over the HTTP MCP transport so a team can stand up a shared central palace with one command. Server capabilities (mempalace/mcp_server.py): - Native TLS via --tls-cert/--tls-key (env MEMPALACE_MCP_TLS_CERT/_KEY): wraps the socket in a TLS 1.2+ context, validated before bind. Token is still required on a non-loopback bind (TLS != auth). - Read-only mode via --read-only (env MEMPALACE_MCP_READ_ONLY): the 24 mutating tools are hidden from tools/list and refused at dispatch (-32003), enforced before arg handling — not merely hidden. Turnkey command (mempalace/cli.py): - Auto-generates a strong bearer token for non-loopback binds, stored 0600 under ~/.mempalace/server/ and printed once; reused across restarts. Token rides in the child env, never argv, so it can't leak via ps. - Prints a ready-to-paste client config (scheme reflects TLS), then foreground-execs the real server so Docker/systemd own the lifecycle. Deployment (deploy/): - docker-compose.server.yml wires the server + Qdrant with a /healthz healthcheck and persistent volumes. - server.env.example documents the env surface. - mempalace-server.service is a hardened systemd unit template. Tests: TLS handshake (openssl-gated), read-only enforcement, token autogen/0600/reuse, token-not-in-argv, secure-by-default gates. Docs: remote-server guide now leads with `mempalace serve` plus Compose and systemd subsections. * test(serve): fix Windows — don't patch os.name; gate 0600 asserts to POSIX Patching os.name to 'posix' broke Path.home() on Windows (pathlib mixed POSIX home resolution with Windows drive parsing). Capture both exec branches (os.execve + subprocess.run) instead, and guard the POSIX permission-bit assertions behind os.name == 'posix' (Windows files report 0o666).
LaTeX source files and BibTeX bibliographies are prose-rich content that benefits from both palace mining and entity detection. Adds the two extensions to the two extension lists most relevant to them, each with a matching test. - ``mempalace/miner.py:READABLE_EXTENSIONS`` — ``.tex`` / ``.bib`` join the mining allowlist (parallel to the Swift/Kotlin PR MemPalace#1368 and the PHP ecosystem PR MemPalace#1819). - ``mempalace/entity_detector.py:PROSE_EXTENSIONS`` — ``.tex`` / ``.bib`` also join the *preferred* entity-detection bucket alongside ``.md`` / ``.rst`` / ``.csv``, NOT the broader code-file fallback. The reason ``PROSE_EXTENSIONS`` exists separately is documented in-code: programming-language files have lots of capitalized identifiers (class names, function names) that produce false-positive person matches. LaTeX/BibTeX don't have that problem — they're typesetting languages for prose documents. ``.bib`` in particular is almost entirely author names, one of the highest real-entity densities of any file type the detector scans. Tests follow the patterns established by the prior extension PRs: ``tests/test_miner.py::test_scan_project_includes_latex_files`` mirrors the Swift/Kotlin scan tests, and ``tests/test_entity_detector.py::test_scan_for_detection_includes_latex_prose`` mirrors ``test_scan_for_detection_finds_prose``. The existing ``test_prose_extensions`` was extended to assert the two new entries. Full env-cleared suite: 3216 passed, 20 skipped. ``ruff check .`` and ``ruff format --check .`` both clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA
Establish guide/configuration.md as the canonical home for per-backend connection settings, with a compatibility table and connection-variable reference for the chroma, sqlite_exact, qdrant, and pgvector backends. remote-server.md already links Postgres + pgvector to /guide/configuration, but the page had no backend section; this populates that target. New backends add one table row plus a connection subsection, keeping README's compatibility table in sync rather than accreting a prose paragraph per backend.
Rename the table's 'Select with' column to 'Configure with' and list each backend's primary connection knob, since a connection variable (e.g. MEMPALACE_QDRANT_URL) configures a backend but does not select it — selection is uniform via --backend / MEMPALACE_BACKEND, covered in the prose below the table. Also state the concrete MEMPALACE_QDRANT_TIMEOUT default (10.0s).
…-config docs(config): add storage backends configuration reference
The custom theme set `.vp-doc table { overflow: hidden }` to clip its
rounded corners, which also overrode VitePress's default `overflow-x: auto`
— so any table wider than the content column was clipped with no way to
scroll to the hidden columns (visible on the storage-backends table). Switch
to `overflow-x: auto` so wide tables scroll, keeping the rounded corners.
Also shorten the storage-backends table's two capability headers
(Namespace isolation -> Namespaces, Lexical search -> Lexical) so the table
fits the content column without needing the scrollbar.
Browser-validated the table layout across desktop (1280) and mobile (375): - Denser doc-table cell padding (8px 16px -> 8px 12px) so comparison tables fit the content column instead of needing a horizontal scrollbar. - `overflow-wrap: break-word` on table-cell code so only genuinely long values (e.g. a Postgres DSN) wrap, while short identifiers like `palace_path` keep natural column sizing and stay on one line. - Drop the redundant 'Configure with' column from the storage-backends table (each backend's connection variables are documented in full in its own subsection right below) and shorten 'Local (exact cosine)' -> 'Local (exact)'. The comparison table is now five columns and fits cleanly. Verified no clipping and no page-level horizontal overflow on the configuration, remote-server, reference (cli/mcp-tools/python-api), claude-code, and knowledge-graph pages; wide tables scroll within their own container on mobile.
…le-layout fix(docs): wide-table layout — stop clipping, fit backend table
feat: add LaTeX (.tex, .bib) to readable and prose extensions
…tchup docs(openclaw): catch up SKILL.md with 8 newer MCP tools
Merge upstream/develop into the fork. Notable upstream additions since the v3.5.0 sync (73e74bf): turnkey secure remote MCP server with TLS + read-only mode (MemPalace#1877/MemPalace#1900), associative-graph auto-population from mined sessions (MemPalace#1895) + cmd_hallways, Qdrant server-side metadata facets (MemPalace#1868), since/before date filters on list_drawers (MemPalace#1128/MemPalace#1891), authored_at timestamp preservation from transcripts (MemPalace#1890), mine_palace_lock re-entrancy for the HTTP transport (MemPalace#1859), pgvector metadata-only fetch fix (MemPalace#1892), SQLite magic-header detect() (MemPalace#1893/MemPalace#1896), FTS5 autoheal (MemPalace#1878), host-root-logger fix (MemPalace#1860/MemPalace#1885), LaTeX extensions, dep bumps (ruff 0.15.20). Fork features composed through, not replaced: auto-query stack, tags filtering (backend + MCP + where-layer $contains_all/$contains_any), RRF/multi-encoder fusion, adaptmem_ft encoder, verbatim mode, PALACE_DAEMON_URL routing + daemon-strict startup announce (#49), postgres tool_status fast path (#267), rename_wing, room-alias resolution, silent-save hooks docs, plugin hooks.json (5 events, ms timeouts), CLI --workers, cli-only mcp_mode docs. tool_list_drawers now carries BOTH upstream date filters and fork tag filters; tool_status keeps the postgres fast path and gains the facets sweep; the HTTP transport keeps host pinning and gains TLS + read-only. Merged TOOLS count stays 39 (upstream 34 + fork tools); all doc/manifest tool-count claims reconciled to 39. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Port upstream's SQLite magic-header ChromaBackend.detect() (MemPalace#1893) that the ours-side resolution of chroma.py dropped; switch the eager-warmup test fixture to make_minimal_chroma_sqlite so fake palaces pass the stricter detection. - Extract _close_backend_palace_clients() from the conftest teardown fixture (C901 under ruff 0.15.20's default budget after composing the fork cache-clear with upstream's handle-close). - Reconcile every doc/manifest tool-count claim to the merged TOOLS count (39); fix integrations/openclaw/SKILL.md straggler. - fork-changes.yaml entry for the sync; regenerate FORK_CHANGELOG, README queue table, llms-full.txt, python-api/; bump README prose to post-v3.5.0 sync (da5a48c) and 4921 tests. Full suite: 4855 passed, 66 skipped (4921 collected). ruff check + format --check clean on 0.15.20. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request performs a major sync with upstream/develop, bringing in significant features like a secure remote MCP server, Qdrant metadata facets, and improved retrieval filtering. It also introduces critical reliability improvements, including process-wide lock management for threaded environments and automated recovery for common FTS5 index corruption scenarios. Highlights
Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
Warning Review limit reached
Next review available in: 21 seconds Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (74)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…footer refs The API docs were rendered before the detect() docstring landed (CI check-docs caught the staleness). The take-theirs CHANGELOG footer carried an orphaned [3.4.0] link def (MD053) and lagged the [3.5.0] body section — restore the consistent footer shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Warning Gemini encountered an error creating the review. You can try again by commenting |
….5.0) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
What
Sync
upstream/developthroughda5a48cinto the fork — 213 upstream commits since the v3.5.0 sync (73e74bf, 2026-06-26).Notable upstream additions
--tls-cert/--tls-key) + read-only server mode (ability to run memapalce on remote server (gpu + embeding + qdrant storage ) MemPalace/mempalace#1877 / feat(serve): turnkey secure remote MCP server (#1877) MemPalace/mempalace#1900)cmd_hallways(feat(graph): auto-populate the associative graph from mined sessions MemPalace/mempalace#1895)since/beforedate filters onlist_drawers(feat: add since/before date filter to list_drawers and search MemPalace/mempalace#1128 / feat(mcp): add since/before date filter to list_drawers (#1128) MemPalace/mempalace#1891)authored_attimestamp preservation from transcripts (feat(convo): preserve authored timestamp from transcripts MemPalace/mempalace#1890)mine_palace_lockre-entrancy for the HTTP transport (fix(palace): process-wide mine_palace_lock re-entrancy so the HTTP transport can write MemPalace/mempalace#1859), pgvector metadata-only fetch fix (fix(pgvector): skip document column for metadata-only fetches (#1840 follow-up) MemPalace/mempalace#1892), SQLite magic-headerdetect()(bug: ChromaBackend.detect() returns True for 0-byte chroma.sqlite3, causes BackendMismatchError on multi-artifact palaces MemPalace/mempalace#1893 / fix(backends): require SQLite magic header for chroma + sqlite_exact detect() (#1893) MemPalace/mempalace#1896), FTS5 auto-heal (fix(repair): auto-heal isolated FTS5 inverted-index corruption (#1596) MemPalace/mempalace#1878), host-root-logger fix (_init_logging() uses force=True which clobbers host app root logger config MemPalace/mempalace#1860 / fix(mcp): stop clobbering host app root logger at import (#1860) MemPalace/mempalace#1885), LaTeX extensions, ruff 0.15.20Composition (fork features kept, not replaced)
~50 conflicted files resolved by composing:
tool_list_drawerscarries both upstream's date filters and the fork's tag filters (schema + validation + body united)tool_statuskeeps the fork's postgres fast path (tool_status sweeps full chroma metadata in Python — route postgres backend through SQL group-by #267) and gains upstream's facets sweepChromaBackend.detect()ported to upstream's magic-header check; fork test fixtures updated tomake_minimal_chroma_sqlitehooks/READMEkeeps the silent-save architecture docsadaptmem_ft, verbatim mode, daemon-strict routing + startup announce (Add MempalaceConfig.daemon_url config-file fallback + log routing decision at CLI/MCP startup #49),rename_wing, room-alias resolution, CLI--workers, cli-onlymcp_modedocsMerged MCP tool surface stays 39 (upstream's 34 + fork tools) — all doc/manifest count claims reconciled against live
mcp_server.TOOLS;check-docs7/7 green.Testing
ruff check+ruff format --checkclean on ruff 0.15.20 (new upstream pin)scripts/check-docs.shcleanNotes for review
commit: TBDin the new fork-changes entry follows the sync-350 precedent; backfill follow-up planned for both.🤖 Generated with Claude Code