Skip to content

sync: upstream/develop through v3.7.0 — 433 commits, 62 conflict files - #394

Merged
jphein merged 436 commits into
mainfrom
sync/upstream-develop-v3.7
Aug 9, 2026
Merged

sync: upstream/develop through v3.7.0 — 433 commits, 62 conflict files#394
jphein merged 436 commits into
mainfrom
sync/upstream-develop-v3.7

Conversation

@jphein

@jphein jphein commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes #383 (scoped there at 349 commits / v3.7.0; upstream moved to 433 commits through 8516db7f by sync time — TS-rewrite watch item checked: upstream's last 50 commits are 60 .py vs 2 .mts, Python tree still primary).

Upstream brought in: mempalace serve hub + hook forwarding, temp-collection repair promotion, round-trippable drawer IDs (MemPalace#2090), kg_supersede, the CLI HNSW-divergence fence, per-conversation dedup, daemon lock retry (MemPalace#2029), repair --dry-run for --mode from-sqlite (MemPalace#2138), Milvus backend.

Fork-preserved through the 62 conflicts (method: per-file 3-way replay of the smaller delta onto the larger side, then hunk-level composition):

  • postgres/pgvector/AGE dispatch, incl. the 96f83d7 union-merge guard composed with upstream's drawer_id field — TestUnionPostgresPath run explicitly per the Upstream sync: develop through v3.7.0 (349 commits, ~59 conflict files) #383 watch item, green
  • daemon-strict hook routing nested inside upstream's one-probe _hook_write_routing_context (both architectures' tests pass)
  • hook_verbatim_mode threaded through upstream's restructured normalize/convo_miner
  • fix(repair): preserve embeddings on rebuild + cap divergence-floor MemPalace/mempalace#1367 embeddings-preserve ported onto the new repair flow
  • adaptmem_ft dispatch, rename_wing wrapper forward, daemon-routed CLI + upstream's --palace hallways scoping
  • Behavior adoptions: precompact mines only the active transcript (upstream v3.6 semantics)
  • MCP surface: 48 tools (fork 39 + upstream additions); current-state docs/manifests reconciled; check-docs now excludes FORK_CHANGELOG + dated specs from the tool-count scan (historical records state counts as of their date)

Test plan

  • Full suite: 5522 passed, 79 skipped, 122 deselected (5609 collected; README/CLAUDE counts updated)
  • tests/test_hybrid_candidate_union.py 17/17 (watch item)
  • ruff check + ruff format --check clean; scripts/check-docs.sh → docs clean 7/7
  • uv.lock regenerated from the merged pyproject
  • Post-merge: rsync-deploy to familiar + live search/hook verification, then backfill the TBD commit hash in fork-changes.yaml

🤖 Generated with Claude Code

mvalentsev and others added 30 commits June 25, 2026 22:20
…ce#923)

The original commit printed SKIP for oversized files to stdout but the
sibling SKIP for symlinks in the same scan_project / scan_convos already
went to stderr. Align the new line with that convention.

Also adds a SKIP-with-error log for the except OSError arm right below the
size check. Files whose stat() raises (permission denied, racing delete,
broken symlink that survived the earlier is_symlink check) were the same
bug class as the silent oversize drop.

Tests switched from captured.out to .err and tightened to the full
template; new test covers the OSError arm via a selective Path.stat
monkeypatch with a follow_symlinks gate for Python 3.10+.
test(wal): cover WAL crash-safety, idempotent setup, and redaction edge paths
…e#1783) (MemPalace#1857)

daemon.py:_detached_kwargs was the last production spawn site still using DETACHED_PROCESS. Swap it to CREATE_NO_WINDOW, matching the hook miner's _detached_popen_kwargs fixed in MemPalace#1848 — the dedicated follow-up the review bot asked for. `grep -rn DETACHED_PROCESS mempalace/` now returns zero production hits.

Survivability is unchanged: CREATE_BREAKAWAY_FROM_JOB (escapes the parent Job Object's kill-on-close) plus the daemon never being attached to the launching console carry survive-terminal-close; CREATE_NEW_PROCESS_GROUP (also kept) isolates Ctrl-C/Break. CREATE_NO_WINDOW is ignored when OR'd with DETACHED_PROCESS, so this replaces the flag rather than adding it. The daemon already redirects stdout/stderr to daemon.log and reads no stdin, so it needs no console.

Adds the first tests for _detached_kwargs (posix + windows, cross-platform monkeypatch of the Windows-only flag constants, mirroring the MemPalace#1848 hooks_cli tests).
* fix: sanitize wing slug for project dirs with special characters

Project folders containing characters outside sanitize_name's set
(e.g. a leading '+') leaked into the derived wing name, producing names
like 'wing_+project' that config.sanitize_name rejects, silently
breaking diary auto-save for that project.

Add _safe_wing_slug(): collapse non-word runs to '_', trim, and fall
back to 'sessions' when a name reduces to nothing. Route the three
wing-derivation sites through it.

Tests: unit cases for the helper plus a hypothesis property test
asserting wing_<slug> always passes sanitize_name for any input.

* fix: preserve dots and apostrophes in wing slug for backward compatibility

The first pass collapsed every non-word character (including dot and
apostrophe) to underscore, renaming existing valid wings — e.g. my.app
became wing_my_app — which would orphan diary entries already filed under
the old name. Keep dot and apostrophe (both accepted by sanitize_name),
collapse consecutive dots to avoid the path-traversal rejection, and trim
edge separators.

Add backward-compatibility tests for previously-valid names plus a
double-dot collapse test.

* fix: cap wing slug length to stay within sanitize_name's limit

sanitize_name rejects names over 128 characters, so a very long project
directory name would produce a wing name that fails validation,
re-triggering the silent auto-save break this PR fixes. Truncate the slug
to 120 chars (the wing_ prefix keeps the total under 128).

Widen the hypothesis property test to max_size=300 so it exercises the
length path, and add an explicit truncation test.

Addresses gemini-code-assist review feedback on PR MemPalace#1852.

---------

Co-authored-by: Ivan Antsimonau <ivan.antsimonau@katim.com>
…ath (MemPalace#1863)

The non-daemon synchronous mine fallback in _mine_sync() spawned the
mine subprocess without CREATE_NO_WINDOW, flashing a visible console
window on every PreCompact fire on Windows. The async paths
(_spawn_mine, _desktop_toast) already pass it via _detached_popen_kwargs();
this sync path was missed. getattr(..., 0) is a no-op off-Windows.

Fixes MemPalace#1862

Co-authored-by: David Finkelstein <david@finkelstein.us>
…aceConfig.palace_path correctly called os.path.expanduser() for\nenv-var paths but not for paths read from config.json. If config.json\nstores palace_path as '~/.mempalace/palace' (the default written by\ninit), the tilde was returned unexpanded.\n\nDownstream callers such as cli.py cmd_mine did call expanduser when\n--palace was passed explicitly, but fell through to MempalaceConfig()\nwhen no flag was given, inheriting the unexpanded string. Python's\nos.makedirs and chromadb.PersistentClient treat a leading tilde as a\nliteral directory name rather than the home directory, so the palace\nwas silently written to a CWD-relative path such as\nmy_project/~/.mempalace/palace.\n\nThe fix is a single os.path.expanduser() call on line 343 of\nconfig.py, mirroring the existing env-var branch on line 342. Since\nDEFAULT_PALACE_PATH is already expanded at module load (line 197),\nexpanduser on an absolute path is a no-op, so the default case is\nunaffected.\n\nSymptoms: scattered {project}/~/.mempalace/palace directories, palace\nalways appears empty after mine, search returns Collection does not\nexist, launchd-driven nightly mine writes to a different location than\ninteractive mine.\n\nCo-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>n (MemPalace#1865)
…lace#1716)

An empty link_lists.bin is not corruption on its own: hnswlib stores the
layer-0 graph inside data_level0.bin and only writes link_lists.bin for
elements promoted to level > 0. A small/low-fanout index where every
element stays on layer 0 serializes an empty link_lists.bin and loads
fine. Flagging that shape as corrupt produced a self-perpetuating
quarantine loop — repair rebuilt the byte-identical all-layer-0 segment,
the next cold start re-quarantined it, accumulating drift dirs (221 MB in
the reported case) with no ingestion involved.

Use the persist-completion marker as the discriminator instead. ChromaDB
writes index_metadata.pickle last, so an intact pickle envelope proves the
flush finished and the empty link_lists.bin is the legitimate all-layer-0
shape. Only treat an empty link_lists.bin as a partial flush when there is
real payload AND no completion marker (absent or truncated pickle). The
MemPalace#1457 partial-flush protection (real payload, no/truncated marker) is
preserved; the byte-sniff is factored into _hnsw_metadata_marker_intact
and reused by _segment_appears_healthy.

Also fixes the related single-writer stale-quarantine false positive
(MemPalace#1564), which shares this all-layer-0 root cause.
…-quarantine

fix(chroma): stop quarantining valid all-layer-0 HNSW segments (MemPalace#1716)
…lace#1596)

Concurrent killed-mid-write mines can leave embedding_fulltext_search in a
malformed-inverted-index state that fails PRAGMA quick_check while the
underlying rows stay intact (integrity_check ok). The repair preflight then
hard-aborts before reaching the FTS5 rebuild step, so `mempalace repair`
refuses to run and full-text search stays broken — the exact loop MemPalace#1596
reports. The MineValidationError banner even promises "repair --yes rebuilds
the FTS5 virtual table automatically," which the preflight abort made false.

Add maybe_autoheal_fts5_index(): when every quick_check error is an isolated
"malformed inverted index for FTS5 table" failure, rebuild the index in place
from the intact embedding_fulltext_search_content table
(INSERT ... VALUES('rebuild')) under mine_palace_lock, then re-run quick_check.
The rebuild touches no drawer rows. Wired into both repair preflights
(rebuild_index and cli cmd_repair). Any non-FTS5 error in the set, a lock held
by a live mine, or a rebuild that does not clear quick_check leaves the errors
unchanged so the caller still aborts with the recovery banner — broader
corruption is never silently rebuilt over.
fix(repair): auto-heal isolated FTS5 inverted-index corruption (MemPalace#1596)
) (MemPalace#1885)

* fix(mcp): stop clobbering host app root logger at import (MemPalace#1860)

_init_logging() ran at import and called logging.basicConfig(force=True),
resetting the root logger's level, format, and handlers unconditionally. An
app that configured logging before importing mempalace.mcp_server lost its
setup: a host on DEBUG dropped to INFO, custom formatters and handlers were
replaced.

force=True existed (MemPalace#1495) only to keep MEMPALACE_LOG_FILE working when root
already had handlers. This keeps that contract without the reset: configure
root only when it is unconfigured (standalone); otherwise attach a
mempalace-filtered file handler additively and leave the host's config alone.

Adds _MempalaceLogFilter so the file captures every mempalace logger (the
dotted mempalace.* family plus the flat mempalace_* names) and nothing else.

* fix(mcp): survive importlib.reload and pin file log format (MemPalace#1860)

Addresses review on MemPalace#1885.

Restore _logging_configured from globals() so the idempotency guard survives
importlib.reload: a reload re-executes the module body, and a plain reset would
let _init_logging() stack a duplicate file handler on root.

Set an explicit "%(message)s" formatter on the file handler so the embedded
path does not depend on logging's default formatter (which already renders the
same, but is now pinned and identical to the standalone path).

Adds a reload regression test and a format-pin assertion.
…ments (MemPalace#1630)

L1's generate() scored drawers by importance/emotional_weight/weight, and
the docstring promised "prefer high importance, recent filing". But no
ingest path (miner, convo_miner, diary, add_drawer) writes any of those
fields, so the sort collapsed to insertion order (oldest first) and
recency was never consulted. A scoped `wake-up --wing X` therefore
surfaced the *oldest* moments: the opposite of useful.

Add filed_at (present on every drawer, ISO-8601, lexically chronological)
as the secondary sort key. Importance stays primary for the day a scoring
pass populates it; filed_at is the effective signal today, making the
"recent filing" half of the promise true with data already present.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
…ace#1648)

On Windows, Path.read_text() and open(path, 'a') use locale encoding
(GBK on Chinese-locale systems) before PEP 686 / Python 3.15. A valid
UTF-8 .gitignore with non-ASCII comments crashes
_ensure_mempalace_files_gitignored() with UnicodeDecodeError, which
aborts 'mempalace init' on Windows for any user whose .gitignore
contains non-ASCII text.

Force encoding='utf-8' on both read and append, with errors='replace'
on read as a defensive fallback for legacy mixed-encoding files.

Co-authored-by: ALaDingAhmad <16530935@qq.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](docker/setup-qemu-action@v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.18 to 0.15.20.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.15.18...0.15.20)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.15.20
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…detect() (MemPalace#1893) (MemPalace#1896)

* fix(chroma): require SQLite magic header for ChromaBackend.detect() (MemPalace#1893)

Closes MemPalace#1893.

ChromaBackend.detect() was returning True for a 0-byte chroma.sqlite3 file
because the check was just os.path.isfile(...). On a palace that has any
other backend marker alongside a stale 0-byte chroma.sqlite3,
resolve_backend_name then raises BackendMismatchError and the palace becomes
unopenable until the user manually rm's the empty file.

The 0-byte file appears as a side effect of any sqlite3.connect() on a
missing path — Python creates the file immediately but writes the SQLite
header only on the first statement. So any code path that touches the
chroma.sqlite3 path with bare sqlite3.connect(), including chromadb's own
PersistentClient lazy-init (see the comment at backends/chroma.py:2052),
can leave a 0-byte artifact behind.

Fix: detect() now reads the first 16 bytes and compares to the SQLite
magic prefix b"SQLite format 3\x00" instead of relying on file presence
alone. One extra open() + 16-byte read; detect() isn't a hot path.

Properties:
- Rejects 0-byte files (the symptom MemPalace#1893 is about).
- Rejects non-SQLite garbage at the canonical path (partial writes, etc.).
- Doesn't false-negative on real chroma palaces: any chroma palace whose
  PersistentClient has done any work has the magic header on disk
  (verified — CREATE TABLE is enough to land the header).
- Doesn't couple detect() to chroma's specific schema; the magic header
  is stable across chromadb releases.

Test sweep: many test files used (chroma.sqlite3).touch() or
.write_bytes(b"") as a "fake palace" shortcut, exploiting the loose
isfile() check (one such site even had the comment "# pass the isfile
guard"). After this change, those stand-ins no longer register as chroma
palaces. Introduced tests/_chroma_palace_helper.py::make_minimal_chroma_sqlite
following the existing _backend_conformance.py precedent, and updated 15
call sites across 8 test files to use it. The existing
test_chroma_detect_matches_palace_with_chroma_sqlite (which encoded the
buggy semantics with write_bytes(b"")) is renamed to
test_chroma_detect_matches_palace_with_sqlite_header and now writes a
real SQLite database via the helper. Added two new tests for the
rejection paths (empty file, non-SQLite garbage).

Full env-cleared suite: 3137 passed, 20 skipped, 0 failed. ruff check
and ruff format --check both clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA

* fix(sqlite_exact): require SQLite magic header for SQLiteExactBackend.detect()

Per gemini-code-assist review on MemPalace#1892 PR MemPalace#1896: SQLiteExactBackend has the
same os.path.isfile() detection pattern as ChromaBackend did, with the same
0-byte-file vulnerability. Mirrors the chroma fix for repo-wide consistency.

- SQLiteExactBackend.detect() now does the same 16-byte SQLite magic-prefix
  check as ChromaBackend.detect().
- _chroma_palace_helper.py: factored its body into a private
  _write_minimal_sqlite_file() and gained a sibling
  make_minimal_sqlite_exact_sqlite() for the sqlite_exact filename. No churn
  to any existing chroma call sites.
- test_sqlite_exact_backend.py:426 (the one site that wrote b"" for
  sqlite_exact.sqlite3) updated to use the new helper.
- Three new tests in test_sqlite_exact_backend.py mirror the chroma trio:
  matches with valid header, rejects empty file, rejects non-SQLite garbage.

Full env-cleared suite: 3140 passed, 20 skipped, 0 failed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…ace#1840 follow-up) (MemPalace#1892)

* fix(pgvector): skip document column for metadata-only fetches (MemPalace#1840 follow-up)

Closes the explicit "separate follow-up to keep this low-risk" callout
in PR MemPalace#1840's description.

For remote pgvector deployments (TLS over WAN), `mempalace_status` and
every other metadata-only consumer was transferring the full `document`
column over the wire even when nothing read it. A single scroll over a
177K-drawer palace on a 175 ms-RTT link moved ~150 MB of document text
plus ~50 MB of metadata; this PR drops that to ~50 MB.

scroll_rows / _scroll gain `with_document: bool = True`. When False,
SELECT projects NULL::text instead of the document column. Positional
_row parser unchanged (record[1] stays the document slot, just receives
NULL). Existing callers default to True and see byte-for-byte identical
behavior.

PgVectorCollection.get_all_metadata override: where=None path goes
single-scroll with with_document=False. Filtered path falls back to base
to keep _matches_where running on array/object metadata values (same
correctness contract as MemPalace#1840's filtered-path decision).

Tests:
- Update _FakePgVectorClient.scroll_rows to accept with_document; mirror
  the NULL-becomes-empty-string semantics when False
- Update 5 existing scroll_calls assertions to include with_document=True
  (unchanged intent)
- test_pgvector_get_all_metadata_skips_document_column: assert exactly
  one scroll call with with_document=False
- test_pgvector_get_all_metadata_filtered_falls_back_to_base: assert
  filtered path preserves with_document=True

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA

* fix(pgvector): extend with_document=False fast path to filtered get_all_metadata

Per gemini-code-assist review feedback on MemPalace#1892: _matches_where only reads
metadata, so the where=None vs where=set conditional fall-back was unnecessary.
The filtered path can use the same single-scroll with_document=False fast path
and apply the post-filter locally on metadata dicts — extending the wire-byte
win to every get_all_metadata caller, not just unfiltered ones.

Mirrors the pushdown + local _matches_where pattern already used by _rows
in the same file: pushdown when _requires_local_filter is False, post-filter
in Python otherwise. Same correctness contract as MemPalace#1840's filtered get path.

Renames test_pgvector_get_all_metadata_filtered_falls_back_to_base to
test_pgvector_get_all_metadata_filtered_uses_fast_path and asserts the new
behavior (with_document=False + pushdown forwards the equality filter to SQL).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…1890)

* feat(convo): preserve authored timestamp from transcripts

Conversation drawers only carried `filed_at` (ingest time), so a bulk
re-mine collapsed every drawer to a single instant and the chronological
signal was lost — even though each Claude Code / Codex JSONL line already
carries an ISO-8601 `timestamp`. The recency-window fallback and any
date-aware consumer then saw ingest order, not when content was written.

- convo_miner: derive `authored_at` (per-file max line `timestamp`) and
  store it as drawer metadata; falls back to `filed_at` when absent
- searcher: surface `authored_at` in search results, and break exact
  hybrid-score ties toward the more recently authored drawer (ISO strings
  sort chronologically; missing dates sort oldest) — benchmark-neutral as
  it only reorders exact ties
- tests: cover `_extract_authored_at` (latest wins, skips/tolerates lines
  without timestamps, non-jsonl/missing -> None) and the tie-break

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(search): surface authored_at in CLI + backfill for existing data

Completes the authored_at work so the field is visible end-to-end and
existing palaces can adopt it without re-mining.

- layers: CLI `search` output shows an `authored:` date line per result
  (peer of the existing date; markdown drawers fall back to filed_at)
- scripts/backfill_authored_at.py: in-place migration that stamps
  authored_at on convos drawers from their source transcripts — metadata
  only (no re-embedding), idempotent, dry-run by default
- docs/authored-at.md: documents created_at (ingest) vs authored_at
  (written) and both backfill paths (in-place / drop-and-recreate)
- tests: backfill integration tests over an ephemeral ChromaDB collection

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(search): address review — non-string timestamp guard + top-level authored_at tiebreak

Two correctness fixes from the PR review:

- _extract_authored_at: only compare when the parsed `timestamp` is a str.
  A non-string timestamp on a malformed/foreign JSONL line previously raised
  TypeError outside the try and could crash the mine.
- _hybrid_rank: the tie-break read `authored_at` only from nested `metadata`,
  but the search_memories path (MCP / Claude Code) carries it at the top level
  of each hit — so the tie-break silently no-op'd there. Read both shapes.
- tests: non-string timestamp cases, and a top-level-shape tie-break test
  (which fails before this fix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* style: apply ruff format to authored_at changes

CI ruff format --check flagged 4 files; ruff check already passed.
Formatting only — no behavior change.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
…ansport can write (MemPalace#1859)

* fix(palace): process-wide mine_palace_lock re-entrancy for threaded HTTP transport

The MCP HTTP transport (ThreadingHTTPServer) acquires the long-lived
writer-lease on one thread (_acquire_mcp_writer_lock) but dispatches each
write request on a different worker thread. The lock re-entrancy guard was
thread-local, so write handlers (add_drawer/update_drawer) failed to see the
process-held lease, re-acquired the flock, and self-conflicted with
"palace ... is held by PID <self>". Reads worked (no lock); writes over the
HTTP transport were impossible.

Make the re-entrancy record process-wide (pid-tagged, guarded by a
threading.Lock) so a write from any thread of the process that already holds
the lease passes through. Safe: flock is per-process and HTTP writes are
serialized by _HTTP_REQUEST_LOCK. Preserves fork-safety, same-thread nesting
(miner.mine -> ChromaCollection.upsert), and cross-process protection
(MineAlreadyRunning still raised between processes).

Add cross-thread same-process regression test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(palace): reset lock guard on fork to avoid inherited-locked deadlock

Address review (PR MemPalace#1859): `_palace_lock_guard` is a threading.Lock, so a child
forked while another thread held it would inherit it locked (the holder thread
is gone in the child) and deadlock on the next acquire. Register an
os.register_at_fork(after_in_child=...) handler that replaces the guard with a
fresh unlocked lock and clears state; the child must reacquire the flock anyway.
Guarded by hasattr(os, "register_at_fork") for Windows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
…) (MemPalace#1891)

* feat(mcp): add since/before date filter to list_drawers (MemPalace#1128)

mempalace_list_drawers previously filtered only by wing/room. This adds
optional since/before ISO date bounds on filed_at: since is inclusive,
before is exclusive.

The filter runs in Python after the rows are fetched. ChromaDB 1.5.7
rejects string operands for $gte/$lt and filed_at is stored as an ISO
string, so a server-side where comparison is not available; the tool
already collapses and paginates the full result set in Python.

Drawers whose filed_at is missing or unparseable are excluded while a
bound is active, and inverted bounds (since >= before) return a clear
error.

* test: close chromadb clients between tests to fix Windows handle leak (MemPalace#1128)

chromadb 1.5.7 caches one System per palace path and only frees the
SQLite/HNSW file handles on client.close(); the collection fixture and
the per-test MCP cache reset only dereferenced the client, so handles
leaked across the session. Harmless on POSIX (rmtree unlinks open files),
but on Windows the handles stay locked and accumulate until an HNSW
segment write in a later test's setup fails, which surfaced here as
TestDeleteBySource::test_commit_purges_matching_closets asserting 0 == 2.

Close the client in the collection fixture and in _reset_mcp_cache so the
handles are released between tests.

* test: release backend chromadb clients between tests (MemPalace#1128)

palace.get_collection() caches one PersistentClient per palace_path on the
process-wide backend singleton and never closes it; sweep, repair and several
CLI tests reach the store through it. chromadb frees the rust-side SQLite/HNSW
file handles only on client.close(), so the handles leak across the whole
session: a 30-palace probe shows ~200 open file descriptors into the palace
tree, dropping to 0 once the clients are closed.

On POSIX the open handles are harmless (rmtree unlinks open files), but on
Windows they stay locked and accumulate until a later test's HNSW segment
write fails ("Failed to apply logs to the hnsw segment writer"), e.g.
test_sweeper.py::TestSweeperTandem::test_sweep_recovers_untaken_message_at_cursor_timestamp.

Drain the cached clients in the autouse _reset_mcp_cache teardown via
close_palace(), which closes each PersistentClient (releasing its handles)
without marking the backend closed so it stays reusable. Complements the
collection-fixture and _client_cache close() added earlier.
…Palace#1868)

* feat: add metadata facet support for qdrant

* added benchmark

* updated benchmark

* chore: remove tracking for local scratch benchmark

* feat: add metadata facet support for qdrant -clean

* Update mempalace/mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update mempalace/backends/qdrant.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update mempalace/backends/qdrant.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_qdrant_backend.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_qdrant_backend.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_qdrant_backend.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* /fix always working tool_status() fallback fixed

* /fix fallback added to tool_list_rooms

* Update mempalace/mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update mempalace/mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_qdrant_backend.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* /fix rebuilt the room populating logic

* /add added temporary files for atomic transactions

* Update mempalace/mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update mempalace/mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* /fix ai slop

* /fix added default facet limit

* Update tests/test_qdrant_backend.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* /fix added max workers pool

* Update mempalace/mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update mempalace/mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update mempalace/backends/qdrant.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* /fix added clear()

* Update tests/test_mcp_server.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix(qdrant): validate facet filter before existence check; fix taxonomy test

- facet_counts now validates the where filter and rejects local-only
  filters before the _remote_exists() short-circuit, so an unsupported
  filter raises UnsupportedCapabilityError even on an unmaterialized
  collection (matches get()/lexical_search() ordering).
- test_tool_get_taxonomy_uses_metadata_facets compared concurrent room
  facet calls via set(), but a call() with a dict kwarg is unhashable;
  compare order-independently via membership instead.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
…emPalace#1895)

* feat(graph): auto-populate the associative graph from mined sessions

Conversation mining never set the `entities` drawer metadata that hallways
consume, so mined sessions produced an empty associative graph (and starved
the entity-navigation / tunnel-recommendation features built on top of it).

Add a no-LLM structural entity extractor and wire it into the convos mine:

- entities: structural-only extractor (author-quoted code spans, URLs, file
  paths, qualified identifiers, CamelCase / snake_case symbols). No wordlists,
  no NLP models, precision-biased so prose doesn't pollute the graph.
- convo_miner: set `entities` per chunk, and compute hallways after a convos
  mine (mirroring the project-file path). Hallways run before the FTS5
  validation, which opens a direct sqlite connection that can invalidate the
  live Chroma collection handle on some Chroma builds.
- cli: `mempalace hallways` lists the associative graph (CLI parity with the
  list_hallways MCP tool).
- tests: extractor precision/ranking, entities metadata at mine time, CLI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(graph): address review — semicolon safety, leading-underscore snake, negative limit

- entities `_clean`: strip `;` out of tokens so a URL query string or backtick
  span can't split the `;`-joined entities metadata field
- entities `_SNAKE`: optional leading/trailing `_?` so `_extract_authored_at`
  and similar are matched in plain text (previously only caught via backticks)
- cli `hallways`: clamp `--limit` with max(0, ...) so a negative value shows
  nothing instead of slicing from the end
- tests for all three

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
… (MemPalace#1897)

Documents running MemPalace as a central memory service for a team:
HTTP MCP transport (--transport http with bearer-token auth), a
networked backend (Qdrant via REST, no extra dep; or pgvector), and
optional GPU embedding. Covers the security model (non-loopback token
requirement, Host/Origin DNS-rebinding guard, TLS-in-front), client
connection, and operating notes. Adds the page to the guide sidebar.

Addresses MemPalace#1877.
…llection

Both methods are concrete on ``BaseCollection`` (``facet_counts`` raises
``UnsupportedCapabilityError``; ``get_all_metadata`` pages through
``self.get(include=["metadatas"])``). Python MRO resolves them on
``EmbeddingCollection`` before ``__getattr__`` ever fires, so without an
explicit forwarder the wrapper silently runs the base default instead of
delegating to the wrapped backend's optimized implementation. The pattern
matches the existing explicit forwarders for ``distance_metric``,
``lexical_search``, and the embedder-identity trio — all added to fix the
same shadow.

What this means in production for the three backends that get wrapped
(``EmbeddingCollection`` only applies to ``requires_explicit_embeddings``
backends — qdrant, pgvector, sqlite_exact; chroma is unwrapped and
unaffected):

- **facet_counts shadow (MemPalace#1868 regression)**: every ``mempalace_status``,
  ``list_wings``, ``list_rooms``, ``get_taxonomy`` call routes through
  the gated ``col.facet_counts(...)`` path. The capability check passes
  (``supports_metadata_facets`` is on the backend), but the call hits the
  wrapper's MRO-resolved ``BaseCollection.facet_counts`` and raises
  ``UnsupportedCapabilityError``. ``mcp_server``'s broad ``except`` swallows
  it, logs ``WARN Failed to fetch metadata facets, falling back to client-
  side loop: backend does not support facet_counts``, and counts via the
  O(n) Python loop — the exact behavior MemPalace#1868 was designed to eliminate.

- **get_all_metadata shadow (MemPalace#1796 / MemPalace#1892 regression)**: the BaseCollection
  default pages through ``self.get(include=["metadatas"])`` — fine for
  Chroma's SQL OFFSET cursor, but on wrapped backends (qdrant, pgvector)
  the inner's overridden ``get_all_metadata`` is unreachable. For pgvector
  specifically, this means MemPalace#1892's ``with_document=False`` fast path is
  never taken even though it's implemented — every metadata-only fetch
  transfers the full document column over the wire. On a 13k-drawer remote
  pgvector palace over WAN that's ~13MB per call, dominating wall time.

Why no test caught it: backend tests (``test_qdrant_backend.py``,
``test_pgvector_backend.py``) call the methods directly on the raw
collection, not through the wrapper. ``test_mcp_server.py`` facet tests
use ``MagicMock()`` for the collection, which synthesizes attributes on
demand and bypasses MRO entirely. Neither path covers the seam where the
bug lives: ``palace.get_collection() -> EmbeddingCollection -> .method()``.

Three tests pin both the fix and the bug class:

- ``test_facet_counts_forwards_to_inner`` — direct integration through the
  wrapper, asserts the inner's recorded call matches.
- ``test_get_all_metadata_forwards_to_inner`` — same shape, plus a sentinel
  ``get()`` on the inner so a missing forwarder would route to the base
  default and pick up the wrong data (observable failure, not silent).
- ``test_wrapper_forwards_all_concrete_basecollection_methods`` — meta-test
  that enumerates every concrete public method on ``BaseCollection`` via
  ``inspect.getmembers`` and asserts each one is explicitly defined on
  ``EmbeddingCollection``. Catches the bug *class*: any future
  ``BaseCollection`` method with a concrete default body becomes a CI
  failure the moment it's added without a wrapper forwarder, with a message
  pointing straight at the file to edit.

Full env-cleared suite: 3205 passed, 20 skipped. ``ruff check`` and
``ruff format --check`` both clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA
Per Gemini review (PR MemPalace#1898 comment r3489013681): the forwarder lacked the
``-> dict[str, int]`` return annotation that ``BaseCollection.facet_counts``
and the sibling ``get_all_metadata`` forwarder both carry. One-line
consistency fix, no behavior change.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA
…lace#1900)

* feat(serve): turnkey secure remote MCP server (MemPalace#1877)

Add `mempalace serve`: a secure-by-default wrapper over the HTTP MCP
transport so a team can stand up a shared central palace with one
command.

Server capabilities (mempalace/mcp_server.py):
- Native TLS via --tls-cert/--tls-key (env MEMPALACE_MCP_TLS_CERT/_KEY):
  wraps the socket in a TLS 1.2+ context, validated before bind. Token
  is still required on a non-loopback bind (TLS != auth).
- Read-only mode via --read-only (env MEMPALACE_MCP_READ_ONLY): the 24
  mutating tools are hidden from tools/list and refused at dispatch
  (-32003), enforced before arg handling — not merely hidden.

Turnkey command (mempalace/cli.py):
- Auto-generates a strong bearer token for non-loopback binds, stored
  0600 under ~/.mempalace/server/ and printed once; reused across
  restarts. Token rides in the child env, never argv, so it can't leak
  via ps.
- Prints a ready-to-paste client config (scheme reflects TLS), then
  foreground-execs the real server so Docker/systemd own the lifecycle.

Deployment (deploy/):
- docker-compose.server.yml wires the server + Qdrant with a /healthz
  healthcheck and persistent volumes.
- server.env.example documents the env surface.
- mempalace-server.service is a hardened systemd unit template.

Tests: TLS handshake (openssl-gated), read-only enforcement, token
autogen/0600/reuse, token-not-in-argv, secure-by-default gates.

Docs: remote-server guide now leads with `mempalace serve` plus Compose
and systemd subsections.

* test(serve): fix Windows — don't patch os.name; gate 0600 asserts to POSIX

Patching os.name to 'posix' broke Path.home() on Windows (pathlib mixed
POSIX home resolution with Windows drive parsing). Capture both exec
branches (os.execve + subprocess.run) instead, and guard the POSIX
permission-bit assertions behind os.name == 'posix' (Windows files
report 0o666).
LaTeX source files and BibTeX bibliographies are prose-rich content that
benefits from both palace mining and entity detection. Adds the two
extensions to the two extension lists most relevant to them, each with a
matching test.

- ``mempalace/miner.py:READABLE_EXTENSIONS`` — ``.tex`` / ``.bib`` join the
  mining allowlist (parallel to the Swift/Kotlin PR MemPalace#1368 and the PHP
  ecosystem PR MemPalace#1819).

- ``mempalace/entity_detector.py:PROSE_EXTENSIONS`` — ``.tex`` / ``.bib``
  also join the *preferred* entity-detection bucket alongside ``.md`` /
  ``.rst`` / ``.csv``, NOT the broader code-file fallback. The reason
  ``PROSE_EXTENSIONS`` exists separately is documented in-code:
  programming-language files have lots of capitalized identifiers (class
  names, function names) that produce false-positive person matches.
  LaTeX/BibTeX don't have that problem — they're typesetting languages
  for prose documents. ``.bib`` in particular is almost entirely author
  names, one of the highest real-entity densities of any file type the
  detector scans.

Tests follow the patterns established by the prior extension PRs:
``tests/test_miner.py::test_scan_project_includes_latex_files`` mirrors
the Swift/Kotlin scan tests, and
``tests/test_entity_detector.py::test_scan_for_detection_includes_latex_prose``
mirrors ``test_scan_for_detection_finds_prose``. The existing
``test_prose_extensions`` was extended to assert the two new entries.

Full env-cleared suite: 3216 passed, 20 skipped. ``ruff check .`` and
``ruff format --check .`` both clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA
igorls and others added 20 commits August 7, 2026 09:23
The dependabot bump moved pyproject's ruff pin to 0.16.1, but nothing else
followed it, so the repo asked for three different versions at once:

- pyproject.toml said 0.16.1 (the bump)
- uv.lock still resolved 0.15.20 (dependabot did not update it)
- .github/workflows/ci.yml installed 0.15.14 by its own literal pin

The lint job never reads pyproject, so CI kept linting with 0.15.14 and
reported this PR green without 0.16.1 ever running. The ci.yml pin had
already drifted from pyproject before this bump, under a comment saying to
keep them identical.

- ci.yml: pin 0.16.1 to match pyproject.
- uv.lock: regenerated so the locked resolution agrees.
- test_ruff_pins_match: assert ci.yml and pyproject stay equal, so the next
  bump that touches only one of them fails loudly instead of passing blind.
- extend-exclude '*.md': 0.16 began formatting Python inside markdown
  fences, taking the formatter from 199 files to 295 and reflowing
  hand-aligned example code in docs/rfcs/002 and three website pages.
  Excluding docs keeps this a version bump rather than a silent
  documentation reflow, and restores the exact file scope the project has
  always formatted.

Verified with 0.16.1 actually installed: ruff check and ruff format --check
both clean over the same 199 files, full suite green (3669 passed).
feat: agent logstream — coordination for a multi-machine agent fleet (RFC 003)
…0.16.1

chore(deps-dev): bump ruff from 0.15.20 to 0.16.1
…2185)

`mempalace_diary_write` returns an `entry_id` for every diary entry, but
for entries large enough to be chunked that id was unusable: get_drawer,
update_drawer and delete_drawer all answered "Drawer not found", and
list_drawers showed the entry as N unrelated chunk rows.

Two metadata conventions never met. The diary chunking path stamped
`parent_entry_id` on each chunk, while the logical-id read paths added in
MemPalace#1782 query only `parent_drawer_id`. Both keys mean the same thing --
"physical chunk of this logical drawer" -- so chunk groups written by
diary_write were invisible to logical-id resolution. Same bug class as
MemPalace#1763, which MemPalace#1782 fixed for `add_drawer` drawers only.

Read paths now resolve either key via `_PARENT_ID_KEYS`:

- `_logical_chunk_group()` matches both with an `$or` (fixes get /
  update / delete). All four backends support `$or`.
- `_collapse_drawer_rows()` groups on either (fixes list_drawers, which
  the `$or` alone does not cover).
- `searcher._result_drawer_id()` resolves either, so a hit on a chunked
  diary entry reports the id that fetches the whole entry rather than
  the single chunk that matched.

New diary writes also stamp `parent_drawer_id` alongside
`parent_entry_id` so the two conventions converge going forward. Because
the read paths still accept the `parent_entry_id`-only shape, palaces
written before this fix are repaired with no data migration.

Diary chunks are written without `source_file`, so neighbor expansion
(MemPalace#1580) returns early on them and is unaffected by the added key.

Also drops the comment telling callers to iterate `chunk_ids` (it
documented the bug as intended behavior) and a stale claim that search
rejoins chunks via `parent_entry_id` -- no search code read that key.
Chroma declares `requires_explicit_embeddings`, so every write on the
default backend routes through `EmbeddingCollection`. `_embed_texts`
built its rows with `list(v)`, and `v` is a float32 `np.ndarray` — that
unpacks into `np.float32` *scalars*, which chromadb's
`normalize_embeddings` rejects:

    ValueError: Expected embeddings to be a list of floats or ints, a
    list of lists, a numpy array, or a list of numpy arrays

`mine` aborted on the first drawer, as did every other write against a
default palace. Convert with `.tolist()` (C-speed), keeping a
`float(x)` branch for embedders that already return plain sequences.

The suite could not see this. conftest's autouse
`_stable_embedding_function_for_tests` monkeypatches
`embedding_wrapper._embed_texts` itself for every module outside
`_REAL_EMBEDDING_TEST_MODULES`, so the defective function was never
executed under test. The regression tests therefore go in
`test_embedding.py`, which is exempt from that stub: one asserts the
returned elements are builtin floats, one drives a real Chroma
collection through `EmbeddingCollection.upsert` and reads the document
back. Both fail against the previous line with the production
ValueError.

Verified end to end outside the suite: mining a project and searching
it back returns the drawer verbatim, on the host and in the container
image built from this tree.
`environment:` was present with nothing but comments beneath it, so YAML
parsed it as null and Compose refused the whole file:

    services.mcp.environment must be a mapping

That is every documented Compose command — `docker compose build`,
`docker compose run --rm mcp`, `docker compose down` — failing before
anything starts, on any machine. The README points at this file.

Comment the key out along with its example entries, and say in the file
why it cannot be left bare. The examples switch to mapping syntax so
uncommenting them yields a valid block.

Verified with `docker compose config` on the shipped file (now valid)
and by running the documented flow against a build of this tree: mine a
mounted directory through `docker compose run --rm mcp cli mine`, then
read it back with `... cli search`, which returns the drawer verbatim.
`deploy/docker-compose.server.yml` was checked for the same defect and
is unaffected.
…-numpy-floats

fix(backends): convert embedding vectors to Python floats before upsert
…ironment-block

fix(docker): drop the null `environment:` block that invalidated compose
The Docker workflow built both images and never started a container,
and never parsed a Compose file. A green run therefore only meant the
Dockerfile compiled. Two defects that break the very first documented
command shipped past it: `docker-compose.yml` carried a bare
`environment:` key that made Compose reject the file outright (MemPalace#2188),
and `_embed_texts` handed chromadb `np.float32` scalars so `mine`
aborted on the first drawer (MemPalace#2187).

Add `scripts/docker-smoke.sh`, which exercises what the README tells
users to run:

  1. `compose config` on docker-compose.yml and the server compose file
  2. entrypoint dispatch for both `cli ...` and bare passthrough
  3. `mine` a mounted directory, asserting a drawer is filed
  4. `search` from a *separate* container, asserting the stored text
     comes back verbatim — this is the assertion that matters, since
     storing user words exactly is the promise the palace makes
  5. a real MCP stdio JSON-RPC handshake: initialize, tools/list, and a
     mempalace_search call whose result must contain the drawer

It asserts on returned content, not just exit codes, and lives in a
script rather than inline YAML so it runs identically on a laptop:
`scripts/docker-smoke.sh <image>`.

The new `smoke` job builds amd64 natively with `load: true` (buildx
cannot load a multi-arch manifest) and reads the publish job's cache
while writing its own scope, so an amd64-only export never lands on top
of the multi-arch one. `build` now needs it, so a failing smoke test
blocks publication rather than being noticed afterwards.

Verified by reintroducing each defect against a real build: the compose
regression fails at step 1, the embedding regression at step 3, and the
current tree passes all five. Failure output is clipped to 500 columns
because a rejected embedding batch otherwise prints a whole 384-dim
vector on one line and buries the message.
The first CI run failed at step 3 with

    PermissionError: [Errno 13] Permission denied: '/work/mempalace.yaml'

`mktemp -d` creates the fixture 0700 owned by the runner user. Bind
mounts carry host ownership through unchanged, and the image runs as
uid 1000, so the container could not stat inside /work. Docker
Desktop's uid mapping hides this on macOS, which is why it passed
locally and only failed on Linux.

Model an ordinary project checkout instead — 0755 dir, 0644 file — which
is the shape that makes the README's `-v /path/to/project:/work` work
against a normal repo.
…-logical-id

fix(mcp): resolve chunked diary entries by their entry_id (MemPalace#2185)
ci(docker): run the image before publishing it
The section only ever documented `docker build`, so every reader
compiled the image locally even though `ghcr.io/mempalace/mempalace` is
published multi-arch. Worse, a clone builds `develop` (the default
branch), not the release — a build and a pull could hand you different
versions with no hint that they differ.

Lead with `docker pull`, and cover the things that actually cost people
an evening:

- the container only sees what you mount, so the MCP client config now
  mounts a transcripts directory; without it the server starts fine and
  every mine finds nothing
- paths become container paths after that, and `~` / `$HOME` are not
  expanded by every MCP client
- the first embedding call downloads ~80 MB (minilm) or ~300 MB
  (embeddinggemma) into /data, which reads as a hung container
- bind mounts keep host ownership and the image runs as uid 1000, so a
  0700 directory fails with a bare PermissionError on Linux; Docker
  Desktop's uid mapping hides this on macOS and Windows. `--user` is
  called out as the wrong fix — /data is mode 700 owned by uid 1000, so
  another uid cannot write the palace at all
- mining never writes to its source, so the examples mount it read-only
- the GPU image is x86_64-only; onnxruntime-gpu has no aarch64 Linux
  wheels, so that build fails on Apple Silicon

Every command in the section was run as written against the published
image, and each claim checked rather than assumed: the uid and mode of
/data read out of the image, `--user` confirmed to break a *fresh*
volume, the read-only mount confirmed to still mine, and the aarch64 GPU
failure reproduced.
…-mounts

docs(readme): lead the Docker section with the published image
…flict files

Second large sync (first: b46f18d, post-v3.5.0). Brings v3.6.0+v3.7.0:
mempalace serve hub + hook forwarding, temp-collection repair promote,
round-trippable drawer IDs (MemPalace#2090), kg_supersede, CLI HNSW-divergence
fence, per-conversation dedup (normalize_conversations), daemon
lock-contention retry (MemPalace#2029), repair --dry-run for --mode from-sqlite
(MemPalace#2138), Milvus backend.

Fork-preserved through the 62 conflicts (resolution: per-file 3-way
replay of the smaller delta onto the larger side, then composition):

- postgres/pgvector/AGE dispatch incl. the 96f83d7 union-merge guard,
  now composed WITH upstream's drawer_id field (TestUnionPostgresPath
  green); _coerce_wing guard intact
- daemon-strict hook routing nested inside upstream's one-probe
  _hook_write_routing_context; fork silent-save block re-spliced;
  session-start = upstream routing warn + fork queue warn
- hook_verbatim_mode threaded through upstream's restructured
  normalize/normalize_conversations/convo_miner
- MemPalace#1367 embeddings-preserve ported onto the new temp-collection repair
  flow (4-tuple _extract_drawers, embeddings reused on promote/rebuild)
- adaptmem_ft dispatch (+ conftest real-embedding opt-out) and an
  explicit rename_wing forward on EmbeddingCollection
- fork CLI surface (daemon-routed tunnels/why/rooms) + upstream's
  --palace scoping on cmd_hallways + upstream hub-forward composed
  into cmd_mine after fork daemon-strict routing
- precompact adopts upstream's active-transcript-only mining
- MCP surface now 48 tools; all current-state doc/manifest counts
  reconciled (check-docs excludes FORK_CHANGELOG + dated specs, which
  state counts as of their date); version 3.7.0; uv.lock regenerated

Tests: 5609 collected; full suite green (5522 passed, 79 skipped,
122 deselected). ruff check + format clean. check-docs 7/7.

Fixes #383.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 9, 2026 21:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 167 files, which is 67 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7ea7a0f8-ca55-486a-b79b-f8faecd7f77d

📥 Commits

Reviewing files that changed from the base of the PR and between 56e3001 and 4f7a004.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (167)
  • .claude-plugin/README.md
  • .claude-plugin/hooks/hooks.json
  • .claude-plugin/marketplace.json
  • .claude-plugin/plugin.json
  • .codex-plugin/README.md
  • .codex-plugin/plugin.json
  • .cursor-plugin/README.md
  • .cursor-plugin/marketplace.json
  • .cursor-plugin/plugin.json
  • .github/workflows/ci.yml
  • .github/workflows/docker-publish.yml
  • CHANGELOG.md
  • CLAUDE.md
  • FORK_CHANGELOG.md
  • README.md
  • docker-compose.yml
  • docs/fork-changes.yaml
  • docs/hook-write-routing.md
  • docs/integrations/opencode.md
  • docs/rfcs/003-agent-logstream-coordination.md
  • docs/rfcs/004-replicated-palace.md
  • docs/rfcs/005-agent-identity-routing.md
  • docs/write-routing-policy.md
  • integrations/openclaw/SKILL.md
  • integrations/shared/coordination-protocol.md
  • integrations/shared/recall-protocol.md
  • lychee.toml
  • mempalace/README.md
  • mempalace/backends/__init__.py
  • mempalace/backends/chroma.py
  • mempalace/backends/embedding_wrapper.py
  • mempalace/backends/milvus.py
  • mempalace/backends/registry.py
  • mempalace/backends/sqlite_exact.py
  • mempalace/cli.py
  • mempalace/closet_llm.py
  • mempalace/config.py
  • mempalace/convo_miner.py
  • mempalace/daemon.py
  • mempalace/dedup.py
  • mempalace/embedding.py
  • mempalace/encoding_repair.py
  • mempalace/entity_detector.py
  • mempalace/format_miner.py
  • mempalace/hallways.py
  • mempalace/hlc.py
  • mempalace/hooks_cli.py
  • mempalace/i18n/__init__.py
  • mempalace/i18n/ja.json
  • mempalace/i18n/zh-CN.json
  • mempalace/knowledge_graph.py
  • mempalace/layers.py
  • mempalace/logstream.py
  • mempalace/logsync.py
  • mempalace/mcp_server.py
  • mempalace/migrate.py
  • mempalace/miner.py
  • mempalace/normalize.py
  • mempalace/palace.py
  • mempalace/palace_graph.py
  • mempalace/repair.py
  • mempalace/replica.py
  • mempalace/searcher.py
  • mempalace/server_registry.py
  • mempalace/service.py
  • mempalace/transport.py
  • mempalace/version.py
  • mempalace/write_routing.py
  • pyproject.toml
  • scripts/check-docs.sh
  • scripts/docker-smoke.sh
  • scripts/mempalace_repair_encoding.py
  • skills/mempalace-recall/SKILL.md
  • skills/mempalace/SKILL.md
  • tests/conftest.py
  • tests/test_backends.py
  • tests/test_clean_nul_bytes.py
  • tests/test_cli.py
  • tests/test_cli_hallways.py
  • tests/test_cli_logstream.py
  • tests/test_closet_llm.py
  • tests/test_closets.py
  • tests/test_config.py
  • tests/test_convo_miner.py
  • tests/test_convo_miner_unit.py
  • tests/test_daemon.py
  • tests/test_dedup.py
  • tests/test_embedder_identity.py
  • tests/test_embedding.py
  • tests/test_embedding_wrapper.py
  • tests/test_embeddinggemma.py
  • tests/test_encoding_repair.py
  • tests/test_entity_detector.py
  • tests/test_format_miner.py
  • tests/test_hallways.py
  • tests/test_hnsw_capacity.py
  • tests/test_hook_write_routing.py
  • tests/test_hooks_cli.py
  • tests/test_hub_forward.py
  • tests/test_hybrid_search.py
  • tests/test_i18n.py
  • tests/test_init_gitignore_protection.py
  • tests/test_knowledge_graph.py
  • tests/test_layers.py
  • tests/test_logstream.py
  • tests/test_logstream_sse.py
  • tests/test_logstream_sync.py
  • tests/test_mcp_http_transport.py
  • tests/test_mcp_logstream.py
  • tests/test_mcp_server.py
  • tests/test_mcp_server_eof.py
  • tests/test_migrate.py
  • tests/test_milvus_backend.py
  • tests/test_miner.py
  • tests/test_miner_fts5_validation.py
  • tests/test_normalize.py
  • tests/test_palace.py
  • tests/test_palace_graph_tunnels.py
  • tests/test_palace_locks.py
  • tests/test_repair.py
  • tests/test_repair_lock_safety.py
  • tests/test_search_drawer_id.py
  • tests/test_searcher.py
  • tests/test_sqlite_exact_backend.py
  • tests/test_transport.py
  • tests/test_vector_write_gate.py
  • tests/test_version_consistency.py
  • tests/test_write_routing.py
  • website/.vitepress/api-sidebar.json
  • website/.vitepress/config.mts
  • website/concepts/agent-logstream.md
  • website/guide/claude-code.md
  • website/guide/configuration.md
  • website/guide/mcp-integration.md
  • website/guide/openclaw.md
  • website/guide/remote-server.md
  • website/guide/shared-brain.md
  • website/public/llms-full.txt
  • website/reference/cli.md
  • website/reference/mcp-tools.md
  • website/reference/modules.md
  • website/reference/python-api/backends/chroma.md
  • website/reference/python-api/backends/embedding_wrapper.md
  • website/reference/python-api/backends/milvus.md
  • website/reference/python-api/cli.md
  • website/reference/python-api/config.md
  • website/reference/python-api/daemon.md
  • website/reference/python-api/dedup.md
  • website/reference/python-api/encoding_repair.md
  • website/reference/python-api/hallways.md
  • website/reference/python-api/hlc.md
  • website/reference/python-api/hooks_cli.md
  • website/reference/python-api/index.md
  • website/reference/python-api/knowledge_graph.md
  • website/reference/python-api/logstream.md
  • website/reference/python-api/logsync.md
  • website/reference/python-api/mcp_server.md
  • website/reference/python-api/miner.md
  • website/reference/python-api/normalize.md
  • website/reference/python-api/palace.md
  • website/reference/python-api/palace_graph.md
  • website/reference/python-api/repair.md
  • website/reference/python-api/replica.md
  • website/reference/python-api/searcher.md
  • website/reference/python-api/server_registry.md
  • website/reference/python-api/transport.md
  • website/reference/python-api/write_routing.md

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Review on demand using usage pricing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

jphein added 2 commits August 9, 2026 14:04
develop's CHANGELOG references v3.6.0/v3.7.0 compare ranges before
upstream tags the releases; 404 until then. Dated exclusion, remove
after upstream tags.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upstream sync: develop through v3.7.0 (349 commits, ~59 conflict files)