diff --git a/CLAUDE.md b/CLAUDE.md index 8659a11f1e..fab3f2672f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -50,14 +50,14 @@ TechEmpower's production fork of [MemPalace/mempalace](https://github.com/MemPal - `~/Projects/mempalace.yaml` — **do not delete**. Mining config with wing/room definitions. Regenerate with `mempalace init ~/Projects --yes` if lost. - `~/.mempalace/config.json` — topic wings and hall keywords, customized for JP's domains (infrastructure, development, tools, creative, projects, system). -- `~/.mempalace/palace/` — ChromaDB vector store. The actual data. +- **Palace data lives in postgres on `disks.jphe.in`**, reached via palace-daemon at `http://disks.jphe.in:8085`. The local `~/.mempalace/palace/` ChromaDB store was retired on the 2026-05-14 pgvector cutover (renamed to `~/.mempalace/palace.retired-pre-pgcutover-2026-05-14/`); a `~/.mempalace/RETIRED` marker refuses default-palace opens so misconfigured tools can't silently fall back to the stale local store. - `~/.mempalace/hook_state/` — stop hook session tracking. ## Development ```bash -source venv/bin/activate -python -m pytest tests/ -q # ~1096 tests (benchmarks deselected) +source .venv/bin/activate +python -m pytest tests/ -q # 2486 tests (benchmarks deselected) mempalace status # check palace state mempalace search "query" # test search python -m mempalace.mcp_server # run MCP server standalone @@ -83,7 +83,7 @@ Workflow for landing new fork-ahead changes lives in [Documentation maintenance] Claude Code has two complementary memory layers, used in tandem: - **Auto-memory** (`~/.claude/projects/*/memory/`) — lightweight preferences, context, feedback. Manual writes only. (Anthropic's "Auto Dream" research-preview shipped late April 2026 in Claude Code `/dream` + the Managed Agents Dreams API; MemPalace deliberately stays un-consolidated and the Dreams API design ratifies the verbatim-vs-derivative axis. See `~/.claude/projects/-home-jp-Projects-memorypalace/memory/project_auto_dream.md`.) -- **MemPalace** (`~/.mempalace/palace/`, ~183K drawers behind the daemon) — verbatim conversations, tool output, code. Write-only archive, searchable via MCP. Completeness is the feature. Wing/room layout follows the canonical 7-room taxonomy (see `palace-taxonomy` skill / `~/Projects/familiar.realm.watch/docs/superpowers/specs/2026-05-13-palace-room-taxonomy.md`). +- **MemPalace** (postgres + pgvector + AGE on `disks.jphe.in`, reached via palace-daemon at `http://disks.jphe.in:8085`, 300K+ drawers) — verbatim conversations, tool output, code. Write-only archive, searchable via MCP. Completeness is the feature. Wing/room layout follows the canonical 7-room taxonomy (see `palace-taxonomy` skill / `~/Projects/familiar.realm.watch/docs/superpowers/specs/2026-05-13-palace-room-taxonomy.md`). Both systems coexist. Hook saves are scoped to MemPalace ("For THIS save, use MemPalace MCP tools only") — this is not a permanent ban on auto-memory. diff --git a/FORK_CHANGELOG.md b/FORK_CHANGELOG.md index 4e9e9c80e0..e88658d90e 100644 --- a/FORK_CHANGELOG.md +++ b/FORK_CHANGELOG.md @@ -1,4 +1,4 @@ -# Fork Changelog (jphein/mempalace) +# Fork Changelog (techempower-org/mempalace) Fork-ahead changes that aren't yet in upstream `MemPalace/mempalace`. Upstream's release history lives in [`CHANGELOG.md`](CHANGELOG.md); @@ -24,7 +24,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added -- **Honor ~/.mempalace/RETIRED marker — refuse default palace, surface retire message** ([`798cf14`](https://github.com/jphein/mempalace/commit/798cf14)) +- **Honor ~/.mempalace/RETIRED marker — refuse default palace, surface retire message** ([`798cf14`](https://github.com/techempower-org/mempalace/commit/798cf14)) Recurring confusion source on this fork: any code path that opens mempalace without ``PALACE_DAEMON_URL`` set silently falls through to the default chroma palace at ``~/.mempalace/palace`` @@ -66,7 +66,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed -- **Empty repo .opencode/opencode.json mcp block — disabled flag wasn't being respected** ([`7133eee`](https://github.com/jphein/mempalace/commit/7133eee)) +- **Empty repo .opencode/opencode.json mcp block — disabled flag wasn't being respected** ([`7133eee`](https://github.com/techempower-org/mempalace/commit/7133eee)) ``.opencode/opencode.json`` from upstream PR #1567 spawns a local ``mempalace.mcp_server`` when opencode launches in the repo root. On daemon-routed setups this creates **two MCP @@ -85,7 +85,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). *Files:* `.opencode/opencode.json` -- **Drop \$comment from .opencode/opencode.json — schema rejects unknown root keys** ([`637bb01`](https://github.com/jphein/mempalace/commit/637bb01)) +- **Drop \$comment from .opencode/opencode.json — schema rejects unknown root keys** ([`637bb01`](https://github.com/techempower-org/mempalace/commit/637bb01)) Commit ``47018e5`` (PR #108) added a ``\$comment`` field to ``.opencode/opencode.json`` explaining why the repo-level MCP entry defaulted to ``enabled: false``. Opencode's strict @@ -104,7 +104,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). *Files:* `.opencode/opencode.json` -- **Disable repo-level MCP entry by default + venv-python fallback** ([`47018e5`](https://github.com/jphein/mempalace/commit/47018e5)) +- **Disable repo-level MCP entry by default + venv-python fallback** ([`47018e5`](https://github.com/techempower-org/mempalace/commit/47018e5)) First attempt at the daemon-routed-vs-local-spawn double-server problem from upstream PR #1567's ``.opencode/opencode.json``. Set ``enabled: false`` on the repo entry; switched the spawn @@ -127,7 +127,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added -- **Bundled OpenCode live-capture plugin that bypasses option-K v1.2.1 bugs (filed upstream as #4, #5)** ([`5522623`](https://github.com/jphein/mempalace/commit/5522623)) +- **Bundled OpenCode live-capture plugin that bypasses option-K v1.2.1 bugs (filed upstream as #4, #5)** ([`5522623`](https://github.com/techempower-org/mempalace/commit/5522623)) Adds `examples/opencode/live-capture/` — a self-contained OpenCode plugin (JS) + Python helper that POSTs verbatim session transcripts to the daemon's `/silent-save` endpoint on @@ -182,7 +182,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). *Files:* `examples/opencode/live-capture/mempalace-live-capture.js`, `examples/opencode/live-capture/capture-session.py`, `examples/opencode/option-k-plugin-daemon-routing.patch`, `examples/opencode/option-k-plugin-message-updated.patch`, `docs/integrations/opencode.md` -- **Documented OpenCode integration recipe (read-side MCP + push plugin + retrospective adapter)** ([`60dc9e6`](https://github.com/jphein/mempalace/commit/60dc9e6)) +- **Documented OpenCode integration recipe (read-side MCP + push plugin + retrospective adapter)** ([`60dc9e6`](https://github.com/techempower-org/mempalace/commit/60dc9e6)) Adds `docs/integrations/opencode.md` and an `examples/opencode/` directory capturing the three-direction OpenCode + MemPalace integration recipe for daemon-routed setups: @@ -222,7 +222,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). *Files:* `docs/integrations/opencode.md`, `examples/opencode/opencode.jsonc.example`, `examples/opencode/option-k-plugin-daemon-routing.patch` -- **.opencode/opencode.json — repo-root MCP config so opencode picks up mempalace automatically** ([`ba16b82`](https://github.com/jphein/mempalace/commit/ba16b82)) +- **.opencode/opencode.json — repo-root MCP config so opencode picks up mempalace automatically** ([`ba16b82`](https://github.com/techempower-org/mempalace/commit/ba16b82)) Cherry-pick of upstream PR #1567 (Dxrk777). Adds `.opencode/opencode.json` so that running `opencode` in the mempalace repo root automatically wires `mempalace` as a local @@ -245,7 +245,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). *Files:* `.opencode/opencode.json` -- **OpenCodeSourceAdapter (RFC 002) — retrospective ingest of OpenCode SQLite sessions** ([`2ffe652`](https://github.com/jphein/mempalace/commit/2ffe652)) +- **OpenCodeSourceAdapter (RFC 002) — retrospective ingest of OpenCode SQLite sessions** ([`2ffe652`](https://github.com/techempower-org/mempalace/commit/2ffe652)) Cherry-pick of upstream PR #1484. Adds `mempalace/sources/opencode.py` — an RFC 002 `BaseSourceAdapter` that ingests OpenCode AI-coding-CLI session transcripts from @@ -284,7 +284,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). *Files:* `mempalace/sources/opencode.py`, `mempalace/sources/transforms.py`, `mempalace/sources/context.py`, `pyproject.toml`, `tests/test_sources_opencode.py`, `tests/fixtures/opencode/sample_session_2026_05_12/README.md`, `tests/fixtures/opencode/sample_session_2026_05_12/build_fixture.py`, `tests/test_corpus_origin_integration.py` -- **Pending-writes journal + replay so daemon outages stop being silent** ([`0c34464`](https://github.com/jphein/mempalace/commit/0c34464)) +- **Pending-writes journal + replay so daemon outages stop being silent** ([`0c34464`](https://github.com/techempower-org/mempalace/commit/0c34464)) Closes a silent-data-loss gap exposed by the 2026-05-17 power event: when ``PALACE_DAEMON_URL`` is set and the daemon's backend is unreachable, the hook write path (``_post_daemon_mine``) @@ -315,7 +315,7 @@ enqueue side effect. ### Fixed -- **Stub resources/list + prompts/list so MCP clients stop ERROR-logging on connect** ([`6ca0670`](https://github.com/jphein/mempalace/commit/6ca0670)) +- **Stub resources/list + prompts/list so MCP clients stop ERROR-logging on connect** ([`6ca0670`](https://github.com/techempower-org/mempalace/commit/6ca0670)) OpenCode 1.15.x and other MCP clients probe ``resources/list`` and ``prompts/list`` on connect to discover server capabilities. mempalace's MCP server only exposes ``tools/*``; @@ -334,7 +334,7 @@ empty-list shape and absence of ERROR. *Files:* `mempalace/mcp_server.py`, `tests/test_mcp_server.py` -- **MCP server distinguishes 'backend unreachable' from 'no palace found'** ([`0c34464`](https://github.com/jphein/mempalace/commit/0c34464)) +- **MCP server distinguishes 'backend unreachable' from 'no palace found'** ([`0c34464`](https://github.com/techempower-org/mempalace/commit/0c34464)) The CLI's misleading ``Run: mempalace init `` hint was the diagnostic blocker during the 2026-05-17 outage: the daemon was up, postgres was down, and every search returned "No palace @@ -369,7 +369,7 @@ on failure, error cleared on successful reopen). ### Added -- **mempalace_walk_palace MCP tool — agent walks the palace via AGE Cypher** ([`8022ecb`](https://github.com/jphein/mempalace/commit/8022ecb)) +- **mempalace_walk_palace MCP tool — agent walks the palace via AGE Cypher** ([`8022ecb`](https://github.com/techempower-org/mempalace/commit/8022ecb)) Phase 6 of the AGE-integration plan. Exposes the "agent walks into the palace finding wings, rooms, drawers" metaphor as a single MCP tool over the unified palace+entity graph (Wing → Room → Drawer → @@ -397,7 +397,7 @@ on failure, error cleared on successful reopen). *Files:* `mempalace/mcp_server.py` -- **Backfill AGE graph from existing drawer table — restartable, checkpointed** ([`b3f0206`](https://github.com/jphein/mempalace/commit/b3f0206)) +- **Backfill AGE graph from existing drawer table — restartable, checkpointed** ([`b3f0206`](https://github.com/techempower-org/mempalace/commit/b3f0206)) Phase 4 of the AGE-integration plan. New module `mempalace/backfill_age.py` with CLI entry point that reads the drawer table once and builds the full Wing/Room/Drawer/MENTIONS @@ -431,7 +431,7 @@ on failure, error cleared on successful reopen). *Files:* `mempalace/backfill_age.py`, `mempalace/knowledge_graph_age.py` -- **Wing/Room/Drawer hierarchy as native AGE nodes; Cypher MATCH walks palace structure** ([`ff583c0`](https://github.com/jphein/mempalace/commit/ff583c0)) +- **Wing/Room/Drawer hierarchy as native AGE nodes; Cypher MATCH walks palace structure** ([`ff583c0`](https://github.com/techempower-org/mempalace/commit/ff583c0)) Phase 3 of the AGE-integration plan. Mirrors `mempalace.palace_graph`'s SQL-aggregation pattern into AGE so Cypher MATCH walks the palace structure natively — no SQL aggregation per query. @@ -467,7 +467,7 @@ on failure, error cleared on successful reopen). *Files:* `mempalace/palace_graph_age.py` -- **Write-through middleware on PostgresCollection — entities populate AGE on every drawer write** ([`3321d83`](https://github.com/jphein/mempalace/commit/3321d83)) +- **Write-through middleware on PostgresCollection — entities populate AGE on every drawer write** ([`3321d83`](https://github.com/techempower-org/mempalace/commit/3321d83)) Phase 2 of the AGE-integration plan. Adds a write-through hook on `PostgresCollection.add`/`upsert` that extracts entities from the document and creates `(Drawer)-[:MENTIONS]->(Entity)` edges in @@ -505,7 +505,7 @@ on failure, error cleared on successful reopen). *Files:* `mempalace/backends/postgres.py`, `mempalace/kg_writethrough.py` -- **KnowledgeGraphAGE API parity with SQLite KG: add_entity, invalidate, query_entity, query_relationship, timeline, seed_from_entity_facts** ([`ff7187d`](https://github.com/jphein/mempalace/commit/ff7187d)) +- **KnowledgeGraphAGE API parity with SQLite KG: add_entity, invalidate, query_entity, query_relationship, timeline, seed_from_entity_facts** ([`ff7187d`](https://github.com/techempower-org/mempalace/commit/ff7187d)) Phase 1 of the AGE-integration plan. Brings `KnowledgeGraphAGE` to API parity with `mempalace.knowledge_graph.KnowledgeGraph` (the SQLite backend). Previously only `add_triple`, `query_triples`, @@ -555,7 +555,7 @@ on failure, error cleared on successful reopen). ### Added -- **KnowledgeGraphAGE skeleton — Apache AGE graph bootstrap over psycopg2** ([`a3ee623`](https://github.com/jphein/mempalace/commit/a3ee623)) +- **KnowledgeGraphAGE skeleton — Apache AGE graph bootstrap over psycopg2** ([`a3ee623`](https://github.com/techempower-org/mempalace/commit/a3ee623)) First commit toward the Apache AGE-backed knowledge graph layer that the migration plan calls for. Skeleton class `KnowledgeGraphAGE` in `mempalace/knowledge_graph_age.py` opens a @@ -608,7 +608,7 @@ set against the homelab mempalace-db at disks.jphe.in:5433). *Files:* `mempalace/knowledge_graph_age.py`, `tests/test_knowledge_graph_age.py` -- **CI: gate postgres-backend tests against a pgvector service container** ([`da0bdbb`](https://github.com/jphein/mempalace/commit/da0bdbb)) +- **CI: gate postgres-backend tests against a pgvector service container** ([`da0bdbb`](https://github.com/techempower-org/mempalace/commit/da0bdbb)) Adds a `test-postgres` job to `.github/workflows/ci.yml` that runs in parallel with the existing `test-linux` / `test-windows` / `test-macos` / `lint` matrix. Service container is the public @@ -640,7 +640,7 @@ set against the homelab mempalace-db at disks.jphe.in:5433). *Files:* `.github/workflows/ci.yml` -- **PostgreSQL backend via #665 cherry-pick + fork-side adaptations + smoke tests** ([`5e90c72`](https://github.com/jphein/mempalace/commit/5e90c72)) +- **PostgreSQL backend via #665 cherry-pick + fork-side adaptations + smoke tests** ([`5e90c72`](https://github.com/techempower-org/mempalace/commit/5e90c72)) Cherry-pick of skuznetsov's upstream PR [#665](https://github.com/MemPalace/mempalace/pull/665) — adds a PostgreSQL backend built on the merged #995 / RFC 001 @@ -724,7 +724,7 @@ regressions in non-postgres paths. ### Changed -- **README pivots to the four-layer model + Auto Dream as vindication of the verbatim-vs-derivative axis** ([`55b36ca`](https://github.com/jphein/mempalace/commit/55b36ca)) +- **README pivots to the four-layer model + Auto Dream as vindication of the verbatim-vs-derivative axis** ([`55b36ca`](https://github.com/techempower-org/mempalace/commit/55b36ca)) Substantial README rewrite (+137/-100) reflecting three things that landed between the previous refresh (`a67be3f`, the 2026-05-10 develop sync) and now: @@ -782,7 +782,7 @@ regressions in non-postgres paths. ### Fixed -- **Defense-in-depth metadata sanitizer at the chromadb-client chokepoint** ([`f499814`](https://github.com/jphein/mempalace/commit/f499814)) +- **Defense-in-depth metadata sanitizer at the chromadb-client chokepoint** ([`f499814`](https://github.com/techempower-org/mempalace/commit/f499814)) Companion to the repair.py sanitizers in #1458 / `949cb20` (which fixed `_extract_drawers` and `_rebuild_one_collection`). A 151,478-drawer rebuild against the canonical palace still @@ -818,7 +818,7 @@ regressions in non-postgres paths. *Files:* `mempalace/backends/chroma.py` -- **Coerce empty + None metadata to sentinel in both rebuild paths** ([`949cb20`](https://github.com/jphein/mempalace/commit/949cb20)) +- **Coerce empty + None metadata to sentinel in both rebuild paths** ([`949cb20`](https://github.com/techempower-org/mempalace/commit/949cb20)) ChromaDB 1.5.x rejects both None and empty-dict entries in the `metadatas` list (raises `ValueError: Expected metadata to be a non-empty dict`). Two functions in `mempalace/repair.py` construct @@ -855,7 +855,7 @@ regressions in non-postgres paths. *Files:* `mempalace/repair.py` -- **Route Stop/PreCompact hooks through palace-daemon/clients/hook.py** ([`42ded2e`](https://github.com/jphein/mempalace/commit/42ded2e)) +- **Route Stop/PreCompact hooks through palace-daemon/clients/hook.py** ([`42ded2e`](https://github.com/techempower-org/mempalace/commit/42ded2e)) Replaces the bash wrapper invocation pattern in `.claude-plugin/hooks/hooks.json` with a single Python entrypoint via the daemon's hook client. Both Stop and PreCompact now invoke @@ -892,7 +892,7 @@ regressions in non-postgres paths. ### Performance -- **Bulk pre-fetch already-mined set instead of N WHERE queries in mine_convos** ([`248854a`](https://github.com/jphein/mempalace/commit/248854a)) +- **Bulk pre-fetch already-mined set instead of N WHERE queries in mine_convos** ([`248854a`](https://github.com/techempower-org/mempalace/commit/248854a)) Replaces the N+1 `col.get(where={"source_file": }, ...)` per-conversation pattern in `mempalace/convo_miner.py:mine_convos` with a single bulk pre-fetch — `col.get(where={"source_file": @@ -923,7 +923,7 @@ regressions in non-postgres paths. ### Added -- **daemon-route `mempalace status` / `search` / `mine` when PALACE_DAEMON_URL is set** ([`22ef562`](https://github.com/jphein/mempalace/commit/22ef562)) +- **daemon-route `mempalace status` / `search` / `mine` when PALACE_DAEMON_URL is set** ([`22ef562`](https://github.com/techempower-org/mempalace/commit/22ef562)) Companion to the `mcp_server` routing in commit `41359ba`. Closes the last desktop-side path that opened a local chromadb client. @@ -969,7 +969,7 @@ when env var is unset. Suite 1591 passed (1577 + 14 new). *Files:* `mempalace/cli.py`, `tests/test_cli_daemon.py` -- **daemon-route `mcp_server.py` via the `handle_request` JSON-RPC chokepoint** ([`41359ba`](https://github.com/jphein/mempalace/commit/41359ba)) +- **daemon-route `mcp_server.py` via the `handle_request` JSON-RPC chokepoint** ([`41359ba`](https://github.com/techempower-org/mempalace/commit/41359ba)) Mirrors the `PALACE_DAEMON_URL` gate that `hooks_cli.py` shipped on 2026-04-24 (the daemon-strict fix for the HNSW drift incident). Closes the last in-process write path inside @@ -1024,7 +1024,7 @@ from the canonical palace. Suite 1577 passed. ### Added -- **mempalace mined + purge --source-file (mining management surface)** ([`2e6ced9`](https://github.com/jphein/mempalace/commit/2e6ced9)) +- **mempalace mined + purge --source-file (mining management surface)** ([`2e6ced9`](https://github.com/techempower-org/mempalace/commit/2e6ced9)) Closes the "removing manually mined data" half of JP's mining-management ask. Adding is already covered by the existing ``mempalace mine ``; this PR adds the symmetric remove + @@ -1052,7 +1052,7 @@ from the canonical palace. Suite 1577 passed. *Files:* `mempalace/cli.py`, `tests/test_cli.py` -- **`hook_verbatim_mode` config flag preserves system tags + full tool I/O during transcript ingest** ([`ef98961`](https://github.com/jphein/mempalace/commit/ef98961)) +- **`hook_verbatim_mode` config flag preserves system tags + full tool I/O during transcript ingest** ([`ef98961`](https://github.com/techempower-org/mempalace/commit/ef98961)) `normalize()` defaults match upstream — system tags, hook chrome, Read/Edit/Write tool results, long Bash output, and large Grep/Glob match lists are stripped or truncated so chunk @@ -1099,7 +1099,7 @@ config-file readback. Suite total 1562 passed. ### Changed -- **Drop wing_ prefix from transcript-derived wings to converge with operator mines** ([`86d4700`](https://github.com/jphein/mempalace/commit/86d4700)) +- **Drop wing_ prefix from transcript-derived wings to converge with operator mines** ([`86d4700`](https://github.com/techempower-org/mempalace/commit/86d4700)) The fork-only ``_wing_from_transcript_path`` returned ``wing_`` for hook-derived wings, but operator-mined content from ``mempalace mine ~/Projects/X`` lands in a bare-name @@ -1127,7 +1127,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/hooks_cli.py`, `tests/test_hooks_cli.py` -- **Retire mempalace_session_recovery collection + read tool** ([`0b945e1`](https://github.com/jphein/mempalace/commit/0b945e1)) +- **Retire mempalace_session_recovery collection + read tool** ([`0b945e1`](https://github.com/techempower-org/mempalace/commit/0b945e1)) Follow-up to drop-checkpoint-write-path. With nothing writing to the recovery collection anymore (hooks moved to verbatim-only on the parent branch), the read paths and migration code that @@ -1163,7 +1163,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/palace.py`, `mempalace/mcp_server.py`, `mempalace/migrate.py`, `mempalace/cli.py`, `website/reference/mcp-tools.md`, `tests/test_session_recovery.py`, `tests/test_migrate.py`, `tests/test_mcp_server.py` -- **Drop hook-side checkpoint diary writes — verbatim-only architecture** ([`69768fc`](https://github.com/jphein/mempalace/commit/69768fc)) +- **Drop hook-side checkpoint diary writes — verbatim-only architecture** ([`69768fc`](https://github.com/techempower-org/mempalace/commit/69768fc)) The Stop hook used to do two things on each fire: (a) write a 1KB checkpoint summary diary entry into the dedicated ``mempalace_session_recovery`` collection AND (b) auto-mine the @@ -1206,7 +1206,7 @@ config-file readback. Suite total 1562 passed. ### Fixed -- **Preserve dashed project names in transcript-derived wings** ([`d76134d`](https://github.com/jphein/mempalace/commit/d76134d)) +- **Preserve dashed project names in transcript-derived wings** ([`d76134d`](https://github.com/techempower-org/mempalace/commit/d76134d)) Two findings from Copilot review on jphein/mempalace#9 that surfaced a real bug: the previous primary regex's ``encoded.rsplit('-', 1)[-1]`` rule collapsed @@ -1232,7 +1232,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/hooks_cli.py`, `tests/test_hooks_cli.py` -- **Restore transcript ingest via daemon /mine when PALACE_DAEMON_URL is set** ([`09d2ca6`](https://github.com/jphein/mempalace/commit/09d2ca6)) +- **Restore transcript ingest via daemon /mine when PALACE_DAEMON_URL is set** ([`09d2ca6`](https://github.com/techempower-org/mempalace/commit/09d2ca6)) Daemon-strict mode (introduced 2026-04-24 in commits ``8c90c0f`` + ``0e97b19`` to fix the HNSW drift incident) skipped all three local mining paths when ``PALACE_DAEMON_URL`` was set, on the @@ -1274,7 +1274,7 @@ config-file readback. Suite total 1562 passed. ### Fixed -- **`cfg.init()` no longer materializes chunking defaults into `config.json`** ([`6ce37c0`](https://github.com/jphein/mempalace/commit/6ce37c0)) +- **`cfg.init()` no longer materializes chunking defaults into `config.json`** ([`6ce37c0`](https://github.com/techempower-org/mempalace/commit/6ce37c0)) `cfg.init()` was unconditionally writing ``chunk_size: 800``, ``chunk_overlap: 100``, and ``min_chunk_size: 50`` into ``config.json`` on first run. The values match ``miner.py``'s @@ -1312,7 +1312,7 @@ config-file readback. Suite total 1562 passed. merges. Amends fork-ahead row 17. *Tests:* 1548/1548 (was 1546/1548 with 2 isolation failures in test_convo_miner) - *Upstream:* [PR #1024](https://github.com/MemPalace/mempalace/pull/1024) (OPEN) + *Upstream:* [PR #1024](https://github.com/MemPalace/mempalace/pull/1024) (MERGED) *Files:* `mempalace/config.py` @@ -1322,7 +1322,7 @@ config-file readback. Suite total 1562 passed. ### Changed -- **Retire the `kind=` filter — structural split made it inert** ([`7ba28dc`](https://github.com/jphein/mempalace/commit/7ba28dc)) +- **Retire the `kind=` filter — structural split made it inert** ([`7ba28dc`](https://github.com/techempower-org/mempalace/commit/7ba28dc)) Phases A–E of the checkpoint collection split (2026-04-25 → 2026-04-26) moved every Stop-hook auto-save checkpoint drawer to the dedicated ``mempalace_session_recovery`` collection. Empirical check on the @@ -1343,7 +1343,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/searcher.py`, `mempalace/mcp_server.py`, `mempalace/palace.py`, `mempalace/migrate.py`, `mempalace/layers.py`, `tests/test_searcher.py` -- **Hoist CLOSET_RANK_BOOSTS to module level + record VecRecall ablation finding** ([`3cb03f3`](https://github.com/jphein/mempalace/commit/3cb03f3)) +- **Hoist CLOSET_RANK_BOOSTS to module level + record VecRecall ablation finding** ([`3cb03f3`](https://github.com/techempower-org/mempalace/commit/3cb03f3)) Two-step refactor + measurement. First (commit ``f558d3c``): hoist ``CLOSET_RANK_BOOSTS = [0.40, 0.25, 0.15, 0.08, 0.04]`` and ``CLOSET_DISTANCE_CAP`` from inside ``search_memories`` to module @@ -1372,7 +1372,7 @@ config-file readback. Suite total 1562 passed. ### Fixed -- **Strip embedded API key from .claude-plugin/ manifests; rely on env inheritance** ([`9f91e18`](https://github.com/jphein/mempalace/commit/9f91e18)) +- **Strip embedded API key from .claude-plugin/ manifests; rely on env inheritance** ([`9f91e18`](https://github.com/techempower-org/mempalace/commit/9f91e18)) ``.claude-plugin/.mcp.json`` and ``.claude-plugin/hooks/hooks.json`` shipped with a real (rotated) API key embedded as a literal in the manifest's ``env`` block, plus my homelab daemon URL. Both are @@ -1401,7 +1401,7 @@ config-file readback. Suite total 1562 passed. ### Added -- **Canonical YAML manifest + renderer for fork-ahead docs** ([`5a01aec`](https://github.com/jphein/mempalace/commit/5a01aec)) +- **Canonical YAML manifest + renderer for fork-ahead docs** ([`5a01aec`](https://github.com/techempower-org/mempalace/commit/5a01aec)) The fork-ahead narrative previously lived (and drifted) across four hand-edited files: README's fork-change-queue table, CLAUDE.md's row inventory, FORK_CHANGELOG.md, and the promises tracker. New @@ -1418,7 +1418,7 @@ config-file readback. Suite total 1562 passed. *Files:* `docs/fork-changes.yaml`, `scripts/render-docs.py`, `scripts/check-docs.sh`, `FORK_CHANGELOG.md`, `CLAUDE.md` -- **Phase D migration + PreCompact recovery write** ([`42817d7`](https://github.com/jphein/mempalace/commit/42817d7)) +- **Phase D migration + PreCompact recovery write** ([`42817d7`](https://github.com/techempower-org/mempalace/commit/42817d7)) ``migrate_checkpoints_to_recovery(palace_path, batch_size=1000)`` walks the main collection in pages, filters drawers with topic in ``_CHECKPOINT_TOPICS`` in Python (avoids the chromadb 1.5.x ``$in``/``$nin`` @@ -1437,7 +1437,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/migrate.py`, `mempalace/cli.py`, `mempalace/hooks_cli.py`, `tests/test_migrate.py` -- **Surface drawer_id in search/diary/recovery payloads** ([`9a8bb77`](https://github.com/jphein/mempalace/commit/9a8bb77)) +- **Surface drawer_id in search/diary/recovery payloads** ([`9a8bb77`](https://github.com/techempower-org/mempalace/commit/9a8bb77)) ChromaDB's primary key was always returned by ``query()`` and ``get()`` but never plumbed into result-building loops; consumers (e.g. familiar.realm.watch's citation-popover loop) couldn't link a hit @@ -1452,7 +1452,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/searcher.py`, `mempalace/mcp_server.py`, `website/reference/mcp-tools.md` -- **scripts/deploy.sh — one-command Syncthing-aware redeploy** ([`8252025`](https://github.com/jphein/mempalace/commit/8252025)) +- **scripts/deploy.sh — one-command Syncthing-aware redeploy** ([`8252025`](https://github.com/techempower-org/mempalace/commit/8252025)) Single command does the right shape: push fork main → wait for Syncthing to reach ``/mnt/raid/projects/memorypalace`` on the deploy host → ``systemctl --user restart palace-daemon`` → poll ``/health`` → @@ -1466,7 +1466,7 @@ config-file readback. Suite total 1562 passed. ### Changed -- **Cherry-pick #1094 — coerce None metadatas at chromadb boundary** ([`43d728d`](https://github.com/jphein/mempalace/commit/43d728d)) +- **Cherry-pick #1094 — coerce None metadatas at chromadb boundary** ([`43d728d`](https://github.com/techempower-org/mempalace/commit/43d728d)) Fork main was carrying the per-site ``meta = meta or {}`` guards from #999 in eight read paths but didn't have the boundary coercion that closes the issue once for all callers. The typed @@ -1486,7 +1486,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/backends/chroma.py`, `tests/test_backends.py` -- **Cherry-pick #1087 rewrite — collection.delete(where=) instead of nuke-and-rebuild** ([`366a9ad`](https://github.com/jphein/mempalace/commit/366a9ad)) +- **Cherry-pick #1087 rewrite — collection.delete(where=) instead of nuke-and-rebuild** ([`366a9ad`](https://github.com/techempower-org/mempalace/commit/366a9ad)) Fork main had been carrying ``cmd_purge``'s nuke-and-rebuild shape (extract survivors, ``shutil.rmtree``, recreate, re-insert). Cherry-picked the post-review rewrite from PR #1087's branch: @@ -1506,7 +1506,7 @@ config-file readback. Suite total 1562 passed. ### Fixed -- **Integrity gate prevents quarantine_stale_hnsw from destroying healthy indexes** ([`645ba20`](https://github.com/jphein/mempalace/commit/645ba20)) +- **Integrity gate prevents quarantine_stale_hnsw from destroying healthy indexes** ([`645ba20`](https://github.com/techempower-org/mempalace/commit/645ba20)) Previous behavior fired whenever ``sqlite_mtime - hnsw_mtime`` exceeded the (lowered, in #1173) 300s threshold. ChromaDB 1.5.x flushes HNSW asynchronously and a clean shutdown does not force-flush, so the @@ -1530,7 +1530,7 @@ config-file readback. Suite total 1562 passed. ### Performance -- **Cherry-pick #1085 — batch ChromaDB inserts in miner (10–30× faster)** ([`6be6fff`](https://github.com/jphein/mempalace/commit/6be6fff)) +- **Cherry-pick #1085 — batch ChromaDB inserts in miner (10–30× faster)** ([`6be6fff`](https://github.com/techempower-org/mempalace/commit/6be6fff)) Cherry-picked from upstream PR [#1085](https://github.com/MemPalace/mempalace/pull/1085) (@midweste, OPEN as of 2026-04-26). New ``_build_drawer()`` helper + ``add_drawers()`` @@ -1539,10 +1539,13 @@ config-file readback. Suite total 1562 passed. and ``os.path.getmtime()`` to file-level (2 syscalls per file instead of 2N). Reported 10–30× mining speedup upstream. Fork-side resolution preserved fork's existing ``DRAWER_UPSERT_BATCH_SIZE=1000``; aliased - upstream's ``CHROMA_BATCH_LIMIT`` to it. Becomes a no-op when #1085 - merges to develop and we next sync. + upstream's ``CHROMA_BATCH_LIMIT`` to it. **2026-05-16:** #1085 was + closed by @midweste, superseded by merged upstream + [#1185](https://github.com/MemPalace/mempalace/pull/1185) (wider + scope: same batching + optional GPU acceleration). The fork-side + cherry-pick is now a no-op against develop; drop on next sync. - *Upstream:* [PR #1085](https://github.com/MemPalace/mempalace/pull/1085) (OPEN) + *Upstream:* [PR #1085](https://github.com/MemPalace/mempalace/pull/1085) (CLOSED) *Files:* `mempalace/miner.py` @@ -1552,7 +1555,7 @@ config-file readback. Suite total 1562 passed. ### Added -- **Phases A–C of the checkpoint collection split** ([`e266365`](https://github.com/jphein/mempalace/commit/e266365)) +- **Phases A–C of the checkpoint collection split** ([`e266365`](https://github.com/techempower-org/mempalace/commit/e266365)) New ``mempalace_session_recovery`` collection adapter (``_SESSION_RECOVERY_COLLECTION`` + ``get_session_recovery_collection`` in ``palace.py``); ``tool_diary_write`` routes ``topic in _CHECKPOINT_TOPICS`` @@ -1570,7 +1573,7 @@ config-file readback. Suite total 1562 passed. ### Fixed -- **Gate quarantine_stale_hnsw to once-per-palace-per-process** ([`70c4bc6`](https://github.com/jphein/mempalace/commit/70c4bc6)) +- **Gate quarantine_stale_hnsw to once-per-palace-per-process** ([`70c4bc6`](https://github.com/techempower-org/mempalace/commit/70c4bc6)) ``make_client()`` previously invoked ``quarantine_stale_hnsw`` on every reconnect; under steady write load the proactive check kept firing, racking up ``.drift-*`` directories every 10–30 minutes. New @@ -1585,7 +1588,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/backends/chroma.py`, `tests/test_backends.py`, `tests/conftest.py` -- **palace_graph.build_graph skips None metadata** ([`5fd15db`](https://github.com/jphein/mempalace/commit/5fd15db)) +- **palace_graph.build_graph skips None metadata** ([`5fd15db`](https://github.com/techempower-org/mempalace/commit/5fd15db)) ``palace_graph.py:95`` was calling ``meta.get("room", "")`` unconditionally; ChromaDB returns ``None`` for legacy/partial-write drawers, taking out every consumer of ``build_graph`` (graph_stats, find_tunnels, traverse, @@ -1597,7 +1600,7 @@ config-file readback. Suite total 1562 passed. *Files:* `mempalace/palace_graph.py` -- **kind= filter on search_memories excludes Stop-hook checkpoints (transitional)** ([`f9f5cc4`](https://github.com/jphein/mempalace/commit/f9f5cc4)) +- **kind= filter on search_memories excludes Stop-hook checkpoints (transitional)** ([`f9f5cc4`](https://github.com/techempower-org/mempalace/commit/f9f5cc4)) Three values: ``"content"`` (default, excludes), ``"checkpoint"`` (recovery/audit only), ``"all"`` (no filter). Two same-day architecture corrections: (a) the where-clause filter (``topic $nin [...]``) tripped @@ -1625,6 +1628,7 @@ config-file readback. Suite total 1562 passed. *See CHANGELOG.md (upstream) for the full released history.* +- [PR #1024](https://github.com/MemPalace/mempalace/pull/1024) — Configurable chunk_size / chunk_overlap / min_chunk_size — 2026-05-15 - [PR #1173](https://github.com/MemPalace/mempalace/pull/1173) — quarantine_stale_hnsw on make_client + cold-start gate + integrity sniff — 2026-04-26 - [PR #1177](https://github.com/MemPalace/mempalace/pull/1177) — `.blob_seq_ids_migrated` marker guard (closes #1090) — 2026-04-26 - [PR #1198](https://github.com/MemPalace/mempalace/pull/1198) — _tokenize None-document guard in BM25 reranker — 2026-04-26 diff --git a/README.md b/README.md index 8e7c70c910..dcff528f53 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ --- -This fork tracks `upstream/develop` through the 2026-05-10 sync (commit `a67be3f`, fork merge via `chore/sync-develop-2026-05-10`) and runs in production on a **273K-drawer Postgres + pgvector + Apache AGE palace** behind [palace-daemon](https://github.com/techempower-org/palace-daemon) on `disks:8085`. It carries ~434 fork-ahead changes that compose with — not replace — bensig's release direction; the v3.3.5 release (2026-05-10) includes our co-authored `_get_collection` retry-once via upstream #1377. ~1850 tests pass on `main`. The new things here are *what we've learned*, not just what we've fixed. +This fork tracks `upstream/develop` through the 2026-05-10 sync (commit `a67be3f`, fork merge via `chore/sync-develop-2026-05-10`) and runs in production on a **300K+ drawer Postgres + pgvector + Apache AGE palace** behind [palace-daemon](https://github.com/techempower-org/palace-daemon) on `disks:8085`. It carries ~434 fork-ahead changes that compose with — not replace — bensig's release direction; the v3.3.5 release (2026-05-10) includes our co-authored `_get_collection` retry-once via upstream #1377. 2486 tests pass on `main`. The new things here are *what we've learned*, not just what we've fixed. **2026-05-15 substrate cutover complete.** What was on the `feat/pgvector-age-impl` branch is now on `main` and serving production traffic. **Hybrid retrieval** (`candidate_strategy="hybrid"` — vector ∪ postgres-`tsvector` BM25 ∪ AGE graph-expanded candidates, hybrid-reranked) shipped end-to-end through `palace-daemon`'s new `/search/keyword` and `/search/hybrid` endpoints. **8 of 9 bench suites pass** on the postgres backend (`test_chromadb_stress` skipped, doesn't apply) — no regressions on the search heap hot path vs the chromadb-era numbers. See [`docs/operators/pgvector-cutover-runbook.md`](docs/operators/pgvector-cutover-runbook.md) for the operator narrative and [`docs/benchmarks/2026-05-15-remaining-benches.json`](docs/benchmarks/2026-05-15-remaining-benches.json) for the bench artifact. @@ -101,7 +101,7 @@ We surveyed the memory-system landscape in April 2026 and found no verbatim-firs | System | Verbatim? | Local? | MCP? | First public | Notes | |---|---|---|---|---|---| -| **MemPalace** | Yes | Yes | Yes | 2026-04-06 (v3.0.0) | What we have. ~160K drawers in production. Verbatim drawers + wings/rooms scope + SQLite KG + BM25/vector hybrid search. | +| **MemPalace** | Yes | Yes | Yes | 2026-04-06 (v3.0.0) | What we have. 300K+ drawers in production. Verbatim drawers + wings/rooms scope + SQLite KG + BM25/vector hybrid search. | | [Longhand](https://glama.ai/mcp/servers/Wynelson94/longhand) | Yes | Yes | Yes, 17-tool MCP | 2026-04-14 (v0.5.5) | Closest cousin. Claude Code-specific — reads `~/.claude/projects/*.jsonl` directly. SQLite (raw JSON per event) + ChromaDB (embeddings of pre-computed "episodes"). Deterministic file-state replay via stored diffs. | | [Celiums](https://celiums.ai/) | Yes | Yes (SQLite, Docker, or DO) | Yes, 6-tool MCP | 2026-04-08 (repo) | Stores full module text with PAD emotional vectors, importance scores, and circadian metadata. Bundles a 500K+ expert-module knowledge base alongside personal memory — different product shape. | | [mcp-memory-service](https://github.com/doobidoo/mcp-memory-service) | Yes by default (opt-in consolidation) | Yes (SQLite) or Cloudflare Workers | Yes | 2024-12-26 | The long-standing verbatim option. Turn-level storage; MiniLM embeddings local. Targets LangGraph / CrewAI / AutoGen plus Claude. | @@ -116,7 +116,7 @@ The April-2026 verbatim cluster (MemPalace, Celiums, Longhand, engram all within ## Substrate: Postgres + pgvector + Apache AGE -*Status (2026-05-15): **shipped on `main`** and serving production traffic. Postgres backend cherry-picked from upstream [#665](https://github.com/MemPalace/mempalace/pull/665) (skuznetsov's PostgreSQL backend on the RFC 001 `BaseBackend` contract) plus our pgvector lazy-index race fix (commit `4566f8a`, also posted to #665 as an operator follow-up). Live on `disks.jphe.in:5433` running PG16 + pgvector 0.8.2 + AGE 1.6.0 via `apache/age:release_PG16_1.6.0` + `postgresql-16-pgvector`. **273K-drawer palace.** Hybrid retrieval (vector ∪ tsvector BM25 ∪ AGE graph) shipped end-to-end as `candidate_strategy="hybrid"`, exposed via `palace-daemon`'s `/search/keyword` and `/search/hybrid` endpoints. 8/9 mempalace bench suites pass on postgres backend ([`docs/benchmarks/2026-05-15-remaining-benches.json`](docs/benchmarks/2026-05-15-remaining-benches.json)). Original WAIT-for-#665-upstream-merge stance superseded; the cutover happened on our timeline. Full operator narrative at [`docs/operators/pgvector-cutover-runbook.md`](docs/operators/pgvector-cutover-runbook.md).* +*Status (2026-05-15): **shipped on `main`** and serving production traffic. Postgres backend cherry-picked from upstream [#665](https://github.com/MemPalace/mempalace/pull/665) (skuznetsov's PostgreSQL backend on the RFC 001 `BaseBackend` contract) plus our pgvector lazy-index race fix (commit `4566f8a`, also posted to #665 as an operator follow-up). Live on `disks.jphe.in:5433` running PG16 + pgvector 0.8.2 + AGE 1.6.0 via `apache/age:release_PG16_1.6.0` + `postgresql-16-pgvector`. **300K+ drawer palace.** Hybrid retrieval (vector ∪ tsvector BM25 ∪ AGE graph) shipped end-to-end as `candidate_strategy="hybrid"`, exposed via `palace-daemon`'s `/search/keyword` and `/search/hybrid` endpoints. 8/9 mempalace bench suites pass on postgres backend ([`docs/benchmarks/2026-05-15-remaining-benches.json`](docs/benchmarks/2026-05-15-remaining-benches.json)). Original WAIT-for-#665-upstream-merge stance superseded; the cutover happened on our timeline. Full operator narrative at [`docs/operators/pgvector-cutover-runbook.md`](docs/operators/pgvector-cutover-runbook.md).* This is composition, not a fork-led architectural shift: `BaseBackend` + `BaseCollection` + `PalaceRef` + the entry-point registry already live in upstream develop at [`mempalace/backends/`](https://github.com/MemPalace/mempalace/tree/develop/mempalace/backends), explicitly designed so third-party backends register via Python entry points without touching core. The architectural decision was upstream's; the fork's contribution is choosing pgvector + AGE as one specific implementation worth picking — and running it. @@ -141,7 +141,7 @@ A six-phase plan to fully integrate the mempalace knowledge base with Apache AGE | 1 | `KnowledgeGraphAGE` API parity with the SQLite KG (6 new methods) | `mempalace/knowledge_graph_age.py` | | 2 | Write-through middleware on `PostgresCollection.add/upsert` (every drawer write extracts entities + creates `:MENTIONS` edges in AGE) | `mempalace/kg_writethrough.py` + `mempalace/backends/postgres.py` | | 3 | Palace structure as native AGE nodes (Wing → Room → Drawer + tunnels) | `mempalace/palace_graph_age.py` | -| 4 | `backfill_age` — restartable, checkpointed one-shot for existing drawer tables (~22h for production 274K palace) | `mempalace/backfill_age.py` | +| 4 | `backfill_age` — restartable, checkpointed one-shot for existing drawer tables (~22h for production 300K+ palace) | `mempalace/backfill_age.py` | | 5 | `POST /search/age-fused` endpoint on palace-daemon — vector ⊕ AGE graph RRF fusion | (palace-daemon repo) | | 6 | `mempalace_walk_palace` MCP tool — `start_wing` / `start_room` / `start_entity` × depth | `mempalace/mcp_server.py` | @@ -168,7 +168,7 @@ Independent same-day cross-fork verification by [@nakata-app](https://github.com Three bands of work, all instances of the principles above. Detail rows in the [appendix](#fork-change-inventory) at the bottom. - **Structural retrieval fixes (Principle 1, Principle 2).** Verbatim-only model: hooks no longer write 1KB checkpoint summaries; auto-mined transcript chunks land in `mempalace_drawers` and `mempalace_search` reaches them directly. The earlier dedicated `mempalace_session_recovery` collection (Apr 25–May 5) and its read-only `mempalace_session_recovery_read` MCP tool have been retired (May 5 — see `docs/superpowers/specs/2026-05-05-verbatim-only-design.md`). Net result: one collection, one search path, no kind-filter / over-fetch hack. `drawer_id` surfacing on every search/diary hit so callers can build citation popovers and follow-ups. -- **Single-writer architecture (Principle 3).** [palace-daemon](https://github.com/jphein/palace-daemon) is the only process that opens the palace; clients connect over HTTP. ChromaDB 1.5.x's HNSW concurrency hazards (`#974`/`#965`/`#823` family) become structurally impossible. Cold-start integrity sniff-test on segment metadata files prevents `quarantine_stale_hnsw` from destroying healthy indexes during async-flush lag. Cherry-pick of upstream [#1085](https://github.com/MemPalace/mempalace/pull/1085) for 10–30× mining speedup; fork-side boundary-level None-metadata coercion that closes a per-site-guard family, filed upstream as [#1094](https://github.com/MemPalace/mempalace/pull/1094). +- **Single-writer architecture (Principle 3).** [palace-daemon](https://github.com/techempower-org/palace-daemon) is the only process that opens the palace; clients connect over HTTP. ChromaDB 1.5.x's HNSW concurrency hazards (`#974`/`#965`/`#823` family) become structurally impossible. Cold-start integrity sniff-test on segment metadata files prevents `quarantine_stale_hnsw` from destroying healthy indexes during async-flush lag. Cherry-pick of upstream [#1085](https://github.com/MemPalace/mempalace/pull/1085) for 10–30× mining speedup; fork-side boundary-level None-metadata coercion that closes a per-site-guard family, filed upstream as [#1094](https://github.com/MemPalace/mempalace/pull/1094). - **Deterministic hook saves (Principles 1+2+3 compose).** Silent saves bypass auto-memory conflicts entirely — the LLM is no longer in the save path, so `decision: "block"` race conditions and Claude's auto-memory winning over MCP tools both go away. Verbatim transcript ingest is the entire save path; the save marker advances on each fire and `systemMessage` reports the wing the ingest landed in. PreCompact does the same — sync-mines the transcript before context boundary, no separate marker write. ## Quickstart @@ -183,7 +183,7 @@ uv run mempalace mine ~/Projects/myproject uv run mempalace search "why did we switch to GraphQL" ``` -For a daemon-fronted deployment (recommended once palace size reaches the multi-thousand-drawer range), see [palace-daemon](https://github.com/jphein/palace-daemon)'s setup. The fork's `scripts/deploy.sh` is a one-command Syncthing-aware redeploy: push fork main, restart palace-daemon, post-restart import-check that the new fork-ahead surface is loaded. +For a daemon-fronted deployment (recommended once palace size reaches the multi-thousand-drawer range), see [palace-daemon](https://github.com/techempower-org/palace-daemon)'s setup. The fork's `scripts/deploy.sh` is a one-command Syncthing-aware redeploy: push fork main, restart palace-daemon, post-restart import-check that the new fork-ahead surface is loaded. ## What it looks like in production @@ -229,7 +229,7 @@ Same instinct as the verbatim-vs-derivative axis. Derivative work belongs *next Hierarchy works when it's derived from unambiguous signals (cwd, transcript path, project directory) — not when it's hand-classified by content inspection. The earlier mistake was conflating "hierarchy is bad" with "mandatory synchronous classification is bad" — different claims. **Good uses of hierarchy, which we keep:** -- **Browseable scope** for serendipitous recall across ~160K drawers. +- **Browseable scope** for serendipitous recall across 300K+ drawers. - **Deletion and retention as a unit.** Purging an abandoned project is one operation, not a risky query-then-delete. - **Disambiguation without query gymnastics.** The same keyword across years of unrelated work. - **Auto-surfacing priors.** A wing derived from cwd is a cheap, unambiguous scoping signal. @@ -253,7 +253,7 @@ Claude Code has two complementary memory layers, used in tandem: | Layer | Storage | Size | Consolidation | Purpose | |---|---|---|---|---| | **Auto-memory** | `~/.claude/projects/*/memory/*.md` | ~17 files (this project) | **Auto Dream** (research preview in both Claude Code and the Managed Agents Dreams API; beta header `dreaming-2026-04-21`) | Preferences, feedback, context | -| **MemPalace** | palace-daemon at `http://disks.jphe.in:8085` (ChromaDB on the daemon host) | ~160K drawers | None — deliberately. See below. | Verbatim conversations, tool output, code | +| **MemPalace** | palace-daemon at `http://disks.jphe.in:8085` (postgres + pgvector + Apache AGE) | 300K+ drawers | None — deliberately. See below. | Verbatim conversations, tool output, code | **Auto Dream's arrival ratifies the verbatim-vs-derivative axis.** Anthropic shipped Auto Dream in two surfaces in late April, both research preview: a [research-preview consolidator inside Claude Code](https://claudefa.st/blog/guide/mechanics/auto-dream) (manual `/dream` or auto-trigger when both 24 hours and 5+ sessions have elapsed since the last run; reads `~/.claude/projects//memory/` + MEMORY.md + JSONL transcripts + daily logs; writes back to the same memory directory) and a [Dreams API in Managed Agents](https://platform.claude.com/docs/en/managed-agents/dreams) (REST research preview, beta header `dreaming-2026-04-21`, supported models `claude-opus-4-7` and `claude-sonnet-4-6`, up to 100 sessions per dream, optional 4096-character `instructions`). @@ -316,7 +316,7 @@ Knowledge lives across 7+ layers: global CLAUDE.md, project CLAUDE.md, auto-memo A meaningful shift in 2026-04 and 2026-05: this fork increasingly *composes with* upstream rather than carrying parallel implementations. -- **Cherry-picks (in-flight upstream PRs we use early):** [#665](https://github.com/MemPalace/mempalace/pull/665) PostgreSQL backend (commit `5e90c72`, the substrate work above), [#1085](https://github.com/MemPalace/mempalace/pull/1085) batched inserts (`6be6fff` — CLOSED 2026-05-05, superseded by merged [#1185](https://github.com/MemPalace/mempalace/pull/1185); safe to drop on next sync), [#1087 rewrite](https://github.com/MemPalace/mempalace/pull/1087) `cmd_purge` via `delete(where=)` (`366a9ad`), [#1094](https://github.com/MemPalace/mempalace/pull/1094) None-metadata coercion (`43d728d`). +- **Cherry-picks (in-flight upstream PRs we use early):** [#665](https://github.com/MemPalace/mempalace/pull/665) PostgreSQL backend (commit `5e90c72`, the substrate work above), [#1085](https://github.com/MemPalace/mempalace/pull/1085) batched inserts (`6be6fff` — CLOSED 2026-05-16, superseded by merged [#1185](https://github.com/MemPalace/mempalace/pull/1185); safe to drop on next sync), [#1087 rewrite](https://github.com/MemPalace/mempalace/pull/1087) `cmd_purge` via `delete(where=)` (`366a9ad`), [#1094](https://github.com/MemPalace/mempalace/pull/1094) None-metadata coercion (`43d728d`). - **Co-authored merges:** [#1377](https://github.com/MemPalace/mempalace/pull/1377) (surgical `_get_collection` retry-once, shipped in v3.3.5 — originated from this fork via #1286 which igorls closed and re-extracted with `Co-authored-by` credit). - **Coordinated reviews:** [#1199](https://github.com/MemPalace/mempalace/pull/1199) (rmdes' unbounded-ingest fix — pulled and tested locally, +1 with composition note), [#1219](https://github.com/MemPalace/mempalace/pull/1219) (pepo72's drawer_id — narrower than ours; offered the diary/recovery extension), [RFC 001 #743](https://github.com/MemPalace/mempalace/pull/743) (storage backend spec — flagged the multi-collection-by-purpose pattern as worth naming explicitly). - **Closed in favor of upstream:** [#1171](https://github.com/MemPalace/mempalace/pull/1171) cross-process write lock (closed 2026-04-25 — Felipe's [#976](https://github.com/MemPalace/mempalace/pull/976) `mine_global_lock` at the right layer plus daemon-strict architecture obsoleted ours). @@ -329,7 +329,7 @@ From a 2026-04-21 sweep of upstream MemPalace issue + comment + discussion histo ### Companion tools (compose with MemPalace, don't replace it) -- **[palace-daemon](https://github.com/rboarescu/palace-daemon)** (@rboarescu) — FastAPI gateway + MCP-over-HTTP proxy. Three asyncio semaphores (read / write / mine). Pins correctness floor at MemPalace ≥3.3.2. **This fork migrated to palace-daemon on 2026-04-24** ([`c09582c`](https://github.com/jphein/mempalace/commit/c09582c) wired MCP + hooks; [`0e97b19`](https://github.com/jphein/mempalace/commit/0e97b19) added daemon-strict mode). All reads and writes from the plugin flow through the daemon; auto-migrate-on-startup of the checkpoint split landed as palace-daemon [`034023c`](https://github.com/jphein/palace-daemon/commit/034023c). JP's deployment runs at [`jphein/palace-daemon`](https://github.com/jphein/palace-daemon). +- **[palace-daemon](https://github.com/rboarescu/palace-daemon)** (@rboarescu) — FastAPI gateway + MCP-over-HTTP proxy. Three asyncio semaphores (read / write / mine). Pins correctness floor at MemPalace ≥3.3.2. **This fork migrated to palace-daemon on 2026-04-24** ([`c09582c`](https://github.com/techempower-org/mempalace/commit/c09582c) wired MCP + hooks; [`0e97b19`](https://github.com/techempower-org/mempalace/commit/0e97b19) added daemon-strict mode). All reads and writes from the plugin flow through the daemon; auto-migrate-on-startup of the checkpoint split landed as palace-daemon [`034023c`](https://github.com/techempower-org/palace-daemon/commit/034023c). JP's deployment runs at [`techempower-org/palace-daemon`](https://github.com/techempower-org/palace-daemon). - **[engram](https://github.com/NickCirv/engram)** (@NickCirv) — File-read interception for AI coding assistants. Uses MemPalace as one of six context providers via `mcp-mempalace mempalace-search`; caches with 1h TTL. Upstream [discussion #798](https://github.com/MemPalace/mempalace/discussions/798). - **[engram](https://github.com/harreh3iesh/engram)** (@harreh3iesh — different project, same name) — Hooks + tools for AI memory, first-class MemPalace backend. **Stuck detector** (`PreToolUse` hook counts Grep/Glob calls and nudges the AI when spinning) is a pattern worth borrowing. Upstream [discussion #748](https://github.com/MemPalace/mempalace/discussions/748). - **[cdd-mempalace](https://github.com/fuzzymoomoo/cdd-mempalace)** (@fuzzymoomoo) — Bridge library mapping Context-Driven Development methodology onto wings/halls/rooms. Multiple active upstream PRs. @@ -358,7 +358,7 @@ Built *on top of* or *alongside* MemPalace by community contributors who use the - **[Kent](https://github.com/kenchambers/kent)** (@kenchambers) — typed async agent runtime with APO training subsystem and recall games. - **[adaptmem](https://github.com/nakata-app/adaptmem)** (@nakata-app) — encoder fine-tune adapter; orthogonal lift across retrieval modes. - **[GraphPalace](https://github.com/web3guru888/GraphPalace)** (@web3guru888) — graph-layer build. Forked at [jphein/GraphPalace](https://github.com/jphein/GraphPalace). -- **[mempalace-viz](https://github.com/JoeDoesJits/mempalace-viz)** (@JoeDoesJits) — visualization layer (wings, rooms, tunnels, drawer counts). Forked at [jphein/mempalace-viz](https://github.com/jphein/mempalace-viz). +- **[mempalace-viz](https://github.com/JoeDoesJits/mempalace-viz)** (@JoeDoesJits) — visualization layer (wings, rooms, tunnels, drawer counts). Forked at [jphein/mempalace-viz](https://github.com/techempower-org/mempalace-viz). - **[AutomataArena](https://github.com/astrutt/AutomataArena)** (@astrutt) — multi-agent orchestration substrate. Forked at [jphein/AutomataArena](https://github.com/jphein/AutomataArena). ### Active forks beyond ours @@ -373,13 +373,12 @@ Built *on top of* or *alongside* MemPalace by community contributors who use the ## Open upstream PRs -Ten open from this fork as of 2026-05-12. Run `gh pr list --repo MemPalace/mempalace --author jphein --state open` for the live list. Two merged today: [#1459](https://github.com/MemPalace/mempalace/pull/1459) (empty-metadata sentinel) and [#1474](https://github.com/MemPalace/mempalace/pull/1474) (convo_miner bulk pre-fetch). +Open from this fork as of 2026-05-22. Run `gh pr list --repo MemPalace/mempalace --author jphein --state open` for the live list. Recently merged: [#1024](https://github.com/MemPalace/mempalace/pull/1024) (configurable chunking, 2026-05-15), [#1459](https://github.com/MemPalace/mempalace/pull/1459) (empty-metadata sentinel, 2026-05-12) and [#1474](https://github.com/MemPalace/mempalace/pull/1474) (convo_miner bulk pre-fetch, 2026-05-12). | PR | Status | Description | |---|---|---| | [#660](https://github.com/MemPalace/mempalace/pull/660) | CI green, awaiting review | L1 importance pre-filter | | [#1005](https://github.com/MemPalace/mempalace/pull/1005) | CI green, Dialectician-acked | Warnings + sqlite BM25 top-up — never silently return fewer results than scope contains | -| [#1024](https://github.com/MemPalace/mempalace/pull/1024) | CI green, qodo-acked | Configurable `chunk_size` / `chunk_overlap` / `min_chunk_size` | | [#1086](https://github.com/MemPalace/mempalace/pull/1086) | CI green, awaiting review | `mempalace export` CLI wrapper | | [#1087](https://github.com/MemPalace/mempalace/pull/1087) | CI green, **rewritten 2026-04-26** per @igorls's review | `mempalace purge --wing/--room` via `delete(where=)` (no nuke-and-rebuild) | | [#1094](https://github.com/MemPalace/mempalace/pull/1094) | CI green, awaiting review | Coerce `None` metadatas to `{}` at `ChromaCollection` boundary | @@ -432,17 +431,15 @@ The canonical source is [`docs/fork-changes.yaml`](docs/fork-changes.yaml); [`FO | Area | Change | Status | Files | |---|---|---|---| -| **Backend** | **Postgres + pgvector + Apache AGE substrate implementation.** Cherry-pick of upstream [#665](https://github.com/MemPalace/mempalace/pull/665) (skuznetsov's PostgreSQL backend on the RFC 001 `BaseBackend` contract). Smoke tests + BaseCollection conformance test added. Live test container on JP's homelab LAN (`10.0.6.120:5433`) running PG16 + pgvector 0.8.2 + AGE 1.6.0. Composition stance: WAIT for #665 to merge; Plan-B trigger 2026-06-08. | In-flight on `feat/pgvector-age-impl` — PR [#21](https://github.com/jphein/mempalace/pull/21). Fork commits `5e90c72` (cherry-pick), `04c6294` (smoke), `1d5486f` (docstring cleanup). | `mempalace/backends/postgres.py`, `mempalace/palace.py`, `tests/test_backends_postgres.py`, `docs/internal/pgvector-665-decision.md` | -| **Reliability** | **Daemon-strict migration completion** (May 7). Closes the last desktop-side write paths that bypassed palace-daemon. `mcp_server.py` gates at the `handle_request` JSON-RPC chokepoint and forwards every method to the daemon's `/mcp` proxy when `PALACE_DAEMON_URL` is set; `cli.py` gates `cmd_status`, `cmd_search`, `cmd_mine` against the same env var. Mirrors the gate `hooks_cli.py` already uses (2026-04-24, drift-incident fix). With the local `mempalace-data/` no longer pinned by `~/.mempalace/config.json`, the canonical palace at `disks.jphe.in:8085` is the single writer for every desktop entry point. CLI `init`/`repair`/`export`/`sweep`/`purge`/`mined`/`wakeup` stay local because they need on-host filesystem access. | Fork-ahead, pitchable upstream as a single-file replacement for `palace-daemon/clients/mempalace-mcp.py`. Fork commits [`41359ba`](https://github.com/jphein/mempalace/commit/41359ba) (mcp_server) + [`22ef562`](https://github.com/jphein/mempalace/commit/22ef562) (CLI). | `mcp_server.py`, `cli.py`, `tests/conftest.py`, `tests/test_mcp_server_daemon.py`, `tests/test_cli_daemon.py` | -| **Search** | **Verbatim-only retrieval** (May 5). Hooks write only verbatim transcript chunks; the dedicated `mempalace_session_recovery` collection and `mempalace_session_recovery_read` MCP tool are retired. `mempalace_search` reaches all session content directly. Replaces the earlier multi-collection split (Apr 25 → May 5). Spec: `docs/superpowers/specs/2026-05-05-verbatim-only-design.md`. | PRs in review — [`#2`](https://github.com/jphein/mempalace/pull/2) (transcript ingest restore), [`#3`](https://github.com/jphein/mempalace/pull/3) (drop checkpoint writes), [`#5`](https://github.com/jphein/mempalace/pull/5) (retire collection); palace-daemon [`#1`](https://github.com/jphein/palace-daemon/pull/1) (path translation) | `hooks_cli.py`, `mcp_server.py`, `palace.py`, `migrate.py`, `cli.py` | -| **Search** | Surface `drawer_id` in `mempalace_search` results and `mempalace_diary_read` entries. ChromaDB primary key was returned but never plumbed into the result-building loop. | PR pending — fork commit [`9a8bb77`](https://github.com/jphein/mempalace/commit/9a8bb77); upstream [#1219](https://github.com/MemPalace/mempalace/pull/1219) (@pepo72) is the narrower searcher-only equivalent. | `searcher.py`, `mcp_server.py`, `tests/...`, `website/reference/mcp-tools.md` | -| **CLI** | `mempalace mined` lists mined source files grouped by wing × source_file; `mempalace purge --source-file` deletes drawers from a specific file. Closes the "removing manually mined data" half of the mining-management ask. | [`#4`](https://github.com/jphein/mempalace/pull/4) | `cli.py`, `tests/test_cli.py` | -| **Performance** | Cherry-picked upstream [#1085](https://github.com/MemPalace/mempalace/pull/1085) (@midweste) — batch ChromaDB inserts in miner. New `_build_drawer()` + `add_drawers()`. Reported 10–30× mining speedup. | Cherry-pick — fork commit [`6be6fff`](https://github.com/jphein/mempalace/commit/6be6fff). **2026-05-16:** #1085 closed by author, superseded by merged upstream [#1185](https://github.com/MemPalace/mempalace/pull/1185) (wider scope: same batching + optional GPU acceleration). Our cherry-pick is now a no-op against develop; drop on next sync. | `mempalace/miner.py` | -| **Reliability** | Coerce None metadatas at chromadb boundary. Closes the per-site-guard family of None-metadata bugs (#999, #1198, #1201) at one site instead of N. Fork-authored; filed upstream as [#1094](https://github.com/MemPalace/mempalace/pull/1094). | Fork commit [`43d728d`](https://github.com/jphein/mempalace/commit/43d728d); upstream PR open | `backends/chroma.py`, `tests/test_backends.py` | +| **Backend** | **Postgres + pgvector + Apache AGE substrate implementation.** Cherry-pick of upstream [#665](https://github.com/MemPalace/mempalace/pull/665) (skuznetsov's PostgreSQL backend on the RFC 001 `BaseBackend` contract). Smoke tests + BaseCollection conformance test added. Live test container on JP's homelab LAN (`10.0.6.120:5433`) running PG16 + pgvector 0.8.2 + AGE 1.6.0. Composition stance: WAIT for #665 to merge; Plan-B trigger 2026-06-08. | In-flight on `feat/pgvector-age-impl` — PR [#21](https://github.com/techempower-org/mempalace/pull/21). Fork commits `5e90c72` (cherry-pick), `04c6294` (smoke), `1d5486f` (docstring cleanup). | `mempalace/backends/postgres.py`, `mempalace/palace.py`, `tests/test_backends_postgres.py`, `docs/internal/pgvector-665-decision.md` | +| **Reliability** | **Daemon-strict migration completion** (May 7). Closes the last desktop-side write paths that bypassed palace-daemon. `mcp_server.py` gates at the `handle_request` JSON-RPC chokepoint and forwards every method to the daemon's `/mcp` proxy when `PALACE_DAEMON_URL` is set; `cli.py` gates `cmd_status`, `cmd_search`, `cmd_mine` against the same env var. Mirrors the gate `hooks_cli.py` already uses (2026-04-24, drift-incident fix). With the local `mempalace-data/` no longer pinned by `~/.mempalace/config.json`, the canonical palace at `disks.jphe.in:8085` is the single writer for every desktop entry point. CLI `init`/`repair`/`export`/`sweep`/`purge`/`mined`/`wakeup` stay local because they need on-host filesystem access. | Fork-ahead, pitchable upstream as a single-file replacement for `palace-daemon/clients/mempalace-mcp.py`. Fork commits [`41359ba`](https://github.com/techempower-org/mempalace/commit/41359ba) (mcp_server) + [`22ef562`](https://github.com/techempower-org/mempalace/commit/22ef562) (CLI). | `mcp_server.py`, `cli.py`, `tests/conftest.py`, `tests/test_mcp_server_daemon.py`, `tests/test_cli_daemon.py` | +| **Search** | **Verbatim-only retrieval** (May 5). Hooks write only verbatim transcript chunks; the dedicated `mempalace_session_recovery` collection and `mempalace_session_recovery_read` MCP tool are retired. `mempalace_search` reaches all session content directly. Replaces the earlier multi-collection split (Apr 25 → May 5). Spec: `docs/superpowers/specs/2026-05-05-verbatim-only-design.md`. | PRs in review — [`#2`](https://github.com/techempower-org/mempalace/pull/2) (transcript ingest restore), [`#3`](https://github.com/techempower-org/mempalace/pull/3) (drop checkpoint writes), [`#5`](https://github.com/techempower-org/mempalace/pull/5) (retire collection); palace-daemon [`#1`](https://github.com/techempower-org/palace-daemon/pull/1) (path translation) | `hooks_cli.py`, `mcp_server.py`, `palace.py`, `migrate.py`, `cli.py` | +| **Search** | Surface `drawer_id` in `mempalace_search` results and `mempalace_diary_read` entries. ChromaDB primary key was returned but never plumbed into the result-building loop. | PR pending — fork commit [`9a8bb77`](https://github.com/techempower-org/mempalace/commit/9a8bb77); upstream [#1219](https://github.com/MemPalace/mempalace/pull/1219) (@pepo72) is the narrower searcher-only equivalent. | `searcher.py`, `mcp_server.py`, `tests/...`, `website/reference/mcp-tools.md` | +| **CLI** | `mempalace mined` lists mined source files grouped by wing × source_file; `mempalace purge --source-file` deletes drawers from a specific file. Closes the "removing manually mined data" half of the mining-management ask. | [`#4`](https://github.com/techempower-org/mempalace/pull/4) | `cli.py`, `tests/test_cli.py` | +| **Reliability** | Coerce None metadatas at chromadb boundary. Closes the per-site-guard family of None-metadata bugs (#999, #1198, #1201) at one site instead of N. Fork-authored; filed upstream as [#1094](https://github.com/MemPalace/mempalace/pull/1094). | Fork commit [`43d728d`](https://github.com/techempower-org/mempalace/commit/43d728d); upstream PR open | `backends/chroma.py`, `tests/test_backends.py` | | **CLI** | `mempalace purge --wing/--room` via `collection.delete(where=...)`. Earlier nuke-and-rebuild draft predicated on #521's race; @igorls's review traced the stack. Simpler version preserves embedding fn, no rmtree window, routes through `ChromaBackend`. | [#1087](https://github.com/MemPalace/mempalace/pull/1087), rewritten 2026-04-26 per review | `cli.py`, `tests/test_cli.py` | | **CLI** | `mempalace export` CLI wrapper for upstream's existing `export_palace()`. | [#1086](https://github.com/MemPalace/mempalace/pull/1086) | `cli.py` | | **Performance** | L1 importance pre-filter — `importance >= 3` first, full scan fallback. | [#660](https://github.com/MemPalace/mempalace/pull/660) | `layers.py` | -| **Config** | Configurable chunking parameters — `chunk_size` (800), `chunk_overlap` (100), `min_chunk_size` (50) in `config.json`, exposed via `MempalaceConfig`. | [#1024](https://github.com/MemPalace/mempalace/pull/1024) | `config.py`, `miner.py`, `convo_miner.py` | | **Search** | Warnings + sqlite BM25 top-up when vector underdelivers — `search_memories` returns `warnings: [...]` + `available_in_scope`; fallback hits tagged `matched_via: "sqlite_bm25_fallback"`. The palace never silently returns fewer results than the scope contains. | [#1005](https://github.com/MemPalace/mempalace/pull/1005) | `searcher.py` | | **Docs** | `docs/RELEASING.md` with `mempalace-mcp` pre-release grep. | [#1142](https://github.com/MemPalace/mempalace/pull/1142), accepted by @bensig 2026-04-23 | `docs/RELEASING.md` | | **Hooks** | `mempal_save_hook.sh` Python auto-detection (`MEMPAL_PYTHON` → repo venv → system `python3`). Same pattern in `.claude-plugin/`. Replied on [#1049](https://github.com/MemPalace/mempalace/issues/1049) offering autodetect, awaiting maintainer arbitration on [#1069](https://github.com/MemPalace/mempalace/issues/1069). | PR pending after #1069 direction | `hooks/mempal_save_hook.sh`, `.claude-plugin/hooks/...` | @@ -450,6 +447,7 @@ The canonical source is [`docs/fork-changes.yaml`](docs/fork-changes.yaml); [`FO ### Recently merged into upstream +- **2026-05-15:** [#1024](https://github.com/MemPalace/mempalace/pull/1024) — Configurable `chunk_size` / `chunk_overlap` / `min_chunk_size` exposed via `MempalaceConfig` - **2026-05-06 (in v3.3.5):** [#1377](https://github.com/MemPalace/mempalace/pull/1377) — `_get_collection` retry-once + log-on-failure (~25 LOC + 2 tests; co-authored from this fork via the closed #1286) - **2026-05-01 (post-v3.3.4):** [#1262](https://github.com/MemPalace/mempalace/pull/1262) (get-then-create at chromadb backend boundary), [#1289](https://github.com/MemPalace/mempalace/pull/1289) (MCP-server-side companion), [#1303](https://github.com/MemPalace/mempalace/pull/1303) (`embedding_function=` plumbing — prevents ONNX SIGSEGV on Py3.14) - **2026-04-26:** [#1173](https://github.com/MemPalace/mempalace/pull/1173) (`quarantine_stale_hnsw` cold-start gate + integrity sniff), [#1177](https://github.com/MemPalace/mempalace/pull/1177) (`.blob_seq_ids_migrated` marker), [#1198](https://github.com/MemPalace/mempalace/pull/1198) (`_tokenize` None guard), [#1201](https://github.com/MemPalace/mempalace/pull/1201) (`palace_graph` None metadata) @@ -461,6 +459,7 @@ The canonical source is [`docs/fork-changes.yaml`](docs/fork-changes.yaml); [`FO ### Closed (superseded or withdrawn) +- [#1085](https://github.com/MemPalace/mempalace/pull/1085) (cherry-pick — closed by @midweste 2026-05-16, superseded by merged upstream [#1185](https://github.com/MemPalace/mempalace/pull/1185) with wider GPU-acceleration scope; fork's `6be6fff` is a no-op against develop, safe to drop on next sync) - [#1286](https://github.com/MemPalace/mempalace/pull/1286) (drifted; @igorls closed and re-extracted the surgical fix as #1377 with co-author credit) - [#1171](https://github.com/MemPalace/mempalace/pull/1171) (cross-process write lock — superseded by #976 + daemon-strict) - [#1146](https://github.com/MemPalace/mempalace/pull/1146) (duplicate of @igorls's [#1147](https://github.com/MemPalace/mempalace/pull/1147)) @@ -492,7 +491,7 @@ Articles, surveys, and research that shaped the fork's direction. - [**Claude Code Auto Dream guide**](https://claudefa.st/blog/guide/mechanics/auto-dream) — research-preview consolidation inside Claude Code; reads memory + JSONL, writes consolidated index. - [**Microsoft Tech Community — Combining pgvector and Apache AGE: knowledge graph & semantic intelligence in a single engine**](https://techcommunity.microsoft.com/blog/adforpostgresql/combining-pgvector-and-apache-age---knowledge-graph--semantic-intelligence-in-a-/4508781) (Raunak, 2026-04-15) — bridge-pattern reference for the substrate work. - [**Dave's Garage — "My Custom AI Went Superhuman Yesterday..."**](https://www.youtube.com/watch?v=TdbpoDjIvPk) (Dave Plummer, 2026-02-28) — conceptual reference for why graph structure matters in retrieval: vectors get you "topically nearby"; the graph gets you "actually related." -- [**Phil Karlton's two hard things**](https://martinfowler.com/bliki/TwoHardThings.html) — naming and cache invalidation. Cited in "What this fork has learned" because, even at ~160K drawers, the day-to-day operational work is still mostly these two. +- [**Phil Karlton's two hard things**](https://martinfowler.com/bliki/TwoHardThings.html) — naming and cache invalidation. Cited in "What this fork has learned" because, even at 300K+ drawers, the day-to-day operational work is still mostly these two. - [**Recursive Language Models**](https://arxiv.org/abs/2512.24601) (Zhang, Kraska, Khattab, 2025) — the RLM paper. The four-layer section's invocation-ceiling finding is measured against this paper's mechanism. - [**Think, But Don't Overthink: Reproducing Recursive Language Models**](https://arxiv.org/abs/2603.02615) — depth-2 collapse finding behind the recommendation to keep RLM at composition rather than retrieval. diff --git a/docs/fork-changes.yaml b/docs/fork-changes.yaml index 93466b8b31..d1b7ace002 100644 --- a/docs/fork-changes.yaml +++ b/docs/fork-changes.yaml @@ -1497,7 +1497,8 @@ entries: merges. Amends fork-ahead row 17. tests: "1548/1548 (was 1546/1548 with 2 isolation failures in test_convo_miner)" pr: 1024 - pr_state: OPEN + pr_state: MERGED + merged_date: 2026-05-15 files: - mempalace/config.py supersedes: [] @@ -1763,12 +1764,16 @@ entries: and ``os.path.getmtime()`` to file-level (2 syscalls per file instead of 2N). Reported 10–30× mining speedup upstream. Fork-side resolution preserved fork's existing ``DRAWER_UPSERT_BATCH_SIZE=1000``; aliased - upstream's ``CHROMA_BATCH_LIMIT`` to it. Becomes a no-op when #1085 - merges to develop and we next sync. + upstream's ``CHROMA_BATCH_LIMIT`` to it. **2026-05-16:** #1085 was + closed by @midweste, superseded by merged upstream + [#1185](https://github.com/MemPalace/mempalace/pull/1185) (wider + scope: same batching + optional GPU acceleration). The fork-side + cherry-pick is now a no-op against develop; drop on next sync. pr: 1085 - pr_state: OPEN + pr_state: CLOSED files: - mempalace/miner.py + supersedes: [] - id: deploy-script date: 2026-04-26 @@ -1885,6 +1890,7 @@ merged_upstream: - "Trim entries from this list once they're more than ~30 days old." - "See CHANGELOG.md (upstream) for the full released history." entries: + - { pr: 1024, title: "Configurable chunk_size / chunk_overlap / min_chunk_size", merged: 2026-05-15 } - { pr: 1173, title: "quarantine_stale_hnsw on make_client + cold-start gate + integrity sniff", merged: 2026-04-26 } - { pr: 1177, title: "`.blob_seq_ids_migrated` marker guard (closes #1090)", merged: 2026-04-26 } - { pr: 1198, title: "_tokenize None-document guard in BM25 reranker", merged: 2026-04-26 } diff --git a/scripts/check-docs.sh b/scripts/check-docs.sh index ff065f7200..e428c7e951 100755 --- a/scripts/check-docs.sh +++ b/scripts/check-docs.sh @@ -45,13 +45,13 @@ failures=0 # ── 1. test count ──────────────────────────────────────────────────────── step "1/4 test count in README" -readme_count=$(grep -oE '^[0-9]+ tests pass on `main`' README.md | grep -oE '^[0-9]+' || echo "") +readme_count=$(grep -oE '[0-9]+ tests pass on `main`' README.md | grep -oE '^[0-9]+' || echo "") if [ -z "$readme_count" ]; then - warn "README has no ' tests pass on \`main\`' line — skipping" + warn "README has no ' tests pass on \`main\`' phrase — skipping" else # Prefer the repo venv's pytest so the check works without an # activated environment. Falls back to whatever pytest is on PATH. - pytest_bin="$REPO_ROOT/venv/bin/pytest" + pytest_bin="$REPO_ROOT/.venv/bin/pytest" [ -x "$pytest_bin" ] || pytest_bin="$(command -v pytest 2>/dev/null || true)" if [ -z "$pytest_bin" ]; then warn "no pytest available — skipping" @@ -79,7 +79,7 @@ step "2/4 commit hashes referenced in docs resolve" docs=(README.md CLAUDE.md FORK_CHANGELOG.md) # Strip cross-repo URLs first so we only check hashes that should resolve # in *this* fork. Pattern: anything inside (https://github.com///commit/HASH) -# where / is not jphein/mempalace. +# where / is not techempower-org/mempalace. # For each line, skip the line entirely if it mentions a sibling repo # (palace-daemon / multipass-structural-memory-eval) — we can't tell which # hashes on that line are fork-mempalace vs cross-repo without parsing @@ -88,7 +88,7 @@ docs=(README.md CLAUDE.md FORK_CHANGELOG.md) # adjacent to a sibling-repo mention) but no false positives. mapfile -t hashes < <( for d in "${docs[@]}"; do - grep -v -E 'palace-daemon|multipass-structural-memory-eval|/jphein/[a-z-]+/commit/' "$d" 2>/dev/null + grep -v -E 'palace-daemon|multipass-structural-memory-eval|/(jphein|techempower-org)/[a-z-]+/commit/' "$d" 2>/dev/null done | grep -hoE '`[0-9a-f]{7,40}`' | tr -d '`' | sort -u ) unresolved=0 @@ -106,7 +106,7 @@ fi step "3/4 FORK_CHANGELOG.md regenerates clean" render_bin="$REPO_ROOT/scripts/render-docs.py" if [ -x "$render_bin" ]; then - py="$REPO_ROOT/venv/bin/python" + py="$REPO_ROOT/.venv/bin/python" [ -x "$py" ] || py="$(command -v python3 2>/dev/null || true)" if [ -z "$py" ]; then warn "no python interpreter — skipping render check" @@ -201,7 +201,7 @@ fi # Check 5 (fork-only YAML commits → CLAUDE.md row inventory) retired # 2026-05-11: the CLAUDE.md row inventory it validated was removed in -# favor of a pointer block to FORK_CHANGELOG.md + jphein/mempalace +# favor of a pointer block to FORK_CHANGELOG.md + techempower-org/mempalace # issues. Check 3 (FORK_CHANGELOG.md ↔ YAML) already guarantees the # meaningful sync property — every fork-only YAML commit appears in the # rendered FORK_CHANGELOG.md by construction, so a separate CLAUDE.md diff --git a/scripts/preflight.sh b/scripts/preflight.sh index 9e6e11cfe3..6888fa1ba7 100755 --- a/scripts/preflight.sh +++ b/scripts/preflight.sh @@ -11,9 +11,9 @@ set -e cd "$(dirname "$0")/.." -if [ -x venv/bin/python ]; then +if [ -x .venv/bin/python ]; then # shellcheck disable=SC1091 - source venv/bin/activate + source .venv/bin/activate fi echo "→ ruff check ." diff --git a/scripts/render-docs.py b/scripts/render-docs.py index a6f4df53a4..e5faacf261 100755 --- a/scripts/render-docs.py +++ b/scripts/render-docs.py @@ -39,7 +39,7 @@ CHANGELOG_PATH = REPO_ROOT / "FORK_CHANGELOG.md" CHANGELOG_HEADER = """\ -# Fork Changelog (jphein/mempalace) +# Fork Changelog (techempower-org/mempalace) Fork-ahead changes that aren't yet in upstream `MemPalace/mempalace`. Upstream's release history lives in [`CHANGELOG.md`](CHANGELOG.md); @@ -74,7 +74,7 @@ def load_manifest(path: Path = YAML_PATH) -> dict: def commit_link(sha: str) -> str: """Render a 7-char SHA as a markdown link to the fork commit.""" - return f"[`{sha}`](https://github.com/jphein/mempalace/commit/{sha})" + return f"[`{sha}`](https://github.com/techempower-org/mempalace/commit/{sha})" def render_entry(entry: dict[str, Any]) -> str: