Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,18 @@ The sections above are the fork's inherited mission and principles — same word

## What This Is

TechEmpower's production fork of [MemPalace/mempalace](https://github.com/MemPalace/mempalace) — a local AI memory system. Transferred from `jphein/mempalace` to `techempower-org/mempalace` in May 2026. Drawer storage is postgres + pgvector behind a palace-daemon at `disks.jphe.in:8085`; the legacy ChromaDB-on-katana palace has been retired (see `~/.mempalace/RETIRED`).
TechEmpower's production fork of [MemPalace/mempalace](https://github.com/MemPalace/mempalace) — a local AI memory system. Transferred from `jphein/mempalace` to `techempower-org/mempalace` in May 2026. Drawer storage is postgres + pgvector behind a palace-daemon at `familiar.jphe.in:8085`; the legacy ChromaDB-on-katana palace has been retired (see `~/.mempalace/RETIRED`).

- **Fork**: `techempower-org/mempalace` (origin) / `MemPalace/mempalace` (upstream)
- **Version + sync state**: `cat mempalace/__init__.py` for fork version; `git log --oneline upstream/develop ^HEAD | head -5` for unmerged upstream commits. Release/landed-PR history in `FORK_CHANGELOG.md`.
- **Python**: venv at `./.venv/` (uv-managed), editable install with dev deps. Activate via `source .venv/bin/activate` or invoke binaries directly (`.venv/bin/python -m pytest`, `.venv/bin/mempalace status`).
- **Palace data**: production lives in postgres on `disks.jphe.in`; the local `~/.mempalace/palace/` chroma store was retired on the 2026-05-14 pgvector cutover and renamed to `~/.mempalace/palace.retired-pre-pgcutover-2026-05-14/`. Daemon connection via `~/.config/palace-daemon/env` (sourced by the MCP wrapper at `~/Projects/palace-daemon/clients/mempalace-mcp-wrapper.sh`).
- **Palace data**: production lives in postgres on `familiar.jphe.in`; the local `~/.mempalace/palace/` chroma store was retired on the 2026-05-14 pgvector cutover and renamed to `~/.mempalace/palace.retired-pre-pgcutover-2026-05-14/`. Daemon connection via `~/.config/palace-daemon/env` (sourced by the MCP wrapper at `~/Projects/palace-daemon/clients/mempalace-mcp-wrapper.sh`).

## Key Files

- `~/Projects/mempalace.yaml` — **do not delete**. Mining config with wing/room definitions. Regenerate with `mempalace init ~/Projects --yes` if lost.
- `~/.mempalace/config.json` — topic wings and hall keywords, customized for JP's domains (infrastructure, development, tools, creative, projects, system).
- **Palace data lives in postgres on `disks.jphe.in`**, reached via palace-daemon at `http://disks.jphe.in:8085`. The local `~/.mempalace/palace/` ChromaDB store was retired on the 2026-05-14 pgvector cutover (renamed to `~/.mempalace/palace.retired-pre-pgcutover-2026-05-14/`); a `~/.mempalace/RETIRED` marker refuses default-palace opens so misconfigured tools can't silently fall back to the stale local store.
- **Palace data lives in postgres on `familiar.jphe.in`**, reached via palace-daemon at `http://familiar.jphe.in:8085`. The local `~/.mempalace/palace/` ChromaDB store was retired on the 2026-05-14 pgvector cutover (renamed to `~/.mempalace/palace.retired-pre-pgcutover-2026-05-14/`); a `~/.mempalace/RETIRED` marker refuses default-palace opens so misconfigured tools can't silently fall back to the stale local store.
- `~/.mempalace/hook_state/` — stop hook session tracking.

## Development
Expand Down Expand Up @@ -83,7 +83,7 @@ Workflow for landing new fork-ahead changes lives in [Documentation maintenance]
Claude Code has two complementary memory layers, used in tandem:

- **Auto-memory** (`~/.claude/projects/*/memory/`) — lightweight preferences, context, feedback. Manual writes only. (Anthropic's "Auto Dream" research-preview shipped late April 2026 in Claude Code `/dream` + the Managed Agents Dreams API; MemPalace deliberately stays un-consolidated and the Dreams API design ratifies the verbatim-vs-derivative axis. See `~/.claude/projects/-home-jp-Projects-memorypalace/memory/project_auto_dream.md`.)
- **MemPalace** (postgres + pgvector + AGE on `disks.jphe.in`, reached via palace-daemon at `http://disks.jphe.in:8085`, 300K+ drawers) — verbatim conversations, tool output, code. Write-only archive, searchable via MCP. Completeness is the feature. Wing/room layout follows the canonical 7-room taxonomy (see `palace-taxonomy` skill / `~/Projects/familiar.realm.watch/docs/superpowers/specs/2026-05-13-palace-room-taxonomy.md`).
- **MemPalace** (postgres + pgvector + AGE on `familiar.jphe.in`, reached via palace-daemon at `http://familiar.jphe.in:8085`, 300K+ drawers) — verbatim conversations, tool output, code. Write-only archive, searchable via MCP. Completeness is the feature. Wing/room layout follows the canonical 7-room taxonomy (see `palace-taxonomy` skill / `~/Projects/familiar.realm.watch/docs/superpowers/specs/2026-05-13-palace-room-taxonomy.md`).

Both systems coexist. Hook saves are scoped to MemPalace ("For THIS save, use MemPalace MCP tools only") — this is not a permanent ban on auto-memory.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@

## Current state

**Substrate (2026-05-15).** Postgres + pgvector + Apache AGE shipped on `main` and serving production traffic. PG16 + pgvector 0.8.2 + AGE 1.6.0 on `disks.jphe.in:5433`. One engine consolidates vector search, full-text search (tsvector BM25), graph traversal, and the temporal entity-relationship store — previously four separate systems (ChromaDB + SQLite + graph cache). 8/9 bench suites pass. Full operator narrative at [`docs/operators/pgvector-cutover-runbook.md`](docs/operators/pgvector-cutover-runbook.md).
**Substrate (2026-05-15).** Postgres + pgvector + Apache AGE shipped on `main` and serving production traffic. PG16 + pgvector 0.8.2 + AGE 1.6.0 on `familiar.jphe.in:5433`. One engine consolidates vector search, full-text search (tsvector BM25), graph traversal, and the temporal entity-relationship store — previously four separate systems (ChromaDB + SQLite + graph cache). 8/9 bench suites pass. Full operator narrative at [`docs/operators/pgvector-cutover-runbook.md`](docs/operators/pgvector-cutover-runbook.md).

Check failure on line 131 in README.md

View workflow job for this annotation

GitHub Actions / vale (advisory)

[vale] reported by reviewdog 🐶 [Vale.Terms] Use 'chromadb' instead of 'ChromaDB'. Raw Output: {"message": "[Vale.Terms] Use 'chromadb' instead of 'ChromaDB'.", "location": {"path": "README.md", "range": {"start": {"line": 131, "column": 343}}}, "severity": "ERROR"}

**AGE integration (2026-05-22).** [PR #101](https://github.com/techempower-org/mempalace/pull/101) merged — six-phase AGE integration complete. Writethrough middleware on every drawer write extracts entities and creates `:MENTIONS` edges in the AGE graph. Backfill running against 335K+ existing drawers at ~5/s. The `mempalace_walk_palace` MCP tool enables Cypher traversal by wing, room, or entity. A [2026-05-17 spike](https://github.com/techempower-org/multipass-structural-memory-eval/blob/feat/rlm-adapter/docs/benchmarks/2026-05-17-age-write-through-spike.md) showed graph signal adds **+9pp R@5** over vector-only retrieval.

Expand Down
12 changes: 6 additions & 6 deletions docs/designs/multi-palace-separation.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ the partition surface proves insufficient.

Hooks today flow:

- Stop-hook: `mempal-stop-hook.sh` → `palace-daemon/clients/hook.py` → `disks.jphe.in:8085/silent-save`
- Stop-hook: `mempal-stop-hook.sh` → `palace-daemon/clients/hook.py` → `familiar.jphe.in:8085/silent-save`
- Manual mine: `mempalace mine <dir>` → `miner.run_mine` → `get_collection(...)`

Each writer needs to declare its target partition. The minimal surface:
Expand Down Expand Up @@ -251,7 +251,7 @@ out of the default federation.
### Q4 — Daemon routing

Today the daemon (`palace-daemon`) is one process, one palace
(`PALACE_DAEMON_URL → http://disks.jphe.in:8085`). It serves `/search`, `/silent-save`,
(`PALACE_DAEMON_URL → http://familiar.jphe.in:8085`). It serves `/search`, `/silent-save`,
`/mcp`, etc., all rooted on a single `palace_path` baked into its config at startup.

**For collection partitioning:** the daemon stays one-process-one-palace. The
Expand Down Expand Up @@ -377,7 +377,7 @@ until multi-palace promotion is on the table; when it lands, alias the legacy

```
┌──────────────┐ ┌──────────────────────────────┐
│ Claude Code │ │ palace-daemon (disks:8085) │
│ Claude Code │ │ palace-daemon (familiar:8085) │
│ │ POST │ │
│ Stop-hook ├────────▶│ /silent-save │
│ shell shim │ │ │ │
Expand All @@ -396,7 +396,7 @@ until multi-palace promotion is on the table; when it lands, alias the legacy

```
┌──────────────┐ ┌─────────────────────────────────────────┐
│ Claude Code │ │ palace-daemon (disks:8085) │
│ Claude Code │ │ palace-daemon (familiar:8085) │
│ │ POST │ │
│ Stop-hook ├────────▶│ /silent-save?partition=drawers │
│ ↑ writes │ │ │ │
Expand Down Expand Up @@ -442,8 +442,8 @@ The Stop-hook shim grows a `PALACE_DAEMON_URL_BY_TARGET` env or config table:
```bash
# In palace-daemon/clients/hook.py
PALACE_DAEMON_URL_BY_TARGET = {
"chat": "http://disks.jphe.in:8085",
"authority": "http://disks.jphe.in:8086",
"chat": "http://familiar.jphe.in:8085",
"authority": "http://familiar.jphe.in:8086",
}
```

Expand Down
8 changes: 4 additions & 4 deletions docs/fork-decisions/sh-shim-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ move all hook logic into the `mempalace` Python CLI.
This fork went the **opposite direction**: the `.sh` shims delegate
to **`palace-daemon/clients/hook.py`** (a stdlib-only Python script
in a separate repo) which talks HTTP to a single FastAPI gateway
(`disks.jphe.in:8085`). The `mempalace` Python package is no longer
(`familiar.jphe.in:8085`). The `mempalace` Python package is no longer
in the hook call path at all.

We are not asking upstream to do this. We are documenting the
Expand Down Expand Up @@ -66,7 +66,7 @@ The single-writer-gateway property is the load-bearing one.
With 300K+ drawers and multiple harnesses (Claude Code, codex,
gemini-cli, opencode, the MCP server) all writing concurrently,
serialization through a single FastAPI process running on
`disks.jphe.in:8085` is the only thing that keeps ChromaDB's
`familiar.jphe.in:8085` is the only thing that keeps ChromaDB's
HNSW from corrupting under concurrent mine jobs (upstream #1161).

## The delegation pattern
Expand All @@ -84,15 +84,15 @@ mempal-stop-hook.sh ◄── thin .sh shim, ~5 lines of logic
▼
palace-daemon/clients/hook.py ◄── stdlib Python, no mempalace import
│
│ urllib.request → POST http://disks.jphe.in:8085/mine
│ urllib.request → POST http://familiar.jphe.in:8085/mine
▼
palace-daemon (FastAPI) ◄── single writer, _mine_sem=1
│
▼
mempalace.mcp_server (in-process)
│
▼
postgres + pgvector + AGE (on disks.jphe.in)
postgres + pgvector + AGE (on familiar.jphe.in)
```

Two properties matter:
Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/opencode.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ OpenCode launched from this repo root logs the following on session start. None

| Log line | What it is |
|---|---|
| `mcp stderr: palace-daemon: connected at http://disks.jphe.in:8085` | The bridge announcing it found the daemon. Good. |
| `mcp stderr: palace-daemon: connected at http://familiar.jphe.in:8085` | The bridge announcing it found the daemon. Good. |
| (Older sessions) `mcp stderr: routing → local palace @ ~/.mempalace/palace` | Pre-2026-05-21 behavior. Indicates the spawn was opening the legacy local palace. Fixed by clearing the repo `.opencode/opencode.json` `mcp` block. If you still see it, your repo checkout is on an older commit — `git pull`. |
| (Older sessions) `mcp stderr: HNSW capacity divergence detected` | Same root cause as the line above — the bridge was opening the legacy local ChromaDB store whose HNSW index lagged its sqlite. Gone after the local palace was archived to `~/.mempalace/palace.retired-pre-pgcutover-2026-05-14/` and the `~/.mempalace/RETIRED` marker was added. |

Expand Down
50 changes: 25 additions & 25 deletions docs/operators/pgvector-cutover-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ Plan: [`docs/superpowers/plans/2026-05-10-pgvector-age-migration-impl.md`](../su

**If you're stopping the daemon** (cleanest):
```bash
ssh disks 'sudo systemctl stop palace-daemon && \
ssh familiar 'sudo systemctl stop palace-daemon && \
cp -al /mnt/raid/projects/mempalace-data/palace \
/mnt/raid/projects/mempalace-data/palace.dry-run-$(date +%Y-%m-%d)'
```
`cp -al` is O(N) hardlinks — instant.

**If you're leaving the daemon running** (use `cp -a`, NOT `cp -al`):
```bash
ssh disks 'cp -a /mnt/raid/projects/mempalace-data/palace \
ssh familiar 'cp -a /mnt/raid/projects/mempalace-data/palace \
/mnt/raid/projects/mempalace-data/palace.dry-run-$(date +%Y-%m-%d)'
```
Real copy (~2 min for 8 GB on local SSD). Hardlinks share inodes with the live palace — ChromaDB 1.5.x's concurrent-writer SIGSEGV will trip when the daemon's client and the migration's client touch the same HNSW files.
Expand All @@ -35,19 +35,19 @@ Symptom: faulthandler trace ends in `chromadb/api/rust.py:440 in _get` returning
Workaround: rebuild HNSW from sqlite (which has all the drawer data intact):

```bash
ssh disks '/mnt/raid/projects/mempalace-dryrun-venv/bin/mempalace \
ssh familiar '/mnt/raid/projects/mempalace-dryrun-venv/bin/mempalace \
--palace /mnt/raid/projects/mempalace-data/palace.dry-run-$(date +%Y-%m-%d) \
repair --mode from-sqlite --archive-existing --yes'
```

This moves the broken snapshot to `palace.dry-run-<date>.pre-rebuild-<timestamp>` and constructs a fresh palace at the original path with the sqlite data re-vectored into clean HNSW segments. Expect 30–60 min for ~270K embeddings.

### 2. Stand up Postgres on disks
### 2. Stand up Postgres on familiar

Pragmatic option:

```bash
ssh disks 'docker run -d --name mempalace-dryrun-pg \
ssh familiar 'docker run -d --name mempalace-dryrun-pg \
-e POSTGRES_PASSWORD=palace \
-e POSTGRES_DB=mempalace_dryrun \
-p 5433:5432 \
Expand All @@ -57,7 +57,7 @@ ssh disks 'docker run -d --name mempalace-dryrun-pg \
`pgvector/pgvector:pg16` ships pgvector but NOT AGE. For full migration:

```bash
ssh disks 'docker run -d --name mempalace-dryrun-pg \
ssh familiar 'docker run -d --name mempalace-dryrun-pg \
-e POSTGRES_PASSWORD=palace \
-e POSTGRES_DB=mempalace_dryrun \
-p 5433:5432 \
Expand All @@ -67,7 +67,7 @@ ssh disks 'docker run -d --name mempalace-dryrun-pg \
Then install pgvector inside the AGE container:

```bash
ssh disks 'docker exec mempalace-dryrun-pg \
ssh familiar 'docker exec mempalace-dryrun-pg \
apt-get update && apt-get install -y postgresql-16-pgvector'
```

Expand All @@ -76,13 +76,13 @@ ssh disks 'docker exec mempalace-dryrun-pg \
### 3. Stop palace-daemon (the migration refuses to run while it's responsive)

```bash
ssh disks 'sudo systemctl stop palace-daemon'
ssh familiar 'sudo systemctl stop palace-daemon'
```

### 4. Run the migration

```bash
ssh disks '/home/jp/.local/share/palace-daemon/venv/bin/mempalace \
ssh familiar '/home/jp/.local/share/palace-daemon/venv/bin/mempalace \
migrate-to-postgres \
--from /mnt/raid/projects/mempalace-data/palace.dry-run-$(date +%Y-%m-%d) \
--to "postgresql://postgres:palace@localhost:5433/mempalace_dryrun" \
Expand All @@ -104,7 +104,7 @@ Record total duration per phase. Phase 2 is the long pole (drawer batch copy thr
### 5. Smoke-test the dry-run Postgres palace

```bash
ssh disks 'MEMPALACE_BACKEND=postgres \
ssh familiar 'MEMPALACE_BACKEND=postgres \
MEMPALACE_POSTGRES_DSN="postgresql://postgres:palace@localhost:5433/mempalace_dryrun" \
MEMPALACE_KG_BACKEND=age \
/home/jp/.local/share/palace-daemon/venv/bin/mempalace search "palace taxonomy"'
Expand All @@ -127,7 +127,7 @@ The migration tool's phase 7 output prints the exact steps. Reproduced here for
### 2. Snapshot the production palace one more time

```bash
ssh disks 'cp -al /mnt/raid/projects/mempalace-data/palace \
ssh familiar 'cp -al /mnt/raid/projects/mempalace-data/palace \
/mnt/raid/projects/mempalace-data/palace.pre-cutover-$(date +%Y-%m-%d)'
```

Expand All @@ -138,7 +138,7 @@ Decision point: same container with a different DB name, or a separate system Po
### 4. Stop palace-daemon

```bash
ssh disks 'sudo systemctl stop palace-daemon'
ssh familiar 'sudo systemctl stop palace-daemon'
```

### 5. Run the migration against production
Expand All @@ -158,40 +158,40 @@ MEMPALACE_KG_BACKEND=age
### 7. Reload + start

```bash
ssh disks 'sudo systemctl daemon-reload && sudo systemctl start palace-daemon'
ssh familiar 'sudo systemctl daemon-reload && sudo systemctl start palace-daemon'
```

### 8. Smoke

```bash
curl http://disks:8085/health
curl -H "X-API-Key: $PALACE_API_KEY" 'http://disks:8085/search?q=palace+taxonomy'
curl http://familiar:8085/health
curl -H "X-API-Key: $PALACE_API_KEY" 'http://familiar:8085/search?q=palace+taxonomy'
```

### 9. Watch hook activity for an hour

```bash
ssh disks 'journalctl -u palace-daemon -f'
ssh familiar 'journalctl -u palace-daemon -f'
```

Confirm Stop hooks, transcript ingests, diary writes all succeed. Time them — postgres should match or beat ChromaDB.

### 10. After 24h of clean operation, archive the chromadb backup

```bash
ssh disks 'mv /mnt/raid/projects/mempalace-data/palace \
ssh familiar 'mv /mnt/raid/projects/mempalace-data/palace \
/mnt/raid/projects/mempalace-data/palace.chromadb-backup-$(date +%Y-%m-%d)'
```

The borg backup on disks picks the directory up by name; the rename means the next backup snapshots BOTH the new postgres palace AND the chromadb backup until you decide to drop the latter.
The borg backup on familiar picks the directory up by name; the rename means the next backup snapshots BOTH the new postgres palace AND the chromadb backup until you decide to drop the latter.

---

## Rollback (if cutover goes wrong)

1. `ssh disks 'sudo systemctl stop palace-daemon'`
1. `ssh familiar 'sudo systemctl stop palace-daemon'`
2. Remove the three `MEMPALACE_*` env additions from the daemon's `EnvironmentFile`
3. `ssh disks 'sudo systemctl daemon-reload && sudo systemctl start palace-daemon'`
3. `ssh familiar 'sudo systemctl daemon-reload && sudo systemctl start palace-daemon'`
4. The daemon resumes against the ChromaDB palace as if nothing happened.

The Postgres palace stays around; you can re-attempt or investigate.
Expand All @@ -208,8 +208,8 @@ this on a fresh palace.
### 1. chromadb open SIGSEGVs on long-lived palaces — even with repair

The runbook's Step 1b suggests `mempalace repair --mode from-sqlite` to
rebuild HNSW from sqlite before migration. On disks's 270k-drawer
palace, that repair script estimated **7+ hours** of wall time (disks's
rebuild HNSW from sqlite before migration. On familiar's 270k-drawer
palace, that repair script estimated **7+ hours** of wall time (familiar's
2011 i5 maxes at ~6 vectors/sec for ONNX embedding). Migration timed
out before it finished.

Expand Down Expand Up @@ -286,7 +286,7 @@ mismatch. Tracked at [#71](https://github.com/techempower-org/mempalace/issues/7

Daemon's systemd `WorkingDirectory` is `/mnt/raid/projects/palace-daemon/`,
but I'd been scp-ing my hand-patches to `/home/jp/.local/share/palace-daemon/`
trusting syncthing to mirror. It does, eventually, but a fresh disks
trusting syncthing to mirror. It does, eventually, but a fresh familiar
reboot could read either side first. Resolved by writing a proper
deploy script (palace-daemon `b303b29`) that rsyncs directly to the
canonical `/mnt/raid/projects/palace-daemon/`.
Expand All @@ -298,8 +298,8 @@ canonical `/mnt/raid/projects/palace-daemon/`.
After the above commits land, the canonical happy path is:

```bash
ssh disks 'sudo systemctl stop palace-daemon'
ssh disks '/home/jp/.local/share/palace-daemon/venv/bin/mempalace \
ssh familiar 'sudo systemctl stop palace-daemon'
ssh familiar '/home/jp/.local/share/palace-daemon/venv/bin/mempalace \
migrate-to-postgres \
--from /mnt/raid/projects/mempalace-data/palace \
--to "postgresql://postgres:<pass>@localhost:5432/mempalace" \
Expand Down
Loading
Loading