fix(storage): stop ChromaDB from crashing when reopening an existing …

[Legion345]

What does this PR do?

Fixes #1089.

When opening an existing palace, get_or_create_collection was called with hnsw metadata on every open. In ChromaDB 1.5.x, if that metadata differs from what's already stored, the Rust bindings segfault — no traceback, just a core dump. This is what causes the stop hook to crash at session end.

Fix: try get_collection first, fall back to create_collection only when the collection doesn't exist yet. Existing palaces open without touching their metadata; new ones are created with the full settings as before.

Credit to @jphein who described this exact approach in #1089 and has been running it on their fork since April 10 with zero crashes across 400+ starts.

How to test

Open a palace twice in the same session:

from mempalace.backends.chroma import ChromaBackend
import tempfile, os

with tempfile.TemporaryDirectory() as d:
    palace = os.path.join(d, "palace")
    backend = ChromaBackend()
    backend.get_collection(palace, collection_name="mempalace_drawers", create=True)
    backend.get_collection(palace, collection_name="mempalace_drawers", create=True)
    print("no crash") 

Or run the new regression tests directly:
python -m pytest tests/test_backends.py -v -k "idempotent or preserves_existing"

Checklist

• Tests pass (python -m pytest tests/ -v)
• No hardcoded paths
• Linter passes (ruff check .)

[jphein]

Thanks for picking this up @Legion345 — and for the kind credit, that was generous.

A couple of corrections about the field data, since you anchored the PR description on it and I want to make sure the maintainer isn't reviewing under a wrong impression:

The "since April 10" / "400+ starts, zero crashes" line came from my #1089 issue body, and both numbers are off. The guard actually landed on our fork at d3a2d22 on 2026-04-18 (8 days later than I wrote there), and it was dropped during a develop merge-conflict resolution at 7e18a70 on 2026-04-25. So the guard was alive in fork main for 7 days, not 20. I've edited #1089 with the correction so the record there is right.

What's actually true today (and apologies — this paragraph has been edited twice already; the prior versions overcorrected in two different directions). Tracing the call graph: chromadb's get_or_create_collection is called from two places on current fork main — backends/chroma.py:1061 and mcp_server.py:295 — both with identical metadata (hnsw:space, hnsw:num_threads: 1, plus the bloat-guard keys from #1191). The fork's 2-arg ChromaBackend.get_or_create_collection(path, name) legacy shim at chroma.py:1101 (used by migrate.py:237 and cli.py:1002) delegates through get_collection(create=True) to the same metadata-passing line, so metadata is consistent across opens. The residual exposure is legacy palaces created under older chromadb whose stored metadata differs from the current expanded set — a per-palace-history risk, not a structural one. I don't have a clean recurrence count to share. Net: this PR is still genuinely needed because the chromadb-side crash class isn't fixed, and the "validated by JP's fork" framing isn't load-bearing in either direction.

On the diff itself: the approach is right. One small thing I'd flag as a positive — you're catching chromadb.errors.NotFoundError, which is the 1.5.x-renamed exception. Our older code referenced InvalidCollectionException, and yours is the right one going forward. Both regression tests cover the case cleanly. Once this merges I'll pull it in via the next develop sync.

Thanks again for taking this on, and apologies for the back-and-forth on this comment — I shouldn't have edited my own correction with another wrong claim.