fix(backends): require SQLite magic header for chroma + sqlite_exact detect() (#1893)

[messelink]

What does this PR do?

Fixes #1893.

ChromaBackend.detect() and SQLiteExactBackend.detect() were both returning True for a 0-byte SQLite marker file (chroma.sqlite3 / sqlite_exact.sqlite3) because each check was just os.path.isfile(...). On a palace that has any other backend marker alongside one of those stale 0-byte files, resolve_backend_name then raises BackendMismatchError: palace at '<path>' contains multiple backend artifacts: <a>, <b> and the palace becomes unopenable until the user manually rms the empty file.

The 0-byte file appears as a side effect of any sqlite3.connect() on a missing path — Python creates the file immediately but writes the SQLite header only on the first statement. So any code path that touches the marker path with bare sqlite3.connect(), including chromadb's own PersistentClient lazy-init (see the comment at backends/chroma.py:2052), can leave a 0-byte artifact behind.

Change

Both ChromaBackend.detect and SQLiteExactBackend.detect now check the SQLite magic header (b"SQLite format 3\x00", the first 16 bytes) instead of file presence alone. One extra open() + 16-byte read() per backend — detection isn't a hot path.

Properties of the new check (identical for both backends):

• Rejects 0-byte files (the symptom bug: ChromaBackend.detect() returns True for 0-byte chroma.sqlite3, causes BackendMismatchError on multi-artifact palaces #1893 is about)
• Rejects non-SQLite garbage at the canonical path (partial writes, leftover content)
• Doesn't false-negative on real palaces: any palace whose backend has done any work has the magic header on disk (verified empirically — CREATE TABLE is enough to land the header)
• Doesn't couple detect() to any backend-specific schema — magic header is stable across chromadb releases / sqlite_exact schema changes

Both backends fixed together for repo-wide consistency, per the review request on the initial chroma-only revision of this PR.

Test sweep — context for the larger-than-expected diff

Many existing tests used (<path> / "chroma.sqlite3").touch() or .write_bytes(b"") (and one site (<path> / "sqlite_exact.sqlite3").write_bytes(b"")) as a "fake palace" shortcut, exploiting the loose os.path.isfile() semantics. One of these sites even has the comment:

(palace / "chroma.sqlite3").touch()  # pass the isfile guard

— signaling that the convention was a known shortcut, not a contract. After the magic-header check, those stand-ins no longer register, so each call site needed to create a real (if minimal) SQLite database.

To avoid duplicating the same sqlite3.connect + CREATE TABLE boilerplate, this PR introduces tests/_chroma_palace_helper.py following the existing tests/_backend_conformance.py precedent for private-module test utilities. It exports:

• make_minimal_chroma_sqlite(palace_path) — chroma.sqlite3 with header
• make_minimal_sqlite_exact_sqlite(palace_path) — sqlite_exact.sqlite3 with header

Both wrap a shared _write_minimal_sqlite_file(db_path) private helper.

Each previously-shortcut call site now reads make_minimal_<...>_sqlite(palace) — same one-line shape, just honest about producing a real SQLite file.

Files updated for the sweep: test_backends.py, test_daemon.py, test_mcp_server.py, test_palace.py (4 sites), test_qdrant_backend.py, test_repair.py (3 sites), test_searcher.py, test_sqlite_exact_backend.py (3 sites), test_sync.py (2 sites).

test_migrate.py also writes to chroma.sqlite3 paths, but with non-empty payloads (b"original") used to verify rename/relocation behavior, not detection. Those tests don't exercise the detect path and continue to work unchanged.

How to test

uv run pytest tests/ -v --ignore=tests/benchmarks

For each backend the trio of dedicated detection tests:

chroma:

• test_chroma_detect_matches_palace_with_sqlite_header
• test_chroma_detect_rejects_empty_chroma_sqlite
• test_chroma_detect_rejects_non_sqlite_file

sqlite_exact:

• test_sqlite_exact_detect_matches_palace_with_sqlite_header
• test_sqlite_exact_detect_rejects_empty_sqlite_exact_sqlite
• test_sqlite_exact_detect_rejects_non_sqlite_file

Full env-cleared suite: 3140 passed, 20 skipped, 0 failed. ruff check . and ruff format --check . both clean.

Checklist

• Tests pass (uv run pytest tests/ -v --ignore=tests/benchmarks)
• No hardcoded paths
• Linter passes (uv run ruff check .)
• Format check passes (uv run ruff format --check .)