fix(chroma): stop quarantining valid all-layer-0 HNSW segments (#1716)

[igorls]

Problem

_hnsw_link_lists_is_usable_for_payload() misclassified a legitimate 0-byte link_lists.bin as structural corruption, causing a self-perpetuating quarantine loop on small / low-fanout HNSW indexes (#1716).

A zero-byte link_lists.bin is not corruption on its own: hnswlib stores the entire layer-0 graph inside data_level0.bin and only writes link_lists.bin for elements promoted to level > 0. An index where every element stays on layer 0 therefore serializes an empty link_lists.bin and loads/searches fine.

The old heuristic (data_level0.bin > floor AND link_lists.bin == 0 → corrupt) triggered the loop the reporter observed:

1. Segment quarantined as <uuid>.drift-<ts> (data_level0.bin = 167,600 B, link_lists.bin = 0 B).
2. Self-repair rebuilds → the rebuilt live segment has the byte-identical all-layer-0 shape.
3. Next cold-start check quarantines it again → loop (20 drift dirs / 221 MB before being noticed), with no ingestion involved. The per-process _quarantined_paths gate resets on every MCP process spawn, so it re-fires every session.

Fix

Use the persist-completion marker as the discriminator. ChromaDB writes index_metadata.pickle last during a flush, so an intact pickle envelope (0x80 head, 0x2e STOP tail) proves the persist finished — the empty link_lists.bin is then the legitimate all-layer-0 shape, not a half-written one.

An empty link_lists.bin is treated as a partial flush only when there is real payload and no completion marker (pickle absent or truncated). This preserves the #1457 partial-flush protection while eliminating the false positive.

The byte-sniff is factored into _hnsw_metadata_marker_intact() and reused by _segment_appears_healthy() (no behavior change there — same 0x80…0x2e envelope check, deduplicated).

Also resolves the related single-writer stale-quarantine false positive (#1564), which shares the all-layer-0 root cause: an idle small collection failed the health sniff after the 300s mtime gap.

Tests

• test_segment_health_accepts_zero_byte_link_lists_with_valid_pickle — all-layer-0 + complete pickle is healthy (Self-repair false positive: empty link_lists.bin (all-layer-0 HNSW) quarantined as corruption, self-perpetuating loop #1716).
• test_quarantine_leaves_zero_byte_link_lists_with_valid_pickle — stale all-layer-0 segment is left in place (breaks the loop).
• test_segment_health_rejects_zero_byte_link_lists_with_truncated_pickle — empty link_lists + truncated marker is still rejected (partial-flush protection intact).
• Existing missing-metadata / link-bloat / divergence tests unchanged and passing.

Full suite: 3125 passed, 20 skipped.

Closes #1716

cc #1564

[gemini-code-assist]

Code Review

This pull request refactors the HNSW segment health check by introducing the _hnsw_metadata_marker_intact helper function. This function verifies that the index_metadata.pickle file has a complete envelope, indicating a finished persist operation. Consequently, an empty link_lists.bin is now accepted as healthy for an all-layer-0 index if the metadata marker is intact, preventing unnecessary quarantining of valid segments. The corresponding tests have been updated and added to verify this behavior. There are no review comments, and I have no additional feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.