ci(test-windows): retry the transient ChromaDB HNSW compaction flake - #1854
Merged
Conversation
ChromaDB's rust HNSW core intermittently fails compaction on Windows with "Failed to apply logs to the hnsw segment writer" during add/update — a long-standing, non-reproducible-on-Linux/macOS flake that hits different tests (test_migrate_wings, test_closets) across unrelated commits and has been turning otherwise-green release/CI runs red at random. Add pytest-rerunfailures and wire `--reruns 2 --only-rerun "Failed to apply logs to the hnsw segment writer"` into the test-windows job only. The --only-rerun scope means a real, deterministic failure still fails on the first run; only this specific transient native-dependency error is retried. The Linux and macOS jobs deliberately keep zero reruns so genuine regressions surface there loudly.
Contributor
There was a problem hiding this comment.
Code Review
This pull request adds the pytest-rerunfailures dependency to the development dependencies in pyproject.toml to handle transient ChromaDB-on-Windows HNSW compaction failures in CI. There are no review comments, and I have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
test-windowsgoes red at random. The failure is always the same family —ChromaDB's rust HNSW core failing compaction on Windows:
It is not caused by our code:
migrate.py/ the affected tests areunchanged across the runs, it hits different tests (
test_migrate_wings,test_closets) on unrelated commits, and it does not reproduce onLinux/macOS. Across a single batch of merges this week roughly half the
test-windowsruns passed and half failed on the same code. CLAUDE.md alreadydocuments Windows ChromaDB instability (the Windows coverage threshold is
lowered for the same reason). It has been turning otherwise-green release runs
red and forcing manual re-runs.
Fix
Add
pytest-rerunfailuresand, only on thetest-windowsjob, retry thatspecific transient error:
--only-rerunscopes the retry to this one native-dependency error, so areal, deterministic test failure still fails on the first attempt — nothing
is masked.
there loudly and immediately.
pytest-rerunfailuresis added to both dev dependency lists(
[project.optional-dependencies].devfor pip/CI and[dependency-groups].devfor uv); zero runtime cost (test-only).
Notes
If a different Windows flake recurs (e.g. the occasional
test_closetsranking assertion), extend the
--only-rerunpattern or investigate that oneon its own merits rather than broadening to a blanket retry.