Skip to content

feat(bam-io): unify the single-threaded raw-BAM reader onto fgumi-bgzf (#800) - #842

Merged
nh13 merged 1 commit into
mainfrom
800/nhomer/unify-raw-reader-fgumi-bgzf
Aug 21, 2026
Merged

nh13 merged 1 commit into
mainfrom
800/nhomer/unify-raw-reader-fgumi-bgzf

Conversation

@nh13

@nh13 nh13 commented Aug 20, 2026

Copy link
Copy Markdown
Member

Closes #800. Supersedes #801 (auto-closed when its base branch 786/nhomer/perf-bgzf-crc-skip was deleted by the #798 squash-merge; GitHub cannot reopen a PR whose base branch was deleted). Same head branch, rebased onto main — the one now-redundant doc commit dropped as already upstream, leaving the single substantive change.

Problem

fgumi-bam-io had two BGZF decoder stacks. The single-threaded raw-BAM reader (create_raw_bam_reader[_with_opts]) decoded via noodles-bgzf, which always verifies CRC32 and has no skip knob — so the --check-crc/--no-check-crc policy from #798 (backed by our own fgumi-bgzf) never reached the single-threaded fast paths, and PipelineReaderOpts.verify_crc was a dead carrier field.

Change

Add FgumiBgzfReader, a safe streaming Read/BufRead adapter over fgumi-bgzf's read_raw_blocks + decompress_block_into_opts (no unsafe; one block decoded per refill, so a CRC/size error is tied to the bytes the caller actually reads), and route the single-threaded raw reader through it via a new BgzfReaderEnum::Fgumi arm. PipelineReaderOpts.verify_crc is now live.

Coverage after this change (--check-crc/--no-check-crc):

  • clip/codec/duplex/simplex/downsample — honored single- or multi-threaded (their single-threaded reader now decodes through fgumi-bgzf).
  • correct/group — honored only with --threads N; their single-threaded mode reads through create_bam_reader_for_pipeline_with_opts + a manual noodles decode, so they keep the noodles path (help text says so).
  • threads > 1 still uses noodles' multithreaded decoder (a threaded fgumi-bgzf raw reader is out of scope).

fgumi-sort and extract have their own reader stacks and remain follow-ups.

Tests

TDD: the reader-level verify_crc test (corrupted-CRC file accepted with verify_crc=false, rejected with true) was written first and confirmed red against the noodles path before implementing. Plus a multi-block roundtrip test (exercises block-boundary buffering) and three end-to-end downsample CRC tests (--no-check-crc accepts a corrupted file; default and --check-crc reject it). The reader also gains a sticky-poison test: after a mid-stream decode failure both read and fill_buf stay in error. All 228 fgumi-bam-io + fgumi-bgzf tests green on the rebased tree.

…f, honoring verify_crc (#800)

Add FgumiBgzfReader, a safe streaming Read/BufRead adapter over fgumi-bgzf's
read_raw_blocks + decompress_block_into_opts, and route the single-threaded
raw-BAM reader (create_raw_bam_reader[_with_opts]) through it instead of
noodles-bgzf. This makes PipelineReaderOpts.verify_crc live: clip, codec,
duplex, simplex, and downsample now honor --check-crc/--no-check-crc in their
single-threaded fast paths (previously they always verified via noodles, which
has no skip knob). correct and group keep the noodles path (their single-
threaded mode reads through create_bam_reader_for_pipeline_with_opts), so the
help text lists them as --threads-only; threads>1 still uses noodles'
multithreaded decoder.

Decode is at parity with noodles single-threaded (~2% either way, within CI);
the deliverable is bringing the CRC-skip policy to these paths. Retires half of
the second-decoder inconsistency (fgumi-sort/extract remain follow-ups).
@nh13
nh13 deployed to github-actions August 20, 2026 05:26 — with GitHub Actions Active
@coderabbitai

coderabbitai Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6fa17c4e-3436-4c07-992a-afe05525ba95

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@nh13

nh13 commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai pause

@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.87302% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.46%. Comparing base (e51dc4b) to head (c222506).

Files with missing lines Patch % Lines
crates/fgumi-bam-io/src/reader.rs 95.10% 12 Missing ⚠️
src/lib/commands/clip.rs 83.33% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main     #842    +/-   ##
========================================
  Coverage   94.46%   94.46%            
========================================
  Files         187      187            
  Lines      115301   115597   +296     
========================================
+ Hits       108921   109201   +280     
- Misses       6380     6396    +16     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai

coderabbitai Bot commented Aug 20, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews paused.

@nh13
nh13 added this pull request to the merge queue Aug 21, 2026
Merged via the queue into main with commit 35e2a0e Aug 21, 2026
16 checks passed
@nh13
nh13 deleted the 800/nhomer/unify-raw-reader-fgumi-bgzf branch August 21, 2026 23:12
@nh13 nh13 mentioned this pull request Aug 21, 2026

This branch was successfully deployed

1 active deployment
github-actions — c2225069 Deployed Aug 20, 2026 by nh13 via coverage #3749
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unify the raw-BAM reader onto fgumi-bgzf (retire the second noodles-bgzf decode path)

1 participant