Skip to content

fix(runall): honor --extract::no-check-crc on BGZF FASTQ input - #996

Merged
nh13 merged 1 commit into
mainfrom
nh/fix-runall-fastq-check-crc
Sep 29, 2026
Merged

nh13 merged 1 commit into
mainfrom
nh/fix-runall-fastq-check-crc

Conversation

@nh13

@nh13 nh13 commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

fgumi runall --start-from extract --extract::no-check-crc aborted with a BGZF CRC32 mismatch on all-BGZF FASTQ file input — while logging CRC verify: off — where standalone fgumi extract --no-check-crc succeeds.

The all-file BGZF FASTQ parallel-decode split (build_bgzf_fastq_split) took its CRC policy from ChainSpec::verify_crc, while the fused FASTQ readers resolved theirs from the extract options' check_crc/no_check_crc. runall set verify_crc to a hardcoded true for a FASTQ source (commented as inert); standalone extract only worked because it resolved the field a second time itself.

The split's policy is now resolved once in open_fastq_source from the same extract options the fused readers use (via the shared resolve_check_crc) and carried in PendingSource::Fastq, so no FASTQ command can leave the two decode fronts out of sync. ChainSpec::verify_crc is now genuinely inert for FASTQ sources (doc corrected), extract's redundant resolution is removed, and open_fastq_reader uses the shared resolve_check_crc/check_crc_reason instead of inline copies. The settled default policy (verify a file, skip trusted stdin; #786/#798) is unchanged.

Found by a sweep for siblings of #991. Blocks v0.8.0.

Test plan

  • runall integration test: a 350k-record BGZF FASTQ with a corrupted CRC in its last data block (past quality-detection's window) is accepted under --extract::no-check-crc with every record present in order and identical to standalone extract, and rejected by default with the mismatch raised by FastqDecompress. Fails without the builder change, as does the existing standalone split test.
  • A BAM start without -i still reports --input is required.
  • cargo ci-fmt, ci-lint, ci-doc, ci-test (10474 passed), ci-doctest.

Risk: Command output data: none; CRC rejection behavior changes for BGZF FASTQ, and integration tests pin record names, order, and parity with standalone extract. unsafe: none added or modified; CLAUDE.md’s allowlist does not need an update. Memory bounds, queue capacity, and thread/backpressure policy: none.

Fix: BGZF FASTQ split decoding now uses the CRC policy resolved from extract options. This aligns runall with fused FASTQ readers. The default CRC policy remains unchanged.

Coverage: Added integration tests for corrupted BGZF FASTQ with CRC checks disabled and enabled, and for missing input when starting from sort. Test execution results were not provided.

@nh13
nh13 deployed to github-actions September 29, 2026 02:24 — with GitHub Actions Active
@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: fulcrumgenomics/fgumi/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 3ec4ae80-6087-4773-a856-5da61e0114af

📥 Commits

Reviewing files that changed from the base of the PR and between 16ac4c9 and fd74432.

📒 Files selected for processing (1)
  • src/lib/pipeline/chains/builder.rs

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.


Walkthrough

FASTQ BGZF split decoding now uses CRC policy resolved from extract options rather than ChainSpec.verify_crc. Integration tests cover corrupted CRC handling and a missing-input error from runall.

Changes

FASTQ CRC policy

Layer / File(s) Summary
Resolve and document FASTQ CRC policy
src/lib/commands/extract.rs, src/lib/commands/runall.rs, src/lib/pipeline/chains/spec.rs
Shared helpers resolve CRC policy. ChainSpec.verify_crc is set to true, and documentation states that FASTQ CRC policy comes from extract options. The test for deriving ChainSpec.verify_crc from CRC flags was removed.
Pass policy to BGZF split decoding
src/lib/pipeline/chains/builder.rs
open_fastq_source stores the resolved policy in PendingSource::Fastq. The BGZF split decoder uses that policy instead of ChainSpec.verify_crc.
Exercise CRC and runall error paths
tests/integration/helpers/fastq.rs, tests/integration/test_runall_command.rs
A new fixture corrupts the final BGZF CRC. Integration tests check that --extract::no-check-crc accepts the input and preserves record order, that default CRC checking rejects it, and that runall reports the missing --input option.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to fd744

The runall FASTQ path follows the requested CRC policy, and the added tests cover both disabled-check acceptance and default rejection. No actionable merge-readiness risk remains.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title follows Conventional Commit format, uses the valid fix type and runall scope, clearly describes the CRC-policy fix, uses a lowercase imperative description, and has no trailing period.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@nh13
nh13 force-pushed the nh/fix-runall-fastq-check-crc branch from 1025773 to 948c232 Compare September 29, 2026 02:26
@nh13
nh13 deployed to github-actions September 29, 2026 02:26 — with GitHub Actions Active
@nh13

nh13 commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai pause

@coderabbitai

coderabbitai Bot commented Sep 29, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews paused.

@codecov

codecov Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 96.27%. Comparing base (27c3a4f) to head (fd74432).

Files with missing lines Patch % Lines
src/lib/pipeline/chains/builder.rs 96.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #996      +/-   ##
==========================================
- Coverage   96.29%   96.27%   -0.02%     
==========================================
  Files         296      296              
  Lines      149023   149032       +9     
==========================================
- Hits       143496   143475      -21     
- Misses       5527     5557      +30     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13

nh13 commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 added this pull request to the merge queue Sep 29, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 29, 2026
@nh13
nh13 force-pushed the nh/fix-runall-fastq-check-crc branch from 948c232 to 16ac4c9 Compare September 29, 2026 07:08
@nh13
nh13 deployed to github-actions September 29, 2026 07:09 — with GitHub Actions Active
@nh13

nh13 commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 added this pull request to the merge queue Sep 29, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Sep 29, 2026
The all-file BGZF FASTQ parallel-decode split (build_bgzf_fastq_split)
took its CRC32 policy from ChainSpec::verify_crc, while the fused FASTQ
readers resolved theirs from the extract options' check_crc /
no_check_crc. runall set verify_crc to a hardcoded true for a FASTQ
source, believing it inert, so under --extract::no-check-crc the split
aborted with a CRC mismatch while the run logged "CRC verify: off";
standalone extract only worked because it resolved the field a second
time itself.

Resolve the split's policy in the builder from the same extract options
the fused readers use, so no FASTQ command can leave the two decode
fronts out of sync. ChainSpec::verify_crc is now genuinely inert for a
FASTQ source, and extract's own copy of the resolution is removed. The
default policy (verify a file, skip stdin) is unchanged.
@nh13
nh13 force-pushed the nh/fix-runall-fastq-check-crc branch from 16ac4c9 to fd74432 Compare September 29, 2026 17:06
@nh13
nh13 deployed to github-actions September 29, 2026 17:06 — with GitHub Actions Active
@nh13 nh13 mentioned this pull request Sep 29, 2026
2 tasks done
@nh13

nh13 commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 added this pull request to the merge queue Sep 29, 2026
Merged via the queue into main with commit 39ea409 Sep 29, 2026
17 checks passed
@nh13
nh13 deleted the nh/fix-runall-fastq-check-crc branch September 29, 2026 17:24

This branch was successfully deployed

1 active deployment
github-actions — fd744325 Deployed Sep 29, 2026 by nh13 via coverage #4683
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant