fix(fastq): accept CRLF and unterminated final records (W9b: EXT3-06/07)

[nh13]

What / why

Both FASTQ parsers split records on \n only, so two valid inputs that fgbio FastqToBam accepts were rejected by fgumi:

• EXT3-06 (CRLF): on a \r\n-terminated file the \r before each \n was left as the last byte of the name, sequence, and quality. A \r (byte 13) in the quality is below the printable-ASCII floor, so fgumi extract rejected the whole file: Invalid quality scores detected: range [13, …].
• EXT3-07 (no trailing newline): SimdFastqReader errored Truncated FASTQ record at EOF on a final record lacking its terminating \n, even though the sibling from_slice parser already accepted it — the two paths disagreed (this is the exact inconsistency the audit flagged: reader.rs vs fastq_parse.rs).

The fix, applied to both the fgumi-simd-fastq crate (extract's path) and src/lib/fastq_parse.rs (the grouper / unified-pipeline path):

• Strip one trailing \r per field (strip_trailing_cr), matching fgbio's Source.getLines, and compare CRLF-stripped lengths in the seq/qual validation so an unterminated final CRLF record isn't spuriously rejected.
• In the reader's EOF branch, yield a complete-but-unterminated leftover (starts with @, does not end in \n, has exactly the three internal newlines) instead of erroring; a genuinely short/malformed leftover still errors. Matches fgbio's lines.take(4).

Verification (§0)

Real-tool repro — a CRLF FASTQ and a newline-less-final-record FASTQ:

input	fgumi before	fgumi after	fgbio
CRLF	error (0 records)	2 records	2 records
no trailing \n	error "Truncated…" (0)	2 records	2 records

fgumi-after name/sequence/quality are byte-identical to fgbio FastqToBam on the CRLF input. RED-first unit tests in both crates.

Reviewers

Both ran. In-diff fixes folded by amend:

• CodeRabbit-CLI (major, real bug it caught): the first EOF-accept condition would have parsed a leftover like @n\nseq\n+\n — a newline-terminated + line with the quality line absent — which has three internal newlines but an empty quality span, panicking parse_single_record with slice index starts at 11 but ends at 10. Added leftover.last() != Some(&b'\n') (a genuine unterminated record ends with quality bytes, never \n) + a RED-first regression test that reproduces the panic.
• CodeRabbit-CLI (minor): corrected the from_slice doc — the trailing newline is optional and CRLF is accepted.

cargo ci-fmt && ci-lint && ci-test green (2234 tests).

Scope

W9b of the extract/FASTQ residuals. Stacked on #512 (nh/fix-fastqgrouper-readname-sync, which also edits fastq_parse.rs) — auto-retargets to main on its merge. Remaining W9: FASTQ3-04 (suffix strip, main-based) and the FASTQ3-02/03 product decisions; EXT3-03 deferred on #496.

Summary by CodeRabbit

• API Changes
  • FASTQ record parsing now accepts an at_eof flag to control how an unterminated final quality line is handled.
• Bug Fixes
  • Added CRLF compatibility so \r no longer appears in parsed name, sequence, or quality fields.
  • Final FASTQ records missing a trailing newline are now accepted when appropriate, both for streaming and chunked inputs.
  • Quality-less or malformed records now return a clean parsing error instead of failing unexpectedly.
  • Improved sequence/quality length validation across LF and CRLF inputs.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f1d0be9d-15e0-417a-aafd-fe3f7067e0e1

📥 Commits

Reviewing files that changed from the base of the PR and between f9cdafc and 8a79547.

📒 Files selected for processing (4)

• crates/fgumi-simd-fastq/src/parser.rs
• crates/fgumi-simd-fastq/src/reader.rs
• src/lib/fastq_parse.rs
• src/lib/grouper.rs

---

Walkthrough

FASTQ parsing now normalizes CRLF fields, accepts complete final records without a trailing newline, preserves incomplete chunks for reassembly, and rejects malformed or quality-less records with InvalidData.

Changes

FASTQ parsing behavior

Layer / File(s)	Summary
Record normalization and EOF contract src/lib/fastq_parse.rs	FastqRecord strips trailing carriage returns, validates normalized sequence and quality lengths, rejects quality-less records safely, and distinguishes incremental parsing from final EOF parsing.
SIMD parser CRLF handling crates/fgumi-simd-fastq/src/parser.rs	Single-record and streaming parsing remove CRLF terminators from names, sequences, and qualities, with coverage for final records lacking a newline.
Unterminated final-record handling crates/fgumi-simd-fastq/src/reader.rs	SimdFastqReader parses structurally complete EOF leftovers without a trailing newline and reports malformed final records as InvalidData.
Chunked input reassembly src/lib/grouper.rs	FastqGrouper retains non-final fragments during streaming input and re-parses leftovers at finish() with EOF semantics.

Estimated code review effort: 4 (Complex) | ~45 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: FASTQ parsing now accepts CRLF line endings and unterminated final records.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch nh/fix-fastq-crlf-eof

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.91%. Comparing base (f55ac4b) to head (8a79547).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #554      +/-   ##
==========================================
+ Coverage   92.84%   92.91%   +0.06%     
==========================================
  Files         166      166              
  Lines      102064   102468     +404     
==========================================
+ Hits        94765    95207     +442     
+ Misses       7299     7261      -38     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/fastq_parse.rs`:
- Around line 277-284: The no-trailing-newline branch in parse_fastq_records
must not treat an incremental chunk’s remaining quality bytes as a complete
record. Update the parser and its interaction with add_bytes_for_stream so this
path is accepted only when the input is known to be final, otherwise preserve
and return the bytes as leftover for the next chunk; retain normal
newline-terminated parsing.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 90428a63-fceb-4564-a646-f66002f3fb44

📥 Commits

Reviewing files that changed from the base of the PR and between f55ac4b and f9cdafc.

📒 Files selected for processing (3)

• crates/fgumi-simd-fastq/src/parser.rs
• crates/fgumi-simd-fastq/src/reader.rs
• src/lib/fastq_parse.rs

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.