feat(group,dedup): add opt-in --verify for strict template-coordinate sort-order

[nh13]

Summary

Adds an opt-in --verify flag to fgumi group and fgumi dedup that checks the input is in strict fgumi sort --order template-coordinate order and fails fast (non-zero exit) on the first out-of-order record, before writing a wrong result. Off by default.

fgumi group/dedup already assume template-coordinate input but only validate the header's sort tags (SO/GO/SS), not the actual record order — so a mislabeled or mangled file is silently mis-grouped. --verify closes that gap for callers who want a hard guarantee.

Design

• One insertion point. The check runs inline, single-pass, at the top of RecordPositionGrouper::process_record — the serial chokepoint shared by all four paths (group/dedup × non-chain / --threads N chain). It runs before the secondary/supplementary skip, so the whole file's canonical order is checked, including sec/supp reads that grouping later drops.
• Reuses the sort's own key. It calls fgumi_sort::extract_template_key_inline and compares with TemplateKey::core_cmp (the same tolerant comparison fgumi sort --verify uses, which ignores the name_hash tie-break so both fgumi- and samtools-sorted inputs pass). It keys on the CB cell tag — matching what fgumi sort --order template-coordinate emits by default (parse_cell_tag), using the same fixed-seed hasher — so a correctly-sorted single-cell/multi-library input is not spuriously rejected.
• Strict by design → opt-in. The check is stricter than grouping/dedup strictly require (they only need a template's mates adjacent by position). That strictness is why it is opt-in, and the help text says so. For a standalone check-only pass, fgumi sort --verify already exists; here --verify is a precondition gate that composes with --output.

Testing

• Grouper unit tests: in-order accept, out-of-order reject (including a mid-run regression), equal-position ties, cross-reference order, and out-of-order secondary reads rejected in both grouper modes (pinning that the check runs before the sec/supp skip).
• Property tests (proptest, 256 cases each): --verify accepts a random (tid,pos) stream iff it is non-decreasing under the template key, and any key-sorted stream is always accepted — fuzzing the fold over arbitrary orders, shuffles, and ties.
• Integration tests (both commands, single-threaded and --threads chain): --verify accepts a correctly-sorted input and leaves output identical to the non-verify run; rejects an out-of-order input with a clear error while the same input is accepted without --verify.
• Multi-batch (600-position) chain acceptance for both commands — guards cross-worker reorder across pipeline batch boundaries.
• CB / multi-library regression: an input fgumi sort orders by cell-barcode hash (overriding library order) is accepted by --verify; a CB-blind check would spuriously reject it.
• Coordinate-mislabel: a coordinate-ordered (mate-interleaved) file under a template-coordinate header is rejected.

Notes

• Independent of fix(group): filter secondary/supplementary reads by flag, not UNKNOWN_REF #903 (the bug(group): Supplementary/secondary alignments change the reported molecule count #901 secondary/supplementary molecule-count fix) and test(dedup): fix inconsistent mate-strand flags in duplicate-group fixtures #908 (dedup fixture mate-flag corrections); this branch is off main.

Risk: output changes only when --verify rejects out-of-order input; unsafe changes are none, and no CLAUDE.md allowlist update applies; memory bounds, queue capacity, and thread/backpressure policy changes are none.

Fix: Use --verify with fgumi group or fgumi dedup to fail fast on input that is not in strict template-coordinate order. Verification is disabled by default.

• Reuses the fgumi sort --order template-coordinate key and comparison logic.
• Checks records before secondary and supplementary reads are skipped.
• Applies to single-threaded, threaded-chain, and multi-batch processing.
• Uses the CB tag and fixed-seed hash for ordering.
• Covers sorted input, out-of-order input, ties, references, secondary reads, multi-library data, threaded execution, and misleading headers.

[coderabbitai]

Note

Reviews paused

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: ae1ae309-7bb0-4bd2-a6b3-d14ec360782a

📥 Commits

Reviewing files that changed from the base of the PR and between f2123ef and 6f80b42.

📒 Files selected for processing (3)

• src/lib/grouper.rs
• tests/integration/test_dedup_command.rs
• tests/integration/test_group_command.rs

Included review availability: 0 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

---

Walkthrough

The PR adds opt-in template-coordinate order verification to group and deduplication. The verifier uses header, library, and cell-barcode ordering data, checks records before filtering, and reports out-of-order input.

Changes

Template-coordinate verification

Layer / File(s)	Summary
Verifier core and unit coverage src/lib/grouper.rs	RecordPositionGrouper derives and compares template-coordinate keys. It reports InvalidData for regressions and checks secondary and supplementary records before filtering. Unit and property tests cover ordering and disabled verification.
CLI and pipeline wiring src/lib/commands/group.rs, src/lib/commands/dedup.rs, src/lib/pipeline/steps/group/position.rs, src/lib/pipeline/chains/builder.rs	group and deduplication expose --verify, default it to disabled, and pass it to header-aware GroupByPosition steps.
Command integration validation tests/integration/test_group_command.rs, tests/integration/test_dedup_command.rs	Integration tests cover sorted and unsorted input, CB-aware ordering, mate-interleaved input, unchanged non-verified behavior, and threaded multi-batch execution.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 6f80b

The opt-in verification is comprehensively covered, with no identified merge-blocking risk.

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant PipelineBuilder
  participant GroupByPosition
  participant RecordPositionGrouper
  participant OrderVerifier
  CLI->>PipelineBuilder: pass verify option
  PipelineBuilder->>GroupByPosition: configure verifying(header)
  GroupByPosition->>RecordPositionGrouper: enable_verify(header)
  RecordPositionGrouper->>OrderVerifier: validate each record
  OrderVerifier-->>RecordPositionGrouper: accept or InvalidData

Loading

Suggested labels: fgumi group

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title follows the required Conventional Commit format. It uses the valid type feat, names the affected group and dedup commands, and gives a concise lowercase imperative description of the m…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[nh13]

@coderabbitai pause

[coderabbitai]

✅ Action performed

Reviews paused.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.53%. Comparing base (247404b) to head (6f80b42).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #909      +/-   ##
==========================================
- Coverage   93.55%   93.53%   -0.03%     
==========================================
  Files         301      301              
  Lines      150626   150745     +119     
==========================================
+ Hits       140923   141001      +78     
- Misses       9703     9744      +41     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.