Skip to content

refactor(bam-io): collapse the duplicated grouping domain types onto one definition (R0b) - #741

Merged
nh13 merged 1 commit into
main-runallfrom
nh/runall-15-unify-decoded-record
Aug 14, 2026
Merged

nh13 merged 1 commit into
main-runallfrom
nh/runall-15-unify-decoded-record

Conversation

@nh13

@nh13 nh13 commented Aug 12, 2026 •

Copy link
Copy Markdown
Member

Prerequisite for R1c. Discovered while porting steps/group/, which is the first ported code that cannot be written without it.

Why

#734 moved DecodedRecord, GroupKey, GroupKeyConfig, and the library-lookup helpers into fgumi-bam-io, and deliberately left the unified_pipeline / read_info copies in place so nothing had to be rewired. That was the right call at the time: the ported step library only ever produced these values, so two identical definitions coexisted harmlessly.

steps/group/ breaks the arrangement. GroupBam calls TemplateGrouper::add_records, which is fgumi_bam_io::Grouper on one side and the umbrella Grouper on the other — typed over two different DecodedRecords:

error[E0308]: expected `fgumi_bam_io::DecodedRecord`, found `base::DecodedRecord`

Two textually identical definitions are still two types. The call cannot be written until they are one.

What changed

Keep the fgumi-bam-io definition; re-export it from both former homes so every existing crate::unified_pipeline::DecodedRecord and crate::read_info::LibraryIndex path resolves unchanged. Collapsed: DecodedRecord, GroupKey, GroupKeyConfig, LibraryIndex, LibraryLookup, build_library_lookup, unknown_library, and the Grouper trait.

Before removing each copy I diffed the two. All were identical apart from field visibility and doc-comment formatting; where they differed in API the fgumi-bam-io side was already a superset (GroupKeyConfig::name_hash_only, DecodedRecord::{record, raw_bytes_mut, cached_umi_position_opt}, GroupKey::name_hash_only). One exception went the other way and was ported across:

  • GroupKey::has_mate_position existed only on the umbrella copy, so it moved to fgumi-bam-io. The surviving definition is now a superset of both.

grouper.rs reached into DecodedRecord's fields directly. Those are private on the fgumi-bam-io copy — made so during #734's review — so five sites now go through into_raw_bytes() / record() / raw_bytes() instead.

Risk

No behavior change; this is deletion plus re-export. Net -483 lines. The suite is unchanged at 8322 passing, which is the main evidence: every caller of these types kept compiling and kept passing against a single definition.

The duplication ledger from the landing plan shrinks accordingly — R6 has this much less to unpick when unified_pipeline is deleted.

Gate

ci-fmt, ci-lint, ci-tag-literals, ci-doc (-D warnings), publish-crates.sh --check, ci-test — all exit 0.

Risk: command output changes none; unsafe changes none and the CLAUDE.md allowlist is unchanged; memory bounds, queue capacities, and thread/backpressure policies change none.

Fix: Consolidate grouping types in fgumi-bam-io and preserve existing import paths through re-exports.

  • Resolve incompatible duplicate types used by GroupBam and TemplateGrouper.
  • Move GroupKey::has_mate_position into fgumi-bam-io.
  • Replace direct DecodedRecord field access with accessors.
  • Share unknown_library() across ReadInfo fallbacks.
  • Remove 483 lines without behavior changes.
  • CI checks and 8,439 tests pass.

@nh13
nh13 temporarily deployed to github-actions August 12, 2026 08:36 — with GitHub Actions Inactive
@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 50 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: be889c7b-0b01-4a01-8953-65f2d2f1163d

📥 Commits

Reviewing files that changed from the base of the PR and between 747ad39 and 607a8eb.

📒 Files selected for processing (6)
  • crates/fgumi-bam-io/src/grouping.rs
  • crates/fgumi-bam-io/src/lib.rs
  • src/lib/grouper.rs
  • src/lib/read_info.rs
  • src/lib/unified_pipeline/bam.rs
  • src/lib/unified_pipeline/base.rs

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: f6c69acc-2bee-4f4c-b0ef-5a6de0aa635e

📥 Commits

Reviewing files that changed from the base of the PR and between 410ca1f and c84c92d.

📒 Files selected for processing (1)
  • src/lib/read_info.rs

Walkthrough

The change centralizes grouping and library APIs in fgumi-bam-io, preserves existing module paths through re-exports, adds mate-position detection, and updates grouper code to use DecodedRecord accessors.

Changes

Shared BAM I/O API migration

Layer / File(s) Summary
Shared grouping and library exports
crates/fgumi-bam-io/src/grouping.rs, crates/fgumi-bam-io/src/lib.rs, src/lib/unified_pipeline/base.rs, src/lib/unified_pipeline/bam.rs
GroupKey, DecodedRecord, GroupKeyConfig, and Grouper now use shared fgumi-bam-io definitions. GroupKey::has_mate_position and unknown_library are publicly available.
Library API path migration
src/lib/read_info.rs
Library lookup APIs are re-exported from fgumi-bam-io. ReadInfo::from uses the shared unknown-library fallback. Tests cover declared, unknown, and missing read-group identifiers.
DecodedRecord accessor migration
src/lib/grouper.rs
Grouping, MC validation, QNAME comparison, and template construction use into_raw_bytes(), record(), and raw_bytes().

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to c84c9

This change consolidates duplicate grouping types through re-exports without changing behavior. No actionable merge-blocking risk remains after normal checks and review.

Suggested labels: fgumi group

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title uses valid Conventional Commit syntax, names the affected crate, and accurately describes the consolidation of duplicated grouping types.

Comment @coderabbitai help to get the list of available commands.

@nh13

nh13 commented Aug 12, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai pause

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews paused.

@codecov

codecov Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main-runall@747ad39). Learn more about missing BASE report.

Additional details and impacted files
@@              Coverage Diff               @@
##             main-runall     #741   +/-   ##
==============================================
  Coverage               ?   94.02%           
==============================================
  Files                  ?      248           
  Lines                  ?   130233           
  Branches               ?        0           
==============================================
  Hits                   ?   122454           
  Misses                 ?     7779           
  Partials               ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from 35c0479 to 0d307d1 Compare August 12, 2026 16:56
@nh13
nh13 changed the base branch from main-runall to nh/runall-13-pipeline-source August 12, 2026 16:56
@nh13
nh13 temporarily deployed to github-actions August 12, 2026 16:56 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/runall-13-pipeline-source branch from 972ecea to c55ac1d Compare August 12, 2026 17:27
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from 0d307d1 to dc37ab7 Compare August 12, 2026 17:27
@nh13
nh13 temporarily deployed to github-actions August 12, 2026 17:27 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/runall-13-pipeline-source branch from c55ac1d to b0fc25c Compare August 12, 2026 18:43
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from dc37ab7 to 410ca1f Compare August 12, 2026 18:43
@nh13
nh13 temporarily deployed to github-actions August 12, 2026 18:43 — with GitHub Actions Inactive
@nh13

nh13 commented Aug 12, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/read_info.rs`:
- Around line 20-26: Remove the local UNKNOWN_LIBRARY static and update every
ReadInfo fallback that references it to call the re-exported unknown_library()
function from fgumi_bam_io. Preserve the existing fallback behavior and keep the
public re-export unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5dac1291-161d-439b-b48f-8007e27eab14

📥 Commits

Reviewing files that changed from the base of the PR and between b0fc25c and 410ca1f.

📒 Files selected for processing (6)
  • crates/fgumi-bam-io/src/grouping.rs
  • crates/fgumi-bam-io/src/lib.rs
  • src/lib/grouper.rs
  • src/lib/read_info.rs
  • src/lib/unified_pipeline/bam.rs
  • src/lib/unified_pipeline/base.rs

Comment thread src/lib/read_info.rs
@nh13
nh13 force-pushed the nh/runall-13-pipeline-source branch from b0fc25c to dfe6609 Compare August 13, 2026 00:57
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from 410ca1f to ba6ab62 Compare August 13, 2026 00:57
@nh13
nh13 temporarily deployed to github-actions August 13, 2026 00:57 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/runall-13-pipeline-source branch from dfe6609 to 538a950 Compare August 13, 2026 02:11
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from ba6ab62 to 94a0192 Compare August 13, 2026 02:11
@nh13
nh13 temporarily deployed to github-actions August 13, 2026 02:11 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/runall-13-pipeline-source branch from 538a950 to 2f80104 Compare August 13, 2026 04:18
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from 94a0192 to fbccad8 Compare August 13, 2026 04:18
@nh13
nh13 temporarily deployed to github-actions August 13, 2026 04:18 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from fbccad8 to ee99214 Compare August 13, 2026 04:23
@nh13
nh13 temporarily deployed to github-actions August 13, 2026 04:23 — with GitHub Actions Inactive
@nh13

nh13 commented Aug 13, 2026

Copy link
Copy Markdown
Member Author

Fixed. The local UNKNOWN_LIBRARY static is gone and both ReadInfo fallbacks call the re-exported unknown_library().

Worth naming why this one stings: it is the PR's own thesis violated one level down. The change collapses duplicated types onto a single definition, and I re-exported unknown_library while leaving a second LazyLock<Arc<str>> for the same logical value sitting right beside it — so "unknown" existed as two allocations, one of which build_library_lookup reserves as index 0 and the other of which ReadInfo::from fell back to.

No behavior changed: nothing compares library Arcs by pointer (checked — the only ptr_eq uses in the tree are on sort arenas and pipeline storage, not libraries), so the two allocations always compared equal by value. That is also why there is no new test: the fix is allocation sharing with no observable difference, which is the cosmetic case the review conventions exempt. Flagging that explicitly rather than leaving it as a silent omission.

Sibling audit for the same mistake across the rest of the change: the other statics in the touched files are THREAD_ID_UNSET, NUM_STEPS, and three buffer-size constants — unrelated tuning, not duplicates of anything this PR re-exports. Clean.

The re-export comment now explains the value-level duplication too, so the next reader sees why there is no local static.

8439 tests passing, all static gates green.

@nh13

nh13 commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 force-pushed the nh/runall-13-pipeline-source branch from 2f80104 to a548ad0 Compare August 14, 2026 05:43
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from ee99214 to 35ad7ab Compare August 14, 2026 05:43
@nh13
nh13 temporarily deployed to github-actions August 14, 2026 05:43 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from 35ad7ab to c84c92d Compare August 14, 2026 06:16
@nh13
nh13 temporarily deployed to github-actions August 14, 2026 06:17 — with GitHub Actions Inactive
@nh13

nh13 commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 force-pushed the nh/runall-13-pipeline-source branch from a548ad0 to 559b2f7 Compare August 14, 2026 15:55
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from c84c92d to c4bfe36 Compare August 14, 2026 15:55
@nh13
nh13 temporarily deployed to github-actions August 14, 2026 15:55 — with GitHub Actions Inactive
Base automatically changed from nh/runall-13-pipeline-source to main-runall August 14, 2026 16:02
…one definition

`DecodedRecord`, `GroupKey`, `GroupKeyConfig`, `LibraryIndex`, `LibraryLookup`,
`build_library_lookup`, and the `Grouper` trait each existed twice: once in
`fgumi-bam-io` (ported in #734) and once in `unified_pipeline` / `read_info`.
The copies were textually identical apart from field visibility and doc
formatting, and the duplication was deliberate — #734 left the umbrella copies
in place so nothing had to be rewired.

That worked while the ported step library only ever *produced* these values.
It stops working at the first ported step that hands one to an umbrella
grouper: `GroupBam` calls `TemplateGrouper::add_records`, and the two identical
definitions are two incompatible types, so the call cannot be written at all.

Keep the `fgumi-bam-io` definition and re-export it from both former homes, so
every existing `crate::unified_pipeline::DecodedRecord` and
`crate::read_info::LibraryIndex` path resolves unchanged. Two supporting
changes fall out:

- `GroupKey::has_mate_position` only existed on the umbrella copy; ported to
  `fgumi-bam-io` so the surviving definition is a superset of both.
- `grouper.rs` reached into `DecodedRecord`'s fields directly, which are
  private on the `fgumi-bam-io` copy (made so during #734's review). Those five
  sites now go through `into_raw_bytes` / `record` / `raw_bytes`, which is what
  the accessors are for.

Net -483 lines with no behavior change; the full suite is unchanged at 8322.
@nh13
nh13 force-pushed the nh/runall-15-unify-decoded-record branch from c4bfe36 to 607a8eb Compare August 14, 2026 16:09
@nh13
nh13 temporarily deployed to github-actions August 14, 2026 16:09 — with GitHub Actions Inactive
@nh13

nh13 commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

No files to review.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Aug 14, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 14, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.


Your included review limit is currently reached under our Fair Usage Limits Policy. Your recent PR review activity is in the 95th percentile or higher among CodeRabbit users, so adaptive limits apply. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 50 minutes.

@nh13
nh13 merged commit de53667 into main-runall Aug 14, 2026
17 checks passed
@nh13
nh13 deleted the nh/runall-15-unify-decoded-record branch August 14, 2026 20:35
nh13 added a commit that referenced this pull request Aug 19, 2026
…one definition (#741)

`DecodedRecord`, `GroupKey`, `GroupKeyConfig`, `LibraryIndex`, `LibraryLookup`,
`build_library_lookup`, and the `Grouper` trait each existed twice: once in
`fgumi-bam-io` (ported in #734) and once in `unified_pipeline` / `read_info`.
The copies were textually identical apart from field visibility and doc
formatting, and the duplication was deliberate — #734 left the umbrella copies
in place so nothing had to be rewired.

That worked while the ported step library only ever *produced* these values.
It stops working at the first ported step that hands one to an umbrella
grouper: `GroupBam` calls `TemplateGrouper::add_records`, and the two identical
definitions are two incompatible types, so the call cannot be written at all.

Keep the `fgumi-bam-io` definition and re-export it from both former homes, so
every existing `crate::unified_pipeline::DecodedRecord` and
`crate::read_info::LibraryIndex` path resolves unchanged. Two supporting
changes fall out:

- `GroupKey::has_mate_position` only existed on the umbrella copy; ported to
  `fgumi-bam-io` so the surviving definition is a superset of both.
- `grouper.rs` reached into `DecodedRecord`'s fields directly, which are
  private on the `fgumi-bam-io` copy (made so during #734's review). Those five
  sites now go through `into_raw_bytes` / `record` / `raw_bytes`, which is what
  the accessors are for.

Net -483 lines with no behavior change; the full suite is unchanged at 8322.
nh13 added a commit that referenced this pull request Aug 19, 2026
…one definition (#741)

`DecodedRecord`, `GroupKey`, `GroupKeyConfig`, `LibraryIndex`, `LibraryLookup`,
`build_library_lookup`, and the `Grouper` trait each existed twice: once in
`fgumi-bam-io` (ported in #734) and once in `unified_pipeline` / `read_info`.
The copies were textually identical apart from field visibility and doc
formatting, and the duplication was deliberate — #734 left the umbrella copies
in place so nothing had to be rewired.

That worked while the ported step library only ever *produced* these values.
It stops working at the first ported step that hands one to an umbrella
grouper: `GroupBam` calls `TemplateGrouper::add_records`, and the two identical
definitions are two incompatible types, so the call cannot be written at all.

Keep the `fgumi-bam-io` definition and re-export it from both former homes, so
every existing `crate::unified_pipeline::DecodedRecord` and
`crate::read_info::LibraryIndex` path resolves unchanged. Two supporting
changes fall out:

- `GroupKey::has_mate_position` only existed on the umbrella copy; ported to
  `fgumi-bam-io` so the surviving definition is a superset of both.
- `grouper.rs` reached into `DecodedRecord`'s fields directly, which are
  private on the `fgumi-bam-io` copy (made so during #734's review). Those five
  sites now go through `into_raw_bytes` / `record` / `raw_bytes`, which is what
  the accessors are for.

Net -483 lines with no behavior change; the full suite is unchanged at 8322.
nh13 added a commit that referenced this pull request Aug 23, 2026
…one definition (#741)

`DecodedRecord`, `GroupKey`, `GroupKeyConfig`, `LibraryIndex`, `LibraryLookup`,
`build_library_lookup`, and the `Grouper` trait each existed twice: once in
`fgumi-bam-io` (ported in #734) and once in `unified_pipeline` / `read_info`.
The copies were textually identical apart from field visibility and doc
formatting, and the duplication was deliberate — #734 left the umbrella copies
in place so nothing had to be rewired.

That worked while the ported step library only ever *produced* these values.
It stops working at the first ported step that hands one to an umbrella
grouper: `GroupBam` calls `TemplateGrouper::add_records`, and the two identical
definitions are two incompatible types, so the call cannot be written at all.

Keep the `fgumi-bam-io` definition and re-export it from both former homes, so
every existing `crate::unified_pipeline::DecodedRecord` and
`crate::read_info::LibraryIndex` path resolves unchanged. Two supporting
changes fall out:

- `GroupKey::has_mate_position` only existed on the umbrella copy; ported to
  `fgumi-bam-io` so the surviving definition is a superset of both.
- `grouper.rs` reached into `DecodedRecord`'s fields directly, which are
  private on the `fgumi-bam-io` copy (made so during #734's review). Those five
  sites now go through `into_raw_bytes` / `record` / `raw_bytes`, which is what
  the accessors are for.

Net -483 lines with no behavior change; the full suite is unchanged at 8322.

This branch was previously deployed

1 inactive deployment
github-actions — 607a8eb8 Deployed Aug 14, 2026 by nh13 via coverage #3490
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant