Skip to content

fix(raw-bam): classify dovetail FR pairs with coincident 5' ends as FR - #1022

Merged
nh13 merged 1 commit into
mainfrom
nh/fix-dovetail-coincident-five-prime
Oct 8, 2026
Merged

nh13 merged 1 commit into
mainfrom
nh/fix-dovetail-coincident-five-prime

Conversation

@nh13

@nh13 nh13 commented Oct 5, 2026

Copy link
Copy Markdown
Member

Changes output for codec, clip (overlap and past-mate clipping), simplex and duplex on dovetail FR pairs whose 5' ends coincide. The new output matches fgbio.

htsjdk 5.0.0 SamPairUtil.getPairOrientation (samtools/htsjdk#1771, pinned by fgbio) classifies a pair as FR when the positive-strand 5' position is <= the negative-strand 5' position. fgumi used a strict <. So a dovetail FR pair whose reverse read's aligned end equals the forward read's aligned start was classified RF from the reverse record. This is the HEK293T CODEC geometry #505 targeted; see fgbio CodecConsensusCallerTest.scala:210 and :359. As a result:

  • codec dropped these pairs as NotPrimaryFrPair.
  • clip skipped overlap and past-mate clipping for them.
  • simplex/duplex (num_bases_extending_past_mate_raw, MC arm) did not trim read-through bases past the mate.

Changes

  • Inclusive 5' comparison everywhere. Every copy of the orientation check now uses <=: is_fr_pair_raw, is_fr_pair_with_mate_cigar_raw, record_utils::get_pair_orientation and template::get_pair_orientation_raw.
    • The forward TLEN arms use htsjdk's CoordMath.getEnd(start, tlen) = start + tlen - 1. Combined with <=, this is identical to the old start + tlen with <, so only the tie changes.
    • Arithmetic is in i64 everywhere.
  • Ends computed as htsjdk does. Every copy now computes alignment ends as start + refLength - 1, unclamped. The raw and typed copies clamped a zero reference span to end = start. With <=, that would have made a mapped 100S reverse read at its mate's start FR, and its forward mate would have been clipped to one base. htsjdk gives start - 1, so the pair is RF and left untouched.
  • Overlap clipping at a tie. At a tie, the midpoint equals the reverse read's alignment end. fgbio's readPosAtRefPos then returns 0 and leaves the reverse read unclipped, with no upgrade or masking. clip_overlapping_reads now does the same instead of clipping away the reverse read's whole alignment.

Tests

Each test cites its fgbio or htsjdk source. A mutation check of each change made at least one test fail.

  • FR classification:
    • is_primary_fr_pair_raw on the fgbio pair, both argument orders.
    • The per-record arms, and the forward TLEN boundary (TLEN 1 is FR, TLEN 0 is RF).
    • htsjdk 5.0.0's "dovetail 5' tie" vector, in the template and typed copies.
    • A zero-reference-span reverse read is RF in all copies.
    • Extreme TLEN does not overflow.
  • codec: the pair is no longer rejected as non-FR. With the default window it gives 1 consensus read. Under --legacy-overlap-window it is rejected as Dovetail.
  • clip:
    • Overlap clipping on the tie pair in Soft, SoftWithMask and Hard modes, both argument orders: (52, 0), and the reverse read is untouched.
    • Past-mate clipping on htsjdk's tie pair: (99, 99).
    • A zero-span read at its mate's start is not clipped.
  • simplex/duplex read-through: num_bases_extending_past_mate_raw on tie pairs gives 7/7 (HEK293T pair) and 99/99 (htsjdk pair).

Out of scope

The typed and template per-record helpers still use TLEN on the forward arm, where htsjdk 5.0.0 prefers MC when present. Their docs now say so. Consolidating the four orientation copies into one helper is left for a follow-up.

htsjdk 5.0.0 SamPairUtil.getPairOrientation (htsjdk#1771), which fgbio
pins, classifies a pair as FR when the positive-strand 5' position is
<= the negative-strand 5' position. fgumi used a strict <, so a dovetail
FR pair whose reverse read's aligned end equals the forward read's
aligned start (the HEK293T geometry from #505) was classified RF from
the reverse record. Because is_primary_fr_pair_raw evaluates the reverse
record's arm, such pairs were rejected:

- fgumi codec dropped them as NotPrimaryFrPair;
- fgumi clip skipped overlap and past-mate clipping for them;
- the simplex and duplex callers (num_bases_extending_past_mate_raw, via
  the MC-tag forward arm) did not clip read-through bases past the mate.

Use the inclusive comparison in is_fr_pair_raw and the MC-based forward
arm (is_fr_pair_with_mate_cigar_raw), and in the TLEN-based per-record
orientation helpers (record_utils::get_pair_orientation and
template::get_pair_orientation_raw). The forward TLEN arms now compute
the mate 5' as htsjdk's CoordMath.getEnd(start, tlen) = start + tlen - 1,
which with <= is identical to the previous start + tlen with <, so only
the 5' tie changes. That arithmetic is now done in i64 in every copy.

Compute alignment ends exactly as htsjdk does (start + refLength - 1,
unclamped) in every orientation copy. The raw and typed copies clamped
a zero reference span to end = start; with the inclusive comparison
that would have made a mapped reverse read with no reference-consuming
ops (e.g. 100S) at its forward mate's start FR and clipped the forward
read down to one base. htsjdk ends such a read at start - 1, so the
pair is RF and left untouched.

Admitting tie pairs makes a degenerate overlap-clip case reachable:
the midpoint equals the reverse read's alignment end, so there is no
reference position after it to keep. fgbio's readPosAtRefPos returns 0
there and leaves the reverse read unclipped (no upgrade or masking in
any mode); clip_overlapping_reads now does the same instead of clipping
away the reverse read's whole alignment.

This changes output for these pairs, matching fgbio: codec now emits
consensus reads for them, clip now clips overlap and past-mate bases,
and simplex/duplex now trim read-through bases (e.g. 7 bases per read
on the HEK293T pair) before consensus.
@nh13
nh13 deployed to github-actions October 5, 2026 20:16 — with GitHub Actions Active
@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: fulcrumgenomics/fgumi/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Essentials
  • Run ID: 814ee53b-d950-4d26-bbab-eb1634e85437
📥 Commits

Reviewing files that changed from the base of the PR and between 8083173 and 01264c7.

📒 Files selected for processing (5)
  • crates/fgumi-consensus/src/codec_caller.rs
  • crates/fgumi-raw-bam/src/overlap.rs
  • crates/fgumi-sam/src/clipper.rs
  • crates/fgumi-sam/src/record_utils.rs
  • src/lib/template.rs

Included review availability: This review used your included allowance. 2 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.


⚠️ A high-level summary could not be generated for this review. CodeRabbit will regenerate it on the next update, or you can request a refresh with @coderabbitai summary.

Walkthrough

FR orientation now treats coincident 5′ ends as FR across raw BAM, SAM, and template checks. Overlap clipping handles dovetail and zero-reference-span cases. Consensus tests cover how dovetail pairs are handled under two overlap-window modes.

Changes

FR orientation and dovetail handling

Layer / File(s) Summary
FR coordinate rules
crates/fgumi-raw-bam/src/overlap.rs, crates/fgumi-sam/src/record_utils.rs, src/lib/template.rs
FR checks now use inclusive 5′ comparisons and widened coordinate calculations. Zero-reference-span reverse reads end one position before their start. Tests cover ties, RF boundaries, and TLEN and mate-CIGAR differences.
Dovetail overlap clipping
crates/fgumi-sam/src/clipper.rs
Clipping skips the negative-strand read when the midpoint reaches its alignment end. Tests cover coincident 5′ ends, clipping modes, argument orders, and zero-reference-span reads.
Consensus handling for dovetails
crates/fgumi-consensus/src/codec_caller.rs
Tests check coincident-end FR pairs in both overlap-window modes. The intersection mode emits one consensus; legacy mode rejects both reads as dovetails.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Suggested labels: raw-bam

Merge Risk: ⚪ Minimal · up to 01264

No actionable issue is established for this change; it is mergeable after normal checks.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title uses the required conventional-commit format. The fix(raw-bam) scope is relevant, and the lowercase imperative description summarizes the change without a trailing period.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@nh13

nh13 commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai pause

@nh13

nh13 commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews paused.

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@codecov

codecov Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.46%. Comparing base (8083173) to head (01264c7).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1022      +/-   ##
==========================================
- Coverage   96.47%   96.46%   -0.01%     
==========================================
  Files         299      299              
  Lines      152124   152297     +173     
==========================================
+ Hits       146756   146920     +164     
- Misses       5368     5377       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13
nh13 added this pull request to the merge queue Oct 8, 2026
Merged via the queue into main with commit 3d33dfb Oct 8, 2026
35 of 43 checks passed
@nh13
nh13 deleted the nh/fix-dovetail-coincident-five-prime branch October 8, 2026 00:41
@nh13 nh13 mentioned this pull request Oct 8, 2026

This branch was successfully deployed

1 active deployment
github-actions — 01264c75 Deployed Oct 5, 2026 by nh13 via coverage #4830
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant