Skip to content

fix(simulate): emit faithful duplex consensus tags - #603

Merged
nh13 merged 1 commit into
mainfrom
nh/fix-simulate-duplex-tag-fidelity
Jul 21, 2026
Merged

nh13 merged 1 commit into
mainfrom
nh/fix-simulate-duplex-tag-fidelity

Conversation

@nh13

@nh13 nh13 commented Jul 20, 2026 •

Copy link
Copy Markdown
Member

Two defects made fgumi simulate consensus-reads --duplex produce records that no real consensus caller emits, so simulated duplex BAMs were unsound fixtures for validating fgumi filter and for fgbio-parity harnesses.

Combined per-base arrays on duplex records

CD_BASES/CE_BASES were written unconditionally, but duplex_caller emits only the per-strand AD_BASES/AE_BASES/BD_BASES/BE_BASES arrays alongside the scalar cD/cM/cE (duplex_caller.rs:1132-1177 vs :1207-1209). Now emitted only in simplex mode.

This is safe for the documented "suitable for input to fgumi filter" contract: filter branches on is_duplex_consensus and masks duplex reads through mask_duplex_bases, which reads the four per-strand arrays (crates/fgumi-consensus/src/filter.rs:819-825) and never the combined ones — while simplex reads still get the CD_BASES/CE_BASES that mask_bases requires.

The duplex cD/cM/cE scalars are now derived from the per-strand sums rather than the independently sampled simplex values, matching duplex_caller, which computes them over the combined per-base depth (ab_i + ba_i).

Strand minimum depths could exceed the combined minimum

aM and bM were each computed as strand_depth.min(cM), so both could equal cM and aM + bM routinely exceeded it. A duplex position's combined depth is the sum of its two strand depths, so the truth TSV and the BAM tags disagreed with each other — anyone validating a real run against --truth got impossible expectations.

cM is now split by the same strand fraction already used for cD, clamping the A share to [cM - bD, min(aD, cM)]. That range is feasible because cM <= cD == aD + bD, and it guarantees aM + bM == cM, aM <= aD, and bM <= bD.

Testing

Tests sweep 64 seeds — the strand fraction is sampled per read, so a single seed proves little — asserting the summation and ordering invariants, and that the truth tuple's cD/cM agree with the emitted tags (the truth file carries sampled values while the tags are now derived; only the aM + bM == cM invariant keeps them equal, so it is pinned explicitly). A second test asserts duplex records carry no CD_BASES/CE_BASES while retaining all four per-strand arrays, and that simplex records still carry the combined arrays.

  • cargo ci-test: 5536 passed, 22 skipped
  • cargo ci-fmt, cargo ci-lint: clean

Summary by CodeRabbit

  • Bug Fixes
    • Improved duplex consensus depth calculations to ensure strand-level and combined minimums remain consistent.
    • Corrected duplex combined consensus summary tags for more accurate BAM metadata.
    • Updated duplex record tag output to omit combined per-base depth/error tags where not applicable (simplex output remains unchanged).
  • Tests
    • Added regression and property-based coverage validating duplex invariants and correct combined tag generation.

@nh13
nh13 temporarily deployed to github-actions July 20, 2026 21:41 — with GitHub Actions Inactive
@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: ba09ae74-7f5f-478f-9cba-5d7d6022cc10

📥 Commits

Reviewing files that changed from the base of the PR and between 785ce43 and aff6d77.

📒 Files selected for processing (1)
  • src/lib/commands/simulate/consensus_reads.rs

Walkthrough

Duplex consensus generation now enforces aM + bM == cM, derives combined summary tags from summed strand arrays, and omits duplex combined per-base tags. Simplex output retains those tags, with regression and property-based tests covering both paths.

Changes

Duplex consensus tags

Layer / File(s) Summary
Strand minimum consistency
src/lib/commands/simulate/consensus_reads.rs
Calculates strand minimum depths from a clamped split of cM, preserving strand bounds and enforcing aM + bM == cM.
BAM tag emission and regression coverage
src/lib/commands/simulate/consensus_reads.rs
Derives duplex CD, CM, and CE from summed strand arrays, omits CD_BASES and CE_BASES for duplex records, preserves them for simplex records, and tests these invariants across generated cases.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: making simulated duplex consensus tags faithful to caller output.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch nh/fix-simulate-duplex-tag-fidelity

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.42%. Comparing base (f3b0c78) to head (aff6d77).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #603      +/-   ##
==========================================
- Coverage   93.42%   93.42%   -0.01%     
==========================================
  Files         175      175              
  Lines      104807   104935     +128     
==========================================
+ Hits        97919    98031     +112     
- Misses       6888     6904      +16     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13

nh13 commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/commands/simulate/consensus_reads.rs`:
- Around line 2002-2046: The invariant test
test_duplex_strand_minimums_sum_to_combined_minimum should use proptest instead
of iterating over 64 fixed seeds. Convert the seed-driven assertions into a
proptest property with generated seeds or relevant inputs, preserving all
existing truth/emitted and strand-sum invariants while allowing failing cases to
shrink.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a863afb3-d89e-4c53-9115-08c7797f62b3

📥 Commits

Reviewing files that changed from the base of the PR and between f3b0c78 and 785ce43.

📒 Files selected for processing (1)
  • src/lib/commands/simulate/consensus_reads.rs

Comment thread src/lib/commands/simulate/consensus_reads.rs
Two defects made `fgumi simulate consensus-reads --duplex` produce records that no
real consensus caller would emit, so simulated duplex BAMs were unsound fixtures
for validating `fgumi filter` and for fgbio-parity harnesses.

Combined per-base arrays on duplex records. `CD_BASES`/`CE_BASES` were written
unconditionally, but `duplex_caller` emits only the per-strand `AD_BASES`/
`AE_BASES`/`BD_BASES`/`BE_BASES` arrays alongside the scalar cD/cM/cE. Emit the
combined arrays only in simplex mode. This is safe for the documented
"suitable for input to `fgumi filter`" contract: filter branches on
`is_duplex_consensus` and masks duplex reads through `mask_duplex_bases`, which
reads the four per-strand arrays and never the combined ones, while simplex reads
still get the `CD_BASES`/`CE_BASES` that `mask_bases` requires.

Derive the duplex cD/cM/cE scalars from the per-strand sums rather than from the
independently sampled simplex values, matching `duplex_caller`, which computes
them over the combined per-base depth (ab_i + ba_i).

Strand minimum depths. `aM` and `bM` were each computed as `strand_depth.min(cM)`,
so both could equal `cM` and `aM + bM` routinely exceeded it. A duplex position's
combined depth is the sum of its two strand depths, so the truth TSV and the BAM
tags disagreed with each other and anyone validating a real run against `--truth`
got impossible expectations. Split `cM` by the same strand fraction already used
for `cD`, clamping the A share to `[cM - bD, min(aD, cM)]` — feasible because
`cM <= cD == aD + bD` — which guarantees `aM + bM == cM`, `aM <= aD`, and
`bM <= bD`.

The combined scalars are summed from the per-strand scalars rather than reduced
over the per-base arrays: the two agree only when the arrays carry the min anchor,
which `per_base_arrays` adds solely for `read_len >= 2`, so a `--read-length 1`
run would have reported `cM == cD` and drifted from the truth TSV again.

A proptest over generated seeds (the strand fraction is sampled per read) asserts
the summation and ordering invariants, an `rstest` case table pins the truth/tag
agreement at read lengths 1, 2, and 50, and further tests assert duplex records
carry no `CD_BASES`/`CE_BASES` while retaining all four per-strand arrays, and
that simplex records still carry the combined arrays.
@nh13
nh13 force-pushed the nh/fix-simulate-duplex-tag-fidelity branch from 785ce43 to aff6d77 Compare July 21, 2026 06:26
@nh13
nh13 temporarily deployed to github-actions July 21, 2026 06:26 — with GitHub Actions Inactive
@nh13

nh13 commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 merged commit d6eca14 into main Jul 21, 2026
14 checks passed
@nh13
nh13 deleted the nh/fix-simulate-duplex-tag-fidelity branch July 21, 2026 07:36
@nh13 nh13 mentioned this pull request Jul 21, 2026

This branch was previously deployed

1 inactive deployment
github-actions — aff6d77c Deployed Jul 21, 2026 by nh13 via coverage #2834
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant