Skip to content

feat(dedup): sample column and marginal duplication-ladder columns (#802) - #841

Merged
nh13 merged 1 commit into
mainfrom
799/nhomer/dedup-metrics-parity
Aug 21, 2026
Merged

nh13 merged 1 commit into
mainfrom
799/nhomer/dedup-metrics-parity

Conversation

@nh13

@nh13 nh13 commented Aug 20, 2026

Copy link
Copy Markdown
Member

Closes #802. Supersedes #803 (auto-closed when its base branch 786/nhomer/feat-dedup-complexity-metrics was deleted by the #799 squash-merge; GitHub cannot reopen a PR whose base branch was deleted). Same head branch and commit, now rebased onto main.

Toward Picard DuplicationMetrics / dupblaster --stats parity, two dupblaster-derived additions to fgumi dedup's QC output:

sample column

--metrics gains a leading sample column, resolved from the header's unique @RG SM: values (comma-joined), with an optional --sample override. Constant across every row of a run; leads the schema, matching dupblaster/Picard.

Marginal duplication-ladder columns

--duplication-ladder gains per-window columns (window_templates, window_duplicate_fraction) alongside the cumulative ones. The marginal rate isolates each depth band instead of averaging over all prior templates — "usually the more legible view" of the saturation curve (dupblaster's complexity.rs). The windows partition the cumulative total exactly.

Not included

The pair/orphan duplicate breakdown (Picard READ_PAIRS_EXAMINED / READ_PAIR_DUPLICATES / UNPAIRED_*) is a distinct algorithmic change — it requires classifying each template as pair vs orphan during marking — so it's tracked as its own follow-up rather than bundled here.

Verification

New tests: sample resolved from @RG SM, via --sample override, and the empty case (no @RG SM: and no --sample → empty column); ladder window columns pinned exactly (each window_templates equals the increment since the previous snapshot, marginal fractions pinned to num/den, windows summing to the per-library total). The pinned column-order test is updated (sample first). The distinct-sample and override tests now pin exact row identities (["libA","libB","All Reads"] / ["libA","All Reads"]), not just non-empty.

#802)

Toward Picard DuplicationMetrics / dupblaster --stats parity, building on the
per-library metrics + saturation ladder in #799:

- `--metrics` gains a leading `sample` column, resolved from the header's
  unique `@RG SM:` values (comma-joined) with an optional `--sample` override.
- `--duplication-ladder` gains marginal per-window columns
  (`window_templates`, `window_duplicate_fraction`) alongside the cumulative
  ones — the marginal duplication rate isolates each depth band instead of
  averaging over all prior ones (dupblaster's complexity.rs view).

The per-reason pair/orphan duplicate breakdown (the third dupblaster metrics
gap) is a distinct algorithmic change and is tracked separately.

Column order is pinned by a test and updated accordingly (sample first).
@nh13
nh13 deployed to github-actions August 20, 2026 05:24 — with GitHub Actions Active
@coderabbitai

coderabbitai Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 34669677-b56a-4c73-9a70-bd1d1b4fe054

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@nh13

nh13 commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai pause

@coderabbitai

coderabbitai Bot commented Aug 20, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews paused.

@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.47%. Comparing base (e51dc4b) to head (e191081).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #841   +/-   ##
=======================================
  Coverage   94.46%   94.47%           
=======================================
  Files         187      187           
  Lines      115301   115361   +60     
=======================================
+ Hits       108921   108982   +61     
+ Misses       6380     6379    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13
nh13 added this pull request to the merge queue Aug 21, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 21, 2026
@nh13
nh13 added this pull request to the merge queue Aug 21, 2026
Merged via the queue into main with commit 05a61ee Aug 21, 2026
16 checks passed
@nh13
nh13 deleted the 799/nhomer/dedup-metrics-parity branch August 21, 2026 23:23
@nh13 nh13 mentioned this pull request Aug 21, 2026

This branch was successfully deployed

1 active deployment
github-actions — e191081f Deployed Aug 20, 2026 by nh13 via coverage #3748
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dedup: enrich metrics toward Picard/dupblaster parity (sample, pair/orphan breakdown, marginal ladder)

1 participant