Skip to content

perf(zipper): build the merge tag bitsets once per step, not per template - #917

Merged
nh13 merged 1 commit into
mainfrom
nh/perf-zipper-tags-once
Sep 6, 2026
Merged

nh13 merged 1 commit into
mainfrom
nh/perf-zipper-tags-once

Conversation

@nh13

@nh13 nh13 commented Sep 4, 2026 •

Copy link
Copy Markdown
Member

What

Build the zipper merge tag bitsets (ZipperTags — three TagBitset allocations derived from cfg.tag_info) once per step instead of once per template.

cfg.tag_info is fixed for a step's whole lifetime, so rebuilding ZipperTags inside the per-template merge loop allocated and populated three bitsets for every template with no change in result. Both zipper hot paths are fixed:

  • AlignAndMergeStep (zipper spawns the aligner) holds Arc<ZipperTags> built once in new().
  • ZipperMergeStep::emit_merged (standalone fgumi zipper -i mapped -u unmapped) caches it and drives merge_one_template_with.

The dead merge_one_template(&TagInfo) per-call wrapper is removed and merge_raw_with reverted to private.

Measured

Mac Studio (M3 Ultra, arm64), 8 threads, CPU-seconds via /usr/bin/time -l, 3 reps, byte-identical samtools view | md5 verified:

  • standalone fgumi zipper over 12.0M reads (6M templates; uBAM from fgumi extract, mapped stream from bwa-mem3), --tags-to-reverse cd,ce,ad,ae,bd,be,aq,bq --tags-to-revcomp ac,bc: baseline 47.65 → branch 47.28 CPU-s, −0.8%, branch consistently ≤ baseline across reps. Output byte-identical.

Small — the per-template allocation is a minor slice of zipper's total (BGZF codec + tag merge dominate) — but real, byte-identical, and removes an obvious per-template allocation from both hot paths.

Tests

Regression guard that ZipperMergeStep::new builds ZipperTags once and emit_merged reuses the cached value across multiple (unmapped, mapped) pairs, driving the exact cached path emit_merged uses. Full gate green.

Risk: grouping, consensus, sort order, corrected UMI, and metrics output changes are none; zipper output is byte-identical, covered by regression tests and the full test gate. unsafe changes are none, and CLAUDE.md needs no allowlist update. Memory bounds, queue capacity, and backpressure policy changes are none.

Cache ZipperTags once per merge step and reuse them across template pairs through Arc and merge_one_template_with. This reduces standalone zipper CPU time by 0.8%.

@coderabbitai

coderabbitai Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 626df5f2-75a7-4e4f-8b42-9b597a7eeee6

📥 Commits

Reviewing files that changed from the base of the PR and between 247404b and 7ef243c.

📒 Files selected for processing (2)
  • src/lib/commands/zipper.rs
  • src/lib/pipeline/steps/align_and_merge.rs

Included review availability: 0 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.


Walkthrough

Risk: template merges rebuilt tag lookups repeatedly. Fix: cache ZipperTags in merge steps and pass it to each template merge.

Changes

Zipper tag lookup reuse

Layer / File(s) Summary
Cache tags in zipper merging
src/lib/commands/zipper.rs
ZipperTags and its constructor are crate-visible. merge_one_template_with accepts cached tags. ZipperMergeStep constructs and reuses the tags. A regression test covers tag transformations across multiple templates.
Pass shared tags through alignment merging
src/lib/pipeline/steps/align_and_merge.rs
AlignAndMergeStep stores shared tags and passes them through normal and completion batch drains. Tests cover tag transfer, length mismatches, removal, reversal, and reverse-complement operations.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 7ef24

Zipper tag lookups are now reused across template merges to reduce processing overhead while retaining the existing merge behavior. Both zipper and alignment paths are covered for multi-template tag transformations, with no current merge-blocking risk identified.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title follows Conventional Commit format, uses the valid perf(zipper) scope, and accurately describes the cached merge tag bitsets.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@nh13
nh13 deployed to github-actions September 4, 2026 18:37 — with GitHub Actions Active
@nh13

nh13 commented Sep 4, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai pause

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews paused.

@codecov

codecov Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.74797% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.55%. Comparing base (247404b) to head (7ef243c).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
src/lib/commands/zipper.rs 96.15% 2 Missing ⚠️
src/lib/pipeline/steps/align_and_merge.rs 97.18% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #917      +/-   ##
==========================================
- Coverage   93.55%   93.55%   -0.01%     
==========================================
  Files         301      301              
  Lines      150626   150735     +109     
==========================================
+ Hits       140923   141020      +97     
- Misses       9703     9715      +12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13

nh13 commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 added this pull request to the merge queue Sep 6, 2026
Merged via the queue into main with commit 3ad8f98 Sep 6, 2026
17 checks passed
@nh13
nh13 deleted the nh/perf-zipper-tags-once branch September 6, 2026 17:32
@nh13 nh13 mentioned this pull request Sep 6, 2026

This branch was successfully deployed

1 active deployment
github-actions — 7ef243c7 Deployed Sep 4, 2026 by nh13 via coverage #4180
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant