Skip to content

refactor(sort): rename SpillCompress → SpillBlockCompress to disambiguate from CompressSpill - #557

Merged
nh13 merged 1 commit into
feat-runallfrom
nh/audit-f-spillcompress-rename
Jul 27, 2026
Merged

nh13 merged 1 commit into
feat-runallfrom
nh/audit-f-spillcompress-rename

Conversation

@nh13

@nh13 nh13 commented Jul 10, 2026 •

Copy link
Copy Markdown
Member

What

Two distinct spill-compression steps had confusable, word-swapped names:

  • CompressSpill (chunk-granularity) — compresses a whole sorted chunk and writes the spill file, in the P6 Phase-1 split (SortBuffer → CompressSpill → SortSpillDecompress → SortMerge).
  • SpillCompress (block-granularity) — compresses one raw SpillBlockEvent::Block in place (no file write; SpillWrite writes), in the block-parallel split (SpillGather → SpillCompress → SpillWrite).

They do different things, but the near-mirrored names invited mix-ups. Rename the block-parallel step to SpillBlockCompress (and its module/file to spill_block_compress), which reads naturally against the SpillBlockCompressor kernel it drives and the SpillBlockEvent::Block it consumes.

Notes

  • Pure rename, no behavior change. fgumi-pipeline-io/fgumi-sort are feat-runall-only, so no main-reconcile conflict. Targets nh/audit-3-parity.

Verification

cargo ci-test 2332 passed; cargo ci-lint clean.

Summary by CodeRabbit

  • New Features

    • Updated the sorting pipeline to use block-level spill compression, enabling clearer separation between compression and disk writing.
    • Renamed the spill compression stage to better reflect its block-oriented behavior.
  • Documentation

    • Updated pipeline descriptions and inline documentation to reflect the revised processing stages.
  • Tests

    • Expanded coverage to verify compressed block metadata, residual event passthrough, and clone behavior.

@nh13
nh13 temporarily deployed to github-actions July 10, 2026 19:59 — with GitHub Actions Inactive
@coderabbitai

coderabbitai Bot commented Jul 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: b38da3d6-2867-4ec3-8161-c3b9c72cbfe2

📥 Commits

Reviewing files that changed from the base of the PR and between 6362996 and 37b6451.

📒 Files selected for processing (14)
  • crates/fgumi-pipeline-io/src/sort/compress_spill.rs
  • crates/fgumi-pipeline-io/src/sort/mod.rs
  • crates/fgumi-pipeline-io/src/sort/protocol.rs
  • crates/fgumi-pipeline-io/src/sort/spill_block_compress.rs
  • crates/fgumi-pipeline-io/src/sort/spill_block_compress/tests.rs
  • crates/fgumi-pipeline-io/src/sort/spill_gather.rs
  • crates/fgumi-pipeline-io/src/sort/spill_write.rs
  • crates/fgumi-pipeline-io/src/sort/spill_write/tests.rs
  • crates/fgumi-sort/src/lib.rs
  • crates/fgumi-sort/src/spill_block.rs
  • src/lib/pipeline/chains/build.rs
  • src/lib/pipeline/chains/builder.rs
  • src/lib/pipeline/chains/commands/sort.rs
  • src/lib/pipeline/steps/sort/mod.rs

Walkthrough

The spill compression stage is renamed from SpillCompress to SpillBlockCompress, with updated exports, pipeline construction, tests, profiling names, and documentation. Compression and event pass-through behavior remain unchanged.

Changes

SpillBlockCompress stage

Layer / File(s) Summary
Compressor rename and behavior validation
crates/fgumi-pipeline-io/src/sort/spill_block_compress.rs, crates/fgumi-pipeline-io/src/sort/spill_block_compress/tests.rs
Renames the public step and profile name, while tests verify block compression, metadata preservation, pass-through events, and clone initialization.
Public exports and pipeline wiring
crates/fgumi-pipeline-io/src/sort/mod.rs, src/lib/pipeline/steps/sort/mod.rs, src/lib/pipeline/chains/builder.rs
Replaces SpillCompress exports and constructs SpillBlockCompress in the sort pipeline.
Pipeline documentation alignment
crates/fgumi-pipeline-io/src/sort/*.rs, crates/fgumi-sort/src/*.rs, src/lib/pipeline/chains/*.rs, src/lib/pipeline/chains/commands/sort.rs
Updates spill-stage descriptions and topology comments to use SpillBlockCompress.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • fulcrumgenomics/fgumi#474: Introduced the original SpillCompress spill-chain implementation that this change renames and refines.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed Accurately summarizes the main change: renaming SpillCompress to SpillBlockCompress to distinguish it from CompressSpill.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch nh/audit-f-spillcompress-rename

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (feat-runall@6362996). Learn more about missing BASE report.

Additional details and impacted files
@@              Coverage Diff               @@
##             feat-runall     #557   +/-   ##
==============================================
  Coverage               ?   94.22%           
==============================================
  Files                  ?      111           
  Lines                  ?    50988           
  Branches               ?        0           
==============================================
  Hits                   ?    48045           
  Misses                 ?     2943           
  Partials               ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13
nh13 force-pushed the nh/audit-3-parity branch from 28856c6 to f697a33 Compare July 11, 2026 05:26
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from 897a269 to 8b5f1e7 Compare July 11, 2026 05:55
@nh13
nh13 temporarily deployed to github-actions July 11, 2026 05:55 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from f697a33 to 30af4db Compare July 11, 2026 23:21
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from 8b5f1e7 to d5e811b Compare July 11, 2026 23:23
@nh13
nh13 temporarily deployed to github-actions July 11, 2026 23:24 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from 30af4db to a0c216b Compare July 12, 2026 00:43
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from d5e811b to bdaaffe Compare July 12, 2026 00:44
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 00:45 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from a0c216b to 4fe2c7b Compare July 12, 2026 05:39
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from bdaaffe to 161f50c Compare July 12, 2026 05:39
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 05:39 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from 4fe2c7b to 8f5b2ae Compare July 12, 2026 15:40
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from 161f50c to fbec2ab Compare July 12, 2026 16:44
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 16:44 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from 8f5b2ae to a1323bd Compare July 12, 2026 17:36
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from fbec2ab to 51484c5 Compare July 12, 2026 17:36
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 17:36 — with GitHub Actions Inactive
Base automatically changed from nh/audit-3-parity to feat-runall July 13, 2026 02:38
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from 51484c5 to b3d2de3 Compare July 13, 2026 02:40
@nh13
nh13 temporarily deployed to github-actions July 13, 2026 02:41 — with GitHub Actions Inactive
…guate from CompressSpill

Two distinct spill-compression steps had confusable word-swapped names:

  - CompressSpill  (chunk-granularity): compresses a whole sorted chunk AND
    writes the spill file, in the P6 Phase-1 split
    (SortBuffer -> CompressSpill -> SortSpillDecompress -> SortMerge).
  - SpillCompress  (block-granularity): compresses one raw SpillBlockEvent::Block
    in place (no file write; SpillWrite writes), in the block-parallel split
    (SpillGather -> SpillCompress -> SpillWrite).

They do different things, but the near-mirrored names invited confusion. Rename
the block-parallel step to SpillBlockCompress (and its module/file to
spill_block_compress), which also reads naturally against the SpillBlockCompressor
kernel it drives and the SpillBlockEvent::Block it consumes. Pure rename; no
behavior change.
@nh13
nh13 force-pushed the nh/audit-f-spillcompress-rename branch from b3d2de3 to 37b6451 Compare July 13, 2026 20:02
@nh13
nh13 temporarily deployed to github-actions July 13, 2026 20:03 — with GitHub Actions Inactive
@nh13

nh13 commented Jul 27, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 27, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 merged commit 3f08860 into feat-runall Jul 27, 2026
11 checks passed
@nh13
nh13 deleted the nh/audit-f-spillcompress-rename branch July 27, 2026 23:51

This branch was previously deployed

1 inactive deployment
github-actions — 37b64512 Deployed Jul 13, 2026 by nh13 via coverage #2532
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant