Skip to content

test: close spill/cancellation coverage gaps + rstest-ify reject loop (audit E1/E2/E3) - #543

Merged
nh13 merged 1 commit into
feat-runallfrom
nh/audit-e-test-coverage
Jul 27, 2026
Merged

nh13 merged 1 commit into
feat-runallfrom
nh/audit-e-test-coverage

Conversation

@nh13

@nh13 nh13 commented Jul 10, 2026 •

Copy link
Copy Markdown
Member

What

Three test-suite gaps from the feat-runall audit (findings/13).

E1 (CG-3) — the sort spill matrix drives -m down to force spills but never asserts one happened. If memory accounting regressed so the engine ignored -m and held everything in RAM, every matrix case would still pass (output is still correct) — silently deleting all spill/merge coverage and the memory-bound guarantee. Added sort_spill_actually_occurs_under_small_memory: a subprocess run with RUST_LOG=info + --threads 1 observes the arena SortMerge step's INFO log — a tiny -m must take the disk-spill merge path ("Sort merge complete"), a large -m the single-source in-memory fast path ("in-memory fast path complete") — pinning the lever in both directions.

E2 (CG-5) — every runall failure test rejects before the pipeline runs (bad stage pair, missing flag). Added runall_group_simplex_fails_cleanly_on_truncated_bam: feed a BAM truncated mid-stream to runall --start-from group --stop-after simplex and assert a clean non-zero exit with no panicked/unreachable in stderr, under a wall-clock watchdog so a hang fails fast instead of wedging CI.

E3 (TQ-4) — convert the test_single_threaded_consensus_rejects_sam_with_threads_hint sequential-assert loop (codec/simplex/duplex) into an #[rstest] case table so a regression names the command instead of dying on the first bad assert (repo convention). The two for (input, output, …) loops at :612/:700 are intentionally left as-is: they produce both BAM and SAM outputs and cross-compare them, so they are setup-then-compare, not independent per-scenario asserts (rstest-ifying would break the comparison).

Target rationale (main vs feat-runall)

test_runall_parity.rs is feat-runall-only; test_sort_correctness.rs/test_streaming_input.rs exist on main but the specific spill-matrix / new-pipeline SAM tests these touch are feat-runall test surface. Feat-runall-only, so this targets the audit stack (nh/audit-3-parity).

Tests

All three new/converted tests pass; cargo ci-test 2338 passed / 8 skipped; cargo ci-lint clean.

Note: this commit is currently unsigned — the local 1Password signing agent was wedged this session. It will be re-signed before merge.

Summary by CodeRabbit

  • Bug Fixes

    • Improved handling of truncated BAM input so runall fails cleanly (no hang, no panics) when data is cut mid-record.
  • Tests

    • Added integration coverage to validate sort correctness across both in-memory and merge/spill paths, including expected stderr log behavior.
    • Added/parameterized integration tests ensuring single-threaded consensus rejects unsuitable SAM inputs and emits clear --threads guidance.
    • Added a new parity-focused integration test to confirm failure behavior under truncated inputs.

@coderabbitai

coderabbitai Bot commented Jul 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 583ab5e3-1180-4fc6-b859-61c8efe41a78

📥 Commits

Reviewing files that changed from the base of the PR and between cbf969e and 604df09.

📒 Files selected for processing (3)
  • tests/integration/test_runall_parity.rs
  • tests/integration/test_sort_correctness.rs
  • tests/integration/test_streaming_input.rs

Walkthrough

Integration coverage tests clean failure for truncated BAM input, distinguishes sort spill and in-memory paths while checking output correctness, and parameterizes single-threaded consensus SAM-input checks across codec, simplex, and duplex commands.

Changes

Integration test coverage

Layer / File(s) Summary
Runall truncated-BAM failure handling
tests/integration/test_runall_parity.rs
Builds a mid-record BGZF truncation, runs the group-to-simplex consensus pipeline with concurrent stderr draining and a 60-second watchdog, and checks for unsuccessful exit without panic markers.
Sort spill-path and output validation
tests/integration/test_sort_correctness.rs
Exercises parameterized small-memory sort paths, validates merge and in-memory INFO logs plus source counts, and checks coordinate ordering and record preservation.
Streaming-input command parameterization
tests/integration/test_streaming_input.rs
Runs the single-threaded consensus SAM-input failure test independently for codec, simplex, and duplex commands while checking the --threads hint.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main changes: spill-path coverage, cancellation handling, and rstest-based rejection coverage.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch nh/audit-e-test-coverage

Comment @coderabbitai help to get the list of available commands.

@nh13
nh13 temporarily deployed to github-actions July 10, 2026 07:11 — with GitHub Actions Inactive
@codecov

codecov Bot commented Jul 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (feat-runall@a3862c5). Learn more about missing BASE report.

Additional details and impacted files
@@              Coverage Diff               @@
##             feat-runall     #543   +/-   ##
==============================================
  Coverage               ?   94.24%           
==============================================
  Files                  ?      111           
  Lines                  ?    51055           
  Branches               ?        0           
==============================================
  Hits                   ?    48116           
  Misses                 ?     2939           
  Partials               ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from a6b4ecf to 0c8f6bd Compare July 10, 2026 16:24
@nh13
nh13 temporarily deployed to github-actions July 10, 2026 16:24 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from 28856c6 to f697a33 Compare July 11, 2026 05:26
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from 0c8f6bd to 5d988b1 Compare July 11, 2026 05:55
@nh13
nh13 temporarily deployed to github-actions July 11, 2026 05:55 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from f697a33 to 30af4db Compare July 11, 2026 23:21
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from 5d988b1 to 38f4b0f Compare July 11, 2026 23:23
@nh13
nh13 temporarily deployed to github-actions July 11, 2026 23:23 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from 30af4db to a0c216b Compare July 12, 2026 00:43
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from 38f4b0f to e20b112 Compare July 12, 2026 00:44
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 00:44 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from a0c216b to 4fe2c7b Compare July 12, 2026 05:39
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from e20b112 to f5ed73f Compare July 12, 2026 05:39
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 05:39 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from 4fe2c7b to 8f5b2ae Compare July 12, 2026 15:40
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from f5ed73f to 7c209a0 Compare July 12, 2026 16:44
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 16:44 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-3-parity branch from 8f5b2ae to a1323bd Compare July 12, 2026 17:36
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from 7c209a0 to 634cbce Compare July 12, 2026 17:36
@nh13
nh13 temporarily deployed to github-actions July 12, 2026 17:36 — with GitHub Actions Inactive
Base automatically changed from nh/audit-3-parity to feat-runall July 13, 2026 02:38
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from 634cbce to ce1ffa6 Compare July 13, 2026 02:40
@nh13
nh13 temporarily deployed to github-actions July 13, 2026 02:40 — with GitHub Actions Inactive
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from ce1ffa6 to cbf969e Compare July 13, 2026 03:18
@nh13
nh13 temporarily deployed to github-actions July 13, 2026 03:18 — with GitHub Actions Inactive
@nh13

nh13 commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Jul 20, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Jul 26, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 26, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Jul 26, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 26, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Jul 26, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 26, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/test_streaming_input.rs`:
- Around line 378-380: Update the argument construction in the streaming input
test to explicitly include the “--threads 1” option alongside the existing
compression setting, ensuring the rejection assertion exercises the stated
single-threaded configuration rather than relying on the default.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: ba42972c-d562-4149-8dab-055f785ad299

📥 Commits

Reviewing files that changed from the base of the PR and between 6362996 and cbf969e.

📒 Files selected for processing (3)
  • tests/integration/test_runall_parity.rs
  • tests/integration/test_sort_correctness.rs
  • tests/integration/test_streaming_input.rs

Comment thread tests/integration/test_streaming_input.rs
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from cbf969e to ee00a3b Compare July 26, 2026 17:00
@nh13
nh13 temporarily deployed to github-actions July 26, 2026 17:00 — with GitHub Actions Inactive
…loop (E1, E2, E3)

Three test-suite gaps from the feat-runall audit (findings/13).

E1 (CG-3) — the sort spill matrix drives `-m` down to force spills but never
asserts one happened; if memory accounting regressed so the engine ignored `-m`,
every matrix case would still pass while silently deleting all spill/merge
coverage. Add `sort_spill_actually_occurs_under_small_memory`: a subprocess run
with `RUST_LOG=info` + `--threads 1` observes the arena `SortMerge` step's log —
a tiny `-m` must take the disk-spill merge path ("Sort merge complete"), a large
`-m` the single-source in-memory fast path ("in-memory fast path complete") —
pinning the memory-bound lever in both directions.

E2 (CG-5) — every runall failure test rejects before the pipeline runs. Add
`runall_group_simplex_fails_cleanly_on_truncated_bam`: feed a BAM truncated
mid-stream to `runall --start-from group --stop-after simplex` and assert a clean
non-zero exit with no `panicked`/`unreachable` in stderr, under a wall-clock
watchdog so a hang fails fast instead of wedging CI.

E3 (TQ-4) — convert the `test_single_threaded_consensus_rejects_sam_with_threads_hint`
sequential-assert loop (codec/simplex/duplex) into an `#[rstest]` case table so a
regression names the command instead of dying on the first bad assert (repo
convention). The two `for (input, output, …)` loops at :612/:700 are left as-is:
they produce both BAM and SAM outputs and cross-compare them, so they are
setup-then-compare, not independent per-scenario asserts.
@nh13
nh13 force-pushed the nh/audit-e-test-coverage branch from ee00a3b to 604df09 Compare July 26, 2026 18:29
@nh13
nh13 temporarily deployed to github-actions July 26, 2026 18:29 — with GitHub Actions Inactive
@nh13

nh13 commented Jul 27, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 27, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13

nh13 commented Jul 27, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 27, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 merged commit 4588365 into feat-runall Jul 27, 2026
11 checks passed
@nh13
nh13 deleted the nh/audit-e-test-coverage branch July 27, 2026 01:08

This branch was previously deployed

1 inactive deployment
github-actions — 604df09a Deployed Jul 26, 2026 by nh13 via coverage #3116
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant