Skip to content

fix(cargo): keep nextest failed/skipped counts when a test is slow - #4467

Open
Cedric921 wants to merge 1 commit into
rtk-ai:developfrom
Cedric921:fix/nextest-slow-summary
Open

Cedric921 wants to merge 1 commit into
rtk-ai:developfrom
Cedric921:fix/nextest-slow-summary

Conversation

@Cedric921

Copy link
Copy Markdown

Summary

nextest puts its in-place annotation right after passed (2 passed (1 slow), 1 failed, 1 skipped). The summary regex expected failed and skipped to come straight after passed, so once a test went slow both counts parsed as 0. With failed == 0 the filter took its green branch and also dropped the collected failure blocks, so a failing run showed up as cargo nextest: 2 passed (1 binary, 2.503s).

The fix adds an optional non-capturing (?:\s+\([^)]*\))? group right after passed. It matches (1 slow) and any other parenthesised annotation nextest puts there, such as a combined (1 slow, 1 flaky). Capture-group numbering doesn't change, so the rest of the parser is untouched.

Overlap with #3577: that PR adds (N slow) rendering by capturing \((\d+) slow\) at the end of the regex, after skipped. That position doesn't match what nextest actually prints, so its slow capture would still miss real output. This PR only fixes correctness and doesn't render slow counts. If #3577 lands afterwards, its slow capture should move into the group after passed that this PR adds.

Fixes #4420

Test plan

  • New test_filter_cargo_nextest_slow_annotation_keeps_failed_and_skipped uses the real 0.9.143 line Summary [ 2.503s] 3 tests run: 2 passed (1 slow), 1 failed, 1 skipped inside a realistic run with a FAIL block and a SLOW line. It asserts 2 passed, 1 failed, 1 skipped and that the failure header and stderr are kept. It failed before the fix with cargo nextest: 2 passed (1 binary, 2.503s), which is the exact output from the issue.
  • New test_filter_cargo_nextest_slow_annotation_all_pass covers an all-green run with (1 slow), which still renders the compact line.
  • Existing test_filter_cargo_nextest_with_failures (2 passed, 2 failed, 1 skipped, no annotation) still passes as the non-slow non-regression.
  • cargo fmt --all --check
  • cargo clippy --all-targets (no warnings)
  • cargo test

nextest annotates `passed` in place (`2 passed (1 slow), 1 failed, 1 skipped`).
The summary regex expected `failed` to follow `passed` directly, so a slow test
zeroed the failed/skipped counts and the green branch dropped failure blocks.
Allow an optional parenthesised annotation after `passed`.

Fixes rtk-ai#4420
@rtk-wshm-sync-bot rtk-wshm-sync-bot Bot added bug Something isn't working cargo rust testing labels Oct 6, 2026
@rtk-wshm-sync-bot

Copy link
Copy Markdown

wshm · Automated triage by AI

📊 Automated PR Analysis

🐛 Type bug-fix
🟢 Risk low

Summary

Fixes a regex in the cargo nextest output filter so that when a test is slow, nextest's inline '(N slow)' annotation after 'passed' no longer breaks parsing of the subsequent failed/skipped counts. Previously this caused failing/skipped runs with a slow test to be misreported as a clean all-passed summary. Adds two new tests covering the slow-annotation case with failures and an all-pass case.

Review Checklist

  • Tests present
  • Breaking change
  • Docs updated

Linked issues: #4420


Analyzed automatically by wshm · This is an automated analysis, not a human review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cargo rust testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cargo nextest: N failed / N skipped are dropped from the summary whenever a test is slow

1 participant