Skip to content

fix(ci): judge a panic ratchet red by its per-lint rows, not the packages it names - #29057

Merged
teamleaderleo merged 2 commits into
feat-cmux-nextfrom
drain-acpmux/ratchet-excuse-lint-lines
Oct 31, 2026
Merged

teamleaderleo merged 2 commits into
feat-cmux-nextfrom
drain-acpmux/ratchet-excuse-lint-lines

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

scripts/ci/base_red_excuse.py never excused a Rust panic ratchet red on clippy (changed cmux-tui packages) (step Rust panic ratchet on changed cmux-tui packages). It compared the ratchet's ::error line, which names the packages the run checked. A base run can be a whole-workspace dispatch ("the panic ratchet is red for the whole cmux-tui workspace"), but a PR run names only its changed packages. So the line never matched, even when every failure row was the same on both runs.

Concrete case: cmux#26849 at f81fd0a, run 41176610742, fails on exactly two rows:

  • acpmux: clippy::indexing_slicing 692 > baseline 686
  • cmux-tui: clippy::indexing_slicing 836 > baseline 831

Base run 41176544319 (at 734e99a) fails on those same two rows plus cmux-rd-link: clippy::indexing_slicing 7 > new crate (must be 0). The classifier still said not excused: errors not on feat-cmux-next run 41176544319.

The ratchet red is now judged by the rows in its rust-panic-ratchet: RED block:

  • Each head row CRATE: clippy::LINT N > WHERE must be on the base with the same crate, the same lint and the same WHERE (baseline M or new crate (must be 0)), and the base's count must be at least the head's. The base may have extra rows.
  • Hit-location lines (crates/x/src/y.rs:L:C) are not compared. Any other RED line (for example an allow with no reason) must be on the base word for word.
  • A ratchet red with no row is never excused. The ratchet prints no row when its clippy run did not finish.
  • Other error lines in the same job (a clippy compile error, say) are still compared as before. So are failed steps. The crash and cut-short rules are unchanged.

This also closes a gap that loosened the check: when the head and base had the same scope line, the old code excused the head even when its count was higher than the base's (see the ..._with_the_same_scope_too test). That case now blocks.

No open PR fixes this classifier. #28958 lowers the base counts. It does not change the classifier.

Testing

Command: python3 -m unittest tests.test_base_red_excuse.PanicRatchet

  • At the test commit (fd5ad6b, fix not applied): FAILED (failures=7, errors=2) of 10 tests.
  • At the head: Ran 10 tests ... OK. The whole file python3 -m unittest tests.test_base_red_excuse passes: 106 tests, OK.

Observed / Not run

  • Observed: the new tests red and then green, as above. I read the real job logs of run 41176610742 (job 124465364731) and base run 41176544319 (job 124465669565) and ran the new parser on them. ratchet_red read 2 head rows and 3 base rows, and ratchet_unmatched returned no unmatched lines.
  • Not run: the end-to-end base_red_excuse.py --run 41176610742 against GitHub. The shared REST quota ran out (HTTP 403) when I tried it. I will re-check after the reset and report the result in a PR comment.

Changelog

none

risk:normal: CI tooling only. It loosens the base-red excuse for one kind of check and tightens it in the same-scope case.

Agent session: agent session

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Fixes the base-red excuse for Rust panic ratchet failures so ratchet reds are judged by their per-lint rows instead of the scope line that names the checked packages.

A base run may cover the whole cmux-tui workspace while a PR run covers only its changed packages, so the scope line never matched and valid excuses were refused. Now each head row (CRATE: clippy::LINT N > WHERE) must appear on the base with the same crate, lint, and WHERE and a count at least the head's; other RED lines match word for word, and a red with no rows is never excused.

Behavior changes

  • Tightens the check: a head count above the base's now blocks even when both runs share the same scope.
  • Other error lines in the same job and failed steps are still compared as before.

Written for commit 317aa68. Summary will update on new commits.

View guided diff Turn on auto-fix

teamleaderleo and others added 2 commits October 30, 2026 23:58
…kspace base run

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-710a4a518c009a6ad126
…e line

The ratchet's ::error line names the packages a run checked. A base run can
check the whole cmux-tui workspace while a PR checks only its changed
packages, so the line never matched and no ratchet red was ever excused
(cmux#26849's run 41176610742 failed on two rows that base run 41176544319
failed on word for word). Each head row must now be on the base with the
same crate, lint and baseline and a count at least the head's; other RED
lines must match word for word, and a red with no row is never excused.
A head count above the base's now blocks even when both runs share a scope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-710a4a518c009a6ad126
@teamleaderleo teamleaderleo added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 317aa68: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr29057 -f ref=drain-acpmux/ratchet-excuse-lint-lines -f platform=macos -f backend_mode=local

Shipped in:

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6.1-sol high fast, review-router; started by the backlog lander): SHIP at 317aa68

SHIP

  1. No P1, P2, or P3 findings.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

End-to-end check after the quota reset: this branch's base_red_excuse.py --sha f81fd0a3e4386ddc13eea076dc9a5315abb4c798 --run 41176610742 (cmux#26849) exits 0 with:

excused 'clippy (changed cmux-tui packages)': feat-cmux-next run 41176544319 at 734e99a7e6 ... fails the same way: acpmux: clippy::indexing_slicing 692 > baseline 686; cmux-tui: clippy::indexing_slicing 836 > baseline 831

On origin/feat-cmux-next the same command said not excused: errors not on feat-cmux-next run 41176544319.

@blacksmith-sh

blacksmith-sh Bot commented Oct 31, 2026

Copy link
Copy Markdown

Found 3 test failures on Blacksmith runners:

Failures

Test View Logs
test_a_picked_signing_label_has_a_rerun_alternative (main.RunsOnFallbackTests.test_
a_picked_signing_label_has_a_rerun_alternative)/
test_a_picked_signing_label_has_a_rerun_alternative (main.RunsOnFallbackTests.test_
a_picked_signing_label_has_a_rerun_alternative)
View Logs
test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt
epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)/
test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt
epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)
View Logs
test_no_runs_on_chain_ends_on_the_reserved_side_label (main.RunsOnFallbackTests.tes
t_no_runs_on_chain_ends_on_the_reserved_side_label)/
test_no_runs_on_chain_ends_on_the_reserved_side_label (main.RunsOnFallbackTests.tes
t_no_runs_on_chain_ends_on_the_reserved_side_label)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo
teamleaderleo merged commit d10348a into feat-cmux-next Oct 31, 2026
172 of 195 checks passed
@teamleaderleo
teamleaderleo deleted the drain-acpmux/ratchet-excuse-lint-lines branch October 31, 2026 07:16

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 317aa685 Deployed Oct 31, 2026 by teamleaderleo via wait for same-key tree publisher #88097
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant