ci(canary): remove provider-matrix lanes and zizmor scan - #7418
Merged
Merged
Conversation
github-actions
Bot
added
scope: ci
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe live-canary workflow no longer schedules or runs the provider-matrix lane. Runner dispatch, documentation, workflow contracts, and timeout filters reflect the remaining lanes. Changes to ChangesCI execution updates
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🧭 IronLoop Run · ReviewThis comment updates in place as the Run moves through its stages. ⬛ Final result · Stopped
Automatic trigger · attempt 1 of 3 · stopped after 4m 56s IronLoop stopped because the pull request target branch or head changed while this Run was active. |
.config/nextest.toml is read by every Tests (Reborn) lane, so the fail-closed planner arm raised 'unclassified pull-request path' on any PR touching it, skipping all downstream Reborn lanes. Widen it to the exhaustive plan like crate deletions.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/ci/test_reborn_pr_test_plan.py`:
- Around line 918-935: Expand test_nextest_config_widens_to_exhaustive_plan to
assert every field returned by _full_plan, including package buckets,
run_group_tests, run_qa_replay, run_sandbox_docker, and coverage_mode, in
addition to the existing exhaustive-plan fields. Use the canonical exhaustive
values so regressions in any part of the plan fail the test.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 94e6e1e7-ffee-4a06-938c-5f0b1888ddc3
📒 Files selected for processing (2)
scripts/ci/reborn_pr_test_plan.pyscripts/ci/test_reborn_pr_test_plan.py
Comment on lines
+918
to
+935
| def test_nextest_config_widens_to_exhaustive_plan(self) -> None: | ||
| """`.config/nextest.toml` is runner config every test lane reads. | ||
|
|
||
| Regression for the provider-matrix retirement PR: deleting the dead | ||
| `live_tests::zizmor_scan*` overrides from `.config/nextest.toml` made | ||
| the fail-closed arm raise `unclassified pull-request path`, which | ||
| failed `Detect Reborn test scope` and skipped every downstream Reborn | ||
| lane. The file is read by every `Tests (Reborn)` lane, so no narrow | ||
| lane can exercise a change to it; it must NOT go to static control | ||
| (whose membership rule is "no Reborn test lane reads the file"). | ||
| Widening to the exhaustive plan is the safe resolution — a superset | ||
| can never under-select. | ||
| """ | ||
| plan = self.plan("pull_request", [".config/nextest.toml"]) | ||
| self.assertEqual(plan["mode"], "full") | ||
| self.assertEqual(plan["root_partitions"], [0, 1, 2, 3]) | ||
| self.assertEqual(plan["integration_lanes"], [0, 1, 2, 3, "groups"]) | ||
| self.assertIn("nextest runner config changed", plan["reasons"][0]) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win
Pin the complete exhaustive-plan contract.
The test checks only mode, root partitions, integration lanes, and the reason. It can still pass if package buckets, run_group_tests, run_qa_replay, run_sandbox_docker, or coverage_mode regress.
Add assertions for all fields that make _full_plan exhaustive.
Proposed assertions
plan = self.plan("pull_request", [".config/nextest.toml"])
self.assertEqual(plan["mode"], "full")
+ self.assertEqual(plan["changed_packages"], [])
+ self.assertEqual(plan["affected_packages"], self.canonical)
+ self.assertEqual(
+ plan["crate_buckets"],
+ [{"name": "selected", "packages": self.canonical}],
+ )
self.assertEqual(plan["root_partitions"], [0, 1, 2, 3])
self.assertEqual(plan["integration_lanes"], [0, 1, 2, 3, "groups"])
+ self.assertTrue(plan["run_group_tests"])
+ self.assertTrue(plan["run_qa_replay"])
+ self.assertTrue(plan["run_sandbox_docker"])
+ self.assertEqual(plan["coverage_mode"], "full")
self.assertIn("nextest runner config changed", plan["reasons"][0])📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def test_nextest_config_widens_to_exhaustive_plan(self) -> None: | |
| """`.config/nextest.toml` is runner config every test lane reads. | |
| Regression for the provider-matrix retirement PR: deleting the dead | |
| `live_tests::zizmor_scan*` overrides from `.config/nextest.toml` made | |
| the fail-closed arm raise `unclassified pull-request path`, which | |
| failed `Detect Reborn test scope` and skipped every downstream Reborn | |
| lane. The file is read by every `Tests (Reborn)` lane, so no narrow | |
| lane can exercise a change to it; it must NOT go to static control | |
| (whose membership rule is "no Reborn test lane reads the file"). | |
| Widening to the exhaustive plan is the safe resolution — a superset | |
| can never under-select. | |
| """ | |
| plan = self.plan("pull_request", [".config/nextest.toml"]) | |
| self.assertEqual(plan["mode"], "full") | |
| self.assertEqual(plan["root_partitions"], [0, 1, 2, 3]) | |
| self.assertEqual(plan["integration_lanes"], [0, 1, 2, 3, "groups"]) | |
| self.assertIn("nextest runner config changed", plan["reasons"][0]) | |
| def test_nextest_config_widens_to_exhaustive_plan(self) -> None: | |
| """`.config/nextest.toml` is runner config every test lane reads. | |
| Regression for the provider-matrix retirement PR: deleting the dead | |
| `live_tests::zizmor_scan*` overrides from `.config/nextest.toml` made | |
| the fail-closed arm raise `unclassified pull-request path`, which | |
| failed `Detect Reborn test scope` and skipped every downstream Reborn | |
| lane. The file is read by every `Tests (Reborn)` lane, so no narrow | |
| lane can exercise a change to it; it must NOT go to static control | |
| (whose membership rule is "no Reborn test lane reads the file"). | |
| Widening to the exhaustive plan is the safe resolution — a superset | |
| can never under-select. | |
| """ | |
| plan = self.plan("pull_request", [".config/nextest.toml"]) | |
| self.assertEqual(plan["mode"], "full") | |
| self.assertEqual(plan["changed_packages"], []) | |
| self.assertEqual(plan["affected_packages"], self.canonical) | |
| self.assertEqual( | |
| plan["crate_buckets"], | |
| [{"name": "selected", "packages": self.canonical}], | |
| ) | |
| self.assertEqual(plan["root_partitions"], [0, 1, 2, 3]) | |
| self.assertEqual(plan["integration_lanes"], [0, 1, 2, 3, "groups"]) | |
| self.assertTrue(plan["run_group_tests"]) | |
| self.assertTrue(plan["run_qa_replay"]) | |
| self.assertTrue(plan["run_sandbox_docker"]) | |
| self.assertEqual(plan["coverage_mode"], "full") | |
| self.assertIn("nextest runner config changed", plan["reasons"][0]) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/ci/test_reborn_pr_test_plan.py` around lines 918 - 935, Expand
test_nextest_config_widens_to_exhaustive_plan to assert every field returned by
_full_plan, including package buckets, run_group_tests, run_qa_replay,
run_sandbox_docker, and coverage_mode, in addition to the existing
exhaustive-plan fields. Use the canonical exhaustive values so regressions in
any part of the plan fail the test.
serrrfirat
enabled auto-merge
August 10, 2026 07:58
think-in-universe
approved these changes
Aug 10, 2026
7 tasks
Kampouse
pushed a commit
to Kampouse/ironclaw
that referenced
this pull request
Aug 13, 2026
* ci(canary): remove provider-matrix lanes and zizmor scan * fix(ci): classify nextest config as exhaustive-plan change .config/nextest.toml is read by every Tests (Reborn) lane, so the fail-closed planner arm raised 'unclassified pull-request path' on any PR touching it, skipping all downstream Reborn lanes. Widen it to the exhaustive plan like crate deletions.
Kampouse
pushed a commit
to Kampouse/ironclaw
that referenced
this pull request
Aug 13, 2026
* ci(canary): remove provider-matrix lanes and zizmor scan * fix(ci): classify nextest config as exhaustive-plan change .config/nextest.toml is read by every Tests (Reborn) lane, so the fail-closed planner arm raised 'unclassified pull-request path' on any PR touching it, skipping all downstream Reborn lanes. Widen it to the exhaustive plan like crate deletions.
l3ocifer
pushed a commit
to l3ocifer/frick-ironclaw
that referenced
this pull request
Sep 3, 2026
* ci(canary): remove provider-matrix lanes and zizmor scan * fix(ci): classify nextest config as exhaustive-plan change .config/nextest.toml is read by every Tests (Reborn) lane, so the fail-closed planner arm raised 'unclassified pull-request path' on any PR touching it, skipping all downstream Reborn lanes. Widen it to the exhaustive plan like crate deletions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
provider-matrixjob fromlive-canary.ymland its weekly cron (30 5 * * 1) — both provider lanes were failing in seconds because they invokecargo test --test e2e_liveagainst test targets deleted with the v1 monolith (refactor(tier-b): delete v1 legacy monolith (src/) and cut deploy over to Reborn #6375), so they were running dead code paths.Pre-install zizmorsteps inpublic-smoke/release-public-full, thezizmor_scan/zizmor_scan_v2scenario runs inrun.sh, the deadtests/fixtures/llm_traces/live/zizmor_scan*.jsonfixtures, and the stalenextest.tomloverrides referencing the deletedlive_tests::zizmor_scan*tests.ws12_workflow_contracts.py(3-hourly Reborn WebUI cadence pins kept) and update the canary docs/lane lists.reborn-webui-v2-live-qalane, its 3-hourly schedule,/canaryPR command, and all other lanes are untouched.Change Type
Linked Issue
None — maintainer-requested retirement of the failing canary lanes.
Validation
cargo fmt --all -- --checkcargo clippy --all --benches --tests --examples --all-features -- -D warningscargo buildpython3 scripts/ci/test_ws12_workflow_contracts.py(52/52),scripts/reborn_webui_v2_live_qa/test_run_live_qa.py::test_live_canary_workflow_shards_cover_non_telegram_qa_suite(OK, py3.14),python3 scripts/ci/test-check-regression-promotions.py(11/11),python3 scripts/live-canary/test_emit_results_json.py(27/27)cargo test -p <owning-crate> --features integrationif database-backed or runtime-integration behavior changed (the rootintegrationfeature is empty — the flag is per-crate)bash -n scripts/live-canary/run.sh;live-canary.ymlparsed — 17 jobs, single 3-hourly cron,provider-matrixabsent from jobs /canary-reportneeds / dispatch optionsreview-prorpr-shepherd --fixwas run before requesting reviewTest Strategy
User behavior: no user-facing behavior changes — removes two failing scheduled CI lanes and the zizmor scenario from the live canary.
Risk areas:
Tests added or updated:
test_ws12_workflow_contracts.pymarkers unpinned; full suite passes with the edited workflowzizmor_scan*.jsondeleted (consumingtests/e2e_live*.rswere removed in refactor(tier-b): delete v1 legacy monolith (src/) and cut deploy over to Reborn #6375; no remaining consumers)reborn-webui-v2-live-qashard contract test still passesWhat the tests prove: the workflow-contract sabotage suite still passes with the retired markers unpinned (i.e., no required schedule disappeared), and the live QA shard coverage contract is unaffected.
Commands run:
python3 scripts/ci/test_ws12_workflow_contracts.py/opt/homebrew/bin/python3.14 -m unittest scripts.reborn_webui_v2_live_qa.test_run_live_qa.RebornWebUiV2LiveQaRunnerTests.test_live_canary_workflow_shards_cover_non_telegram_qa_suitepython3 scripts/ci/test-check-regression-promotions.pypython3 scripts/live-canary/test_emit_results_json.pybash -n scripts/live-canary/run.shlive-canary.yml(jobs, schedule, needs)Security Impact
None. Retires CI lanes that consumed live provider credentials (ANTHROPIC_API_KEY, LIVE_OPENAI_COMPATIBLE_API_KEY); no new permissions, network calls, or secret handling introduced.
Reborn Trust-Boundary Checklist
N/A — no Rust, runtime, or trust-boundary code touched; CI workflow/scripts/docs only.
Database Impact
None.
Blast Radius
live-canary.yml— scheduled runs lose the weekly provider-matrix jobs; manual dispatch option removed;canary-reportno longer depends on itscripts/live-canary/run.sh—provider-matrixlane and zizmor scenarios removed; remaining lanes unchangedscripts/ci/ws12_workflow_contracts.py— weekly-cron markers unpinned (would otherwise fail CI after the workflow edit)scripts/ci/check-regression-promotions.py,test_run_live_qa.py— untouched, verified still passingRollback Plan
Revert this commit (
git revert) — all changes are additive-restore (job block, cron, scenario lines, fixtures, contract pins). The deleted fixtures are recoverable from git history if the zizmor lanes are ever reinstated.Review Follow-Through
None — reviewer judgment only on whether the remaining
zizmor_scanstrings inscripts/live-canary/emit_results_json.py/test_emit_results_json.py(log-parser sample fixtures, not live wiring) and.coderabbit.yaml(CodeRabbit's own scanner) should stay; both intentionally left.Review track: C (CI)