test(live-canary): promote QA 9/10 Slack canaries into the cron rotation - #5949
Conversation
The qa-9 and qa-10 shards were dispatch_only (and their probe cases default_enabled=False) while expected-red pre-fix, so they could not page the 3-hourly cron. Both fix PRs are now merged and live-verified green (9/9 QA 10, 4/4 QA 9), so the shards join the scheduled rotation and the cases join bare local default runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🔎 IronLoop Review StatusHead: Current reviewers:
Reviewer summaries
Recent activity
Available commands
Run metadataAdmission: webhook accepted the request and IronLoop persisted reviewer state before this projection. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughSummary by CodeRabbit
WalkthroughQA 9 and QA 10 cases are promoted into default runs. Their live-canary shards no longer use ChangesLive QA promotion
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
scope: ci
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request promotes the QA 9 and QA 10 families of Slack tool-correctness and delivery-routing live QA test cases to run by default. This is achieved by removing the default_enabled=False attribute from their respective CaseSpec definitions in run_live_qa.py. Corresponding unit tests in test_run_live_qa.py have been updated to assert that these cases are now enabled by default, and a new assertion has been added to verify that no live QA shard in the workflow matrix is marked as dispatch_only anymore. I have no feedback to provide on these changes.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![ironloopai[bot]](https://avatars.githubusercontent.com/in/4116490?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ IronLoop Review: reviewer
Review at a glance
| Verdict | Blocking | Notes | Inline | Head |
|---|---|---|---|---|
| ✅ Approved | 0 | 0 | 0 | 94682a2a888f |
Head: 94682a2a888fc626f791fa6ed4e7f3819bacaacb
Next: No reviewer action needed.
Run details
Status: Current
Needs human: no
Needs validation: no
Summary
No actionable issues found in the QA 9/10 live-canary promotion. The workflow matrix, case default metadata, and matching unit-test expectations are consistent.
Findings
None.
Developer follow-up
After fixing this feedback:
- Push the fix to this PR branch.
- Re-run this reviewer with
@ironloopai review --agent reviewerif you only changed this reviewer's findings. - Re-run all reviewers with
@ironloopai reviewwhen the fix may affect multiple areas. - Use
@ironloopai statusto check queued/running/completed/failed/superseded state while reviewers run.
This branch was successfully deployed
1 participant
Merge ordering
This PR was staged to MERGE LAST — only after #5899 (the QA 9/10 canary suite) and #5898 (the Slack delivery-routing + tool-surface fixes) were both on main, because on pre-fix servers these shards are red and would page the 3-hourly cron. Both merged today, so that precondition is now satisfied. It was originally based on
canary/automation-probes(so #5899's merge would retarget it tomain), but #5899 squash-merged and its branch was deleted before this PR opened, so it targetsmaindirectly with the same one-commit diff.What
Promotes the QA 9 (automation delivery) and QA 10 (Slack tool correctness) canary shards from dispatch-only into the scheduled rotation, now that the fixes they guard are merged and live-verified green (9/9 QA 10, 4/4 QA 9):
.github/workflows/live-canary.yml: dropdispatch_only: truefrom the qa-9 and qa-10 matrix entries; their comments now state they are in the 3-hourly cron rotation and which failure classes they guard. The resolve-step dispatch_only guard itself stays for any future expected-red shard.scripts/reborn_webui_v2_live_qa/run_live_qa.py: dropdefault_enabled=Falsefrom qa_9b/qa_9c/qa_9d and all nine qa_10* CaseSpec entries, so they run in bare local default runs too.scripts/reborn_webui_v2_live_qa/test_run_live_qa.py: flip the promoted cases'default_enabledpins to True, and pin in the workflow shard-parity test that no matrix shard is dispatch_only (every shard runs on schedule / cases=all) while the resolve-step guard remains.No probe logic, assertions, or helpers change.
Out of scope
Fixture cleanup (deleting seeded messages post-assertion) and realistic fixture envelopes ride in a separate follow-up.
🤖 Generated with Claude Code