fix(ci): clone qwen-autofix into a recovery workflow entity - #9482
Merged
Conversation
The Actions backend wedged the original qwen-autofix.yml workflow entity on 2026-08-19: runs stick "queued" with zero jobs and cannot be cancelled or deleted via API, schedule ticks stopped being created, and event- triggered runs are dropped. Same-repo control workflows run normally, so the failure is bound to that one workflow entity. A byte-identical copy under a new path registers as a fresh entity and resumes the loop; the original file stays untouched so the revert is deleting this one file.
Collaborator
wenshao
enabled auto-merge
August 19, 2026 14:25
Collaborator
|
Thanks for the fast incident response! Template looks good ✓ Problem: observed and independently verified, not theoretical. I checked the Actions API directly: the original Direction: aligned. The autonomous-fix loop is core CI infrastructure for this repo, cloning the file to force-register a fresh workflow entity is the standard self-serve remedy for a wedged entity, and leaving the original untouched makes revert trivial (delete one file once GitHub Support heals it). CI infrastructure, so no CHANGELOG signal applies. Size: one file, +7,699 lines — but 7,689 of those are a verbatim copy of the existing, battle-tested Approach: scope is exactly right, and I verified the central claim: diffing the new file against Risk: no elevated risk signals — CI config only, none of the revert-correlated paths are touched. The documented tradeoffs (fork-bridge dispatches target the original path and stay harmless no-ops while it's wedged; occasional duplicate dispatches burn one build before per-PR concurrency cancels them) are inherent to temporarily running two entities, and the delete-on-recovery plan is written into the file itself. Moving on to code review. 🔍 中文说明感谢快速响应事故！ 模板完整 ✓ 问题： 已观测并由我独立核实，不是理论问题。我直接查了 Actions API：原 方向： 对齐。自治修复流水线是本仓库的核心 CI 基础设施，克隆文件强制注册新 workflow 实体是对卡死实体的标准自救手段，且原文件保持不动使还原很简单（GitHub Support 修复后删除这一个文件即可）。属于 CI 基础设施，不涉及 CHANGELOG 信号。 规模： 单文件 +7,699 行——其中 7,689 行是现有、久经考验的 方案： 范围恰当，且我核实了核心声明：新文件与 main 上的 风险： 无升级风险信号——仅 CI 配置，未触及任何与 revert 相关的高风险路径。已记录的权衡（fork-bridge 的 dispatch 指向原路径，卡死期间是无害空操作；偶发重复 dispatch 会浪费一次 build 后被 per-PR 并发组取消）是临时双实体共存的固有代价，恢复后删除的计划也已写进文件本身。 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
Code reviewMy independent proposal for "resume the autofix loop while its workflow entity is wedged" was exactly what this PR does: add a verbatim copy under a new path so GitHub registers a fresh entity, keep the display name so name-keyed consumers keep matching, leave the original file intact for a one-file revert. Comparing against the diff, the PR matches that proposal; the only addition is a 10-line comment header documenting the incident and the delete-on-recovery criteria, which I'd have added too. Verification details:
No blockers, no convention issues. Test evidence (PR's own CI, via API — unattended run, no PR code executed)At fetch time: no failures on Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 Sandboxed lanes: the one thing no pre-merge check can settle is whether the Actions backend will actually schedule the new entity — that's GitHub-side behavior, observable only post-merge (the PR's own oracle: next cron tick fires, or a manual dispatch gets jobs within a minute, in contrast to the original's zero-job queue). Neither Real-scenario testing: N/A — unattended CI run; nothing user-visible in a workflows-file clone. 中文说明代码审查面对"在 workflow 实体卡死期间恢复 autofix 流水线"这个问题，我的独立方案与本 PR 完全一致：在新路径下放一份逐字拷贝让 GitHub 注册新实体，保留显示名让按名字匹配的消费方继续生效，原文件保持不动以便单文件还原。对比 diff，PR 与我的方案一致；唯一的额外内容是 10 行注释头，记录事故背景与恢复后的删除标准——我也会这么写。 核实细节：
无阻塞项，无规范问题。 测试证据（PR 自己的 CI，经 API 读取——无人值守运行，未执行任何 PR 代码）读取时点： （CI 表格见上方英文部分的标记区域） 沙箱验证通道： 所有 pre-merge 检查都无法验证的唯一事项，是 Actions 后端是否真的会调度新实体——那是 GitHub 侧行为，只能在合并后观测（即 PR 自己的验证标准：下一次 cron tick 触发，或手动 dispatch 在一分钟内拿到 job，与原实体 0-job 排队形成对照）。 真实场景测试： N/A——无人值守 CI 运行；workflows 文件克隆没有任何用户可见行为。 — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
|
Confidence: 5/5 — verified incident, minimal diff, zero new logic; would merge without hesitation. Stepping back: this is exactly what an incident-response PR should look like. I verified the wedged entity myself through the Actions API rather than taking the description on faith — zero schedule ticks today and four zero-job queued dispatches, the freshest 32 minutes before this PR was filed — so the problem is real and ongoing. The fix matches what I would have done independently: verbatim clone under a new path, display name preserved so the finalize filter and in-file exclusions keep matching, original untouched so revert is deleting one file. I confirmed the clone differs from the proven production workflow by nothing but a 10-line comment header, which means there is no new behavior to get wrong. The two factual nits from earlier stages (the description's "byte-identical" wording, and review-event runs actually still completing) don't change the justification — the schedule + dispatch wedge alone is enough. In six months this file either doesn't exist anymore (the intended outcome) or its header comment tells whoever finds it exactly why it's there and when to delete it. That's the right shape for a temporary recovery artifact. Approval is deferred until CI lands green on 中文说明置信度：5/5 —— 已核实的事故、最小 diff、零新增逻辑；毫不犹豫地合并。 退一步看：这正是事故响应 PR 该有的样子。我没有轻信描述，而是自己通过 Actions API 核实了卡死的实体——今天零 schedule tick、4 个 0-job 的排队 dispatch，最新一个就在本 PR 提交前 32 分钟——所以问题真实且仍在持续。修复方案与我独立想到的做法一致：在新路径下逐字克隆，保留显示名以让 finalize 过滤器与文件内排除规则继续匹配，原文件不动使还原只需删除一个文件。我确认了克隆体与已被验证的线上 workflow 之间除了 10 行注释头外别无差异，这意味着不存在任何可能出错的新行为。前面提到的两处事实性小问题（描述中"字节级一致"的措辞，以及 review 事件 run 实际仍在正常完成）不改变结论——单是 schedule + dispatch 卡死就足以支撑本次变更。 六个月后，这个文件要么已不存在（预期结局），要么其头部注释会告诉发现它的人它为何在此、何时该删。这是临时恢复工件该有的形态。 审批将推迟到 CI 在 — Qwen Code · qwen3.8-max Reviewed at |
yiliang114
approved these changes
Aug 19, 2026
yiliang114
left a comment
yiliang114
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed. This is a recovery clone of qwen-autofix.yml → qwen-autofix-recovery.yml.
Verified it is a faithful byte-identical copy: diffing the recovery file against the original (excluding the 10-line recovery-comment header) yields zero content differences — the only addition is the recovery comment, so no workflow logic changed.
The workaround is sound: a new path registers as a NEW workflow entity on GitHub's side (escaping the wedged original), and keeping the same name: 'Qwen Autofix' preserves the name-keyed consumers (triage-finalize's workflow_run filter, in-file check-run exclusions). The explicit "DELETE THIS FILE once the original recovers" note covers the temporary-duplication maintenance risk.
0 unresolved threads.
yiliang114
approved these changes
Aug 19, 2026
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
doudouOUC
approved these changes
Aug 19, 2026
doudouOUC
left a comment
doudouOUC
left a comment
Collaborator
There was a problem hiding this comment.
Independent review — verified against HEAD 5b537971
Verdict: APPROVE. I diffed the new file against main's qwen-autofix.yml line by line and traced every consumer of the workflow's identity. This is a clean incident-response clone.
Content verification
The only delta versus main's qwen-autofix.yml is the 10-line RECOVERY CLONE header comment (lines 3–12); every other one of the 7,699 lines is identical. So "byte-identical" holds for everything behavioral — the added comment is the sole deviation, and it is the right deviation, since it documents the deletion criterion inline.
Identity-keyed consumers keep matching
The clone's entire safety case rests on keeping name: 'Qwen Autofix', and both consumer classes are indeed name-keyed, not path-keyed:
qwen-triage-finalize.yml'sworkflow_runfilter lists'Qwen Autofix'by display name — the clone matches.- Every check-run exclusion inside the file keys on
workflowName == "Qwen Autofix"(I count ten such jq filters), never on the workflow file path — the clone's runs stay excluded exactly like the original's.
No other path self-reference exists: the only qwen-autofix.yml string inside the file is the busy-detection's gh run list --workflow qwen-autofix.yml, which deliberately enumerates the ORIGINAL entity — that is the intended shape while the original is wedged.
The known tradeoffs are real but contained
- Busy-detection can't see the clone's own runs, so a PR whose review-address leg is live under the clone reads as idle on the next tick and may be re-dispatched. The per-PR concurrency groups cancel the duplicate; the cost is one build-cli per occurrence, exactly as the PR body states.
- Concurrency groups are repo-global by name, which works in this PR's favor: both entities share groups like
qwen-autofix-route-cronandqwen-autofix-route-pr-{n}, so if the original entity heals while the clone still exists, latest-wins suppression dedupes their overlapping cron/event ticks instead of letting both loops run uncoordinated. The window of genuine duplication is limited tocancel-in-progress: falsegroups (issue phase), and the PR commits to deleting this file once the original schedules normally again — that deletion is the real fix, and the header comment records it. - Fork-bridge/fleet dispatches still target the dead original (harmless no-ops); the 10-minute cron and the event triggers on the clone are the recovery path. Acceptable for the incident window.
CI
Green — 45 checks completed (SUCCESS/SKIPPED), only the bot's own review-pr still in flight; actionlint accepted the file. Secrets are repo-scoped, so the clone inherits them with no extra setup.
Ship it, and track the deletion follow-up so the clone doesn't outlive the incident.
Collaborator
|
Released in v0.21.15. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Adds a byte-identical copy of the autonomous-fix workflow under a new path, so GitHub registers it as a brand-new workflow entity. The display name stays the same so name-keyed consumers (the triage finalize filter and the in-file check-run exclusions) keep matching. The original workflow file is untouched, so reverting is deleting this one file once the backend heals.
Why it's needed
Since ~01:00 UTC 2026-08-19 the Actions backend has wedged the original workflow entity: runs stick in "queued" with zero jobs and cannot be cancelled (HTTP 500) or deleted (HTTP 403); schedule ticks stopped being created entirely after ~02:00; event-triggered runs (takeover labels, reviews) are dropped; even a manual dispatch never gets jobs. Same-repo control workflows on identical cron cadences run normally, so the failure is bound to that one workflow entity, not the repo. The autonomous-fix loop (8 open takeover PRs and the issue scan) has been dead ~12 hours; a fresh entity is the only self-serve way to resume it while GitHub Support investigates.
Reviewer Test Plan
How to verify
After merge, confirm the new workflow entity appears in Actions and either its next cron tick fires or a manual dispatch gets jobs within a minute — in contrast to the original entity where new runs stay at zero jobs.
Evidence (Before & After)
N/A — CI infrastructure change.
Tested on
Risk & Scope
Linked Issues
None — incident response.
中文说明
这个 PR 做了什么
在同一个仓库里为自治修复工作流新增了一份字节级一致的副本（新路径），让 GitHub 把它注册为一个全新的 workflow 实体。显示名保持不变，以便按名字匹配的消费方（triage finalize 的 workflow_run 过滤器、文件内的 check-run 排除规则）继续生效。原 workflow 文件未动，后端恢复后删掉这一个文件即可还原。
为什么需要
从 2026-08-19 约 01:00 UTC 起，Actions 后端把原 workflow 实体卡死了：run 永远停在 queued、0 jobs，API 取消返回 500、删除返回 403；约 02:00 后 schedule tick 完全不再创建 run；事件触发的 run（takeover label、review）被丢弃；手动 dispatch 也不分配 job。同仓库相同 cron 频率的对照 workflow 全部正常，说明故障绑定在这一个 workflow 实体上而非仓库层面。自治修复流水线（8 个开着 takeover label 的 PR 和 issue 扫描）已停摆约 12 小时；新实体是在等 GitHub Support 处理期间唯一能自救恢复的方式。
验证方式
合并后确认 Actions 里出现新 workflow 实体，其下一次 cron tick 触发、或手动 dispatch 在一分钟内拿到 job——与原实体（新 run 始终 0 jobs）形成对照。
风险与范围