fix(ci): stop triaging the autofix bot's own deferred-finding tracking issues (#9264) - #9271
Merged
Conversation
…g issues (#9264) Every PR that defers findings for the first time opens a tracking issue upserted by the autofix bot, and the issues trigger (opened/edited/reopened) ran a full triage agent on that bookkeeping issue per deferral — the authorize gate exempts the issues path as read-only, so nothing stopped it. Condition the triage job's issues clause on the creator not being the autofix bot (the same vars.AUTOFIX_BOT_LOGIN || 'qwen-code-dev-bot' identity qwen-autofix.yml upserts under), and route bot-created issues runs to a per-run concurrency group: GitHub evaluates concurrency before the job if, so a run left in the shared per-number group would still cancel an in-progress triage of the same issue before its own skip is evaluated. Pins: the issues-clause guard, the group routing, and the cross-workflow identity sync, all on the parsed document.
Collaborator
Collaborator
|
Thanks for the PR! Template looks good ✓ Problem: observed, not theoretical. The bot-opened bookkeeping issue #7167 ("Fleet Shepherd Dashboard", authored by Direction: aligned — pure CI hygiene: it stops an agent run and a runner being spent on an issue whose content is a deferral ledger, without touching any product surface. The autofix workflow itself is verified unaffected: it triggers on Size: not core paths — 20 production lines in the workflow (17+/3−) and 43 test lines. Well under every threshold. Approach: the scope is exactly right. The guard keys on the same Risk: no elevated risk signals — neither changed file matches the revert-correlated path patterns. Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题：真实存在，不是理论问题。机器人创建的记账 issue #7167（"Fleet Shepherd Dashboard"，作者 方向：对齐——纯 CI 卫生改动：避免把 agent 运行和 runner 浪费在内容为延期清单的 issue 上，不影响任何产品功能。autofix 工作流本身已验证不受影响：它只在 规模：非核心路径——工作流 20 行生产代码（17+/3−），测试 43 行。远低于所有阈值。 方案：范围恰到好处。守卫条件使用与 风险：无升级风险信号——两个改动文件均不命中与 revert 相关的路径模式。 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
Code reviewRe-run pass: the workflow YAML is byte-for-byte what the first pass reviewed — the two commits since then (
Testing evidence — this PR's own CI (the triage gate never runs PR code)CI is settled on the reviewed commit — 15 checks green, the rest skipped by design for a CI-config-plus-test diff (macOS/Windows suites, sandbox integration, fork-only jobs), nothing red, nothing pending. Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 What the pins can and cannot prove: they are parsed-YAML assertions, which is the right oracle for a workflow change — they prove the guard and the routing are present, correctly connected, in the exact expressions GitHub will evaluate. The live event-layer behavior (bot issue opened/edited → no triage run) is only observable on the next bot-created issue event after merge; neither sandboxed lane ( Real-scenario testing: N/A — unattended CI run, and nothing user-visible to drive (CI-config change; live behavior is exercised only by the isolated 中文说明代码审查重跑复核：工作流 YAML 与首轮审查的内容逐字节一致——此后的两个提交（
测试证据——来自该 PR 自己的 CI（triage 门禁从不运行 PR 代码）CI 已在被审查提交上全部结束：15 项绿色，其余按 CI 配置+测试类 diff 的设计跳过（macOS/Windows 套件、沙箱集成、仅 fork 触发的 job），无失败、无进行中。承载三个加固 pin 的 （CI 表格见上方英文部分） pin 能证明与不能证明的：它们是对解析后 YAML 的断言，这对工作流改动是正确的 oracle——证明守卫与路由确实存在、连接正确，且正是 GitHub 将求值的表达式。事件层的实际行为（机器人 issue 创建/编辑 → 无 triage 运行）只能在合并后下一次机器人 issue 事件时观察到；任何沙箱通道（ 真实场景测试：N/A——无人值守 CI 运行，且无用户可见行为可驱动（CI 配置改动；真实行为仅由隔离的 — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
|
Confidence: 5/5 — the problem is observed (bot-opened bookkeeping issues drew repeated triage runs), the fix is the minimum that works, every path that should stay open was checked to stay open, and the follow-up commits hardened the tests exactly where the first pass had one soft spot. Stepping back: my independent proposal for "stop triaging the bot's ledger issues" was exactly this shape — a creator guard on the issues clause, keyed on the shared autofix identity — with one addition the PR got right that I'd have missed on first pass: the concurrency routing. Guarding only the job The re-run delta is the right kind of delta: two test-only commits that pin the connectors around the guard and upgrade the identity-sync check from a raw-substring search to a parsed-YAML assertion on Verdict: approve, attested now — CI is fully settled on the reviewed commit, so no deferred-approval marker this round. (The earlier approval on 中文说明置信度：5/5 —— 问题是观测到的（机器人创建的记账 issue 反复触发 triage 运行），修复是最小可行方案，所有应保持畅通的路径都已核实未被误伤，后续提交恰好加固了首轮唯一偏软的地方。 回头看：我对"停止 triage 机器人台账 issue"的独立方案与此形状完全一致——在 issues 分支上加创建者守卫、复用 autofix 的共享身份——但 PR 还做对了一处我第一遍会漏掉的改动：并发路由。只守 job 重跑增量是正确的增量：两个仅改测试的提交，为守卫周边补上连接符 pin，并把身份同步检查从原始子串查找升级为对 结论：批准，且现在即可背书——CI 已在被审查提交上全部结束，本轮无需延迟批准标记。（早先在 — Qwen Code · qwen3.8-max Reviewed at |
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
previously approved these changes
Aug 16, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
— qwen3.8-max via Qwen Code /review (v0.21.12)
Collaborator
Author
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
— qwen3.8-max via Qwen Code /review (v0.21.12)
Collaborator
Author
|
Closeout update from resolve-pr-comments automation: Changed: hardened the parsed YAML pins for the autofix bot identity and per-run concurrency routing. Verified: focused #9264 workflow tests passed; git diff --check passed. Pending: post-push CI/review are running. |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
— qwen3.8-max via Qwen Code /review (v0.21.12)
| it('keeps the guard identity in sync with the autofix workflow', () => { | ||
| // qwen-autofix.yml defines AUTOFIX_BOT as the same variable-with-fallback | ||
| // (inside a bare `${{ }}`, so without the expression's parentheses). | ||
| expect(botIdentity).toContain(botIdentityCore); |
Collaborator
There was a problem hiding this comment.
[Suggestion] Tautological assertion: botIdentity is defined two lines above as (${botIdentityCore}), and "(" + s + ")" contains s for every possible string — and this line performs no file I/O, so no mutation of either workflow file can falsify it. The test title's guarantee is carried by the adjacent toBe against parsed qwen-autofix.yml, not by this line. — Failure scenario: the assertion reads as verification but is unfalsifiable; a future refactor that weakens or breaks the adjacent load-bearing toBe leaves this line green, still appearing to vouch for the sync — this suite's own doctrine is that a control that cannot fail proves nothing about the assertions it vouches for. Deleting the line loses nothing: the toBe plus the identity embedded in the first two tests carry the guarantee.
Suggested change
| expect(botIdentity).toContain(botIdentityCore); |
— qwen3.8-max via Qwen Code /review (v0.21.12)
| // same issue even though its own job skips. | ||
| const doc = parse(workflow); | ||
| expect(flat(doc.jobs.triage.concurrency.group)).toContain( | ||
| `!startsWith(github.event.comment.body, '@qwen-code /triage'))) || (github.event_name == 'issues' && github.event.issue.user.login == ${botIdentity}) ) && format('{0}-run-{1}', github.workflow, github.run_id)`, |
Collaborator
There was a problem hiding this comment.
[Suggestion] This pin stops one clause short: it anchors the bot-side routing to the per-run group but not the complement — that every other event still falls through to the shared per-number group format('{0}-{1}', …). Grep confirms this pin is the only {0}- format string in scripts/tests, so nothing covers the fallback clause. — Failure scenario: demonstrated surviving mutant — changing the workflow's shared-group fallback from format('{0}-{1}', …) to format('{0}-run-{1}', …) routes every triage run into a unique per-run group, silently disabling the cancel-in-progress supersede the workflow relies on for issues/dispatch triggers (duplicate concurrent full agent runs on the same issue, no cancellation) — and all 134 tests still pass. Extending the pin by one clause kills the mutant. Witness (probe flip): baseline 134/134 pass; the fallback mutant survives 134/134; the extended pin below fails against the mutant (expected …format('{0}-{1}'…, received …format('{0}-run-{1}'…) and passes 134/134 against the restored tree — the suggested fix is verified sound, not hypothesized.
Suggested change
| `!startsWith(github.event.comment.body, '@qwen-code /triage'))) || (github.event_name == 'issues' && github.event.issue.user.login == ${botIdentity}) ) && format('{0}-run-{1}', github.workflow, github.run_id)`, | |
| `!startsWith(github.event.comment.body, '@qwen-code /triage'))) || (github.event_name == 'issues' && github.event.issue.user.login == ${botIdentity}) ) && format('{0}-run-{1}', github.workflow, github.run_id) || format('{0}-{1}', github.workflow, github.event.issue.number || github.event.pull_request.number || github.event.inputs.number)`, |
— qwen3.8-max via Qwen Code /review (v0.21.12)
wenshao
approved these changes
Aug 16, 2026
Collaborator
|
@qwen-code /triage |
wenshao
enabled auto-merge
August 16, 2026 16:55
Collaborator
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 56 passed · 0 failed · 56 total 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：56 通过 · 0 失败 · 56 总计 Verification reportPR 9271 Deep Verification —
|
| cell | payload | arm | if (runs?) |
concurrency group | verdict |
|---|---|---|---|---|---|
| C1 central | issues/opened by qwen-code-dev-bot |
base | true (bug: full triage runs) | Triage-777 (shared, cancel-in-progress=true) |
base fails as predicted |
| C1 central | same | head | false (skipped) | Triage-run-42 (per-run) |
head fixed |
| C3/C4 | issues/edited / reopened by bot |
base → head | true → false | per-number → per-run | fixed (upserts edit the ledger) |
| C5 | bot, vars.AUTOFIX_BOT_LOGIN set (vars path, not fallback) |
head | false | per-run | guard works through the knob |
| C10 | bot, vars.AUTOFIX_BOT_LOGIN='' |
head | false | per-run | empty var falls back to literal |
| C7 | login Qwen-Code-Dev-Bot (case variant) |
head | false | per-run | == is case-insensitive |
| C2 | issues/opened by human |
both | true | per-number | unchanged |
| C11 | human edits a bot-created issue | head | false | per-run | guard keys on creator, not actor (verified boundary) |
| C11b | bot edits a human issue | both | true | per-number | symmetric boundary, arm-identical |
| C8/C9 | creator ghost / login null |
both | true | per-number | fail-open, safe direction |
| C12–C16 | issue_comment magic/unmagic/unauthorized/bot-issue/closed |
both | identical on both arms | identical | unchanged (incl. explicit @qwen-code /triage on a bot issue still runs — human command path intact) |
| C17/C18 | workflow_dispatch number / tmux lane |
both | identical | identical | unchanged |
| C19–C21 | pull_request_target open/draft/unauthorized |
both | identical | identical | unchanged |
132/132 cell×arm×expression expectations hold; the base arm reproduces the waste (run + shared group with cancel-in-progress=true) and the head arm flips exactly the bot-issues rows. The identity the guard keys on is real: qwen-autofix.yml routes every GitHub write through CI_DEV_BOT_PAT and asserts the PAT authenticates as AUTOFIX_BOT in its actor-checking jobs; upsert-deferred-issue.sh creates the tracking issue via gh api repos/…/issues with that PAT and — verified — with -f title/-f body only (no labels, no assignees), so the autofix workflow's own issues: [labeled, assigned] trigger never fires on the ledger, as the PR description claims.
Test pins — vacuity and mutation matrix
Witnesses: 03-mixed-cell-head-tests-vs-base-source.png, 02-mutation-matrix-all-mutants-killed.png.
- Head gate:
scripts/tests/qwen-triage-workflow.test.js134/134 green (author's claimed count reproduced). Base arm: 131/131 green. - Mixed cell (head tests vs base source): exactly 2 red — the guard pin and the routing pin, each failing its intended assertion (
expected '<parsed base expression>' to contain '<guard text>'). The third pin (identity sync) is green on base by design: its target fileqwen-autofix.ymlis untouched by this PR; the pin exists to catch a future rename, and mutation M5 below proves it is live. - Mutation matrix (one-point mutants of head; suite = the 134-test file):
| mutant | result | red pins | designed target | match |
|---|---|---|---|---|
M1 if-guard != → == |
1F/133P | PIN1 | PIN1 | ✅ |
| M2′ guard present only as a YAML comment | 1F/133P | PIN1 | PIN1 | ✅ (+ control: guard text does occur in raw file, grep -c = 1 — a substring pin would have been fooled, the parse-based pin was not) |
| M3 group: delete issues disjunct | 1F/133P | PIN2 | PIN2 | ✅ |
M4 group: == → != |
1F/133P | PIN2 | PIN2 | ✅ |
| M5 autofix env fallback renamed | 1F/133P | PIN3 | PIN3 | ✅ |
| M6 triage-side fallback renamed (2 sites) | 2F/132P | PIN1+PIN2 | PIN1+PIN2 (PIN3 stays green — sync coverage is split across the shared constant exactly as the pins intend) | ✅ |
| M7 semantics-preserving re-wrap of both hunks | 134/134 green | — | re-wrap must not defeat pins | ✅ (and the re-wrapped file re-passes the full 132-assertion A/B matrix — behavior identical) |
No survivors. Positive controls: unmutated head 134/134, base 131/131. The author's "mutation-verified both ways" claim is reproduced and extended.
Full scripts-suite gate
Witness: 04-full-suite-gate-head-vs-base.png. Raw logs: logs/full-suite-{head,base}.log.
| arm | tests passed | tests failed | failing file |
|---|---|---|---|
| head | 1109 | 0 | install-script.test.js fails to load (top-level guard throws) |
| base | 1106 | 0 | same file, same reason |
Δ = +3 passing, exactly the new pins, and zero new failures. The load failure is environmental and A/A-identical: this container ships unzip but no zip binary, so the archive guard aborts collection of that file on both arms. The author's cited "1201 pass; 3 pre-existing failures" run is consistent with this: 1106 + the ~95 tests of install-script.test.js (runnable where zip exists) = 1201, and their three named failures (audio-capture packaging; two unwritable-directory tests that cannot fail as root) are environment shapes this container does not share — this container runs as root and lacks zip.
Lint gates
- actionlint 1.7.12 (repo wrapper
node scripts/lint.js --setup+--actionlint; repo-wide run at head exited 0): headqwen-triage.ymlclean, base snapshot clean, re-wrapped snapshot clean. Live-check: dropping one)from the PR's new guard made actionlint reportunexpected end of input while parsing closing ')' of nested expression … expecting ")"(exit 1); the restored file passes — the gate is proven live. - shellcheck/bash -n: the diff touches no
run:blocks — not applicable. - yamllint: could not be installed (
pip3: Permission deniedin this container). YAML validity is still covered twice over: both files parse with theyamlparser in every harness, and actionlint parses them. See Not covered.
Findings
No blocking findings. One observation, non-blocking:
- Observation (non-blocking, PR-acknowledged): guard misses under repository-variable drift. Cell C6 measures the semantics: if
vars.AUTOFIX_BOT_LOGINis set to a login different from the account the PAT authenticates as, the head guard evaluatestruefor the real bot and the waste silently returns. This is the PR's own stated risk note, and it is bounded: both workflows read the same variable, the file-level fallback literals are pinned against each other (M5/M6), andqwen-autofix.ymlloudly assertsCI_DEV_BOT_PAT authenticates as … expected AUTOFIX_BOTin its actor-checked jobs. The one path without such an assertion is the upsert script itself (a wrong variable would first show up as duplicated tracking issues, since itscreator=lookup would miss) — pre-existing behavior, not changed by this PR.
No injection-style instructions were present in the PR metadata; none recorded.
Not covered
- Per-commit attribution: the checkout is depth 2 (
git rev-parse --is-shallow-repository= true); the metadata lists 3 commits but only the merge commit and the two tips are reachable locally, so the aggregateHEAD^1..HEADdiff was verified and per-commit behavior was not. - yamllint — install blocked by container permissions (
pip3: Permission denied); compensated as described above. - Live GitHub runtime behavior (real event delivery, actual concurrency cancellation) — no token and no API in this sandbox; behavior was proven by evaluating the real parsed expressions against a documented-semantics evaluator plus actionlint's expression parser, not by observing a live run. The evaluator is a faithful subset (calibrated), not the runner's own C# implementation — residual risk is small but named.
- "GitHub evaluates concurrency before the job
if" — documented platform behavior the existing workflow already relies on; not independently provable from inside the sandbox. - ESLint/typecheck: the changed files are YAML and a
.jstest; repo CI covers ESLint andnpm run typechecktargets package TypeScript, so neither was re-run here. - Rest of autofix: review backlog from #9189 (pin strength, comment accuracy, triage trigger) #9264 (35+ items) — explicitly out of scope by the PR description.
- The tmux lane and other jobs of
qwen-triage.ymlwere inspected forissues-event paths (none besides the triage job) but not executed.
Methodology
Environment: the CI verify container (node:22-bookworm class, running as root), merge-ref checkout at depth 2; npm ci + npm run build pre-done. The A/B drove the actual parsed YAML expressions — js-yaml-family yaml parse of jobs.triage.{if, concurrency.group, concurrency['cancel-in-progress']} from git show HEAD^1:… (base snapshot) and the head file — through a strict subset evaluator of GitHub Actions expression semantics (harness/expr-eval.mjs) whose 20 calibration cases reproduce the documented operator examples before any matrix cell is trusted; 22 event payloads covered every trigger family plus boundary shapes (case variants, null/ghost creators, vars unset/set/empty/drifted, creator-vs-actor splits). Vacuity used scratch git worktrees (tmp/base-tree at HEAD^1, tmp/head-mut at HEAD^2, both removed afterwards) with the root node_modules shared — clean because the PR touches no lockfile. actionlint 1.7.12 and shellcheck were installed via the repo's pinned-hash wrapper; yamllint could not be. Raw logs: logs/ (matrix result JSON, per-mutation vitest logs, full-suite logs, mixed-cell log, assertion detail); harnesses rerunnable from harness/. Evidence images: evidence/01…05*.png.
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
qwen-code-ci-bot
approved these changes
Aug 16, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
Collaborator
|
Released in v0.21.13. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Implements item 1 of #9264 (the one cross-workflow effect that could not land in #9189): every PR that defers findings for the first time opens a tracking issue upserted by the autofix bot, and
qwen-triage.yml'sissues: [opened, edited, reopened]trigger ran a full triage agent on that bookkeeping issue on every deferral — the authorize gate deliberately exempts the issues path ("triage is read-only"), so nothing stopped it.Two changes, both keyed on the same identity
qwen-autofix.ymlupserts under (vars.AUTOFIX_BOT_LOGIN || 'qwen-code-dev-bot'):issuesclause now requiresgithub.event.issue.user.login != (vars.AUTOFIX_BOT_LOGIN || 'qwen-code-dev-bot'). Human-opened issues triage exactly as before.if, so a bot bookkeeping run left in the shared per-number group would still cancel an in-progress triage of the same issue before its own skip is evaluated.Why it's needed
Each deferral spawned a full triage-agent run against its own bookkeeping issue — model budget and a runner spent on an issue whose content is a deferral ledger, with a triage verdict nobody acts on. The autofix workflow itself is unaffected (it triggers on
issues: [labeled, assigned], which the unlabeled/unassigned tracking issues never fire).Reviewer Test Plan
How to verify
Three new pins in
scripts/tests/qwen-triage-workflow.test.js, asserted on the parsed document (a commented-out guard cannot satisfy them), with whitespace normalized so a re-wrap cannot defeat them:qwen-autofix.yml'sAUTOFIX_BOTdefinition.npx vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/qwen-triage-workflow.test.js # 134 passMutation-verified both ways: removing the issues-clause guard turns the first pin red; replacing the group clause turns the second red.
Evidence (Before & After)
Before: bot-opened tracking issues (e.g. the ones this very backlog series is filed under) match
issues: [opened]and run the triage agent. After: the job skips them at theif, and their runs cannot cancel an authorized triage of the same number. Observable on the next first-deferral of any PR.Tested on
Environment (optional)
Full scripts suite: 1201 pass; the 3 failures are pre-existing on clean main (stash-baselined: install-script audio-capture packaging, and the two unwritable-directory tests that cannot fail for root).
Risk & Scope
vars.AUTOFIX_BOT_LOGINon the autofix side without updating this guard, the waste silently returns — the identity-sync pin exists to catch the rename on either side.Linked Issues
Part of #9264