Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/scripts/check-workflow-size.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# GitHub does not START RUNS for a workflow file larger than 500 KB (512,000
# bytes), and it says nothing when it stops: schedule ticks vanish, dispatches
# sit "queued" forever with zero jobs, `issues`/`issue_comment` go quiet, and
# only PR-event runs keep working — those resolve the workflow from the PR's
# own branch, so an older, smaller copy runs and the workflow looks half-alive.
# qwen-autofix.yml crossed the line on 2026-08-19 and the autofix loop went
# dark for a day before anyone read the size.
#
# The gate below sits well under the real ceiling on purpose: a PR that trips
# it still has room to land the fix, instead of discovering the wall with no
# space left to move prose out.
set -uo pipefail

GITHUB_LIMIT_BYTES=512000
GATE_BYTES="${WORKFLOW_SIZE_GATE_BYTES:-470000}"
WARN_BYTES=$((GATE_BYTES - 25000))

status=0
shopt -s nullglob
for file in .github/workflows/*.yml .github/workflows/*.yaml; do
if ! size="$(wc -c <"${file}")"; then
echo "::error file=${file}::unable to read ${file}"
status=1
continue
fi
pct=$((size * 100 / GITHUB_LIMIT_BYTES))
if ((size > GATE_BYTES)); then
echo "::error file=${file}::${file} is ${size} bytes — ${pct}% of GitHub's ${GITHUB_LIMIT_BYTES}-byte start-runs limit, past this repo's ${GATE_BYTES}-byte gate. Move prose into a sibling .md and long steps into .github/scripts/; do not raise the gate."
status=1
elif ((size > WARN_BYTES)); then
echo "::warning file=${file}::${file} is ${size} bytes (${pct}% of GitHub's limit) — approaching the ${GATE_BYTES}-byte gate."
fi
done

if ((status == 0)); then
echo "✅ every workflow file is under the ${GATE_BYTES}-byte gate"
fi
exit "${status}"
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,15 @@ jobs:
echo "ci_profile=${profile}" >> "${GITHUB_OUTPUT}"
echo "Selected CI profile: ${profile}"

# Deliberately gated on skip_ci ONLY, not on the CI profile: a PR that
# touches nothing but .github/ classifies as `github_ci_only`, and that
# is exactly the PR most likely to push a workflow file past GitHub's
Comment on lines +292 to +294

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] This comment (and its twin in scripts/tests/workflow-size.test.js) claims a PR touching nothing but .github/ classifies as github_ci_only, but classify-profile.mjs defines GITHUB_CI_ONLY_FILES as a hardcoded 3-file allowlist (pr-safety-precheck.mjs, its test, and qwen-pr-safety-precheck.yml), and classify-profile.test.mjs pins classifyChangedFiles(['.github/workflows/ci.yml']) === 'full'. Probe against the real classifier at this commit: ci.yml only → full, qwen-autofix.yml only → full, all 3 allowlisted → github_ci_only. The step's actual gating (skip_ci only, runs on every profile) is correct and test-pinned — only the recorded rationale is wrong. Failure scenario: a maintainer trusting this comment re-gates the step onto the github_ci_only profile and silently removes the size gate from every workflow-editing PR — exactly the PRs that can trip the 500 KB limit. Reword to match the classifier: github_ci_only is a 3-file allowlist (one of which is a workflow file — the narrow case this gate additionally covers), while workflow-file PRs classify as full.

中文说明

这条注释(以及 scripts/tests/workflow-size.test.js 中的孪生注释)声称只改 .github/ 的 PR 会被分类为 github_ci_only,但 classify-profile.mjs 把 GITHUB_CI_ONLY_FILES 定义为硬编码的 3 文件白名单(pr-safety-precheck.mjs、其测试、qwen-pr-safety-precheck.yml),且 classify-profile.test.mjs 钉住了 classifyChangedFiles(['.github/workflows/ci.yml']) === 'full'。在本提交上对真实分类器探测:仅 ci.yml → full、仅 qwen-autofix.yml → full、三个白名单文件 → github_ci_only。该 step 的实际门控(仅 skip_ci、所有 profile 都跑)是正确且有测试钉住的——只是注释记录的理由错了。失败场景:维护者相信这条注释,把该 step 改为仅在 github_ci_only profile 下运行,就会静默地让所有改动 workflow 的 PR 失去大小门禁——而那恰恰是最可能触发 500 KB 限制的 PR。请按分类器实情改写:github_ci_only 是 3 文件白名单(其中之一是 workflow 文件——正是本门禁额外覆盖的窄场景),而改 workflow 的 PR 分类为 full。

— qwen3.8-max via Qwen Code /review (v0.21.11)

# 500 KB start-runs limit — the limit that silently killed the autofix
# loop for a day on 2026-08-19. Costs one `wc -c` per workflow file.
- name: 'Check workflow file size'
if: "${{ needs.classify_pr.outputs.skip_ci != 'true' }}"
run: '.github/scripts/check-workflow-size.sh'

- name: 'Docs-only CI'
if: "${{ needs.classify_pr.outputs.skip_ci != 'true' && steps.ci_profile.outputs.ci_profile == 'docs_only' }}"
run: 'echo "Docs-only change; full CI skipped."'
Expand Down
7,699 changes: 0 additions & 7,699 deletions .github/workflows/qwen-autofix-recovery.yml

This file was deleted.

2,151 changes: 2,151 additions & 0 deletions .github/workflows/qwen-autofix.md

Large diffs are not rendered by default.

1,416 changes: 90 additions & 1,326 deletions .github/workflows/qwen-autofix.yml

Large diffs are not rendered by default.

12 changes: 7 additions & 5 deletions scripts/tests/qwen-autofix-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ import { describe, expect, it } from 'vitest';
import { getWorkflowJob } from './workflow-helpers.js';

const workflow = readFileSync('.github/workflows/qwen-autofix.yml', 'utf8');
// Long-form rationale moved out of the YAML when the file approached
// GitHub's 500 KB start-runs limit; assertions that pin a REASON (rather
// than a code line) read it here.
const designDoc = readFileSync('.github/workflows/qwen-autofix.md', 'utf8');
const ciWorkflow = readFileSync('.github/workflows/ci.yml', 'utf8');
const releaseWorkflow = readFileSync('.github/workflows/release.yml', 'utf8');
const sandboxImageResolverScript = readFileSync(
Expand Down Expand Up @@ -3864,13 +3868,11 @@ describe('qwen-autofix workflow', () => {
// idle candidates hit `continue` before the TARGETS append, so they
// never contend for it; the real win is inspection budget + walk
// latency, and the comment says so.
expect(reviewScanJob).toContain(
'Idle PRs never reach the\n # 10-target budget',
);
expect(designDoc).toContain('Idle PRs never reach the\n10-target budget');
// The two scan-only signals updatedAt cannot see are named, not
// papered over by an absolute invariant.
expect(reviewScanJob).toContain('base conflict');
expect(reviewScanJob).toContain('still-red checks');
expect(designDoc).toContain('base conflict');
expect(designDoc).toContain('still-red checks');
Comment on lines +3874 to +3875

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The migrated assertion lost its scope: it previously matched only reviewScanJob (pinning the idle-backoff rationale where the comment lived); now expect(designDoc).toContain('base conflict') matches the whole 2152-line doc, where the string occurs three times — af-002 (md L192, 'base conflicts may drive code changes'), af-027 (L926, the intended referent), and af-066 (L1991) — the first and third as substrings of 'base conflicts'. Dropping the af-027 enumeration of the two scan-only signals keeps this green via the unrelated occurrences, so the documented invariant the test exists to catch can be deleted unflagged. (still-red checks occurs once and stays effectively pinned.) Scope the assertion to the section:

Suggested change
expect(designDoc).toContain('base conflict');
expect(designDoc).toContain('still-red checks');
const idleBackoff = designDoc.slice(
designDoc.indexOf('<a id="af-027">'),
designDoc.indexOf('<a id="af-029">'),
);
expect(idleBackoff).toContain('base conflict');
expect(idleBackoff).toContain('still-red checks');
中文说明

迁移后的断言丢失了作用域:原先只匹配 reviewScanJob(在注释原所处位置钉住 idle-backoff 理由);现在 expect(designDoc).toContain('base conflict') 匹配整份 2152 行的记录,而该字符串出现三次——af-002(md L192,'base conflicts may drive code changes')、af-027(L926,预期指称)、af-066(L1991)——首尾两处还是 'base conflicts' 的子串。删除 af-027 中关于两个 scan-only 信号的列举,断言仍会经由无关出现处保持绿色,该测试本要捕捉的已记录不变量可以被无声删除。(still-red checks 仅出现一次,实际仍被钉住。)请把断言限定到该章节:

— qwen3.8-max via Qwen Code /review (v0.21.11)

});

it('fails closed on busy-enumeration failure, keeps explicit dispatches, and signals the issue phase', () => {
Expand Down
81 changes: 81 additions & 0 deletions scripts/tests/workflow-size.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/

import { readFileSync, readdirSync } from 'node:fs';
import { join } from 'node:path';
import { describe, expect, it } from 'vitest';

// GitHub does not start runs for a workflow file over 500 KB (512,000 bytes)
// and reports nothing when it stops — see .github/scripts/check-workflow-size.sh
// and .github/workflows/qwen-autofix.md for the incident this encodes.
const GITHUB_LIMIT_BYTES = 512_000;
const WORKFLOW_DIR = '.github/workflows';
const gateScript = readFileSync(
'.github/scripts/check-workflow-size.sh',
'utf8',
);
const ciWorkflow = readFileSync(join(WORKFLOW_DIR, 'ci.yml'), 'utf8');

const gateBytes = Number(
gateScript.match(/GATE_BYTES="\$\{WORKFLOW_SIZE_GATE_BYTES:-(\d+)\}"/)?.[1],
);
Comment on lines +22 to +24

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] No test executes .github/scripts/check-workflow-size.sh — this suite regex-parses it as data, so its runtime behaviour (exit codes, the ::error/::warning branches, the scanned-file glob, the read-failure path) is untested. On the github_ci_only profile the vitest suite does not run at all, and that lane (which today reaches qwen-pr-safety-precheck.yml) is where this script is the ONLY size check. Concrete breakage: a glob typo (.github/workflow/*.yml) that shopt -s nullglob turns into zero files checked still exits 0 with a green '✅ every workflow file is under the … gate' message — a dead gate nothing catches. Probe at this commit confirms both arms are exercisable as-is: WORKFLOW_SIZE_GATE_BYTES=1 → ::error per file + exit 1; default env → exit 0. Add a spawn-based test for both arms.

中文说明

没有任何测试会执行 .github/scripts/check-workflow-size.sh——本套件只是用正则把它当作文本解析,因此其运行时行为(退出码、::error/::warning 分支、扫描的文件 glob、读取失败路径)未被测试。在 github_ci_only profile 下 vitest 套件根本不会运行,而该通道(目前会触及 qwen-pr-safety-precheck.yml)恰恰只有这个脚本作为唯一的大小检查。具体损坏方式:glob 拼写错误(.github/workflow/*.yml)在 shopt -s nullglob 下变成零文件受检,却仍以退出码 0 输出绿色的 '✅ every workflow file is under the … gate'——门禁失效且无人察觉。已在本提交上探测确认两个分支都可以直接测:WORKFLOW_SIZE_GATE_BYTES=1 → 每文件一条 ::error + 退出码 1;默认环境 → 退出码 0。建议补一个 spawn 脚本的测试覆盖两个分支。

— qwen3.8-max via Qwen Code /review (v0.21.11)


const workflowFiles = readdirSync(WORKFLOW_DIR)
.filter((name) => name.endsWith('.yml') || name.endsWith('.yaml'))
.map((name) => join(WORKFLOW_DIR, name));

describe('workflow file size', () => {
it('keeps the gate below GitHub 500 KB start-runs limit', () => {
expect(gateBytes).toBeGreaterThan(0);
expect(gateBytes).toBeLessThan(GITHUB_LIMIT_BYTES);
});

it.each(workflowFiles)('%s stays under the gate', (file) => {
const bytes = Buffer.byteLength(readFileSync(file));
expect(bytes).toBeLessThan(gateBytes);
});

it('runs the gate on every CI profile, not just full', () => {
// A .github-only PR classifies as `github_ci_only`; gating the check on the
// `full` profile would skip it for exactly the changes that can trip it.
Comment on lines +42 to +43

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] Same false classifier claim as the ci.yml comment discussed in this review: a .github-only PR does NOT classify as github_ci_only — that profile is a hardcoded 3-file allowlist (pr-safety-precheck.mjs, its test, qwen-pr-safety-precheck.yml), verified by probe against classify-profile.mjs at this commit. Please reword together with the ci.yml comment so the two stay consistent.

中文说明

与 ci.yml 中讨论过的注释是同一个错误的分类器断言:只改 .github/ 的 PR 并不会分类为 github_ci_only——该 profile 是硬编码的 3 文件白名单(pr-safety-precheck.mjs、其测试、qwen-pr-safety-precheck.yml),已在本提交上对 classify-profile.mjs 探测证实。请与 ci.yml 注释一并改写,保持一致。

— qwen3.8-max via Qwen Code /review (v0.21.11)

const step = ciWorkflow.match(
/- name: 'Check workflow file size'[\s\S]*?run: '(.+?)'/,
);
expect(step?.[1]).toBe('.github/scripts/check-workflow-size.sh');
expect(step?.[0]).toContain(
'if: "${{ needs.classify_pr.outputs.skip_ci != \'true\' }}"',
);
expect(step?.[0]).not.toContain('ci_profile');
});
});

describe('qwen-autofix.yml design-record pointers', () => {
const workflow = readFileSync(join(WORKFLOW_DIR, 'qwen-autofix.yml'), 'utf8');
const doc = readFileSync(join(WORKFLOW_DIR, 'qwen-autofix.md'), 'utf8');

const pointers = [...workflow.matchAll(/qwen-autofix\.md#(af-\d+)/g)].map(
(m) => m[1],
);
const anchors = [...doc.matchAll(/<a id="(af-\d+)"><\/a>/g)].map((m) => m[1]);

it('every pointer resolves to a section', () => {
expect(pointers.length).toBeGreaterThan(0);
expect(
[...new Set(pointers)].filter((id) => !anchors.includes(id)),
).toEqual([]);
});

it('every section is still pointed at from the workflow', () => {
expect(anchors.filter((id) => !pointers.includes(id))).toEqual([]);
});

it('lists every section in the contents table', () => {
const listed = [...doc.matchAll(/^- \[\d+\..*?\]\(#(af-\d+)\)$/gm)].map(
(m) => m[1],
);
expect(listed).toEqual(anchors);
});
});
Loading