Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,12 @@ jobs:
# Kill-switch: MAINTAINER_ECS_RUNNER_DISABLED.
runs-on: '${{ (github.repository == ''QwenLM/qwen-code'' && vars.MAINTAINER_ECS_RUNNER_DISABLED != ''true'') && fromJSON(''["self-hosted", "linux", "x64", "ecs-qwen"]'') || fromJSON(''["ubuntu-latest"]'') }}'
# A wedged shard must free the pool runner well before GitHub's
# 360-minute default; 60 matches ci.yml's pool jobs.
timeout-minutes: 60
# 360-minute default. The ceiling is extended on the ECS pool only,
# mirroring ci.yml's pool lanes: run 33345905817 (issue #10591) expired
# at the old flat 60 on a contended shared host before its log reported
# any test result. Hosted fallbacks keep the pre-contention ceiling so
# a genuine hang there does not burn the extra 30 minutes.
timeout-minutes: "${{ fromJSON((github.repository == 'QwenLM/qwen-code' && vars.MAINTAINER_ECS_RUNNER_DISABLED != 'true') && '90' || '60') }}"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] R1-1: [fails-closed] [regression] This change turns e2e-test-linux's timeout-minutes from the number 60 into this expression string, but the existing lane-contract test .github/scripts/ci-runner-routing.test.mjs:552 still pins assert.equal(job['timeout-minutes'], 60) for this exact job and is never updated by this PR. That file imports node:assert/strict (so assert.equal is strictEqual), which means the assertion now fails unconditionally in every routing state. The file is listed in HELPER_TESTS (.github/workflows/ci.yml:82) and executed via node --test by both helper steps — "GitHub CI helper checks" (ci.yml:436) and "Run .github/scripts helper tests" (ci.yml:586); this PR classifies into the full profile, so the required Test (ubuntu-latest, Node 22.x) check goes red on this PR's own CI run.

Witness:

$ node --test .github/scripts/ci-runner-routing.test.mjs   (at HEAD 64a703c)
# tests 23
# pass 22
# fail 1
not ok 5 - carries the pool hygiene and capability steps in order
  operator: strictEqual
  expected: 60
  actual: "${{ fromJSON((github.repository == 'QwenLM/qwen-code' && vars.MAINTAINER_ECS_RUNNER_DISABLED != 'true') && '90' || '60') }}"
  at: .github/scripts/ci-runner-routing.test.mjs:552

A/B on the contested assertion: BASE (pre-change e2e.yml) passes with value=60; the PR arm fails with the expression string.

Update .github/scripts/ci-runner-routing.test.mjs:552 in this same PR: replace the bare 60 pin with an evaluation of the new routing-scoped expression — the same substitute-then-evaluate technique this PR adds in scripts/tests/e2e-workflow.test.js — asserting 90 on the pool routing and 60 on both hosted fallbacks (fork and kill-switch), and fix the adjacent stale comment ("Pin the ci.yml pool precedent"), which now contradicts ci.yml's actual pool precedent of 90.

The updated pin must keep the two ceilings mirrored from .github/workflows/ci.yml:219 (90 on ecs-qwen routing, 60 hosted) and agree with the routing condition the timeout mirrors (runs-on at .github/workflows/e2e.yml:47). The updated assertion itself must go red if timeout-minutes regresses to an unconditional value (a flat 60 or flat 90 without the routing condition) — please remove the routing condition from the assertion and run that subtest to prove it.

中文说明

本改动把 e2e-test-linuxtimeout-minutes 从数字 60 改成了这个表达式字符串,但既有的通道契约测试 .github/scripts/ci-runner-routing.test.mjs:552 仍然对同一个任务钉死 assert.equal(job['timeout-minutes'], 60),且本 PR 没有更新它。该文件导入 node:assert/strict(因此 assert.equalstrictEqual),所以这条断言现在在任何路由状态下都会无条件失败。该文件位于 HELPER_TESTS(.github/workflows/ci.yml:82)中,由两个助手测试步骤通过 node --test 执行——"GitHub CI helper checks"(ci.yml:436)与 "Run .github/scripts helper tests"(ci.yml:586);本 PR 会被分类为 full profile,因此本 PR 自己的 CI 运行中必需的 Test (ubuntu-latest, Node 22.x) 检查会变红。

见证(在 HEAD 64a703c 上执行):node --test .github/scripts/ci-runner-routing.test.mjs → 23 个测试,22 通过,1 失败——失败子测试为 "carries the pool hygiene and capability steps in order",operator: strictEqual,expected: 60,actual 为新的表达式字符串。对该断言做 A/B:BASE(改动前的 e2e.yml)通过(值为 60),PR 侧失败。

请在同一个 PR 中更新 .github/scripts/ci-runner-routing.test.mjs:552:把裸的 60 钉子替换为对新的路由限定表达式求值——复用本 PR 在 scripts/tests/e2e-workflow.test.js 中新增的"先替换再求值"手法——断言资源池路由为 90、两种托管回退(fork 与紧急开关)均为 60,并修正旁边那条已过时的注释("Pin the ci.yml pool precedent"),它现在与 ci.yml 实际的资源池上限 90 相矛盾。

更新后的钉子必须保持与 .github/workflows/ci.yml:219 一致的两个上限(ecs-qwen 路由 90,托管 60),并与 timeout 所镜像的路由条件(.github/workflows/e2e.yml:47 的 runs-on)保持一致。更新后的断言本身必须在 timeout-minutes 退化为无条件值(不带路由条件的裸 60 或裸 90)时变红——请从断言中去掉路由条件并运行该子测试来证明这一点。

— qwen3.8-max via Qwen Code /review (v0.22.3)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified real at the current head: node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs fails at line 552 (strictEqual expected 60, actual is the new routing-scoped expression string) — 23 tests, 22 pass, 1 fail. The full HELPER_TESTS set confirms this is the sole cause of the red required Test (ubuntu-latest, Node 22.x) check: 500 tests, exactly this one failure.

This round nevertheless does NOT implement the pin update: .github/scripts/ is outside this PR's footprint (the PR touches only .github/workflows/e2e.yml and scripts/tests/e2e-workflow.test.js), and the round-3 attempt at exactly this pin update was rejected by the autofix gate and reverted for that reason. Repeating the rejected change would fail the gate again, and there is no in-footprint change that can turn the check green while the pin still reads 60. This needs a maintainer decision on where the pin update lands:

  1. Allow this PR to expand into .github/scripts/, so a next round updates the pin here (substitute-then-evaluate: 90 on the pool routing, 60 on both hosted fallbacks, mirroring the runs-on condition; plus the stale "Pin the ci.yml pool precedent" comment); or
  2. A maintainer lands the pin update separately (direct commit, or a small standalone PR merged first), and this PR rebases onto it; or
  3. Revert this PR's timeout change entirely (not recommended — it abandons the Main CI failed: E2E Tests on cd5d5af2fbd7 #10591 fix).

Leaving this thread open until a maintainer chooses.

中文说明

已在当前 head 上确认为真实问题:node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs 在第 552 行失败(strictEqual 期望 60,实际为新的路由限定表达式字符串)——23 个测试,22 通过,1 失败。完整 HELPER_TESTS 集合确认这是必需的 Test (ubuntu-latest, Node 22.x) 检查变红的唯一原因:共 500 个测试,仅此一处失败。

但本轮不实施该断言更新:.github/scripts/ 在本 PR 的 footprint 之外(本 PR 只改动 .github/workflows/e2e.ymlscripts/tests/e2e-workflow.test.js),且第 3 轮对同一断言的更新尝试已被 autofix gate 以该理由拒绝并回滚。重复被拒绝的改动只会再次被 gate 拒绝,而在断言仍钉死 60 的情况下,footprint 内没有任何改动能让该检查变绿。需要维护者决定断言更新落在哪里:

  1. 允许本 PR 扩展到 .github/scripts/,下一轮即可在此更新断言(先替换再求值:资源池路由 90、两种托管回退均为 60,与 runs-on 条件保持一致;同时修正过时的 "Pin the ci.yml pool precedent" 注释);或
  2. 由维护者单独落地断言更新(直接提交,或先合并一个小的独立 PR),之后本 PR 重新变基;或
  3. 整体回退本 PR 的 timeout 改动(不建议——等于放弃 Main CI failed: E2E Tests on cd5d5af2fbd7 #10591 的修复)。

在维护者作出选择之前,本线程保持打开。

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] R1-1: [fails-closed] [regression] This change turns e2e-test-linux's timeout-minutes from the number 60 into this expression string, but the existing lane-contract test .github/scripts/ci-runner-routing.test.mjs:552 still pins assert.equal(job['timeout-minutes'], 60) for this exact job and is never updated by this PR. That file imports node:assert/strict (so assert.equal is strictEqual), which means the assertion now fails unconditionally in every routing state. The file is listed in HELPER_TESTS (.github/workflows/ci.yml:82) and executed via node --test by both helper steps — "GitHub CI helper checks" (ci.yml:436) and "Run .github/scripts helper tests" (ci.yml:586); this PR classifies into the full profile, so the required Test (ubuntu-latest, Node 22.x) check goes red on this PR's own CI run.

Re-checked at f2b2a2f7 (this round's reviewed commit): the stale pin is untouched and the failure still fires.

Witness:

$ node --test .github/scripts/ci-runner-routing.test.mjs   (at HEAD f2b2a2f7)
# tests 23
# pass 22
# fail 1
not ok 5 - carries the pool hygiene and capability steps in order
  operator: strictEqual
  expected: 60
  actual: "${{ fromJSON((github.repository == 'QwenLM/qwen-code' && vars.MAINTAINER_ECS_RUNNER_DISABLED != 'true') && '90' || '60') }}"
  at: .github/scripts/ci-runner-routing.test.mjs:552

A/B on the contested assertion: BASE (pre-change e2e.yml) passes with value=60; the PR arm fails with the expression string.

Update .github/scripts/ci-runner-routing.test.mjs:552 in this same PR: replace the bare 60 pin with an evaluation of the new routing-scoped expression — the same substitute-then-evaluate technique this PR adds in scripts/tests/e2e-workflow.test.js — asserting 90 on the pool routing and 60 on both hosted fallbacks (fork and kill-switch), and fix the adjacent stale comment ("Pin the ci.yml pool precedent"), which now contradicts ci.yml's actual pool precedent of 90.

The updated pin must keep the two ceilings mirrored from .github/workflows/ci.yml:219 (90 on ecs-qwen routing, 60 hosted) and agree with the routing condition the timeout mirrors (runs-on at .github/workflows/e2e.yml:47). The updated assertion itself must go red if timeout-minutes regresses to an unconditional value (a flat 60 or flat 90 without the routing condition) — please remove the routing condition from the assertion and run that subtest to prove it.

中文说明

本改动把 e2e-test-linuxtimeout-minutes 从数字 60 改成了这个表达式字符串,但既有的通道契约测试 .github/scripts/ci-runner-routing.test.mjs:552 仍然对同一个任务钉死 assert.equal(job['timeout-minutes'], 60),且本 PR 没有更新它。该文件导入 node:assert/strict(因此 assert.equalstrictEqual),所以这条断言现在在任何路由状态下都会无条件失败。该文件位于 HELPER_TESTS(.github/workflows/ci.yml:82)中,由两个助手测试步骤通过 node --test 执行——"GitHub CI helper checks"(ci.yml:436)与 "Run .github/scripts helper tests"(ci.yml:586);本 PR 会被分类为 full profile,因此本 PR 自己的 CI 运行中必需的 Test (ubuntu-latest, Node 22.x) 检查会变红。

已在本轮审阅的提交 f2b2a2f7 上复核:该过时的断言原样存在,失败仍然触发。

见证(在 HEAD f2b2a2f 上执行):node --test .github/scripts/ci-runner-routing.test.mjs → 23 个测试,22 通过,1 失败——失败子测试为 "carries the pool hygiene and capability steps in order",operator: strictEqual,expected: 60,actual 为新的表达式字符串。对该断言做 A/B:BASE(改动前的 e2e.yml)通过(值为 60),PR 侧失败。

请在同一个 PR 中更新 .github/scripts/ci-runner-routing.test.mjs:552:把裸的 60 钉子替换为对新的路由限定表达式求值——复用本 PR 在 scripts/tests/e2e-workflow.test.js 中新增的"先替换再求值"手法——断言资源池路由为 90、两种托管回退(fork 与紧急开关)均为 60,并修正旁边那条已过时的注释("Pin the ci.yml pool precedent"),它现在与 ci.yml 实际的资源池上限 90 相矛盾。

更新后的钉子必须保持与 .github/workflows/ci.yml:219 一致的两个上限(ecs-qwen 路由 90,托管 60),并与 timeout 所镜像的路由条件(.github/workflows/e2e.yml:47 的 runs-on)保持一致。更新后的断言本身必须在 timeout-minutes 退化为无条件值(不带路由条件的裸 60 或裸 90)时变红——请从断言中去掉路由条件并运行该子测试来证明这一点。

— qwen3.8-max via Qwen Code /review (v0.22.3)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified real at the current head: node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs fails at line 552 (strictEqual expected 60, actual is the new routing-scoped expression string) — 23 tests, 22 pass, 1 fail. The full HELPER_TESTS set confirms this is the sole cause of the red required Test (ubuntu-latest, Node 22.x) check: 500 tests, exactly this one failure.

This round nevertheless does NOT implement the pin update: .github/scripts/ is outside this PR's footprint (the PR touches only .github/workflows/e2e.yml and scripts/tests/e2e-workflow.test.js), and the round-3 attempt at exactly this pin update was rejected by the autofix gate and reverted for that reason. Repeating the rejected change would fail the gate again, and there is no in-footprint change that can turn the check green while the pin still reads 60. This needs a maintainer decision on where the pin update lands:

  1. Allow this PR to expand into .github/scripts/, so a next round updates the pin here (substitute-then-evaluate: 90 on the pool routing, 60 on both hosted fallbacks, mirroring the runs-on condition; plus the stale "Pin the ci.yml pool precedent" comment); or
  2. A maintainer lands the pin update separately (direct commit, or a small standalone PR merged first), and this PR rebases onto it; or
  3. Revert this PR's timeout change entirely (not recommended — it abandons the Main CI failed: E2E Tests on cd5d5af2fbd7 #10591 fix).

Leaving this thread open until a maintainer chooses.

中文说明

已在当前 head 上确认为真实问题:node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs 在第 552 行失败(strictEqual 期望 60,实际为新的路由限定表达式字符串)——23 个测试,22 通过,1 失败。完整 HELPER_TESTS 集合确认这是必需的 Test (ubuntu-latest, Node 22.x) 检查变红的唯一原因:共 500 个测试,仅此一处失败。

但本轮不实施该断言更新:.github/scripts/ 在本 PR 的 footprint 之外(本 PR 只改动 .github/workflows/e2e.ymlscripts/tests/e2e-workflow.test.js),且第 3 轮对同一断言的更新尝试已被 autofix gate 以该理由拒绝并回滚。重复被拒绝的改动只会再次被 gate 拒绝,而在断言仍钉死 60 的情况下,footprint 内没有任何改动能让该检查变绿。需要维护者决定断言更新落在哪里:

  1. 允许本 PR 扩展到 .github/scripts/,下一轮即可在此更新断言(先替换再求值:资源池路由 90、两种托管回退均为 60,与 runs-on 条件保持一致;同时修正过时的 "Pin the ci.yml pool precedent" 注释);或
  2. 由维护者单独落地断言更新(直接提交,或先合并一个小的独立 PR),之后本 PR 重新变基;或
  3. 整体回退本 PR 的 timeout 改动(不建议——等于放弃 Main CI failed: E2E Tests on cd5d5af2fbd7 #10591 的修复)。

在维护者作出选择之前,本线程保持打开。

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] R1-1: [fails-closed] [regression] This change turns e2e-test-linux's timeout-minutes from the number 60 into this expression string, but the existing lane-contract test .github/scripts/ci-runner-routing.test.mjs:552 still pins assert.equal(job['timeout-minutes'], 60) for this exact job and is never updated by this PR. That file imports node:assert/strict (so assert.equal is strictEqual), which means the assertion now fails unconditionally in every routing state. The file is listed in HELPER_TESTS (.github/workflows/ci.yml:82) and executed via node --test by both helper steps — "GitHub CI helper checks" (ci.yml:436) and "Run .github/scripts helper tests" (ci.yml:586); this PR classifies into the full profile, so the required Test (ubuntu-latest, Node 22.x) check goes red on this PR's own CI run and on every run after merge until the pin is updated.

Re-checked at 77eb9c1 (this round's reviewed commit): the stale pin is untouched and the failure still fires.

Witness:

$ node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs   (at HEAD 77eb9c1)
# tests 23
# pass 22
# fail 1
not ok 5 - carries the pool hygiene and capability steps in order
  operator: strictEqual
  expected: 60
  actual: "${{ fromJSON((github.repository == 'QwenLM/qwen-code' && vars.MAINTAINER_ECS_RUNNER_DISABLED != 'true') && '90' || '60') }}"
  at: .github/scripts/ci-runner-routing.test.mjs:552

A/B on the contested assertion: BASE (pre-change e2e.yml, git show HEAD^2) carries the numeric 60 that satisfies the pin; the PR arm fails with the expression string.

Update .github/scripts/ci-runner-routing.test.mjs:552 in this same PR: replace the bare 60 pin with an evaluation of the new routing-scoped expression — the same substitute-then-evaluate technique this PR adds in scripts/tests/e2e-workflow.test.js — asserting 90 on the pool routing and 60 on both hosted fallbacks (fork and kill-switch), and fix the adjacent stale comment ("Pin the ci.yml pool precedent"), which now contradicts ci.yml's actual pool precedent of 90. Note: the round-3 autofix attempt at exactly this fix was rejected by the autofix gate because .github/scripts/ is outside this PR's footprint, and the change was reverted — if that gate still applies to this PR, this update needs a maintainer decision on where the pin change lands.

The replacement must respect the strict-assert import at .github/scripts/ci-runner-routing.test.mjs:17 (node:assert/strict, so compare like-typed values), must keep the pin as a node:test/node:assert assertion (ci.yml:579-583 notes vitest does not collect node:test files), and must keep the two ceilings mirrored from .github/workflows/ci.yml:219 (90 on ecs-qwen routing, 60 hosted), agreeing with the routing condition the timeout mirrors (runs-on at .github/workflows/e2e.yml:47). The updated assertion itself must go red if timeout-minutes regresses to an unconditional value (a flat 60 or flat 90 without the routing condition) — please remove the routing condition from the assertion and run that subtest to prove it.

中文说明

本改动把 e2e-test-linuxtimeout-minutes 从数字 60 改成了这个表达式字符串,但既有的通道契约测试 .github/scripts/ci-runner-routing.test.mjs:552 仍然对同一个任务钉死 assert.equal(job['timeout-minutes'], 60),且本 PR 没有更新它。该文件导入 node:assert/strict(因此 assert.equalstrictEqual),所以这条断言现在在任何路由状态下都会无条件失败。该文件位于 HELPER_TESTS(.github/workflows/ci.yml:82)中,由两个助手测试步骤通过 node --test 执行——"GitHub CI helper checks"(ci.yml:436)与 "Run .github/scripts helper tests"(ci.yml:586);本 PR 会被分类为 full profile,因此本 PR 自己的 CI 运行中必需的 Test (ubuntu-latest, Node 22.x) 检查会变红,且在更新该断言之前,合并后的每次运行也会变红。

已在 77eb9c1(本轮审阅的提交)上复核:该过时的断言原样存在,失败仍然触发。

见证(在 HEAD 77eb9c1 上执行):node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs → 23 个测试,22 通过,1 失败——失败子测试为 "carries the pool hygiene and capability steps in order",operator: strictEqual,expected: 60,actual 为新的表达式字符串。对该断言做 A/B:BASE(改动前的 e2e.yml,git show HEAD^2)是能满足该断言的数字 60;PR 侧因表达式字符串而失败。

请在同一个 PR 中更新 .github/scripts/ci-runner-routing.test.mjs:552:把裸的 60 钉子替换为对新的路由限定表达式求值——复用本 PR 在 scripts/tests/e2e-workflow.test.js 中新增的"先替换再求值"手法——断言资源池路由为 90、两种托管回退(fork 与紧急开关)均为 60,并修正旁边那条已过时的注释("Pin the ci.yml pool precedent"),它现在与 ci.yml 实际的资源池上限 90 相矛盾。注意:第 3 轮 autofix 曾尝试过完全相同的修复,但因 .github/scripts/ 超出本 PR 的改动范围而被 autofix 门拒绝并回退——如果该限制对本 PR 仍然生效,这次更新需要维护者决定断言改动落在哪里。

替换时必须尊重 .github/scripts/ci-runner-routing.test.mjs:17 的 strict 断言导入(node:assert/strict,即比较同类型值);断言必须保持为 node:test/node:assert 形式(ci.yml:579-583 注明 vitest 不会收集 node:test 文件);并保持与 .github/workflows/ci.yml:219 一致的两个上限(ecs-qwen 路由 90,托管 60),与 timeout 所镜像的路由条件(.github/workflows/e2e.yml:47 的 runs-on)保持一致。更新后的断言本身必须在 timeout-minutes 退化为无条件值(不带路由条件的裸 60 或裸 90)时变红——请从断言中去掉路由条件并运行该子测试来证明这一点。

— qwen3.8-max via Qwen Code /review (v0.22.3)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified real at the current head: node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs fails at line 552 (strictEqual expected 60, actual is the new routing-scoped expression string) — 23 tests, 22 pass, 1 fail. The full HELPER_TESTS set confirms this is the sole cause of the red required Test (ubuntu-latest, Node 22.x) check: 500 tests, exactly this one failure.

This round nevertheless does NOT implement the pin update: .github/scripts/ is outside this PR's footprint (the PR touches only .github/workflows/e2e.yml and scripts/tests/e2e-workflow.test.js), and the round-3 attempt at exactly this pin update was rejected by the autofix gate and reverted for that reason, as this comment itself notes. Repeating the rejected change would fail the gate again, and there is no in-footprint change that can turn the check green while the pin still reads 60. This needs a maintainer decision on where the pin update lands:

  1. Allow this PR to expand into .github/scripts/, so a next round updates the pin here (substitute-then-evaluate: 90 on the pool routing, 60 on both hosted fallbacks, mirroring the runs-on condition; plus the stale "Pin the ci.yml pool precedent" comment); or
  2. A maintainer lands the pin update separately (direct commit, or a small standalone PR merged first), and this PR rebases onto it; or
  3. Revert this PR's timeout change entirely (not recommended — it abandons the Main CI failed: E2E Tests on cd5d5af2fbd7 #10591 fix).

Leaving this thread open until a maintainer chooses.

中文说明

已在当前 head 上确认为真实问题:node --test --test-concurrency=1 .github/scripts/ci-runner-routing.test.mjs 在第 552 行失败(strictEqual 期望 60,实际为新的路由限定表达式字符串)——23 个测试,22 通过,1 失败。完整 HELPER_TESTS 集合确认这是必需的 Test (ubuntu-latest, Node 22.x) 检查变红的唯一原因:共 500 个测试,仅此一处失败。

但本轮不实施该断言更新:.github/scripts/ 在本 PR 的 footprint 之外(本 PR 只改动 .github/workflows/e2e.ymlscripts/tests/e2e-workflow.test.js),且正如本评论所述,第 3 轮对同一断言的更新尝试已被 autofix gate 以该理由拒绝并回滚。重复被拒绝的改动只会再次被 gate 拒绝,而在断言仍钉死 60 的情况下,footprint 内没有任何改动能让该检查变绿。需要维护者决定断言更新落在哪里:

  1. 允许本 PR 扩展到 .github/scripts/,下一轮即可在此更新断言(先替换再求值:资源池路由 90、两种托管回退均为 60,与 runs-on 条件保持一致;同时修正过时的 "Pin the ci.yml pool precedent" 注释);或
  2. 由维护者单独落地断言更新(直接提交,或先合并一个小的独立 PR),之后本 PR 重新变基;或
  3. 整体回退本 PR 的 timeout 改动(不建议——等于放弃 Main CI failed: E2E Tests on cd5d5af2fbd7 #10591 的修复)。

在维护者作出选择之前,本线程保持打开。

# Skip on fork PRs: forks have no access to repository secrets
# (OPENAI_*, DOCKERHUB_*), so the matrix would fail unconditionally
# and show misleading red status. Same-repo PRs run normally.
Expand Down
52 changes: 52 additions & 0 deletions scripts/tests/e2e-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,4 +107,56 @@ describe('e2e workflow', () => {
expect(runStep.run).toContain('mktemp -d /var/tmp/qwen-ci-XXXXXX');
expect(runStep.run).toContain('trap \'rm -rf "$TMPDIR"');
});

describe('job timeout lane contract', () => {
// Run 33345905817 (issue #10591) expired at the old flat 60-minute
// ceiling on a contended shared ECS host before its log reported any
// test result. The extended ceiling is
// scoped to the pool routing — the same condition `runs-on` uses:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-2: The new lane-contract test evaluates only the timeout-minutes expression and never reads runs-on, so the timeout↔routing equivalence this comment claims ("the same condition runs-on uses") is documented but unpinned. e2e.yml carries the routing predicate as two hand-written copies (runs-on at e2e.yml:47 and timeout-minutes at e2e.yml:54), unlike ci.yml where both fields derive from one source of truth (needs.classify_pr.outputs.ubuntu_runner). A future edit to the runs-on condition (adding a kill-switch term or widening pool eligibility) that forgets timeout-minutes leaves every test in this suite green: runs newly routed onto the contended ECS pool keep the 60-minute ceiling and silently re-create issue #10591 (a shard expires before its log reports any test result), or drift the other way and hosted runs get 90 minutes, burning the extra 30 the diff's comment explicitly exists to prevent.

Witness:

Mutation probe: flip != 'true' -> == 'true' on runs-on ONLY (timeout expression untouched)
  PR suite on mutated tree: 11/11 passed (green through a complete inversion of routing semantics)
  equivalence probe (assert timeout == 90 iff runs-on resolves to ecs-qwen): 2 failed | 1 passed
    poolRouted=false but timeout=90
    poolRouted=true but timeout=60   <- the exact #10591 re-creation
  flip check on pristine tree: 3 passed

In the same describe block, evaluate the runs-on expression with the same substitution sweep and assert agreement per scenario instead of asserting each half against test-chosen constants — e.g. assert timeoutMinutesOn(s) === 90 iff the evaluated runs-on expression resolves to the ["self-hosted", "linux", "x64", "ecs-qwen"] branch — so the two expressions answer to one oracle. Note that runs-on at .github/workflows/e2e.yml:47 is a single-quoted YAML scalar with doubled quotes, so after YAML parse the embedded literals carry single quotes; the check must compare the two evaluated results, not a third hand-copied condition. The new equivalence assertion must go red when != 'true' is flipped to == 'true' on only the runs-on expression — today's suite stays green under that mutation; please add it and prove it.

中文说明

新增的通道契约测试只对 timeout-minutes 表达式求值,从不读取 runs-on,因此这条注释所声称的 timeout↔路由等价关系("与 runs-on 使用相同的条件")只是写在注释里,并没有被测试钉住。e2e.yml 中的路由谓词以两份手写副本存在(runs-on 在 e2e.yml:47,timeout-minutes 在 e2e.yml:54),而不像 ci.yml 那样两个字段都从 needs.classify_pr.outputs.ubuntu_runner 这一唯一事实源派生。如果未来某次编辑修改了 runs-on 条件(新增紧急开关项或放宽资源池准入)却忘了 timeout-minutes,本套件的所有测试仍会全绿:新路由到争用 ECS 资源池的运行会继续使用 60 分钟上限,悄悄复现 issue #10591(分片在报告任何测试结果之前过期);反向漂移则会让托管运行获得 90 分钟,白白烧掉本 diff 注释明确要避免的额外 30 分钟。

见证:变异探针——只对 runs-on 表达式把 != 'true' 翻转为 == 'true'(timeout 表达式不动),本 PR 的套件在变异树上 11/11 全绿(路由语义被完全反转后依然通过);而等价性探针(断言 timeout == 90 当且仅当 runs-on 解析到 ecs-qwen 分支)在同一变异树上 2 失败 | 1 通过,包括 "poolRouted=false but timeout=90" 与 "poolRouted=true but timeout=60"(即 #10591 的复现形态);在纯净树上做翻转校验:3 通过。

请在同一个 describe 块中,用相同的替换扫描对 runs-on 表达式求值,并按场景断言两者一致,而不是各自对着测试选定的常量断言——例如断言 timeoutMinutesOn(s) === 90 当且仅当求值后的 runs-on 表达式解析到 ["self-hosted", "linux", "x64", "ecs-qwen"] 分支——让两个表达式对同一个裁决者负责。注意:.github/workflows/e2e.yml:47 的 runs-on 是内含双写引号的单引号 YAML 标量,YAML 解析后其中的字面量带单引号;检查应比较两个求值结果,而不是第三份手抄的条件。新的等价断言必须满足:只把 runs-on 表达式中的 != 'true' 翻转为 == 'true' 时变红——当前套件在该变异下保持全绿,请加上断言并运行证明。

— qwen3.8-max via Qwen Code /review (v0.22.3)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-2: The new lane-contract test evaluates only the timeout-minutes expression and never reads runs-on, so the timeout↔routing equivalence this comment claims ("the same condition runs-on uses") is documented but unpinned. e2e.yml carries the routing predicate as two hand-written copies (runs-on at e2e.yml:47 and timeout-minutes at e2e.yml:54), unlike ci.yml where both fields derive from one source of truth (needs.classify_pr.outputs.ubuntu_runner). A future edit to the runs-on condition (adding a kill-switch term or widening pool eligibility) that forgets timeout-minutes leaves every test in this suite green: runs newly routed onto the contended ECS pool keep the 60-minute ceiling and silently re-create issue #10591 (a shard expires before its log reports any test result), or drift the other way and hosted runs get 90 minutes, burning the extra 30 the diff's comment explicitly exists to prevent.

Re-checked at f2b2a2f7 (this round's reviewed commit): the equivalence is still unpinned.

Witness:

Mutation probe: flip != 'true' -> == 'true' on runs-on ONLY (timeout expression untouched)
  PR suite on mutated tree: 11/11 passed (green through a complete inversion of routing semantics)
  equivalence probe (assert timeout == 90 iff runs-on resolves to ecs-qwen): 2 failed | 1 passed
    poolRouted=false but timeout=90
    poolRouted=true but timeout=60   <- the exact #10591 re-creation
  flip check on pristine tree: 3 passed

In the same describe block, evaluate the runs-on expression with the same substitution sweep and assert agreement per scenario instead of asserting each half against test-chosen constants — e.g. assert timeoutMinutesOn(s) === 90 iff the evaluated runs-on expression resolves to the ["self-hosted", "linux", "x64", "ecs-qwen"] branch — so the two expressions answer to one oracle. Note that runs-on at .github/workflows/e2e.yml:47 is a single-quoted YAML scalar with doubled quotes, so after YAML parse the embedded literals carry single quotes; the check must compare the two evaluated results, not a third hand-copied condition. The new equivalence assertion must go red when != 'true' is flipped to == 'true' on only the runs-on expression — today's suite stays green under that mutation; please add it and prove it.

中文说明

新增的通道契约测试只对 timeout-minutes 表达式求值,从不读取 runs-on,因此这条注释所声称的 timeout↔路由等价关系("与 runs-on 使用相同的条件")只是写在注释里,并没有被测试钉住。e2e.yml 中的路由谓词以两份手写副本存在(runs-on 在 e2e.yml:47,timeout-minutes 在 e2e.yml:54),而不像 ci.yml 那样两个字段都从 needs.classify_pr.outputs.ubuntu_runner 这一唯一事实源派生。如果未来某次编辑修改了 runs-on 条件(新增紧急开关项或放宽资源池准入)却忘了 timeout-minutes,本套件的所有测试仍会全绿:新路由到争用 ECS 资源池的运行会继续使用 60 分钟上限,悄悄复现 issue #10591(分片在报告任何测试结果之前过期);反向漂移则会让托管运行获得 90 分钟,白白烧掉本 diff 注释明确要避免的额外 30 分钟。

已在本轮审阅的提交 f2b2a2f7 上复核:该等价关系仍未被测试钉住。

见证:变异探针——只对 runs-on 表达式把 != 'true' 翻转为 == 'true'(timeout 表达式不动),本 PR 的套件在变异树上 11/11 全绿(路由语义被完全反转后依然通过);而等价性探针(断言 timeout == 90 当且仅当 runs-on 解析到 ecs-qwen 分支)在同一变异树上 2 失败 | 1 通过,包括 "poolRouted=false but timeout=90" 与 "poolRouted=true but timeout=60"(即 #10591 的复现形态);在纯净树上做翻转校验:3 通过。

请在同一个 describe 块中,用相同的替换扫描对 runs-on 表达式求值,并按场景断言两者一致,而不是各自对着测试选定的常量断言——例如断言 timeoutMinutesOn(s) === 90 当且仅当求值后的 runs-on 表达式解析到 ["self-hosted", "linux", "x64", "ecs-qwen"] 分支——让两个表达式对同一个裁决者负责。注意:.github/workflows/e2e.yml:47 的 runs-on 是内含双写引号的单引号 YAML 标量,YAML 解析后其中的字面量带单引号;检查应比较两个求值结果,而不是第三份手抄的条件。新的等价断言必须满足:只把 runs-on 表达式中的 != 'true' 翻转为 == 'true' 时变红——当前套件在该变异下保持全绿,请加上断言并运行证明。

— qwen3.8-max via Qwen Code /review (v0.22.3)

// hosted fallbacks (forks and MAINTAINER_ECS_RUNNER_DISABLED=true)
// keep the pre-contention bound, so a genuine hang there does not
// burn the extra 30 minutes. Evaluate the real timeout expression for
// both routings instead of pinning a bare number — the same
// substitute-then-evaluate technique ci-platform-lanes.test.js uses —
// so a regression to an unconditional ceiling fails here instead of
// reading as a passing constant.
const timeoutMinutesOn = ({ repository, ecsDisabled }) => {
const expr = String(yml.jobs['e2e-test-linux']['timeout-minutes'])
.replace(/^\$\{\{\s*/, '')
.replace(/\s*\}\}$/, '')
.replace(/github\.repository/g, JSON.stringify(repository))
.replace(
/vars\.MAINTAINER_ECS_RUNNER_DISABLED/g,
JSON.stringify(ecsDisabled),
)
.replace(/fromJSON\(/g, '(');
if (/github\.|vars\.|needs\.|steps\.|fromJSON\(/.test(expr)) {
throw new Error(
`e2e-test-linux timeout carries a term this guard does not model: ${expr}`,
);
}
return Number(new Function(`return (${expr});`)());
};

it('keeps a contended ECS shard above the install/build/test budget', () => {
expect(
timeoutMinutesOn({ repository: 'QwenLM/qwen-code', ecsDisabled: '' }),
).toBe(90);
});

it('keeps the hosted fallback on the pre-contention ceiling', () => {
expect(
timeoutMinutesOn({
repository: 'some-fork/qwen-code',
ecsDisabled: '',
}),
).toBe(60);
expect(
timeoutMinutesOn({
repository: 'QwenLM/qwen-code',
ecsDisabled: 'true',
}),
).toBe(60);
});
});
});
Loading