test(ci): keep Chromium's random local ports off test loopback connects on Windows

[mrgoonie]

Refs #544

Cause

The ERR_NO_BUFFER_SPACE failures in reference theme browser (windows-latest) do not come from sockets leaking or running out. They come from Chromium's local-port randomization on Windows.

• Chromium 139+ sets SO_RANDOMIZE_PORT before every TCP connect on Windows 11 22H2 and later (kTcpPortRandomizationWin, enabled by default, minimum WIN11_22H2; net/socket/tcp_socket_win.cc, net/base/features.cc). The runner is Windows Server 2025 (windows-2025-vs2026, build 26100), so it is affected. Chromium's Intent to Ship names rapid port reuse (the birthday problem) as the known risk.
• When the random local port collides, connect() fails at once with WSAENOBUFS, which the page reports as net::ERR_NO_BUFFER_SPACE. Both CI failures were the first navigation of a test, 63 ms and 415 ms in, which fits an immediate connect failure rather than gradual exhaustion.

Evidence (Windows 11 build 26200, Playwright 1.63 Chromium)

No leak, no exhaustion. I sampled netstat -ano -p TCP every 300 ms for loopback sockets during the job's commands. The dynamic port range is 16,384 ports.

Suite	Baseline total	Peak total	Peak TIME_WAIT	ESTABLISHED after
test:reference-theme-browser, before the fix	40	97	62	14 (= baseline)
test:reference-theme-browser, after, 3 runs	68 / 68 / 116	83 / 125 / 173	+0 / +42 / +48 per run	14 (= baseline)
test:widget-browser, after	164	245	+45	14 (= baseline)

Every host and browser is already closed in afterEach/afterAll, and every SSE client is ended before server.close(). No code path churns connections.

Port randomization causes the error. I ran a stress harness with Playwright's Chromium against a loopback server, where each request makes a fresh TCP connect. The two arms were alternated run by run.

Chromium launch	Connects	ERR_NO_BUFFER_SPACE
default	540,000	32 (plus 11 ERR_ADDRESS_IN_USE)
TcpPortRandomizationWin disabled	540,000	0

What changed

• tools/chromium-test-args.ts (new) provides chromiumTestArgs(launcher). On Windows it returns one --disable-features= switch that holds Playwright's own disabled features plus TcpPortRandomizationWin. On other platforms it returns [].
  • Chromium keeps only the last --disable-features switch. I verified this with a third-party storage probe: Playwright's defaults gave "shared", and appending a bare --disable-features=TcpPortRandomizationWin gave null, because it silently re-enabled ThirdPartyStoragePartitioning, PaintHolding, HttpsUpgrades and the rest. The helper therefore reads Playwright's list from the command line Chromium was actually launched with (Browser.getBrowserCommandLine, from one probe launch per process, Windows only) and extends that list. It does not copy the list, so it cannot drift. With the helper the probe gives "shared" again.
• All five widget-cli real-browser specs launch with these args: reference-theme-preview, semantic-composition, job-simulator, service-simulator and detach.
• playwright.config.ts adds the same args to the Chromium project. It runs the job's last step (reference-theme-composition and widget-appearance) on the same Windows runner.

What stays the same

• No assertion, timeout or retry changed, and there is no page.goto retry. A retry would only cover navigation, and a colliding subresource connect would still fail a test through requestfailed.
• macOS and Linux launches are unchanged (empty args).
• Product code is untouched.

Verification (on e3288d8)

• corepack pnpm run test:reference-theme-browser ×3: 8/8, 8/8, 8/8 passed.
• corepack pnpm run test:widget-browser: 4/4 passed.
• corepack pnpm vitest run packages/widget-cli/test/detach-real-browser.e2e.spec.ts: 1/1 passed.
• corepack pnpm test:e2e apps/web/e2e/reference-theme-composition.spec.ts apps/web/e2e/widget-appearance.spec.ts: 20/20 passed. The config's top-level await works under Playwright Test.
• corepack pnpm typecheck: passed. corepack pnpm invariants: all 13 passed. eslint on the changed files: clean.
• corepack pnpm verify: results are in a follow-up comment.

Known limitations

• The fix depends on Chromium's feature name TcpPortRandomizationWin. If Chromium renames or removes it, the switch silently does nothing and the flake could return. The CDP probe fails loudly if Playwright stops allowing Browser.getBrowserCommandLine.
• I could not reproduce the collision rate of the CI runner locally. On this machine it takes about 17,000 connects per failure, while the job makes a few hundred. Background socket load on the runner probably raises the rate there.

Docs impact

None. This changes only the test harness, not any user-visible behaviour, command or contract.

[mrgoonie]

Local corepack pnpm verify on e3288d8 (Windows 11): exit 0. All 13 invariants passed, typecheck and lint were clean, and tests: 519 files passed, 1 skipped; 6910 tests passed, 37 skipped, 1 todo.

[mrgoonie]

CI on e3288d8: reference theme browser (windows-latest) passed in both runs (job 112430735077 for pull_request, job 112430507134 for push). widget-dev-host 43/43, widget-browser 4/4, reference-theme-browser 8/8, and no ERR_NO_BUFFER_SPACE anywhere.

The Playwright e2e step reported flaky tests that passed on retry: 3 in the push run and 1 in the pull_request run. All of them failed in Playwright's Node-side request fixture: apiRequestContext.get: read ECONNRESET against the runtime node's /appearance, and once a 60 s timeout that ended in apiRequestContext.put. That client does not go through Chromium's network stack, so this change cannot affect it. The flake predates this PR. In the last 25 CI runs, the same read ECONNRESET retries appear on feat/510 (1, 2), feat/508 (1) and feat/507 (1), none of which touch the browser harness. It is a separate problem, which I will report for tracking.

[mrgoonie]

Review attestation: ready to merge at e3288d8174c543abb68a311bce208623c2b81aec, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at da7c148539e18fc41306d79391f186b3e9048986, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at a3a4ef98d35eedaffc954f0f21147ab7bab2e71f, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at 93c926e06a1cc2c391c42805056dddb1dda55459, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at d8de6e0fb394293d1022fe0ef55351c9fd3b550b, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at b10213f8eb59eacbb516fa140247306b0563efe8, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[claude]

⚠️ Code review skipped — your organization has no extra usage available to pay for this review.

If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.

Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.

[mrgoonie]

Review attestation: ready to merge at 4857266b5064940b54305d970bd8e6706cc8c620, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at e8fe30843b8dce20566c45b2bcdb11369fc18e82, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at f30bc2516ab2e5b603f7c03ed74f8a23c032ab2e, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.

[mrgoonie]

Review attestation: ready to merge at dbc9fd3cd92c8eaefedfaad85880d4e8e07a84c0, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.