Skip to content

Harden the known wall-clock test flakes#2657

Merged
lahma merged 1 commit into
sebastienros:mainfrom
lahma:test-reliability-wallclock-flakes
Jul 13, 2026
Merged

Harden the known wall-clock test flakes#2657
lahma merged 1 commit into
sebastienros:mainfrom
lahma:test-reliability-wallclock-flakes

Conversation

@lahma

@lahma lahma commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Addresses the recurring CI/local test flakes (e.g. the Windows CI failure on #2542). Five flake families were investigated to root cause; all share one pattern: wall-clock deadlines racing starved CPUs on loaded runners.

Test262 slow files (RegExp-*-escape-BMP, decodeURI byte-range loops)

These do seconds of legitimate work (64K-code-point regex loops; ~1.3M decodeURI calls) and pass in ~1–5 s isolated, but blow the 30-second wall-clock TimeoutInterval when the parallel suite starves the CPU — measured locally: 4 failures at exactly 30 s in a full run, 4 passes in 5 s isolated. A known-slow file list now gets a 5-minute budget; every other test keeps the tight runaway-script limit, so hang detection is not weakened.

AsyncTests task-interop / async-wake family

The CI failure above is EvaluateAsyncWithTaskInteropShouldWork: "Promise was rejected with value Timeout of 00:00:10 reached" — a Task.Delay(100) continuation queued past the default 10 s PromiseTimeout under thread-pool starvation on a two-core runner. (The event-loop wake path itself checks out: registration is double-checked and wake is broadcast; this is starvation, not a lost-wakeup bug.) The family now uses a generous 2-minute budget — these tests assert behavior, not latency. Tests asserting a timeout/cancellation fires keep tight budgets (starvation only helps them fire).

Bonus: ShouldRespectCustomProvidedTimeoutWhenUnwrapping — skipped as "Flaky test" — is un-skipped: it raced a 1 ms timeout against a 100 ms delayed task (under load the task could win and no exception was thrown); a never-resolving promise makes it deterministic.

TimeSystemTests.CanUseTimeProvider

Asserted the script''s new Date() within a fixed 100 ms window captured before a cold engine construction — regularly exceeded under load (the known net472 flake). Now bracketed between wall-clock readings taken around the evaluation; the frozen FakeTimeProvider half asserts exact equality.

Atomics.waitAsync early timeout (engine fix)

Surfaced during gating: no-spurious-wakeup-on-store.js failed because Task.Delay can complete slightly ahead of the requested interval (timer granularity/coalescing), letting the waiter resolve "timed-out" before the timeout actually elapsed — script-observable, and test262 asserts lapse >= timeout. The timeout timer now re-delays until a monotonic clock confirms the interval has fully elapsed — the same discipline the sync Atomics.wait loop already had. This complements the existing NonParallelFeatures serialization of the Atomics suites.

Gate

Jint.Tests / PublicInterface / CommonScripts green (including the un-skipped test on both TFMs), plus a fully green 99,429-test parallel Test262 run — previously the BMP quartet failed exactly this kind of run on this machine.

🤖 Generated with Claude Code

…us promise timeouts, bracketed clock asserts, no-early-timeout waitAsync

Five flake families, one root pattern: wall-clock deadlines racing
starved CPUs on loaded runners.

* Test262: the escape-BMP RegExp and 4-level decodeURI byte-range tests
  do seconds of legitimate work and pass in isolation, but blow the
  30-second wall-clock TimeoutInterval when the parallel suite starves
  the CPU. A known-slow file list now gets a 5-minute budget; everything
  else keeps the tight runaway-script limit.
* AsyncTests: the default 10-second PromiseTimeout lost races against
  thread-pool starvation on CI (a Task.Delay(100) continuation was
  observed to queue past 10s on a two-core runner). The async-wake /
  task-interop family now uses a generous 2-minute budget - these tests
  assert behavior, not latency; tests asserting that a timeout FIRES
  keep tight budgets. ShouldRespectCustomProvidedTimeoutWhenUnwrapping
  is un-skipped: a never-resolving promise makes its 1 ms timeout
  deterministic instead of racing a 100 ms delayed task.
* TimeSystemTests.CanUseTimeProvider: the script "now" is asserted
  between wall-clock readings taken around the evaluation instead of
  within a fixed 100 ms window after a cold engine construction; the
  frozen FakeTimeProvider half asserts exact equality.
* Atomics.waitAsync: Task.Delay can complete slightly ahead of the
  requested interval (timer granularity/coalescing), letting a waiter
  resolve "timed-out" before the timeout elapsed - script-observable,
  and test262 asserts lapse >= timeout. The timeout timer now re-delays
  until a monotonic clock confirms the interval has fully elapsed,
  matching the discipline the sync Atomics.wait loop already had.

Full gate green including a clean 99,429-test parallel Test262 run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lahma
lahma force-pushed the test-reliability-wallclock-flakes branch from 2b608ab to 79e4587 Compare July 13, 2026 08:25
@lahma
lahma enabled auto-merge (squash) July 13, 2026 08:25
@lahma
lahma merged commit 681400f into sebastienros:main Jul 13, 2026
4 checks passed
@lahma
lahma deleted the test-reliability-wallclock-flakes branch July 13, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant