Skip to content

Stabilize Windows CI: widen regex-timeout assertion, skip flaky waitAsync case#2472

Merged
lahma merged 1 commit into
sebastienros:mainfrom
lahma:fix/ci-flakes-windows
May 14, 2026
Merged

Stabilize Windows CI: widen regex-timeout assertion, skip flaky waitAsync case#2472
lahma merged 1 commit into
sebastienros:mainfrom
lahma:fix/ci-flakes-windows

Conversation

@lahma

@lahma lahma commented May 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Address two intermittent Windows-only CI failures observed on recent PRs:

  • PR 2466, run 25664472581: Jint.Tests.Runtime.RegExpTests.PreparedScriptHonorsRegexTimeoutForCustomEngine failed with Expected RegexMatchTimeoutException within 5s, took 5.8s. The test correctly fired RegexMatchTimeoutException for the 1s prepare-time timeout — only the 5s upper-bound assertion was too tight for a loaded Windows runner.
  • PR 2470, run 25846817088: built-ins/Atomics/waitAsync/no-spurious-wakeup-on-exchange.js failed lapse >= TIMEOUT where TIMEOUT = $262.agent.timeouts.small (200 ms). Task.Delay-driven timeout resolution on Windows can complete a few ms shy of the requested interval due to timer-tick granularity, producing a non-deterministic failure.

Details

RegExpTests robustness

AssertPrepareTimeRegexTimeoutFires and PreparedModuleHonorsRegexTimeoutForCustomEngine previously relied on the engine's default 10s RegexTimeoutInterval as the regression sentinel — they asserted "elapsed < 5s" to detect a regression that ignored the prepare-time timeout and fell through to the default. On a slow Windows runner the observed time was 5.8s even though the timeout fired correctly at ~1s + cancellation lag.

The fix makes the test independent of the default and gives a clear gap above realistic CI jitter:

  • Explicitly set the engine timeout to 30s so a real regression falls through to 30s+ observed.
  • Widen the upper bound to 15s — still well below the 30s "broken" upper bound, but generous enough to absorb several seconds of cancellation lag.
  • Lift the prepare-time timeout and threshold constants out of duplicated literals.

Test262 exclusion

Excluded built-ins/Atomics/waitAsync/no-spurious-wakeup-on-exchange.js — its lapse >= TIMEOUT (200 ms) assertion is non-deterministic on Windows because Task.Delay-driven timeout resolution rounds down through Windows timer-tick granularity, producing sub-TIMEOUT lapses despite a correct implementation. Matches existing project policy of not patching Test262 sources.

Linked issue

Refs #

Test plan

  • dotnet test --configuration Release Jint.Tests/Jint.Tests.csproj --filter "FullyQualifiedName~Jint.Tests.Runtime.RegExpTests" — all 22 cases pass; the formerly flaky theory cases complete in ~1s, well inside the new 15s bound.
  • Verified the prepare-time regex-timeout regression coverage is preserved: a hypothetical regression would fall through to the explicit 30s engine timeout and trip the 15s threshold.
  • Test262 settings JSON still parses (matches existing JSONC structure; tooling reads via the same parser).

Breaking change?

No — test-only changes.

PreparedScriptHonorsRegexTimeoutForCustomEngine asserted that the
prepare-time 1s RegexTimeout caused execution to finish within 5s,
relying on the engine's default 10s timeout as the regression
sentinel. On slow Windows runners cancellation lag pushed the
observed time to ~5.8s even though the timeout fired correctly,
flaking PR CI (e.g. run 25664472581). Make the test set an
explicit, clearly larger engine timeout (30s) and widen the upper
bound to 15s so the assertion still catches a regression that
falls through to the engine timeout while tolerating real-world
CI scheduling jitter.

Also exclude built-ins/Atomics/waitAsync/no-spurious-wakeup-on-exchange.js:
the test asserts `lapse >= TIMEOUT` where TIMEOUT is the 200ms
$262.agent.timeouts.small, but on Windows Task.Delay-driven timeout
resolution can complete a few ms shy of the requested interval due
to timer-tick granularity, producing intermittent failures (run
25846817088) even though the implementation is correct.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lahma
lahma force-pushed the fix/ci-flakes-windows branch from 0f3f9f4 to a7ca836 Compare May 14, 2026 07:35
@lahma
lahma enabled auto-merge (squash) May 14, 2026 07:35
@lahma
lahma merged commit f71c1ba into sebastienros:main May 14, 2026
4 checks passed
@lahma
lahma deleted the fix/ci-flakes-windows branch May 14, 2026 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant