Tests: a timing assertion measures the property, not the machine it ran on

[lahma]

Across a full day of v5 campaign work, every single CI failure was environmental — not one was a real
regression from the twenty-two pull requests that merged. The cost is not the re-run. It is that a genuine
regression now arrives looking exactly like the noise, and gets waved through.

This is the mechanism behind the repeat offenders. Nothing here deletes an assertion, and nothing here raises
a number until it stops failing. #3301 states the trap explicitly and it applies to all of them: a wait that
ignored its token and ran out its whole ceiling would still throw OperationCanceledException, so deleting
the elapsed check would let a real defect pass. Each row below is instead given a clock it can trust, a
handshake that removes the clock, or a margin expressed as a ratio to the ceiling it discriminates against.

What changed, per test

The stopwatch is started by the thread that releases the wait

Jint.Tests/Runtime/WaitForScheduledWorkTests.cs, Jint.Tests.PublicInterface/HostPumpWaitTests.cs

WaitForScheduledWorkTests.ThrowsOperationCanceledOnTheToken (#3301) missed its discriminator by 11 ms;
HostPumpWaitTests.WaitForScheduledWorkWakesOnACrossThreadPost reported 12 s against a 10 s ceiling it never
actually spent. Both were measuring far more than the wake: the clock started before the releasing thread had
even been scheduled, and ran through its quarter-second settle sleep.

It now starts on that thread, immediately before the release, so the only interval measured is the wake
latency — which is the thing the assertion is about. The property survives exactly: a wait that noticed the
release only when its ceiling ran out still fails, because that interval is measured from the release.

The cancellation row also stops releasing through CancellationTokenSource.CancelAfter. That callback is
delivered by the thread pool, and a saturated pool grows at roughly one worker per 500 ms, so it is the one
release whose latency the test cannot bound — a healthy 250 ms cancellation was measured at 5.011 s against a
5 s midpoint. It is now cancelled from a DedicatedThread the test owns, the same shape the enqueue rows
already used.

Ceiling moves from ten seconds to a minute in both files, with EarlyReturnMargin derived from it as a
half rather than written out again. #3301 asked for exactly this: the ratio is what the assertion is about,
so moving both together widens the margin without weakening it. A healthy run spends none of the ceiling; only
a genuine regression pays the extra fifty seconds before being reported.

The witness that needs no clock is asserted first

Jint.Tests.PublicInterface/HostPromiseTimeoutTests.cs

AnExplicitTimeoutStillWinsOverTheConfiguredOne failed with "Expected elapsed to be less than 5s, but found
1m 3s" — a 200 ms wait, stalled by a factor of three hundred.

UnwrapIfPromiseCore resolves the effective timeout into one local, hands that local to the drain, and formats
the same local into the rejection message. The message is therefore a witness of which budget the wait
actually ran under, not a restatement of the configuration: a wait that consulted the wrong budget cannot
name the right one. That is asserted first, by a named helper.

The elapsed check stays as the other half of the claim — that the wait ended on the budget it names rather
than sitting out the longer one it should have ignored — but its bound is now half of whatever the wrong
answer would have spent instead of a fixed five seconds. For the explicit row the wrong answer is the
configured five minutes, so the midpoint moves to 2 m 30 s and a three-hundred-fold stall no longer reaches it.
For the argument-less row the wrong answer is the ten-second default, which this test cannot change, so its
midpoint stays at five seconds — that is the one residual, and it is now a deliberate, documented consequence
of the rule rather than a magic constant.

The simulated IO ends when the test says so

Jint.Tests/Runtime/PromiseTests.cs

UnwrapIfPromiseAsync_WithIOBoundTask_DoesNotBlockCallerThread asserted that the unwrap task is still pending
while IO is in flight — with the IO being await Task.Delay(100). On a loaded runner the hundred milliseconds
can be gone before the test thread is scheduled again, and a completed unwrap then reads as exactly the defect
the test exists to catch.

It now ends on a TaskCompletionSource the test holds. "In flight" becomes a fact rather than a hope, and the
assertion gets stronger: the unwrap task provably cannot have completed.

Wedge ceilings on budgets nothing asserts

TestBudgets.WedgeCeiling already exists for this, and documents the distinction it turns on: a budget is
either part of what a test asserts, or it is a ceiling that exists only so a wedge is reported instead of a
hang. Five rows were sitting on budgets of the second kind, sized for an idle machine.

• HostStreamBridgeTests.ReadsAStreamWhoseReadsCompleteOnAnotherThread — every chunk arrives as a thread-pool
  continuation, and the whole copy was failing as PromiseRejectedException: Timeout of 00:00:10 reached. The
  assertion is the text that came out of the stream.
• AsyncTests.ShouldTaskAwaitCurrentStack (Three fixed 10-second budgets in tests blow on a contended Windows runner, the way the WPT fetch budget did #3358, seen on net472) and its ValueTask twin
  ShouldValueTaskAwaitCurrentStack — about 1.1 s of intended work inside the ten-second default. The
  assertion is the order the three appends landed in.
• UntrustedCodeProfileTests.OperationScopeCannotEndWhileAsyncWorkOwnsTheEngine — the profile's five-second
  timeout interval and ten-second operation deadline are budgets an embedder sizes for their own work. The
  assertion is that a scope refuses to end while an evaluation is suspended, and ends once it resumes.
• ConstraintReplacementTests.TheLaterTimeoutIsTheOneEnforced — the discriminator is the one-millisecond
  interval that must no longer be registered, which two hundred thousand iterations pass by four orders of
  magnitude on any machine. The widened interval was never a duration the test is about; thirty seconds was a
  wedge ceiling sized for an idle box.
• Jint.Tests.CommonScripts — the fixture is [Parallelizable(ParallelScope.All)], so twenty-eight CPU-bound
  real-world workloads share whatever cores the runner has, and a Windows leg was observed taking 7 m 19 s for
  the twenty-eight against ~20 s unloaded, with RegexMatchTimeoutException as the only symptom. Nothing here
  asserts anything about RegexTimeout. It stays finite deliberately — Regex.InfiniteMatchTimeout would turn
  a genuinely catastrophic pattern into a hung run.

Evidence

A clean run on an idle box proves nothing here, so: a 32-core machine saturated with CPU-bound spinner
processes, the affected tests run in a loop on both net10.0 and net472, before and after, same load and
same iteration count each time. "Before" is upstream/main's copy of the nine files, built in the same tree.

load	build	Jint.Tests net10.0	Jint.Tests net472	PublicInterface net10.0	PublicInterface net472	total
64 spinners (2×), 20 iterations	before	20/20	20/20	19/20	20/20	79/80
64 spinners (2×), 20 iterations	after	20/20	20/20	20/20	20/20	80/80
128 spinners (4×), 15 iterations	before	13/15	15/15	14/15	15/15	57/60
128 spinners (4×), 15 iterations	after	15/15	15/15	15/15	15/15	60/60

136/140 before, 140/140 after. All four baseline failures are named in this pull request, and both are
reproductions of the reported symptom rather than something new:

• PromiseTests.UnwrapIfPromiseAsync_WithIOBoundTask_DoesNotBlockCallerThread — twice, at 128 spinners:
  "Expected unwrapTask.IsCompleted to be False … but found True". This is the Task.Delay(100) race,
  reproduced exactly.
• UntrustedCodeProfileTests.OperationScopeCannotEndWhileAsyncWorkOwnsTheEngine — once at each load:
  PromiseRejectedException: Timeout of 00:00:00.1000000 reached, the profile's 100 ms promise budget.

The remaining rows did not fire in 140 attempts, which is expected — #3301's own failure was one occurrence
on a shared runner and it took six CI attempts on one pull request to establish the last one of these. Their
change is reasoned from the mechanism rather than from a reproduction, and the reasoning is stated per row
above.

#3358's third item is a different mechanism, and is not here

html/webappapis/timers/negative-settimeout.any.js was listed in #3358 with the issue's own caveat that it
should not be assumed to be a budget until someone looked. It was looked at, and it is neither a budget nor a
timer-ordering defect: the engine's clamp and ordering are deterministic and correct, and the driver simply
has no completion boundary for uncaught-callback reports while draining the event loop twice more after the
file is over. Upstream testharness.js has an explicit guard for exactly this. It is a WPT-driver change with
its own shim-symmetry tests and a worker-lane question to settle, so it is filed separately as #3366 — with
the full trace, upstream's guard quoted at the corpus pin, and the minimal fix — rather than folded in here.

One more flake found on the way, also filed rather than folded in

TypeConverterRegistrationTests.AnEngineWithACustomConverterSharesTheStockAccessorCache fails about one run
in three on the net472 leg under JINT_HOST_CONTRACT_VERIFICATION=1, and never when its class is run alone.
It is not a budget either: JsAccessibleRegistry.Generation is a process-wide counter and
TypeResolver.SyncGeneratedMembers drops the whole accessor cache when it moves, so any of the four
RegisterAll() calls elsewhere in the same two assemblies landing between that test's two engines makes its
calls.Should().Be(0) see a re-resolution. Filed with the mechanism and a fix direction as #3368.

Notes

• No public surface is touched: nine test files and nothing under Jint/. PublicApiTest and
  PublicApiDocumentationTest both pass unchanged, and UndocumentedPublicApi.txt is untouched.
• Jint.Tests.Test262 was not run: no engine code changed.
• dotnet build -c Release clean. Jint.Tests 7316/10668/10668 (net472/net8.0/net10.0),
  Jint.Tests.PublicInterface 2964/2339 (net10.0/net472), Jint.Tests.CommonScripts 28/28 on both — and the
  same two suites again with JINT_HOST_CONTRACT_VERIFICATION=1, whose only failure was AnEngineWithACustomConverterSharesTheStockAccessorCache flakes because another test's JsAccessibleRegistry.Register drops the shared cache #3368 above.

Fixes #3358.
Fixes #3301.

🤖 Generated with Claude Code

https://claude.ai/code/session_014W5mbjGhyvgAS4pivXoc4S