Skip to content

Fix the flaky TransientNotice timer test blocking both Dependabot PRs - #113

Merged
bilbospocketses merged 2 commits into
masterfrom
fix/transient-notice-flaky-timer-test
Aug 26, 2026
Merged

Fix the flaky TransientNotice timer test blocking both Dependabot PRs#113
bilbospocketses merged 2 commits into
masterfrom
fix/transient-notice-flaky-timer-test

Conversation

@bilbospocketses

Copy link
Copy Markdown
Owner

What

TransientNoticeTests.Show_TwiceQuickly_LaterMessageSurvivesTheEarlierTimer has been failing CI on both open Dependabot PRs (#110, #111), blocking the whole Dependabot lane. #111 has auto-merge on and is stuck behind it.

It is flaky, not a regression — and the run history proves that rather than merely suggesting it. On PR #110's same commit, CI passed at 2026-08-17T21:39:26Z and failed at 21:43:59Z.

Why it flakes

The test drove a 200ms auto-dismiss with real Task.Delay calls:

t=0      Show("first",  200ms)  -> deadline t=200
t~100    Show("second", 200ms)  -> cancels the first; deadline t~300
t~280    assert Message == "second"     <- 20ms of headroom

Two Task.Delay calls that between them had to not overshoot by more than 20ms, on a shared runner, against ~15.6ms Windows timer granularity. It was always going to fail eventually.

The fix

TransientNotice now takes an optional TimeProvider, defaulting to TimeProvider.System so all six call sites are untouched, and the auto-dismiss awaits Task.Delay against it. The tests drive a FakeTimeProvider and land their assertions exactly on the deadlines that matter instead of sampling somewhere between them.

The dismiss clears state before invoking the re-render callback, so the tests use that callback as their completion signal rather than sleeping. Where a test asserts that a cancelled timer never fires, it first waits a real grace window — safe here precisely because fake time only moves when a test moves it, so a correctly cancelled timer never fires however long we wait.

Clear_RemovesMessageImmediatelyAndStopsTimer gets stronger in passing: it used to prove the timer had not fired within 50ms of a 60s deadline, and now advances five minutes past it.

Verification

  • 710 tests green in Release (39 Common / 25 Launcher / 646 ControlMenu).
  • Mutation-checked — each test catches the bug it names: dropping Cancel() from Show() fails Show_TwiceQuickly_LaterMessageSurvivesTheEarlierTimer; dropping it from Clear() fails Clear_RemovesMessageImmediatelyAndStopsTimer.
  • No changelog entry: this is test determinism plus an internal refactor with no behavior change.

Adds Microsoft.Extensions.TimeProvider.Testing (first-party, test project only) via CPM.

Show_TwiceQuickly_LaterMessageSurvivesTheEarlierTimer has been failing CI
on both open Dependabot PRs. It is flaky, not a regression: PR #110's same
commit passed at 21:39 and failed at 21:44 on 2026-08-17.

The test slept 100ms then 180ms against a 200ms dismiss deadline that
restarted at the 100ms mark, so it asserted ~280ms into a window closing at
~300ms. Twenty milliseconds of headroom across two Task.Delay calls, on a
shared runner, against ~15.6ms timer granularity.

TransientNotice now takes an optional TimeProvider, defaulting to
TimeProvider.System so all six call sites are untouched, and the
auto-dismiss awaits Task.Delay against it. The tests drive a
FakeTimeProvider and land their assertions exactly on the deadlines that
matter instead of sampling somewhere between them.

The dismiss clears state before invoking the re-render callback, so the
tests use that callback as their completion signal rather than sleeping.
Where a test asserts that a cancelled timer never fires, it first waits a
real grace window - safe here because fake time only moves when a test
moves it, so a correctly cancelled timer never fires however long we wait.

Confirmed by mutation: dropping the Cancel() from Show() fails
Show_TwiceQuickly_LaterMessageSurvivesTheEarlierTimer, and dropping it from
Clear() fails Clear_RemovesMessageImmediatelyAndStopsTimer.

710 tests green in Release.
@bilbospocketses
bilbospocketses enabled auto-merge (squash) August 26, 2026 16:31
@bilbospocketses
bilbospocketses merged commit 8f7ec46 into master Aug 26, 2026
8 checks passed
@bilbospocketses
bilbospocketses deleted the fix/transient-notice-flaky-timer-test branch August 26, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant