Skip to content

fix(test): make LlmSession Windows CI flakes deterministic - #1862

Merged
Aaronontheweb merged 2 commits into
devfrom
fix/llm-session-flakes
Aug 11, 2026
Merged

fix(test): make LlmSession Windows CI flakes deterministic#1862
Aaronontheweb merged 2 commits into
devfrom
fix/llm-session-flakes

Conversation

@Aaronontheweb

Copy link
Copy Markdown
Collaborator

Problem

Two LlmSessionIntegrationTests flaked on Windows CI (run 31435126237, PR #1859):

  1. Reminder_redelivery_is_deduped_while_first_turn_is_in_flightAssert.Equal(1, _fakeChatClient.CallCount) got Actual: 0 inside a 3s AwaitAssertAsync poll.
  2. JoinSession_receives_SessionJoined_acknowledgementAskTimeoutException : Timeout after 3.00 seconds.

Both are real-time-budget races on a loaded runner — same family as the #1858 Discord fix. This PR fixes them deterministically, without touching any timeouts.

Fixes

1. Reminder dedup test — event barrier instead of polling.

The CommandAck fires before the first turn's LLM call is scheduled (recall resolution + working-context mailbox hop run first), so polling CallCount == 1 raced the very thing it measured. Added a FirstCallEntered TaskCompletionSource to FakeChatClient, set when the first GetResponseAsync entry increments CallCount (before the gate). The test now awaits the signal — proving the turn is genuinely in flight (blocked on the gate) — then asserts CallCount == 1 directly. Event-driven, not wall-clock.

2. JoinSession test — warm up the cold spawn outside the timed Ask.

The first JoinSession cold-spawns the session actor through DI (fresh SQLite store, persistence recovery, serialization verification) inline in the parent mailbox — unbounded cost on a loaded runner that exceeds the 3s Ask. The test now warms the session first (Tell + probe ExpectMsgAsync<SessionJoined>), so the assertion Ask measures only the hot mailbox path. TurnCount == 0 / Title == null still hold — the warmup runs no turn.

The 30s ceilings are hang guards only, not pass/fail mechanisms — the pass/fail is event-driven (TCS completion / probe message receipt).

Verification

  • Both tests pass; full LlmSessionIntegrationTests class 34/34 passes on Linux.
  • 5x stress loop of the two tests: all green.
  • Windows CI on this PR is the definitive check.

Follow-up

~30 other tests in this file carry the same latent cold-start Ask<SessionJoined>(3s) pattern. Scoping this PR to the two observed flakes; a base-class WarmUpSession(...) helper sweep is a good tracked follow-up.

Two LlmSessionIntegrationTests flaked on Windows CI (run 31435126237)
with real-time budget races — same family as the #1858 Discord fix, but
fixed without touching any timeouts:

Reminder_redelivery_is_deduped_while_first_turn_is_in_flight:
The CommandAck fires BEFORE the first turn's LLM call is scheduled
(recall resolution + working-context mailbox hop run first), so polling
CallCount==1 raced the very thing it measured (got 0 under load). Added
a FirstCallEntered TaskCompletionSource to FakeChatClient, set when the
first GetResponseAsync entry increments CallCount (before the gate);
the test now awaits the signal instead of polling. Event-driven, not
wall-clock.

JoinSession_receives_SessionJoined_acknowledgement:
The first JoinSession cold-spawns the session actor through DI (fresh
SQLite store, persistence recovery, serialization verification) inline
in the parent mailbox — unbounded on a loaded runner, exceeding the 3s
Ask. The test now warms the session with an unbounded-gated Tell +
probe ExpectMsg first, so the assertion Ask measures only the hot path.

The 30s ceilings are hang guards only, not pass/fail mechanisms.
@Aaronontheweb Aaronontheweb added bug Something isn't working sessions LLM session actor, turn lifecycle, pipelines tests All issues related to testing, quality assurance, and smoke testing. labels Aug 10, 2026
@Aaronontheweb
Aaronontheweb enabled auto-merge (squash) August 10, 2026 23:40
@Aaronontheweb
Aaronontheweb disabled auto-merge August 11, 2026 00:23
@Aaronontheweb
Aaronontheweb merged commit 3565655 into dev Aug 11, 2026
21 checks passed
@Aaronontheweb
Aaronontheweb deleted the fix/llm-session-flakes branch August 11, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working sessions LLM session actor, turn lifecycle, pipelines tests All issues related to testing, quality assurance, and smoke testing.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant