Skip to content

test(e2e): fix project-outcomes thread-panel flake — bridge seed race + mock relay ignoring filter.ids - #131

Merged
oscarlehuu merged 4 commits into
mainfrom
devin/1786366224-fix-125-project-outcomes-seed-race
Aug 11, 2026
Merged

test(e2e): fix project-outcomes thread-panel flake — bridge seed race + mock relay ignoring filter.ids#131
oscarlehuu merged 4 commits into
mainfrom
devin/1786366224-fix-125-project-outcomes-seed-race

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 10, 2026

Copy link
Copy Markdown

Summary

It was a spec bug, not a product bug — specifically a test-setup race, not a superseded surface. The outcome-first Projects view from #95 mounts the in-place thread panel correctly; the spec was seeding its thread root before the E2E bridge existed, and swallowing the miss.

project-outcomes.spec.ts navigates with waitUntil: "domcontentloaded" and then immediately calls the bridge global with optional chaining:

await page.goto("/", { waitUntil: "domcontentloaded" });
await page.evaluate(() => {
  window.__BUZZ_E2E_SEED_MOCK_MESSAGE__?.({ /* id: "project-outcome-thread" */ });
});

domcontentloaded does not imply the E2E bridge module has executed. When the bundle is still evaluating — which is what load does — the global is absent, ?. turns the miss into a silent no-op, and the root event is never seeded. The panel's root fetch then finds no such event, thread.threadHead is null, and ProjectOutcomeThreadPanel renders its error branch, which carries project-in-flight-panel but no message-thread-panel. That is exactly the observed shape: line 107 (project-in-flight-panel visible) passes, line 108 (message-thread-panel) fails.

Instrumenting the mock bridge and catching a natural failure captured the smoking gun — at seed time the bridge was { "available": false }, and at the failing root fetch the requested id was absent from the channel store while an unrelated kind-9 event was returned instead.

Two changes:

  1. desktop/tests/e2e/project-outcomes.spec.ts — wait for the bridge globals before using them, and fail loudly instead of silently skipping. This follows the convention already used in the repo (channel-dense-second-reach.spec.ts:35, active-turn-resilience.spec.ts:27) rather than adding a new helper. No assertion is relaxed, skipped, or deleted; the silent ?. no-op is what hid this for days, so the seed now also asserts the returned event id.

  2. desktop/src/testing/e2eBridge.tsemitMockHistory now honors filter.ids (same exact/prefix semantics as the existing id-only path). It previously filtered on kinds/since/until only, so a filter carrying both #h and ids silently degraded to "newest event in the channel". The panel's root fetch is { ids, kinds, "#h", limit: 1 }, so without this the spec would stay latently flaky even with correct seeding — it would only pass while the seeded root happened to sort first under created_at DESC, id ASC. Demonstrated deterministically: a competing same-second event with id 000…0 wins the tiebreak over project-outcome-thread, and the panel renders Unable to load this thread. The only production caller of the combined #h + ids shape is ProjectOutcomeThreadPanel.tsx:53, so the blast radius is contained.

No product code changed. No docs/ change: this is test/harness-only and does not move shipped state, so the STATE.md anti-drift rule doesn't apply. No DECISIONS.md entry — no product decision was forced, so no D-number was consumed (leaving D-028/D-029/D-030 to #120).

Related issue

Fixes #125.

Related but deliberately untouched: #109 (Desktop Smoke E2E (4) dead lane, ~19 upstream project-* specs, owned by another session on a separate branch). This PR does not touch any upstream project-* spec and does not add or duplicate that session's project-plumbing <details> helper.

Testing

Red -> green on the reported failure. The failure is load-dependent — it passes in isolation on main, which is why it read as a hard break only when triaged inside the shard.

RED, on unmodified main, --repeat-each=20 --retries=0:

# --workers=8
12 failed
8 passed (28.0s)

# --workers=1, under 8 background CPU-load processes
16 failed
4 passed (4.6m)

Every failure identical:

Error: expect(locator).toBeVisible() failed

Locator: getByTestId('message-thread-panel')
Expected: visible
Timeout: 5000ms
Error: element(s) not found

at desktop/tests/e2e/project-outcomes.spec.ts:108:58

with the error context showing the panel's error branch:

complementary: Unable to load this thread. Close the panel and try again.

GREEN, same commands on this branch:

# --workers=8
20 passed (23.6s)

# --workers=1, under CPU contention
20 passed (4.3m)

Plain reported command, from desktop/:

xvfb-run -a pnpm exec playwright test --project=smoke project-outcomes.spec.ts
1 passed (10.0s)

Gates (. ./bin/activate-hermit first):

pnpm --filter buzz build:e2e   pass
pnpm --filter buzz check       pass
pnpm --filter buzz typecheck   pass
pnpm --filter buzz test        5045 passed, 0 failed, 1 skipped
just test-unit                 8/8 suites pass

Because emitMockHistory is shared channel-history code, a broader slice was run beyond the target spec: the six project-*.spec.ts files in one invocation (--retries=0) went from 22 failed / 17 passed to 21 failed / 18 passed, with project-outcomes.spec.ts moving into the passed column; the remaining failures are the preexisting shard failures tracked by #109 and are unaffected by this change either way. A 103-spec channel/thread slice ran 83 passed / 20 failed on this branch; the identical slice on main (35af74019, separate build, same workers and --retries=0) also ran 83 passed / 20 failed, and the failing spec sets are identical (empty symmetric difference). So none of those 20 come from making emitMockHistory honor filter.ids.

CI

Red checks on this PR are preexisting on the base commit. main run 31362178966 is on 35af74019 — the exact commit this branch builds on — and already shows:

Desktop Smoke E2E (1): failure
Desktop Smoke E2E (2): success
Desktop Smoke E2E (3): failure
Desktop Smoke E2E (4): cancelled   # 30m timeout, issue #109

Those are the same two shards red here, and shard 4 is the #109 lane. channels.spec.ts:500 is #110. Every individual failing test from shards 1 and 3 was additionally re-run locally against unmodified origin/main and reproduced there.

No UI change, so no screenshots: the product surface is unchanged and already rendered correctly before this PR.

Link to Devin session: https://app.devin.ai/sessions/48126c669ed543be891e81905624328d
Requested by: @oscarlehuu


Open in Devin Review

@oscarlehuu oscarlehuu self-assigned this Aug 10, 2026
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

Wait for the E2E bridge before seeding project outcome threads and make mock history honor id filters.\n\nFixes #125

Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Oscar Le <oscar.lehuu@gmail.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the devin/1786366224-fix-125-project-outcomes-seed-race branch from 23a8a7b to bfae5c1 Compare August 11, 2026 09:29
@oscarlehuu
oscarlehuu merged commit db2052a into main Aug 11, 2026
9 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

project-outcomes.spec.ts:108 fails on main: Crew-owned break in the outcome-first Projects thread panel

1 participant