Skip to content

test(plugin-qrcode): stop QR re-render poll assertions from flaking on CI#2756

Merged
upupming merged 1 commit into
mainfrom
fix/qrcode-poll-timeout-flake
May 29, 2026
Merged

test(plugin-qrcode): stop QR re-render poll assertions from flaking on CI#2756
upupming merged 1 commit into
mainfrom
fix/qrcode-poll-timeout-flake

Conversation

@upupming

@upupming upupming commented May 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

The Vitest (Ubuntu) job intermittently fails on plugin-qrcode:

packages/rspeedy/plugin-qrcode/test/index.test.ts:198 (and :275)
Error: Matcher did not succeed in time.
expected "renderUnicodeCompact" to be called 2 times, but got 1 times

Both sites do await expect.poll(() => renderUnicodeCompact).toBeCalledTimes(2), waiting for the QR re-render triggered when an interactive-prompt promise resolves (resolve('bar') / resolve('main2')). They used expect.poll's default 1000ms timeout, which busy CI runners routinely exceed, so the poll gives up before the 2nd render lands. Locally (fast machine) it passes 9/9.

The dev-server tests also ran on the default 5s test budget, even though waitDevCompileDone alone allows up to 5s — so simply bumping the poll timeout would just hit the test timeout instead.

Failing CI (before this fix)

Fix

  • vitest.config.ts: raise the package testTimeout to 20s (these tests legitimately spin up a real dev server).
  • The two re-render polls: pass { timeout: 10_000 }.

Test-only change (no version bump). Can't be reproduced locally (passes here), so the validation is CI staying green.

Verified fixed

(The Vitest (Windows) failure on this PR is an unrelated CI-infra issue — hashFiles(...) in the workflow template timing out on the Windows runner — not a test failure.)

Test plan

  • Vitest (Ubuntu) passes.
  • Vitest (Windows) — re-run to clear the unrelated hashFiles infra timeout.

…n CI

`expect.poll(() => renderUnicodeCompact).toBeCalledTimes(2)` waits for the QR
re-render triggered by an interactive-prompt promise resolving. It used the
default 1000ms poll timeout, which busy CI runners routinely exceed, so the
Vitest job flaked with "Matcher did not succeed in time". The dev-server tests
also only had the default 5s test budget (the compile wait alone can take 5s).

Raise the package `testTimeout` to 20s and the two re-render polls to 10s.
@upupming upupming requested review from colinaaa and luhc228 as code owners May 29, 2026 09:17
@changeset-bot

changeset-bot Bot commented May 29, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c16d7b7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR increases test timeouts at both the Vitest configuration and individual assertion level to address flaky QR re-render polling assertions in CI. The config increases the per-test baseline to 20 seconds, and two specific test assertions explicitly set a 10-second polling timeout.

Changes

QR Plugin Test Timeout Adjustments

Layer / File(s) Summary
Test timeout configuration and polling adjustments
packages/rspeedy/plugin-qrcode/vitest.config.ts, packages/rspeedy/plugin-qrcode/test/index.test.ts, .changeset/qrcode-test-timeout.md
Vitest project timeout increases to 20 seconds with explanatory CI flakiness comments. Two expect.poll assertions for renderUnicodeCompact in "custom schema object" and "select between entries" tests are updated to use an explicit 10-second polling timeout. Changeset documents the fix.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Suggested reviewers

  • colinaaa
  • luhc228

Poem

A rabbit hops through flaky tests so weary,
"These timeouts are tight, the CI gets dreary!"
With twenty seconds now, and polls set just right,
The QR codes render—no more flaky night! 🐰✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title directly and accurately describes the main change: increasing test timeouts to resolve flaky QR re-render poll assertions in the qrcode plugin tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/qrcode-poll-timeout-flake

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented May 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented May 29, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 14.78%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 80 untouched benchmarks
⏩ 26 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
transform 1000 view elements 46.2 ms 40.2 ms +14.78%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/qrcode-poll-timeout-flake (c16d7b7) with main (bc8c3d4)2

Open in CodSpeed

Footnotes

  1. 26 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (40701bd) during the generation of this report, so bc8c3d4 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@github-actions

Copy link
Copy Markdown
Contributor

UI Judge

GEQI weighted score: 60.3 / 100 across 8 examples.
Average visual-correctness score: 3.4 / 5.

Dimension Weight Average Results Status
Usability & Interaction 30% 2.8 / 5 8 OK
Visual & Aesthetics 25% 3.1 / 5 8 OK
Consistency & Standards 15% 3.1 / 5 8 OK
Architecture & UX Writing 15% 3.1 / 5 8 OK
Accessibility & Performance 15% 3.1 / 5 8 OK
# Example Visual Correctness Usability & Interaction (30%) Visual & Aesthetics (25%) Consistency & Standards (15%) Architecture & UX Writing (15%) Accessibility & Performance (15%) GEQI Page Status
1 recs 2 / 5 2 / 5 3 / 5 2 / 5 2 / 5 2 / 5 45 / 100 preview OK
2 cast-grid 5 / 5 3 / 5 4 / 5 4 / 5 5 / 5 4 / 5 77 / 100 preview OK
3 citywalk-list 2 / 5 2 / 5 3 / 5 2 / 5 2 / 5 3 / 5 48 / 100 preview OK
4 fridge-search 4 / 5 3 / 5 2 / 5 4 / 5 3 / 5 3 / 5 58 / 100 preview OK
5 trip-planner 2 / 5 3 / 5 3 / 5 2 / 5 2 / 5 2 / 5 51 / 100 preview OK
6 weather-current 5 / 5 3 / 5 4 / 5 5 / 5 5 / 5 5 / 5 83 / 100 preview OK
7 product-card 5 / 5 4 / 5 4 / 5 4 / 5 4 / 5 4 / 5 80 / 100 preview OK
8 workout-plan 2 / 5 2 / 5 2 / 5 2 / 5 2 / 5 2 / 5 40 / 100 preview OK
Details

Result 1

  • Example: recs
  • Dimension: visual-correctness
  • Visual correctness: 2 / 5
  • GEQI dimensions:
    • Usability & Interaction: 2 / 5 (30%)
    • Visual & Aesthetics: 3 / 5 (25%)
    • Consistency & Standards: 2 / 5 (15%)
    • Architecture & UX Writing: 2 / 5 (15%)
    • Accessibility & Performance: 2 / 5 (15%)
  • Task: The A2UI playground preview should show date-night dining recommendations for Moonlight Terrace, Pinewood Bistro, and Sea Breeze Kitchen.

Result 2

  • Example: cast-grid
  • Dimension: visual-correctness
  • Visual correctness: 5 / 5
  • GEQI dimensions:
    • Usability & Interaction: 3 / 5 (30%)
    • Visual & Aesthetics: 4 / 5 (25%)
    • Consistency & Standards: 4 / 5 (15%)
    • Architecture & UX Writing: 5 / 5 (15%)
    • Accessibility & Performance: 4 / 5 (15%)
  • Task: The A2UI playground preview should show a cast grid for the short film Night Notes, including Lin Xia and Zhou Ning cast cards.

Result 3

  • Example: citywalk-list
  • Dimension: visual-correctness
  • Visual correctness: 2 / 5
  • GEQI dimensions:
    • Usability & Interaction: 2 / 5 (30%)
    • Visual & Aesthetics: 3 / 5 (25%)
    • Consistency & Standards: 2 / 5 (15%)
    • Architecture & UX Writing: 2 / 5 (15%)
    • Accessibility & Performance: 3 / 5 (15%)
  • Task: The A2UI playground preview should show weekend citywalk coffee picks with Rooftop Brew Room, Corner Canvas Lab, and Late Sun Roastery.

Result 4

  • Example: fridge-search
  • Dimension: visual-correctness
  • Visual correctness: 4 / 5
  • GEQI dimensions:
    • Usability & Interaction: 3 / 5 (30%)
    • Visual & Aesthetics: 2 / 5 (25%)
    • Consistency & Standards: 4 / 5 (15%)
    • Architecture & UX Writing: 3 / 5 (15%)
    • Accessibility & Performance: 3 / 5 (15%)
  • Task: The A2UI playground preview should show refrigerator search results with Siemens, Hualing, Haier, and Midea product cards.

Result 5

  • Example: trip-planner
  • Dimension: visual-correctness
  • Visual correctness: 2 / 5
  • GEQI dimensions:
    • Usability & Interaction: 3 / 5 (30%)
    • Visual & Aesthetics: 3 / 5 (25%)
    • Consistency & Standards: 2 / 5 (15%)
    • Architecture & UX Writing: 2 / 5 (15%)
    • Accessibility & Performance: 2 / 5 (15%)
  • Task: The A2UI playground preview should show a Kyoto 48-hour trip planner with Day 1 and Day 2 itinerary sections, including Monkey Park Viewpoint.

Result 6

  • Example: weather-current
  • Dimension: visual-correctness
  • Visual correctness: 5 / 5
  • GEQI dimensions:
    • Usability & Interaction: 3 / 5 (30%)
    • Visual & Aesthetics: 4 / 5 (25%)
    • Consistency & Standards: 5 / 5 (15%)
    • Architecture & UX Writing: 5 / 5 (15%)
    • Accessibility & Performance: 5 / 5 (15%)
  • Task: The A2UI playground preview should show the current weather for Austin, TX, including clear skies with light breeze.

Result 7

  • Example: product-card
  • Dimension: visual-correctness
  • Visual correctness: 5 / 5
  • GEQI dimensions:
    • Usability & Interaction: 4 / 5 (30%)
    • Visual & Aesthetics: 4 / 5 (25%)
    • Consistency & Standards: 4 / 5 (15%)
    • Architecture & UX Writing: 4 / 5 (15%)
    • Accessibility & Performance: 4 / 5 (15%)
  • Task: The A2UI playground preview should show a Wireless Headphones Pro product card with a visible Add to Cart action.

Result 8

  • Example: workout-plan
  • Dimension: visual-correctness
  • Visual correctness: 2 / 5
  • GEQI dimensions:
    • Usability & Interaction: 2 / 5 (30%)
    • Visual & Aesthetics: 2 / 5 (25%)
    • Consistency & Standards: 2 / 5 (15%)
    • Architecture & UX Writing: 2 / 5 (15%)
    • Accessibility & Performance: 2 / 5 (15%)
  • Task: The A2UI playground preview should show a weekly workout plan with five days from Monday Ramp-Up through Friday Conditioning.

Workflow run

@upupming upupming enabled auto-merge (squash) May 29, 2026 10:23
@upupming upupming merged commit fc217dc into main May 29, 2026
78 of 80 checks passed
@upupming upupming deleted the fix/qrcode-poll-timeout-flake branch May 29, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants