Flaky: test_api_examples[webkit-shiny/api-examples/todo_list/app-core.py] — resource 404 in console

[schloerke]

Occurrence

• Job: https://github.com/posit-dev/py-shiny/actions/runs/31414916798/job/93541539114
• Job name: playwright-examples (3.12, webkit, 1)
• Test: tests/playwright/examples/test_api_examples.py::test_api_examples[webkit-shiny/api-examples/todo_list/app-core.py]
• Date: 2026-08-10
• Seen on: PR fix: Soft teardown of reactives when a session closes #2428 (a docstring-only commit)

Error

E           AssertionError: In app shiny/api-examples/todo_list/app-core.py had JavaScript console errors!
E           Failed to load resource: the server responded with a status of 404 (Not Found)

tests/playwright/examples/example_apps.py:258: AssertionError
========= 1 failed, 96 passed, 4 skipped, 3 rerun in 110.79s (0:01:50) =========

The assertion is the console_errors check in example_apps.py:258. Note the 3 rerun — the in-job pytest-rerunfailures retries were exhausted, so this reproduced several times within the one job before surfacing.

Why this looks like a flake

• Passed on restart. gh run rerun 31414916798 --failed came back fully green (173 passed, 2 skipped, 0 failed) with no code change in between.
• The failure is a static resource 404 in the browser console, not an app-level Python error, and the commit under test changed only docstrings.
• The immediately preceding head on the same branch was fully green at 173 passed.

Notes

The console message does not identify which resource 404'd, which is the main obstacle to diagnosing this. If it recurs, capturing the failing request URL (e.g. logging console_errors alongside the page's failed network requests in example_apps.py) would make the next occurrence actionable — a webkit-only asset race on app startup is the leading guess.

[schloerke]

New occurrence, this time on firefox and with a webfont fetch rather than a 404 — same test, same console_errors assertion.

Failed job: https://github.com/posit-dev/py-shiny/actions/runs/31565383133/job/94016019484 (from #2437)

AssertionError: In app shiny/api-examples/todo_list/app-core.py had JavaScript console errors!
[JavaScript Error: "downloadable font: download failed (font-family: "Montserrat" style:normal weight:500 stretch:100 src index:0): status=2147746065 source: https://fonts.gstatic.com/s/montserrat/..."]

Passed on restart of the same job, so it is the flake and not a real failure.

Two notes that may matter for the fix:

• pytest-rerunfailures did not absorb this one — the log shows 3 rerun and the job still failed.
• Both recorded occurrences are the same shape: the assertion requires zero JS console errors while the example loads an external resource, so the test depends on fonts.gstatic.com (here) or the 404'd resource (original report) being reachable from the runner. Filtering console errors for third-party network failures, or serving the font locally, would address the class rather than the instance.

[schloerke]

New occurrence (2026-08-12), same signature.

• Failed job: https://github.com/posit-dev/py-shiny/actions/runs/31626891028/job/94215473350
• Commit: ed140dd9 (PR feat(session): Add session.allow_reconnect() #2441)
• Job: playwright-examples (3.12, webkit, 1) (run attempt 1)
• Re-run of the same job on the same commit: passed (run attempt 2)

FAILED tests/playwright/examples/test_api_examples.py::test_api_examples[webkit-shiny/api-examples/todo_list/app-core.py] - AssertionError: In app shiny/api-examples/todo_list/app-core.py had JavaScript console errors!
* Failed to load resource: the server responded with a status of 404 (Not Found)
Failed network requests:
* HTTP 404 - GET https://fonts.gstatic.com/s/montserrat/v31/JTU4jIg1_i6t8kCHKm4532VJOt5-QNFgpCtr6C41QYuEPgG_LbnLmMlbcDGy_rGKMQ.woff2
* Load request cancelled - GET https://fonts.gstatic.com/s/montserrat/v31/JTU4jIg1_i6t8kCHKm4532VJOt5-QNFgpCtr6C41QYuEPgG_LbnLmMlbcDGy_rGKMQ.woff2

Same Montserrat .woff2 URL from fonts.gstatic.com as the original report.