fix(e2e): run the suite against a production build - #319
Conversation
Switch the Playwright webServer from `bun run dev` to a real production build served by the standalone server. The Turbopack dev server was the root of two CI-only e2e failures that never reproduced on-device: - cold on-demand route compile under `workers:1` could outlast the 15s expect timeout (the #114 flake), and - on next 16.2 its HMR websocket crashed hydration under Bun (broke chat-popup + terminal-reconnect on the dep-refresh PR). A pre-built, minified standalone server has no HMR and no on-demand compile, so both classes disappear. Verified on the Jetson: chat-popup + terminal-reconnect + 36 specs pass against the prod build, and e2e JS coverage lands at 48.8% (above the 39% floor — the collector's V8 ranges still attribute fine). SESSION_SECRET is unset for the test server so middleware auth stays inactive and `/` renders the desktop (production sets it via production-server.js; e2e drives the UI, not the auth gate). The WebSocket specs mock `window.WebSocket` in-browser, so the bare standalone server needs no gateway/terminal proxy. Drops the now-obsolete global-setup warmup (a dev-Turbopack workaround). Re-enables terminal-reconnect (passes against prod). The five DOM-interaction specs stay fixme'd: they fail on GitHub Actions AND against the prod build on the Jetson, so they're a distinct per-spec issue, still tracked in #114.
📝 WalkthroughWalkthroughPlaywright now builds and starts the standalone production server for E2E tests. The global setup warm-up is removed. Five affected specs update FIXME comments to describe failures during their first DOM interaction. ChangesProduction E2E execution
Estimated code review effort: 2 (Simple) | ~10 minutes Sequence Diagram(s)sequenceDiagram
participant Playwright
participant NextBuild
participant NextServer
participant E2ESpecs
Playwright->>NextBuild: Build production application
Playwright->>NextServer: Start standalone server
E2ESpecs->>NextServer: Run browser interactions
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🦀 ClawReviewClaws waving — here's what this change is about. Switches the Playwright e2e suite to run against a real production build ( At a glance
Good to know
— ClawReview 🦀. I set the scene; CodeRabbit reviews the code; you decide. Conventions: docs. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@playwright.config.ts`:
- Line 54: Update the Playwright webServer command to launch the application
through production-server.js instead of directly invoking
.next/standalone/server.js, preserving the existing environment variables and
build step so E2E tests exercise the production upgrade proxy, TLS, and
WebSocket paths.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 8dc89d4c-a11d-4d9f-a6fd-422763258b3e
📒 Files selected for processing (7)
e2e/browser-vnc.spec.tse2e/clawkeep-interactions.spec.tse2e/desktop-selection.spec.tse2e/global-setup.tse2e/installed-app-settings.spec.tse2e/mascot-context.spec.tsplaywright.config.ts
💤 Files with no reviewable changes (1)
- e2e/global-setup.ts
| // via production-server.js; e2e drives the UI, not the auth gate). The | ||
| // WebSocket-backed specs mock `window.WebSocket` in-browser, so the bare | ||
| // standalone server (no gateway/terminal proxy) is enough. | ||
| command: `bun run build && env -u SESSION_SECRET PORT=${port} HOSTNAME=127.0.0.1 NODE_ENV=production node .next/standalone/server.js`, |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 10 'production-server|Server\.prototype\.listen|upgrade|WebSocket|SESSION_SECRET|terminal|gateway|novnc' \
package.json production-server.js playwright.config.tsRepository: ID-Robots/clawbox
Length of output: 20735
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== playwright config =="
sed -n '1,90p' playwright.config.ts
echo
echo "== production-server startup tail =="
sed -n '230,263p' production-server.js
echo
echo "== terminal reconnect tests and websocket handling =="
rg -n -C 8 'terminal-reconnect|terminal-ws|WebSocket|window\.WebSocket|test\.skip|new WebSocket|connect.*terminal' playwright tests src -SRepository: ID-Robots/clawbox
Length of output: 50373
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== e2e files =="
git ls-files 'e2e/**' | sed -n '1,120p'
echo
echo "== terminal reconnect occurrences in e2e =="
rg -n -C 10 'terminal|connect|reconnect|WebSocket|skip|disable|mock' e2e -S || true
echo
echo "== deterministic check: production-server monkey patch path =="
python3 - <<'PY'
from pathlib import Path
text = Path('production-server.js').read_text()
checks = {
'monkey_patches_http_server_listen': 'http.Server.prototype.listen = function (...args)' in text,
'production_branch_polls_upgrade': '"upgrade"' in text and 'if (IS_DEV) {' in text and 'const poll = setInterval' in text,
'attaches_terminal_proxy': 'prefix: "/terminal-ws"' in text and 'TERMINAL_WS_PORT' in text,
'removes_upgrade_listeners_before_proxy': 'removeAllListeners("upgrade")' in text and 'attachUpgradeProxy(this)' in text,
'starts_https': 'startHttpsServer(this)' in text,
'starts_next_server': 'require("./.next/standalone/server.js")' in text,
}
for k, v in checks.items():
print(f'{k}: {v}')
print('direct_standard_server_command:' in open('playwright.config.ts').read().replace('\n', ' ').replace(' ', ' '))
PYRepository: ID-Robots/clawbox
Length of output: 36892
Run E2E through the same production entrypoint.
production-server.js monkey-patches http.Server.prototype.listen, then in production polls until Next attaches upgrade handlers, strips those handlers, calls attachUpgradeProxy, starts HTTPS/WSS, and finally requires .next/standalone/server.js. Directly starting the standalone server skips the production upgrade proxy and TLS path, including the terminal WebSocket route that terminal-reconnect covers. Use node production‑server.js here, or document why bypassing this entrypoint is safe for all production behavior.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@playwright.config.ts` at line 54, Update the Playwright webServer command to
launch the application through production-server.js instead of directly invoking
.next/standalone/server.js, preserving the existing environment variables and
build step so E2E tests exercise the production upgrade proxy, TLS, and
WebSocket paths.
1 participant
Switches the Playwright e2e suite from
bun run devto a real production build, served by the standalone server. This is the Option-B rework: it fixes the class of CI-only e2e failures that never reproduced on-device.Why
The Turbopack dev server caused two CI-only failures:
workers:1could outlast the 15s expect timeout (the e2e: 6 tests fail only on GitHub Actions despite passing locally #114 flake).socket.addEventListener is not a function), breaking chat-popup + terminal-reconnect on the dep-refresh PR (fix(deps): patch runtime security advisories (next, undici, ws, postcss, nanoid, sharp) #315).A pre-built, minified standalone server has no HMR and no on-demand compile, so both classes disappear.
Verified on the Jetson
Against the prod build: chat-popup (4/4) + terminal-reconnect + 36 specs pass, and e2e JS coverage lands at 48.8% (above the 39% floor — the V8 collector still attributes fine to the hashed chunks).
Details
SESSION_SECRETis unset for the test server so middleware auth stays inactive and/renders the desktop (production sets it viaproduction-server.js; e2e drives the UI, not the auth gate). The WebSocket specs mockwindow.WebSocketin-browser, so the bare standalone server needs no gateway/terminal proxy.global-setupwarmup (a dev-Turbopack workaround).terminal-reconnect(passes against prod).Still open (does NOT close #114)
The five DOM-interaction specs (
browser-vnc,desktop-selection,installed-app-settings,mascot-context,clawkeep-interactions) stayfixme'd: they fail on GitHub Actions and against the prod build on the Jetson, so they're a distinct per-spec issue — not the dev-server class this PR fixes. Tracked in #114.Unblocks
#315 (the dep-security refresh) rebases on this and its chat-popup goes green — landing all 48 runtime CVEs.
Summary by CodeRabbit