chore: add Chrome verify script for Phase 1.7b/c new UI (pre-publish gate)#47
Conversation
Pre-publish production verification — covers the new flows that PRs #43, #44, #45 introduced but never round-tripped through Devvit's actual runtime: 1. Compose form rendering 2. Clarify modal renders the **select** field (was paragraph) 3. Re-compile → composeConfirmForm opens + compiledSummary visible 4. Save click → success toast 5. Dashboard onboarding card + token cost line 6. New "vibe-mod: Manage rules" menu opens with per-rule action select Each step writes a screenshot + JSON record into playwright/.auth/. End-of-run summary: playwright/.auth/verify-phase17b-result.json. Pre-req: `npx devvit upload` (NOT publish) — installs latest code on r/SocialSeeding for playtest. Cookie source is the same as chrome-reddit-v3.py (browser_cookie3 reads the user's local Chrome reddit.com cookies, no creds embedded in the script). Usage: source .venv-chrome-auth/bin/activate python scripts/chrome-reddit-verify-phase17b.py # HEADLESS=0 for visible browser, REDDIT_SUB=other for a different sub Why this is needed: 211 unit/integration tests cover the handler logic, but a production runtime issue (Devvit-specific form-field rendering, redis.watch/multi/exec semantics, new menu/form name registration) can still surface only at devvit upload time. Catching those before `devvit publish --public` saves the Reddit App Directory review round.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new Playwright-based verification script, scripts/chrome-reddit-verify-phase17b.py, designed to automate the testing of new UI flows for rule composition and management. The review feedback focuses on improving the script's reliability and performance by replacing brittle manual interactions—such as coordinate-based clicking and fixed sleep durations—with more robust Playwright features like element-based locators and built-in wait conditions for toasts and form transitions.
| cy = box["y"] + box["height"] / 2 | ||
| await page.mouse.move(cx, cy) | ||
| await page.wait_for_timeout(120) | ||
| await page.mouse.click(cx, cy) |
There was a problem hiding this comment.
Using manual mouse coordinates for clicking is brittle and can fail if the page layout changes slightly or if the element is not correctly aligned. It is generally better to use the Playwright locator.click() method, which automatically handles scrolling and ensures the element is actionable.
| waited = 0 | ||
| while waited < deadline: | ||
| await page.wait_for_timeout(2000) | ||
| waited += 2000 | ||
| for sel in ['faceplate-toast', '[role="alert"]', '[role="status"]']: | ||
| try: | ||
| loc = page.locator(sel).first | ||
| if await loc.count() == 0: | ||
| continue | ||
| t = (await loc.inner_text(timeout=800)).strip() | ||
| if t and len(t) < 800: | ||
| return t | ||
| except Exception: | ||
| pass | ||
| return '' |
There was a problem hiding this comment.
The manual polling loop for the toast message can be replaced with Playwright's built-in wait_for functionality. This is more efficient and idiomatic, as it avoids unnecessary fixed delays and leverages Playwright's internal polling mechanism.
try:
loc = page.locator('faceplate-toast, [role="alert"], [role="status"]').first
await loc.wait_for(state="visible", timeout=timeout_ms)
return (await loc.inner_text()).strip()
except Exception:
return ''|
|
||
| # Wait for either a Clarify modal (next form) or a Confirm form (if | ||
| # input was deterministic) or a toast (error). 20s max. | ||
| await page.wait_for_timeout(8_000) |
There was a problem hiding this comment.
Avoid using page.wait_for_timeout() with large fixed durations. This makes the script slow and prone to flakiness if the network or server response time varies. Instead, wait for a specific element that indicates the next state has been reached (e.g., the next form appearing).
| await page.wait_for_timeout(8_000) | |
| await page.locator('faceplate-form').first.wait_for(state="visible", timeout=15_000) |
| await shot("05-clarify-picked") | ||
| recompiled = await submit_form(page, r"recompile|re-compile|compile") | ||
| report.add(StepResult("clarify-recompile-submit", recompiled, "clicked Re-compile")) | ||
| await page.wait_for_timeout(8_000) |
There was a problem hiding this comment.
Similar to the previous step, replacing this fixed 8-second wait with a targeted wait for the expected form element will improve the script's performance and reliability.
| await page.wait_for_timeout(8_000) | |
| await page.locator('faceplate-form').first.wait_for(state="visible", timeout=15_000) |
…script chore: add Chrome verify script for Phase 1.7b/c new UI (pre-publish gate)
1 participant
Summary
Adds `scripts/chrome-reddit-verify-phase17b.py` — production verification that PRs #43, #44, #45 actually work end-to-end through Devvit's runtime, not just through the in-memory test doubles.
What it covers
The new UI surfaces that have only been tested via doubles so far:
Each step writes a screenshot + JSON record into `playwright/.auth/`. End-of-run summary: `verify-phase17b-result.json`.
Why pre-publish (not post-publish)
Devvit-specific runtime issues (form field rendering, redis.watch/multi/exec semantics, new menu/form name registration) can still surface only at `devvit upload` time. Catching them BEFORE `devvit publish --public` saves a full Reddit App Directory review round if anything's off.
How to run
```bash
npx devvit upload # USER ACTION
source .venv-chrome-auth/bin/activate
python scripts/chrome-reddit-verify-phase17b.py
HEADLESS=0 for visible browser, REDDIT_SUB=other for a different sub
```
Cookies
`browser_cookie3` reads the user's local Chrome reddit.com cookies — no creds embedded in the script. Output cookie file (`playwright/.auth/reddit-com.json`) and screenshots are gitignored (`.gitignore` entry for `playwright/` was added in PR #42).
Test plan