Skip to content

test(e2e): migrate WhatsApp compact QR to Vitest - #5146

Closed
cv wants to merge 18 commits into
mainfrom
codex/e2e-migrate-whatsapp-qr-compact
Closed

test(e2e): migrate WhatsApp compact QR to Vitest#5146
cv wants to merge 18 commits into
mainfrom
codex/e2e-migrate-whatsapp-qr-compact

Conversation

@cv

@cv cv commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Migrates the WhatsApp compact QR reporter workflow regression from the legacy shell script into a Vitest live scenario. The regression workflow now runs the version-pinned renderer proof directly through e2e-scenarios-live and uploads fixture artifacts.

Related Issue

Refs #4941
Refs #4522

Changes

  • Add test/e2e-scenario/live/whatsapp-qr-compact.test.ts with pinned OpenClaw package install, oversized baseline, and compact preload assertions.
  • Replace test/e2e/test-whatsapp-qr-compact-e2e.sh in regression-e2e.yaml with direct Vitest execution and artifact upload.
  • Update QR compact references in test/e2e/test-messaging-providers.sh, test/whatsapp-qr-compact.test.ts, and test/regression-e2e-workflow.test.ts.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

@cv cv self-assigned this Jun 10, 2026
@copy-pr-bot

copy-pr-bot Bot commented Jun 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 26234cd4-714a-4ff7-a957-71a0889669b9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/e2e-migrate-whatsapp-qr-compact

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: whatsapp-qr-compact-e2e
Optional E2E: None

Dispatch hint: whatsapp-qr-compact-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/codex/e2e-simplify-migration-tracking
Head: HEAD
Confidence: high

Required E2E

  • whatsapp-qr-compact-e2e (low; ubuntu-latest with node/npm only, no Docker, GPU, sandbox, or NVIDIA_API_KEY): This PR changes the implementation and workflow wiring for this exact regression lane. Running it validates that the new Vitest live scenario is selected by regression-e2e.yaml, installs dependencies correctly, writes artifacts, and still proves the WhatsApp pairing QR renders compact with the NemoClaw preload.

Optional E2E

  • None.

New E2E recommendations

  • None.

Dispatch hint

  • Workflow: .github/workflows/regression-e2e.yaml
  • jobs input: whatsapp-qr-compact-e2e

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

E2E Scenario Advisor Recommendation

Required scenario E2E: e2e-scenarios-all
Optional scenario E2E: None

Dispatch required scenario E2E:

  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/codex/e2e-simplify-migration-tracking
Head: HEAD
Confidence: medium

Required scenario E2E

  • e2e-scenarios-all: PR adds a Vitest live scenario test under test/e2e-scenario/live/. The new free-standing WhatsApp compact-QR test is not a trusted-main live-supported typed registry scenario ID, so it cannot be targeted safely by scenario id; use the scenario workflow fan-out to validate the live Vitest scenario surface without inventing an unsupported id.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Optional scenario E2E

  • None.

Relevant changed files

  • test/e2e-scenario/live/whatsapp-qr-compact.test.ts

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 0 needs attention, 1 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 1 still applies, 0 new items found

Review findings

🛠️ Needs attention

  • None.

🔎 Worth checking

  • Renderer proof install still runs package lifecycle scripts (test/e2e-scenario/live/whatsapp-qr-compact.test.ts:112): The migrated live test installs exact OpenClaw and WhatsApp package versions, but the nested `npm install` still omits `--ignore-scripts` and runs with `inheritEnv: true`. That preserves the deleted shell behavior, but it means lifecycle scripts from `openclaw`, `@openclaw/whatsapp`, or transitive dependencies can execute on the GitHub runner inside a workflow that has write-scoped permissions at the workflow level. This is the same prior advisor finding and remains applicable.
    • Recommendation: Add `--ignore-scripts` to this nested install if the renderer proof still works. If these packages require lifecycle scripts, document that requirement in the test, avoid inheriting ambient environment where possible, and consider reducing this job's permissions explicitly because this lane does not need repository write scopes or product secrets.
    • Evidence: The new test calls `host.command("npm", ["install", "--no-audit", "--no-fund", `openclaw@${openclawVersion}`, `@openclaw/whatsapp@${openclawVersion}`], { inheritEnv: true, ... })`; the workflow root install uses `npm ci --ignore-scripts`, but that protection does not apply to this later package install.

🌱 Nice ideas

  • None.
Consider writing more tests for
  • **Runtime validation** — Run the `whatsapp-qr-compact-e2e` regression lane and confirm it invokes `test/e2e-scenario/live/whatsapp-qr-compact.test.ts` instead of the deleted shell script.. Static workflow contract coverage is useful, but this change crosses GitHub Actions YAML, root dependency installation, Vitest project selection, live-scenario fixture artifact plumbing, npm package resolution, and the real upstream renderer. Behavioral runtime validation would provide confidence that the migrated lane works end to end.
  • **Runtime validation** — Confirm the migrated lane uploads `scenario.json`, `versions.json`, `baseline-dimensions.json`, `patched-dimensions.json`, and shell probe stdout/stderr/result artifacts under `e2e-artifacts/vitest/whatsapp-qr-compact/`.. Static workflow contract coverage is useful, but this change crosses GitHub Actions YAML, root dependency installation, Vitest project selection, live-scenario fixture artifact plumbing, npm package resolution, and the real upstream renderer. Behavioral runtime validation would provide confidence that the migrated lane works end to end.
  • **Runtime validation** — Confirm the live scenario fails when the baseline renderer is no longer oversized and when the preload does not reduce the QR row count.. Static workflow contract coverage is useful, but this change crosses GitHub Actions YAML, root dependency installation, Vitest project selection, live-scenario fixture artifact plumbing, npm package resolution, and the real upstream renderer. Behavioral runtime validation would provide confidence that the migrated lane works end to end.
  • **Runtime validation** — Add a workflow or static contract check that the nested renderer package install uses `--ignore-scripts`, or that this job has explicitly reduced permissions if lifecycle scripts must remain enabled.. Static workflow contract coverage is useful, but this change crosses GitHub Actions YAML, root dependency installation, Vitest project selection, live-scenario fixture artifact plumbing, npm package resolution, and the real upstream renderer. Behavioral runtime validation would provide confidence that the migrated lane works end to end.
  • **Acceptance clause:** Refs Adopt Vitest fixtures as the E2E scenario execution model #4941 / Refs [DGX Spark][CLI&UX] WhatsApp QR code renders too large in terminal — fills entire screen, impossible to scan #4522 — add test evidence or identify existing coverage. The PR body references these issues, but no linked issue bodies or comments were provided in deterministic context, so there were no literal issue acceptance clauses to verify beyond the PR body's migration claims.
Since last review details

Current findings:

  • Renderer proof install still runs package lifecycle scripts (test/e2e-scenario/live/whatsapp-qr-compact.test.ts:112): The migrated live test installs exact OpenClaw and WhatsApp package versions, but the nested `npm install` still omits `--ignore-scripts` and runs with `inheritEnv: true`. That preserves the deleted shell behavior, but it means lifecycle scripts from `openclaw`, `@openclaw/whatsapp`, or transitive dependencies can execute on the GitHub runner inside a workflow that has write-scoped permissions at the workflow level. This is the same prior advisor finding and remains applicable.
    • Recommendation: Add `--ignore-scripts` to this nested install if the renderer proof still works. If these packages require lifecycle scripts, document that requirement in the test, avoid inheriting ambient environment where possible, and consider reducing this job's permissions explicitly because this lane does not need repository write scopes or product secrets.
    • Evidence: The new test calls `host.command("npm", ["install", "--no-audit", "--no-fund", `openclaw@${openclawVersion}`, `@openclaw/whatsapp@${openclawVersion}`], { inheritEnv: true, ... })`; the workflow root install uses `npm ci --ignore-scripts`, but that protection does not apply to this later package install.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

@wscurran wscurran added area: e2e End-to-end tests, nightly failures, or validation infrastructure integration: whatsapp WhatsApp integration or channel behavior refactor PR restructures code without intended behavior change labels Jun 10, 2026
@wscurran

Copy link
Copy Markdown
Contributor

Base automatically changed from codex/e2e-simplify-migration-tracking to main June 10, 2026 20:53
cv pushed a commit that referenced this pull request Jun 11, 2026
## Summary
Migrate `test/e2e/test-whatsapp-qr-compact-e2e.sh` to a focused Vitest
renderer proof and retire the shell script.

Refs #5098
Refs #4522
Supersedes the simplified subset of #5146.

## Contract mapping
- Legacy assertion: bundled OpenClaw / `@openclaw/whatsapp` version is
resolved from `Dockerfile.base`.
- Replacement: `test/e2e-scenario/live/whatsapp-qr-compact.test.ts`
reads `ARG OPENCLAW_VERSION` and installs `openclaw@<version>` plus
`@openclaw/whatsapp@<version>`.
  - Boundary preserved: pinned upstream renderer package boundary.
- Legacy assertion: the WhatsApp plugin channel-login path renders
through `renderQrTerminal`.
- Replacement: the Vitest test scans installed `@openclaw/whatsapp/dist`
for `renderQrTerminal` before measuring output.
  - Boundary preserved: real upstream plugin distribution path.
- Legacy assertion: baseline QR without the NemoClaw preload is
oversized.
- Replacement: the Vitest test runs the real `renderQrTerminal(qr)`
probe without `NODE_OPTIONS` and requires rows >=
`WHATSAPP_QR_OVERSIZE_MIN_ROWS` (default 50).
  - Boundary preserved: real OpenClaw media-runtime renderer.
- Legacy assertion: QR with the NemoClaw compact preload is
scan-friendly and smaller.
- Replacement: the Vitest test reruns the same probe with
`NODE_OPTIONS=--require
nemoclaw-blueprint/scripts/whatsapp-qr-compact.js`, requires rows <=
`WHATSAPP_QR_COMPACT_MAX_ROWS` (default 40), and requires shrinkage
versus baseline.
  - Boundary preserved: real preload injection and renderer behavior.

## Simplicity check
- Test shape: focused Vitest test with local temp-dir helpers.
- New shared helpers: none.
- New framework/registry/ledger: none.
- No WhatsApp account, phone scan, sandbox, Docker, or `NVIDIA_API_KEY`
required; the renderer proof preserves the legacy contract.
- Workflow changes: existing `whatsapp-qr-compact-e2e` regression lane
now invokes the Vitest test instead of the deleted shell script.

## Verification
- `NEMOCLAW_RUN_E2E_SCENARIOS=1 npx vitest run --project
e2e-scenarios-live test/e2e-scenario/live/whatsapp-qr-compact.test.ts
--silent=false --reporter=default`
- `npx vitest run --project cli test/whatsapp-qr-compact.test.ts
test/regression-e2e-workflow.test.ts test/e2e-script-workflow.test.ts
--silent=false --reporter=default`
- `git diff --check`

Note: full pre-push CLI test started but was killed by signal 9 on this
host; the narrow required checks above passed.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Strengthened WhatsApp QR compact mode validation with comprehensive
E2E testing of terminal output dimensions and scan-frame compatibility,
including baseline and preload comparisons.

* **Chores**
* Modernized test infrastructure and CI/CD workflows, updated
documentation references, and removed legacy test scripts.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@cv cv added the v0.0.64 label Jun 11, 2026
@cv

cv commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by #5187. The focused WhatsApp compact QR Vitest migration has landed there, #5098 now records test/e2e/test-whatsapp-qr-compact-e2e.sh as converted by #5187, and this draft branch also carries older migration-ledger/docs/advisor churn that has been overtaken by later merged work. Any remaining useful idea should come back as a fresh, narrow PR against current main.

@cv cv closed this Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure integration: whatsapp WhatsApp integration or channel behavior refactor PR restructures code without intended behavior change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants