(MOT-3250) feat(harness): test first Console message in quickstart

[ytallo]

Summary

• document ANTHROPIC_API_KEY as the quickstart credential source and guide users to Claude Sonnet 5
• drive the published Console with Playwright, send the first message, reload the conversation, and verify durable Harness completion
• record structured browser/terminal evidence and a secret-safe MP4 for release reporting
• install the required Console, Chromium, and video dependencies in the quickstart GitHub Actions job

Why

The quickstart previously proved installation, worker registration, and Console availability, but it could pass without proving that a new user could complete the first model-backed interaction. This extends the check through the first successful Console conversation while keeping credentials out of persisted configuration and uploaded artifacts.

Impact

A passing quickstart now demonstrates the documented environment-variable setup, exact Sonnet 5 selection, an exact QUICKSTART_OK response, persistence after reload, and a durable completed Harness turn.

Paired delivery

Slack thread updates and verified MP4 upload are implemented in iii-hq/release-control#18. Release Control must be deployed before end-to-end Slack reporting becomes active.

Validation

• clean Docker reproduction against latest/latest: iii 0.22.1, Harness 1.8.1, Console 1.9.3
• Playwright completed the Sonnet 5 conversation and reload successfully
• terminal status reached completed with expects_wake=false
• artifact credential scan passed
• ShellCheck passed
• actionlint passed
• Playwright test collection passed
• git diff --check passed

Fixes MOT-3250

Summary by CodeRabbit

• New Features
  • Quickstart now supports selecting Claude Sonnet 5 in the Console, sending a message, and verifying conversation persistence after reload.
  • Automated checks capture browser, terminal, video, and Slack-compatible evidence.
• Documentation
  • Updated setup instructions to use an Anthropic API key through the engine environment.
  • Added guidance for Playwright, FFmpeg, ports, release channels, and validation requirements.
• Bug Fixes
  • Improved validation, cleanup, artifact handling, and credential-safety checks for quickstart runs.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
workers	Ready	Preview	Aug 14, 2026 1:46am
workers-tech-spec	Ready	Preview	Aug 14, 2026 1:46am

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a271e563-8a2d-4177-8904-9f801c340878

📥 Commits

Reviewing files that changed from the base of the PR and between 4392320 and 515acd5.

📒 Files selected for processing (6)

• .github/workflows/harness-quickstart.yml
• harness/README.md
• harness/tests/quickstart/README.md
• harness/tests/quickstart/console-first-message.spec.ts
• harness/tests/quickstart/playwright.config.ts
• harness/tests/quickstart/run-ci.sh

---

📝 Walkthrough

Walkthrough

The Harness quickstart now runs an Anthropic Claude Sonnet 5 conversation through Console with Playwright, verifies reload persistence and durable completion, and publishes browser, terminal, video, and Slack evidence after credential scanning.

Changes

Anthropic Console quickstart

Layer / File(s)	Summary
Quickstart setup and validation contract .github/workflows/harness-quickstart.yml, harness/README.md, harness/tests/quickstart/README.md, harness/tests/quickstart/playwright.config.ts, harness/tests/quickstart/run-ci.sh	The workflow and documentation define Anthropic credential handling, Playwright dependencies, Claude Sonnet 5 selection, and execution prerequisites.
Console conversation and durable completion harness/tests/quickstart/console-first-message.spec.ts, harness/tests/quickstart/run-ci.sh	The validator selects Claude Sonnet 5, sends the quickstart prompt, verifies the response and reload persistence, and waits for durable Harness completion.
Evidence, secret scanning, and artifact publication harness/tests/quickstart/run-ci.sh, .github/workflows/harness-quickstart.yml, harness/tests/quickstart/README.md	The validator records browser, terminal, video, and Slack evidence, scans artifacts for credentials, and uploads the expanded execution results.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Validator
  participant Anthropic
  participant Playwright
  participant Console
  participant Harness
  Validator->>Anthropic: Poll for Claude Sonnet 5
  Validator->>Playwright: Run Console conversation
  Playwright->>Console: Select model and send prompt
  Console-->>Playwright: Return response and session ID
  Playwright->>Console: Reload persisted conversation
  Console-->>Playwright: Hydrate conversation messages
  Validator->>Harness: Poll session status
  Harness-->>Validator: Return durable completion

Loading

Possibly related PRs

• iii-hq/workers#617: Modifies the Harness quickstart for Console-based live model validation and evidence handling.
• iii-hq/workers#619: Changes the Harness quickstart workflow and validator provider flow.
• iii-hq/workers#679: Modifies the same quickstart validation and evidence paths.

Poem

A rabbit checks the Console bright,
Claude replies with text just right.
Playwright saves each trace and sign,
Secrets stay outside the evidence line.
Durable turns complete on time.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/quickstart-console-test

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

skill-check — worker

0 verified, 59 skipped (no docs/).

Layer	Result
structure	✓
vale	✓
ai	✓
render	✓

Four for four. Nicely done.