Skip to content

(MOT-4277) feat(harness): preview local E2E results - #648

Merged
ytallo merged 1 commit into
mainfrom
feat/harness-e2e-local-dashboard
Jul 30, 2026
Merged

(MOT-4277) feat(harness): preview local E2E results#648
ytallo merged 1 commit into
mainfrom
feat/harness-e2e-local-dashboard

Conversation

@ytallo

@ytallo ytallo commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a local dashboard command that imports existing Harness E2E results.json files and serves the production dashboard shell on port 4173
  • retain an idempotent local execution history with full scenario metrics, failures, transcripts, tokens, function calls, and multi-scenario reports
  • distinguish local data in the UI, suppress sample fixtures, hide unavailable workflow links, and document the local workflow

Impact

Developers can inspect and compare local Harness E2E executions with the same dashboard used by GitHub Pages without running the tests again or publishing artifacts.

Validation

  • PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=.github/scripts python3 .github/scripts/tests/test_serve_harness_e2e_dashboard.py
  • node --test .github/benchmark-site/*.test.cjs
  • git diff --check
  • imported and served 11 existing local reports; verified the execution manifest and retained detail over HTTP

Refs MOT-4277

Summary by CodeRabbit

  • New Features

    • Added a local dashboard option for viewing imported E2E reports without rerunning scenarios.
    • Supports importing individual reports or directories, persistent local history, reset options, and configurable server settings.
    • Added a Local execution filter and clearer empty states for locally imported data.
    • Added safeguards to hide unavailable or unsafe workflow links.
  • Documentation

    • Added instructions for running the local dashboard and importing reports.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7d391fe9-1045-40e4-b41e-d8e009c911b8

📥 Commits

Reviewing files that changed from the base of the PR and between 7ebcd88 and 3f97ce0.

📒 Files selected for processing (10)
  • .github/benchmark-site/README.md
  • .github/benchmark-site/execution-data.js
  • .github/benchmark-site/execution-data.test.cjs
  • .github/benchmark-site/execution.js
  • .github/benchmark-site/index.html
  • .github/benchmark-site/overview.js
  • .github/scripts/publish_harness_e2e_dashboard.py
  • .github/scripts/serve_harness_e2e_dashboard.py
  • .github/scripts/tests/test_serve_harness_e2e_dashboard.py
  • harness/tests/e2e/README.md

📝 Walkthrough

Walkthrough

Adds a local Harness E2E dashboard server that imports reports, accumulates execution history, serves generated sites, and distinguishes local data in the dashboard UI through persisted mode metadata.

Changes

Local dashboard workflow

Layer / File(s) Summary
Report validation and site preparation
.github/scripts/serve_harness_e2e_dashboard.py, .github/scripts/tests/test_serve_harness_e2e_dashboard.py
Validates reports, discovers unique inputs, derives repository metadata, prepares dashboard directories, and tests discovery behavior.
Dashboard import and local serving
.github/scripts/serve_harness_e2e_dashboard.py, .github/scripts/tests/test_serve_harness_e2e_dashboard.py, .github/benchmark-site/README.md, harness/tests/e2e/README.md
Stages and imports reports, regenerates dashboard data, serves the generated site, and tests idempotent and cumulative history behavior.
Mode-aware publishing and dashboard UI
.github/scripts/publish_harness_e2e_dashboard.py, .github/benchmark-site/execution-data.js, .github/benchmark-site/execution-data.test.cjs, .github/benchmark-site/index.html, .github/benchmark-site/overview.js, .github/benchmark-site/execution.js
Persists local or published mode, adds local filtering, updates local-data labels and links, and hides unsafe workflow URLs.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant build_local_dashboard
  participant import_report
  participant publish
  participant StaticHTTPServer
  CLI->>build_local_dashboard: report inputs and site options
  build_local_dashboard->>import_report: validated report paths
  import_report->>publish: local execution metadata and collected outputs
  build_local_dashboard-->>CLI: imported execution identifiers
  CLI->>StaticHTTPServer: serve generated site
Loading

Possibly related PRs

  • iii-hq/workers#637: Introduces the benchmark dashboard extended by this local-import workflow.
  • iii-hq/workers#644: Provides related execution-history and published-data behavior.

Poem

I hop through reports, neat and bright,
Importing runs by candlelight.
Local histories softly grow,
Safe links bloom and statuses show.
The dashboard hums—what a delight!

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/harness-e2e-local-dashboard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
workers Ready Ready Preview Jul 30, 2026 11:03pm
workers-tech-spec Ready Ready Preview Jul 30, 2026 11:03pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

skill-check — worker

0 verified, 50 skipped (no docs/).

Layer Result
structure
vale
ai
render

Four for four. Nicely done.

@ytallo
ytallo marked this pull request as ready for review July 30, 2026 23:13
@ytallo
ytallo merged commit f679b3b into main Jul 30, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant