Skip to content

feat(e2e): run shadow observations on ephemeral GitHub workers - #836

Merged
ytallo merged 2 commits into
mainfrom
feat/release-control-e2e-shadow
Aug 19, 2026
Merged

feat(e2e): run shadow observations on ephemeral GitHub workers#836
ytallo merged 2 commits into
mainfrom
feat/release-control-e2e-shadow

Conversation

@ytallo

@ytallo ytallo commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a trusted workflow_dispatch executor for Release Control shadow E2E campaigns
  • authenticate each execution with a single-use GitHub Actions OIDC admission before any model call
  • install the exact deployed Harness stack and harness-e2e independently from the Registry on an ephemeral GitHub runner
  • materialize and run the native e2e::* control plane, then upload a digest-verified terminal observation bundle
  • keep GitHub Artifact as the D0 retention boundary and preserve factual infrastructure failures

Safety

  • this workflow is observational and never changes deployment or promotion state
  • native GitHub reruns are rejected; an explicit Release Control rerun creates a new attempt
  • provider credentials stay in the protected harness-e2e-trusted environment
  • the target stack is re-pinned after installing the runner and verified against iii.lock

Validation

  • bash -n harness/tests/e2e/run-shadow-control-ci.sh
  • python3 .github/scripts/tests/test_harness_e2e_shadow_contract.py — 4 passed
  • python3 -m py_compile .github/scripts/harness_e2e_shadow_contract.py
  • workflow YAML parse
  • git diff --check

Related PRs and rollout order

  1. feat: add e2e observation contract harness-e2e#19 publishes the versioned observation contract and Registry runner.
  2. This PR provides the ephemeral GitHub execution environment.
  3. iii-hq/release-control#21 dispatches, admits, reconciles, persists, and displays the observations.

The Release Control feature flag remains off until all three changes are deployed and a Registry smoke succeeds.

Summary by CodeRabbit

  • New Features

    • Added an observe-only Harness E2E workflow for deployment shadow observations.
    • Added contract validation, request materialization, execution monitoring, and evidence packaging.
    • Added factual observation bundles with file manifests and SHA-256 digests.
    • Added isolated E2E control-plane execution with retries, timeouts, logging, and cleanup.
  • Documentation

    • Documented the deployment shadow observation workflow and its non-blocking behavior.
  • Tests

    • Added coverage for contract validation, catalog consistency, prerelease versions, workflow inputs, and evidence packaging.

@vercel

vercel Bot commented Aug 19, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
workers Ready Ready Preview Aug 19, 2026 11:51am
workers-tech-spec Ready Ready Preview Aug 19, 2026 11:51am

Request Review

@coderabbitai

coderabbitai Bot commented Aug 19, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d41c098c-bed6-45ed-8355-15829c98846e

📥 Commits

Reviewing files that changed from the base of the PR and between 1bea36f and 532c6c6.

📒 Files selected for processing (6)
  • .github/scripts/harness_e2e_shadow_contract.py
  • .github/scripts/tests/test_harness_e2e_shadow_contract.py
  • .github/scripts/tests/test_release_workflows.py
  • .github/workflows/harness-e2e-shadow.yml
  • harness/tests/e2e/README.md
  • harness/tests/e2e/run-shadow-control-ci.sh

📝 Walkthrough

Walkthrough

Adds an observe-only Harness E2E workflow. The change validates execution contracts, runs an isolated control plane, admits executions through Release Control, and packages factual observation evidence.

Changes

Shadow observation execution

Layer / File(s) Summary
Contract validation and request packaging
.github/scripts/harness_e2e_shadow_contract.py, .github/scripts/tests/test_harness_e2e_shadow_contract.py
Adds contract and scenario-catalog validation, request materialization, bundle packaging, CLI commands, and coverage for catalog drift, prerelease versions, and file digests.
Isolated E2E control plane
harness/tests/e2e/run-shadow-control-ci.sh
Creates an isolated engine environment, installs pinned workers, runs the contracted scenarios, polls execution status, retrieves results, and attempts archival.
Workflow admission and evidence
.github/workflows/harness-e2e-shadow.yml, .github/scripts/tests/test_release_workflows.py, harness/tests/e2e/README.md
Adds dispatch inputs, OIDC-based admission, conditional execution, evidence packaging, failure handling, workflow assertions, and documentation.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Workflow as harness-e2e-shadow.yml
  participant ReleaseControl
  participant Script as run-shadow-control-ci.sh
  participant Engine as iii engine
  participant Artifact as Observation bundle

  Workflow->>ReleaseControl: Validate dispatch and request admission with OIDC
  ReleaseControl-->>Workflow: Return admission result
  Workflow->>Script: Start contracted observe-only execution
  Script->>Engine: Install workers, submit plan, poll status
  Engine-->>Script: Return results and execution metadata
  Script-->>Workflow: Return logs and evidence files
  Workflow->>Artifact: Package and upload observation bundle
Loading

Poem

A rabbit checks each contract line,
Then hops through plans by seed and sign.
An engine wakes in a workspace bright,
OIDC guards the flight.
Digests gather, bundles grow—
Shadow runs report what they know.

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/release-control-e2e-shadow

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

skill-check — worker

0 verified, 61 skipped (no docs/).

Layer Result
structure
vale
ai
render

Four for four. Nicely done.

@ytallo ytallo added the no-ticket PR deliberately has no Linear ticket (bump/typo/CI-only) label Aug 19, 2026
@ytallo
ytallo marked this pull request as ready for review August 19, 2026 12:03
@ytallo
ytallo merged commit df11bfe into main Aug 19, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-ticket PR deliberately has no Linear ticket (bump/typo/CI-only)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant