test(e2e): restore staging Launchable checkpoint by jyaunches · Pull Request #11270 · NVIDIA/NemoClaw

jyaunches · 2026-09-09T03:12:52Z

Summary

Restores the general staging Brev Launchable workflow as a host-side Vitest control-plane checkpoint. The trusted lane resolves the latest staging handoff, creates a uniquely owned workspace, records its control-plane identity, and requires ID-bound deletion with confirmed absence.

The existing live-test path is retained to carry its assertion budget without increasing the live E2E inventory. The workflow, target ID, title, artifacts, and behavior are general staging Launchable coverage and no longer run the issue #9880 product scenario.

Changes

Rename the trusted issue workflow to the general staging Launchable workflow.
Reduce the existing live controller to the last proven passing boundary.
Remove inference credentials, remote execution, onboarding, and product assertions.
Record the exact handoff, Launchable, and workspace control-plane identity.
Retain independent cleanup and artifact upload.

Verification

Brev fixture tests: 28 passed
Semantic E2E phase coverage passed
Vitest project membership passed
Repository checks and assertion ratchet passed
actionlint passed
Commit and push hooks passed

Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

Tests
- Updated staging end-to-end validation to verify control-plane launchability and successful workspace completion.
- Added corresponding fast-test coverage to maintain parity with the live staging check.
- Expanded workspace lifecycle checks to confirm readiness, build completion, and running status.
- Adjusted assertion budgets to reflect the revised validation scenarios.
- Evidence artifacts now capture staging control-plane verification results.
Chores
- Refined staging test execution settings, time limits, cleanup behavior, and concurrency controls for more consistent runs.

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

coderabbitai · 2026-09-09T03:13:19Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 90025ec6-cb17-4d01-b87d-7ad8b3a559cd

📥 Commits

Reviewing files that changed from the base of the PR and between 8288df5 and a389255.

📒 Files selected for processing (2)

ci/e2e-assertion-budget.json
test/e2e/mock-parity.json

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Walkthrough

Walkthrough

The staging E2E test now performs a Launchable control-plane checkpoint. The workflow uses staging-specific paths and evidence names. Assertion budgets, workspace assertions, and mock parity mappings were updated.

Changes

Staging Launchable control-plane flow

Layer / File(s)	Summary
Control-plane checkpoint test `test/e2e/live/issue-9880-staging-launchable.test.ts`	The test resolves the staging handoff, creates a uniquely named workspace, records identity data, registers cleanup, and marks the checkpoint passed.
Staging workflow execution `.github/workflows/staging-launchable-full.yaml`	The workflow runs the control-plane test with updated concurrency, timeouts, staging paths, cleanup, and evidence artifact names.
Validation and parity alignment `test/e2e/support/brev-launchable-fixture.test.ts`, `ci/e2e-assertion-budget.json`, `test/e2e/mock-parity.json`	The workspace fixture checks the returned workspace state. Assertion budgets and the live-to-fast parity mapping were updated.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 0a6d0

This replaces the prior staging scenario with a Launchable control-plane checkpoint that creates, records, and deletes a staging workspace. No concrete current-head merge-blocking risk remains.

Suggested reviewers: ericksoa, cv

Sequence Diagram(s)

sequenceDiagram
  participant LiveTest
  participant StagingHandoff
  participant StagingWorkspace
  participant CheckpointArtifact
  LiveTest->>StagingHandoff: Resolve staging handoff
  LiveTest->>StagingWorkspace: Create workspace and register cleanup
  LiveTest->>CheckpointArtifact: Write identity data
  LiveTest->>LiveTest: Mark checkpoint passed

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: restoring the staging Launchable checkpoint workflow and test coverage.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch test/staging-launchable-control-plane-checkpoint

_{Comment @coderabbitai help to get the list of available commands.}

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

github-code-quality · 2026-09-09T03:19:13Z

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall line coverage in commit 0a6d0dd in the test/staging-launcha... branch remains at 96%, unchanged from commit 091be16 in the main branch.

TypeScript / code-coverage/cli

The overall line coverage in commit 0a6d0dd in the test/staging-launcha... branch remains at 83%, unchanged from commit 729aecc in the main branch.

Show a line coverage summary of the most impacted files.

File	`main` 729aecc	`test/staging-launcha...` 0a6d0dd	+/-
`src/lib/onboard...dns-fallback.ts`	85%	65%	-20%
`src/lib/onboard...wn-authority.ts`	98%	91%	-7%
`src/lib/onboard...box-prebuild.ts`	94%	91%	-3%
`src/lib/inferen...file/cleanup.ts`	82%	81%	-1%
`src/lib/onboard...ce-lifecycle.ts`	84%	83%	-1%
`src/lib/inference/health.ts`	89%	90%	+1%
`src/lib/onboard...u-patch-mode.ts`	88%	90%	+2%
`src/lib/onboard...ntime-marker.ts`	65%	71%	+6%
`src/lib/onboard...eate-journal.ts`	80%	87%	+7%
`src/lib/inferen...ycle-adapter.ts`	77%	86%	+9%

_{Updated September 09, 2026 13:33 UTC}

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

…le-control-plane-checkpoint

github-actions · 2026-09-09T13:38:24Z

PR Review Advisor finished for commit 0a6d0dd. Include the Advisor findings in the complete PR feedback collection. Verify and group valid findings before repair.

All previous runs

## Outcome The general staging Launchable E2E now succeeds only after `brev exec <owned-workspace-id> true` proves remote execution readiness. Failed probes remain bounded and retain only the final bounded, redacted diagnostic, while the existing control-plane checkpoint and confirmed cleanup remain intact. ## Reason The control-plane checkpoint merged in PR #11270 proves workspace creation but does not prove that Brev can execute a remote command. This is the first deferred capability slice in epic #11250. ### Related issues Part of #11250 ## Changes - Add a distinct remote execution readiness phase after workspace creation and before the retained control-plane checkpoint. - Reuse `BrevLaunchableFixture.waitForExec` and its exact-owned-ID, bounded retry, redaction, and diagnostic contracts. - Add fast evidence that readiness succeeds after two failed probes and uses only the owned workspace ID. - Align workflow and artifact names with remote execution readiness and record the retry policy. - Keep runtime identity, onboarding, inference credentials, OpenClaw execution, and issue #9880 classification deferred. ## Verification - `npx vitest run --project e2e-support test/e2e/support/brev-launchable-fixture.test.ts` — 29 tests passed. - `npm run test:changed` — 45 integration tests and 29 E2E-support tests passed. - `npm run test:e2e-phases:check` — 134 tests across 88 files passed semantic phase validation. - `npm run test:projects:check` — 2,632 candidates have exact Vitest project membership. - `npm run e2e:assertions:check` — assertion ratchet passed with 1,800 direct live assertions. - `npm run checks:repository` — repository checks passed. - `actionlint .github/workflows/staging-launchable-full.yaml` — passed. - `node --experimental-strip-types --no-warnings scripts/checks/e2e-mock-parity.mts --base origin/main --head HEAD` — mock/live parity passed. - Normal commit and push hooks passed after local build prerequisites were restored. - The diff contains no secrets, API keys, or credentials. ## Review notes This changes a credentialed, billable remote-execution workflow. Brev credentials remain confined to the workflow-specific `HOME`, command probes persist no raw attempt artifacts, diagnostics are bounded and redacted before persistence or error reporting, operations remain bound to the persisted owned ID, and cleanup still requires two confirmed absence observations. --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>  ## Summary by CodeRabbit - **New Features** - Added validation that remote execution becomes ready after workspace creation. - Added retry handling for temporary execution-readiness probe failures. - **Bug Fixes** - Improved staging verification timing and readiness checks. - Updated monitoring and evidence labeling to reflect remote execution readiness. - **Documentation** - Documented retry timing, conditions, safety guarantees, and diagnostic evidence. - **Tests** - Added coverage for readiness after temporary probe failures. - Added readiness evidence and classification checks for staging end-to-end runs.  --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

jyaunches added 2 commits September 8, 2026 23:02

test(e2e): restore staging Launchable checkpoint

afcd09c

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

fix(e2e): construct staging fixture explicitly

bed7e5e

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

jyaunches self-assigned this Sep 9, 2026

jyaunches added the v0.0.121 label Sep 9, 2026

merge: update staging checkpoint from main

c49fa5e

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

test(e2e): cover staging checkpoint state

8288df5

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

github-actions Bot added v0.0.122 and removed v0.0.121 labels Sep 9, 2026

jyaunches added 2 commits September 9, 2026 08:44

merge: refresh staging checkpoint from main

a389255

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

Merge remote-tracking branch 'origin/main' into test/staging-launchab…

0a6d0dd

…le-control-plane-checkpoint

jyaunches merged commit a4e0605 into main Sep 9, 2026
46 of 47 checks passed

jyaunches deleted the test/staging-launchable-control-plane-checkpoint branch September 9, 2026 13:35

wscurran added the chore Build, CI, dependency, or tooling maintenance label Sep 9, 2026

This was referenced Sep 9, 2026

test(e2e): prove staging Brev exec readiness #11299

Closed

test(e2e): prove staging Brev exec readiness #11300

Closed

test(e2e): prove staging Brev exec readiness #11301

Merged

wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure labels Sep 9, 2026

jyaunches mentioned this pull request Sep 9, 2026

[Epic] Restore staging Brev Launchable E2E #11250

Open

11 tasks

Navigation Menu

Uh oh!

test(e2e): restore staging Launchable checkpoint - #11270

test(e2e): restore staging Launchable checkpoint#11270
jyaunches merged 6 commits into
mainfrom
test/staging-launchable-control-plane-checkpoint

jyaunches commented Sep 9, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

github-code-quality Bot commented Sep 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

jyaunches commented Sep 9, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

github-code-quality Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

TypeScript / code-coverage/plugin

TypeScript / code-coverage/cli

Uh oh!

Uh oh!

github-actions Bot commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jyaunches commented Sep 9, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading

github-code-quality Bot commented Sep 9, 2026 •

edited

Loading