ci(e2e): test staging Brev Launchable - #7270
Conversation
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a gated staging Brev Launchable E2E workflow that verifies immutable image identity, runs preinstalled tests remotely, records evidence, cleans up workspaces, and validates success and failure paths with focused tests. ChangesStaging Brev Launchable E2E
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Workflow
participant GitHub
participant Brev
participant SSH
participant FullE2E
Workflow->>GitHub: Dispatch qualification workflow
GitHub-->>Workflow: Return verified image manifest
Workflow->>Brev: Create and poll Launchable workspace
Brev-->>Workflow: Return ready workspace
Workflow->>SSH: Verify baked identity and run E2E
SSH->>FullE2E: Execute preinstalled full-e2e suite
FullE2E-->>SSH: Return pass marker and logs
Workflow->>Brev: Delete workspace and verify absence
Possibly related issues
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in commit d0c7ada in the TypeScript / code-coverage/cliThe overall coverage in commit d0c7ada in the Show a code coverage summary of the most impacted files.
Updated |
Security review: PASSThe focused nine-category review found no actionable security findings. The draft is safe to continue through CI and maintainer review; activation must remain disabled until the protected environment, standing Launchable, credentials, and Brev ownership are configured.
Files reviewed: |
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
Activation auditRead-only repository audit after opening this PR:
The authorized-PR trigger criterion is implemented: the trusted controller validates Do not add the enable variable until #7270 is approved and merged, the producer WIF pair is configured, the Brev owner accepts the contract, and the first protected run is scheduled. This audit does not claim that the stored credentials are valid or sufficiently scoped; the first protected real run must prove authentication, deployment, post-boot identity, full E2E, evidence upload, and verified deletion. |
Unrelated CI failure:
|
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
|
Security review refresh for exact head |
PR Review Advisor — InformationalAdvisor assessment: Informational / high confidence Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: 1 warning · 0 suggestionsWarningsWarnings do not block.
|
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
|
Audit-blocker update: the fresh exact-head run on #6830 still fails reviewed-npm-audit; its OpenClaw 2026.6.10 locked graph contains high brace-expansion GHSA-3jxr-9vmj-r5cp and critical tar GHSA-23hp-3jrh-7fpw (artifact 8478602808). Therefore #6830 is not yet a valid clean-audit unblock for this PR. #7270 remains intentionally unchanged; it should rebase and rerun only after the dependency owner lands a reviewed fix or a maintainer records an explicit security disposition. |
wscurran
added
area: ci
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/brev-launchable-e2e.test.ts`:
- Around line 172-185: Add a configurable fake baked-SHA value to the fixture
identity used by the boot scenario, set it to differ from the candidate SHA
while keeping imageId valid, and assert that run(boot.env) fails before invoking
preinstalled E2E. Update the stderr assertion as needed to specifically cover
the baked-SHA mismatch, while preserving the existing no-call and no-state
assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 7977e41b-47ff-4715-b417-149750cc4f42
📒 Files selected for processing (1)
test/brev-launchable-e2e.test.ts
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
|
Please remove the in-guest GCP image-ID lookup from this PR. Track immutable boot-image qualification in a separate issue if we still want that stronger platform contract. For #6943, a generic producer success signal is not sufficient. The lane must bind the exact producer run to the PR SHA. After the Launchable boots, the following checks are sufficient for this issue:
The previous live run showed that the default service-account token endpoint returns The producer result plus the post-boot exact-SHA checks prove the behavior required by #6943. They do not claim immutable cloud image-ID qualification. Please move that stronger claim and its infrastructure requirements to a separate issue. |
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
cjagwani
changed the title
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
prekshivyas
left a comment
prekshivyas
left a comment
There was a problem hiding this comment.
Reviewed exact head 664b422 against base c6f46d4. Security PASS across secrets, validation, authorization, dependencies, logging, data protection, configuration, tests, and system boundaries. The trusted workflow checkout, protected-environment secret guards, exact producer/receipt/boot SHA binding, bounded evidence, and verified cleanup all fail closed. Focused validation: 83 tests passed. No actionable findings. This approval does not waive the exact-SHA E2E authorization or #6943 real-Launchable activation evidence.
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
cv
left a comment
cv
left a comment
There was a problem hiding this comment.
Re-reviewed exact head 2aaef13 against current main 1bb15a0. Both base refreshes are Verified, conflict-free, and leave the nine-file effective E2E-hardening diff unchanged. The merged documentation-receipt workflow does not alter this PR's runtime semantics; prior product-scope and nine-category security PASS conclusions remain valid. Awaiting fresh GitHub-hosted gates.
<!-- markdownlint-disable MD041 --> ## Summary Adds the canonical dated `## v0.0.93` release entry to `docs/changelog/2026-07-23.mdx`. The entry records user-visible behavior, release validation, and documentation controls merged after `v0.0.92`, while preserving the pending DGX OS `7.6.x` Station Express qualification caveat. ## Changes - Adds the parser-safe dated release entry with a summary, grouped details, and published-route links. - Reconciles the `v0.0.92..origin/main` commit range with merged `v0.0.93` PRs. - Records that no-OTA DGX OS `7.6.x` passed bounded host preflight, while full Station Express end-to-end qualification remains pending. - Leaves existing product pages unchanged because the source PRs already document their supported behavior. ### Source summary - #7285 -> `docs/changelog/2026-07-23.mdx`: Records the existing-vLLM ownership choice and resumable Station handoff. - #7419 -> `docs/changelog/2026-07-23.mdx`: Records bounded no-OTA DGX OS `7.6.x` recognition and its pending end-to-end qualification. - #7268 -> `docs/changelog/2026-07-23.mdx`: Records optional Hugging Face authentication, output sanitization, and resumable HTTP `429` recovery. - #7442 -> `docs/changelog/2026-07-23.mdx`: Records clean SIGINT handling at hidden credential prompts. - #7299 -> `docs/changelog/2026-07-23.mdx`: Records Intel macOS rejection before ref resolution or network work. - #7296 -> `docs/changelog/2026-07-23.mdx`: Records the DGX Spark non-interactive local-vLLM selection order. - #7342 -> `docs/changelog/2026-07-23.mdx`: Records delegated protected E2E approvals in the grouped release-validation bullet. - #7373 -> `docs/changelog/2026-07-23.mdx`: Records base-image publication gating before final-main fanout. - #7388 -> `docs/changelog/2026-07-23.mdx`: Records semantic phase runtime summaries. - #7397 -> `docs/changelog/2026-07-23.mdx`: Records progress coverage hardening. - #7391 -> `docs/changelog/2026-07-23.mdx`: Records centralized larger-runner routing. - #7423 -> `docs/changelog/2026-07-23.mdx`: Records one retry for confirmed hosted-runner loss. - #7399 -> `docs/changelog/2026-07-23.mdx`: Records runner-comparison telemetry. - #7270 -> `docs/changelog/2026-07-23.mdx`: Records staging Brev Launchable validation. - #7426 -> `docs/changelog/2026-07-23.mdx`: Records filtering of irrelevant base-image run history. - #7333 -> `docs/changelog/2026-07-23.mdx`: Records aligned Quickstart platform guidance. - #7343 -> `docs/changelog/2026-07-23.mdx`: Records documentation-writer receipt collection. - #7400 -> `docs/changelog/2026-07-23.mdx`: Records the documentation-writer receipt requirement for docs-only PRs. - #7413 -> `docs/changelog/2026-07-23.mdx`: Records removal of redundant receipt PR metadata. - #7405 -> `docs/changelog/2026-07-23.mdx`: Records corrected inference CLI references. - #7389 -> `docs/changelog/2026-07-23.mdx`: Records completion of the v0.0.91 documentation audit. `#7384` is an internal refactor with no intended runtime behavior change. `#7401` updates internal CodeQL Actions dependencies. `#7376` is already contained in `v0.0.92`, so it is outside the release-entry scan range despite its retained planning label. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [x] Existing tests cover changed behavior — justification: `test/changelog-docs.test.ts` validates dated changelog structure, SPDX syntax, and version headings. - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Evidence: Reviewed `docs/changelog/2026-07-23.mdx` against `WRITING.md`, `docs/CONTRIBUTING.md`, `docs/.docs-skip`, `docs/index.yml`, the six user-visible source PRs, and the remaining grouped release commits. The review corrected an ambiguous qualification claim, confirmed all published routes, preserved the DGX OS `7.6.x` caveat, and found no remaining action. - Agent: Codex Desktop <!-- docs-review-head-sha: ec0a866 --> <!-- docs-review-agents-blob-sha: 9c9b36d --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable. This PR does not change `scripts/prepare-dgx-station-host.sh`. - Station profile/scenario: Not applicable. - Result: Not applicable. - Supporting evidence: Not applicable. ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npx vitest run test/changelog-docs.test.ts`: 1 file and 6 tests passed. - [ ] Applicable broad gate passed — Not applicable to one native changelog file. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — completed with 0 errors and 2 existing Fern warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) — not applicable because native changelog entries use a parser-safe MDX SPDX comment without frontmatter. --- Signed-off-by: Carlos Villela <cvillela@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added the v0.0.93 changelog covering onboarding and validation improvements. * Documented support for additional DGX Station Express workstation releases and clearer handling of existing vLLM workloads. * Added guidance for optional Hugging Face authentication, resumable rate-limit recovery, and DGX Spark provider selection. * Clarified installer behavior on Intel macOS, release validation requirements, hosted-runner retries, documentation checks, and supported CLI quickstart paths. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
5 participants
Summary
Add one explicit, non-blocking staging Brev Launchable E2E lane targeted for
.92. The lane binds an exact candidate SHA to the exactnemoclaw-imageproducer run, verifies the guest's provisioned SHA and clean NemoClaw checkout after boot, reuses the existing full E2E in preinstalled mode, and verifies workspace deletion. It does not gate production Launchable promotion or claim immutable cloud image-ID qualification.Related Issue
Part of #6943.
Changes
brevdev/nemoclaw-imageworkflow fix: detect sandbox context in status command #80 and validate that its receipt binds the exact producer run to the candidate SHA./etc/nemoclaw/provision.json, the checkout HEAD, and a clean working tree all match the exact candidate.preinstalled-launchablesetup branch.lane.log,qualification.json,full-e2e.log, andcleanup.json.The implementation is 628 added lines: workflow 67, runner 227, existing full-E2E adaptation 43, new lifecycle tests 225, and narrow existing-validator/test changes 66.
Type of Change
Quality Gates
e2e.yamlremains the release-E2E source of truth.Documentation Writer Review
no-docs-neededDGX Station Hardware Evidence
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailable4/4focused lifecycle tests,79/79relevant workflow-boundary tests,10/10Docker-auth boundary tests, and the exact affected E2E-support shard (154/154) passed.npm run docsbuilds without warnings (doc changes only)Signed-off-by: Charan Jagwani cjagwani@nvidia.com
Summary by CodeRabbit