test(e2e): cover v0.0.55 gateway upgrades - #6397
Merged
Merged
Conversation
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe ChangesLegacy matrix and test payload updates
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in the Show a code coverage summary of the most covered files.
TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most covered files.
Updated |
Contributor
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Contributor
E2E Target RecommendationRequired E2E targets: Dispatch required E2E targets:
Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
Contributor
E2E Target Results — ❌ Some jobs failedRun: 28872636484
|
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Contributor
PR Review Advisor — No blocking findingsMerge posture: No blocking advisor findings Action checklist
Test follow-ups to resolve or justifyIf these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.
This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision. |
Contributor
E2E Target Results — ❌ Some jobs failedRun: 28872946003
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
test/e2e/live/openshell-gateway-upgrade.test.ts (1)
163-175: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAmbiguous multi-ARG fallback produces a misleading error.
When
arg_markershas more than one entry (e.g. a multi-stage Dockerfile re-declaringARG OPENCLAW_VERSION=per stage), the code silently falls through to theMIN_VERgate branch. If that gate marker is also absent (likely for an ARG-based fixture like the new v0.0.55 shape), the resulting error"old OpenClaw version gate not found"doesn't reflect the real cause (ambiguous ARG count vs. a genuinely missing gate), which will confuse whoever debugs a future CI failure here.♻️ Suggested clarification
if injection not in text: arg_markers = [ line for line in text.splitlines(keepends=True) if line.startswith("ARG OPENCLAW_VERSION=") ] if len(arg_markers) == 1: marker = arg_markers[0] text = text.replace(marker, marker + "\n" + injection, 1) + elif len(arg_markers) > 1: + raise SystemExit(f"{path}: multiple 'ARG OPENCLAW_VERSION=' lines found, expected exactly one") else: marker = "RUN set -eu; \\\n MIN_VER=$(grep -m 1 'min_openclaw_version'" if marker not in text: raise SystemExit(f"{path}: old OpenClaw version gate not found") text = text.replace(marker, injection + marker, 1)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e/live/openshell-gateway-upgrade.test.ts` around lines 163 - 175, The fallback in the Dockerfile injection logic is too ambiguous: in the branch that handles OpenClaw version markers, a multi-entry arg_markers case currently falls through to the MIN_VER gate path and can raise a misleading “old OpenClaw version gate not found” error. Update the logic around the injection handling so it distinguishes between a single ARG match, multiple ARG matches, and a missing MIN_VER gate, and make the error message in the relevant test helper or fixture code explicitly report the real condition using the arg_markers and marker checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@test/e2e/live/openshell-gateway-upgrade.test.ts`:
- Around line 163-175: The fallback in the Dockerfile injection logic is too
ambiguous: in the branch that handles OpenClaw version markers, a multi-entry
arg_markers case currently falls through to the MIN_VER gate path and can raise
a misleading “old OpenClaw version gate not found” error. Update the logic
around the injection handling so it distinguishes between a single ARG match,
multiple ARG matches, and a missing MIN_VER gate, and make the error message in
the relevant test helper or fixture code explicitly report the real condition
using the arg_markers and marker checks.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: dd7d3ced-ddcc-4e59-b651-23648030ef7a
📒 Files selected for processing (1)
test/e2e/live/openshell-gateway-upgrade.test.ts
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Contributor
E2E Target Results — ❌ Some jobs failedRun: 28873234163
|
Contributor
PR Review Advisor (Nemotron Ultra) — No blocking findingsMerge posture: No blocking advisor findings Action checklist
Findings index
Review findings by urgency: 0 required fixes, 0 items to resolve/justify, 4 in-scope improvements
|
21 tasks
Contributor
E2E Target Results — ✅ All requested jobs passedRun: 28873726294
|
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
cv
requested changes
Jul 7, 2026
cv
left a comment
cv
left a comment
Collaborator
There was a problem hiding this comment.
Maintainer review: the two-row live run is valuable, but exact head is not approval-ready because the source/evidence assertions can prove the wrong thing. In openshell-gateway-upgrade.test.ts, a missing /home/cvillela/.nemoclaw/source/.git yields empty stdout with status 0, and git ls-remote also returns status 0 with empty stdout for an unknown tag; the current trim-to-trim comparison therefore accepts empty === empty. Please require both actual and expected values to match a full 40-hex commit SHA before comparing them. Also make the v0.0.55 base-image evidence truthful: the artifact records digest 104151ff... while the build log used 10433a8... and the wrapper log is empty. Pass the recognized NEMOCLAW_SANDBOX_BASE_IMAGE_REF into the legacy payload or record/assert the actual resolved digest. Then rerun jobs=openshell-gateway-upgrade at the new exact head and obtain an exact-head merge_as_is advisor result.
Closed
27 tasks
wscurran
added
area: ci
Contributor
E2E Target Results — ✅ All requested jobs passedRun: 28956561506
|
Validate fixture overrides before any test side effects. Fetch each historical installer by an exact commit and pinned SHA-256 so the evidence fails closed. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Exercise every historical fixture validation boundary. Make the base-image digest explicit in each workflow matrix row. Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
E2E Target Results —
|
| Job | Result |
|---|---|
| openshell-gateway-upgrade |
Collaborator
|
Maintainer disposition for exact head
Focused support and workflow-contract tests pass, all commits are GitHub-Verified, and the exact-head two-row live run is in progress. |
Contributor
E2E Target Results — ✅ All requested jobs passedRun: 28962268680
|
cv
approved these changes
Jul 8, 2026
cv
left a comment
cv
left a comment
Collaborator
There was a problem hiding this comment.
Exact-head approval for fcc60076408b6fe6746f6d92094eafd4306d880a.
The prior source-head and base-image-evidence concerns are resolved. Historical fixture inputs now fail closed before side effects; v0.0.36 and v0.0.55 installers are pinned by verified commit and SHA-256; the sandbox-base digest is explicit per matrix row; and focused tests cover every validation boundary.
Verification: all 41 PR checks are green, all 8 commits are GitHub-Verified with DCO, CodeRabbit has no unresolved major/critical findings, both review advisors say merge_as_is, and exact-head live run 28962268680 passed both OpenShell gateway upgrade rows (v0.0.36 and v0.0.55).
21 tasks
cv
pushed a commit
that referenced
this pull request
Jul 9, 2026
<!-- markdownlint-disable MD041 --> ## Summary Adds the pre-tag v0.0.79 release notes entry to `docs/about/release-notes.mdx` so the release plan can be generated after docs merge. The entry summarizes the merged v0.0.79 release train across inference, diagnostics, runtime hardening, policies, onboarding recovery, and release validation. ## Changes - Added the v0.0.79 release notes section with linked follow-up documentation for OpenRouter onboarding, managed vLLM changes, completion and logging, Deep Agents runtime limits, policy updates, onboarding recovery, and release validation. - Source summary: - #6461 -> `docs/about/release-notes.mdx`: Documents OpenRouter onboarding support and links to inference/provider references. - #6271 and #6272 -> `docs/about/release-notes.mdx`: Documents shell completion and structured logging highlights. - #6465, #6539, #6570, and #6528 -> `docs/about/release-notes.mdx`: Documents status route-drift, orphaned sandbox, gateway cleanup, and DGX Spark express-install diagnostics. - #6523, #6551, #6484, #6488, #6324, and #6542 -> `docs/about/release-notes.mdx`: Documents managed vLLM, Qwen3.6 tool parser, compaction, and timeout/readiness improvements. - #6559, #6538, #6560, #6568, #6552, #6567, and #6587 -> `docs/about/release-notes.mdx`: Documents runtime, credential, proxy, PID namespace, TOML, and provider-state hardening. - #6541, #5415, #6246, #6496, and #6573 -> `docs/about/release-notes.mdx`: Documents GitHub policy, Gmail policy, MCP allowlist, WhatsApp, and messaging-variant updates. - #6253, #6572, #6444, #6536, and #5860 -> `docs/about/release-notes.mdx`: Documents onboarding resume and create-step recovery improvements. - #6508, #6527, #5506, #6588, #6446, #6447, #6582, #6296, #6367, #6397, and #6505 -> `docs/about/release-notes.mdx`: Documents docs, release-risk, and E2E validation updates. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates <!-- Check exactly one tests line and one docs line. Check other lines when applicable. Add every requested justification or approval reference. --> - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: Release-note prose only. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each applicable item only when supported by the requested evidence. Run targeted tests once per relevant change set and rerun after later edits or hook autofixes that can affect the tested behavior. Do not rerun hook-covered checks. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Tests not applicable, release-note prose only. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Docs validation note: `npm run docs:check-agent-variants && npm run docs:check-routes && git diff --check` passed. Full `npm run docs` is currently blocked before Fern validation because the pinned `fern-api@5.65.2` package is unavailable from npm (`ETARGET No matching version found`). --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added release notes for v0.0.79 with a new summary of recent improvements, including onboarding and inference options, operator/CLI diagnostics, sandbox recovery hardening, runtime limits, network policy behavior, and release validation updates. * Added updated references and links for the latest release. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
## Summary - run the existing live OpenShell gateway upgrade test as a two-case matrix - retain the v0.0.36 legacy fixture - add the exact v0.0.55 / OpenShell 0.0.44 / OpenClaw 2026.5.22 source shape from NVIDIA#6114 - publish version-specific artifacts and record the selected legacy versions in test metadata ## Related Issue Follow-up coverage for NVIDIA#6114 and NVBug 6401602. ## Type of Change - [x] Test and CI coverage - [ ] Product code change - [ ] Documentation only ## Quality Gates - [x] Tests added or updated for changed behavior - [x] Existing live E2E implementation reused to avoid duplicate upgrade logic - [x] Sensitive upgrade and recovery path covered by a real installer, gateway, sandbox, process, and workspace-survival boundary - [x] No secrets, API keys, or credentials committed ## Verification - focused e2e-support Vitest tests: 7 passed - local e2e-live target: 3 passed, Linux live case skipped on macOS as designed - workflow inventory validation passed - actionlint passed after ignoring pre-existing custom runner label warnings - npm run check:diff passed - targeted live upgrade matrix: v0.0.36 and v0.0.55 both passed ([run 28873726294](https://github.com/NVIDIA/NemoClaw/actions/runs/28873726294)) Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved live end-to-end upgrade test reliability by handling varying legacy installer layouts during OpenClaw injection. * Made legacy upgrade tag/head verification more robust with dereferenced-tag-first resolution and fallback. * Refreshed upgrade assertions and expanded the upgrade payload with dynamic legacy reference/version and sandbox/base image details. * **Chores** * Updated the end-to-end upgrade workflow to run across two legacy fixture variants, with clearer per-variant job naming and variant-specific artifact labeling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
<!-- markdownlint-disable MD041 --> ## Summary Adds the pre-tag v0.0.79 release notes entry to `docs/about/release-notes.mdx` so the release plan can be generated after docs merge. The entry summarizes the merged v0.0.79 release train across inference, diagnostics, runtime hardening, policies, onboarding recovery, and release validation. ## Changes - Added the v0.0.79 release notes section with linked follow-up documentation for OpenRouter onboarding, managed vLLM changes, completion and logging, Deep Agents runtime limits, policy updates, onboarding recovery, and release validation. - Source summary: - NVIDIA#6461 -> `docs/about/release-notes.mdx`: Documents OpenRouter onboarding support and links to inference/provider references. - NVIDIA#6271 and NVIDIA#6272 -> `docs/about/release-notes.mdx`: Documents shell completion and structured logging highlights. - NVIDIA#6465, NVIDIA#6539, NVIDIA#6570, and NVIDIA#6528 -> `docs/about/release-notes.mdx`: Documents status route-drift, orphaned sandbox, gateway cleanup, and DGX Spark express-install diagnostics. - NVIDIA#6523, NVIDIA#6551, NVIDIA#6484, NVIDIA#6488, NVIDIA#6324, and NVIDIA#6542 -> `docs/about/release-notes.mdx`: Documents managed vLLM, Qwen3.6 tool parser, compaction, and timeout/readiness improvements. - NVIDIA#6559, NVIDIA#6538, NVIDIA#6560, NVIDIA#6568, NVIDIA#6552, NVIDIA#6567, and NVIDIA#6587 -> `docs/about/release-notes.mdx`: Documents runtime, credential, proxy, PID namespace, TOML, and provider-state hardening. - NVIDIA#6541, NVIDIA#5415, NVIDIA#6246, NVIDIA#6496, and NVIDIA#6573 -> `docs/about/release-notes.mdx`: Documents GitHub policy, Gmail policy, MCP allowlist, WhatsApp, and messaging-variant updates. - NVIDIA#6253, NVIDIA#6572, NVIDIA#6444, NVIDIA#6536, and NVIDIA#5860 -> `docs/about/release-notes.mdx`: Documents onboarding resume and create-step recovery improvements. - NVIDIA#6508, NVIDIA#6527, NVIDIA#5506, NVIDIA#6588, NVIDIA#6446, NVIDIA#6447, NVIDIA#6582, NVIDIA#6296, NVIDIA#6367, NVIDIA#6397, and NVIDIA#6505 -> `docs/about/release-notes.mdx`: Documents docs, release-risk, and E2E validation updates. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates <!-- Check exactly one tests line and one docs line. Check other lines when applicable. Add every requested justification or approval reference. --> - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: Release-note prose only. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each applicable item only when supported by the requested evidence. Run targeted tests once per relevant change set and rerun after later edits or hook autofixes that can affect the tested behavior. Do not rerun hook-covered checks. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Tests not applicable, release-note prose only. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Docs validation note: `npm run docs:check-agent-variants && npm run docs:check-routes && git diff --check` passed. Full `npm run docs` is currently blocked before Fern validation because the pinned `fern-api@5.65.2` package is unavailable from npm (`ETARGET No matching version found`). --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added release notes for v0.0.79 with a new summary of recent improvements, including onboarding and inference options, operator/CLI diagnostics, sandbox recovery hardening, runtime limits, network policy behavior, and release validation updates. * Added updated references and links for the latest release. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Related Issue
Follow-up coverage for #6114 and NVBug 6401602.
Type of Change
Quality Gates
Verification
Signed-off-by: Julie Yaunches jyaunches@nvidia.com
Summary by CodeRabbit