test(e2e): make release fixtures sandbox-reachable - #6999
Conversation
📝 WalkthroughWalkthroughLive OpenShell E2E tests now use gateway-reachable fake OpenAI-compatible servers. The EXDEV test records server metadata and requests, cleans up the server, and uses its base URL; the upgrade test configures explicit host and public host values. ChangesGateway endpoint reachability
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage remains at 96%, unchanged from the TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most impacted files.
Updated |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e/live/openclaw-plugin-runtime-exdev.test.ts`:
- Around line 1021-1027: Update the fake server configuration in the test’s
startFakeOpenAiCompatibleServer call to set requireAuth to true, ensuring
authentication is enabled and the configured apiKey validates the
COMPATIBLE_API_KEY contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 80fa653c-7882-4a4c-b250-d7eaff566342
📒 Files selected for processing (2)
test/e2e/live/openclaw-plugin-runtime-exdev.test.tstest/e2e/live/openshell-gateway-upgrade.test.ts
| const fake = await startFakeOpenAiCompatibleServer({ | ||
| apiKey: "nemoclaw-exdev-dummy-key", | ||
| host: "0.0.0.0", | ||
| model: "nemoclaw-exdev-probe", | ||
| publicHost: "host.openshell.internal", | ||
| responseText: "ok", | ||
| }); |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Enable authentication on the fake server.
startFakeOpenAiCompatibleServer maps options.requireAuth to the server’s authentication flag. Supplying apiKey alone does not enable auth, so this test can pass without validating the configured COMPATIBLE_API_KEY contract. Add requireAuth: true.
Proposed fix
const fake = await startFakeOpenAiCompatibleServer({
apiKey: "nemoclaw-exdev-dummy-key",
+ requireAuth: true,
host: "0.0.0.0",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const fake = await startFakeOpenAiCompatibleServer({ | |
| apiKey: "nemoclaw-exdev-dummy-key", | |
| host: "0.0.0.0", | |
| model: "nemoclaw-exdev-probe", | |
| publicHost: "host.openshell.internal", | |
| responseText: "ok", | |
| }); | |
| const fake = await startFakeOpenAiCompatibleServer({ | |
| apiKey: "nemoclaw-exdev-dummy-key", | |
| requireAuth: true, | |
| host: "0.0.0.0", | |
| model: "nemoclaw-exdev-probe", | |
| publicHost: "host.openshell.internal", | |
| responseText: "ok", | |
| }); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test/e2e/live/openclaw-plugin-runtime-exdev.test.ts` around lines 1021 -
1027, Update the fake server configuration in the test’s
startFakeOpenAiCompatibleServer call to set requireAuth to true, ensuring
authentication is enabled and the configured apiKey validates the
COMPATIBLE_API_KEY contract.
Source: Path instructions
PR Review Advisor — InformationalAdvisor assessment: Informational / high confidence Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
wscurran
added
area: ci
3 participants
Summary
Release-candidate E2E exposed two fixture endpoints that the OpenShell gateway could not reach, producing deterministic inference readiness HTTP 503 failures. This change gives the EXDEV fixture a real authenticated compatible endpoint and advertises both affected mocks through
host.openshell.internal.Changes
openclaw-plugin-runtime-exdevinstead of pointing onboarding at an intentionally dead port.Type of Change
Quality Gates
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablenpx vitest run --project e2e-support test/e2e/support/device-auth-health-helpers.test.ts test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts test/e2e/support/openclaw-plugin-runtime-exdev-workflow-boundary.test.ts(3 files, 8 tests passed)npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result:npm run docsbuilds without warnings (doc changes only)Signed-off-by: Prekshi Vyas prekshiv@nvidia.com
Summary by CodeRabbit