test(e2e): restore recovery kimi scope parity - #5808
Merged
Conversation
Contributor
📝 WalkthroughWalkthroughThe PR adds public-nvidia Kimi inference mode handling to the Vitest workflow, live compatibility helpers, and scenario execution, and extends tests for mode selection, API-key validation, and workflow env wiring. ChangesKimi inference compatibility mode wiring
Sequence Diagram(s)sequenceDiagram
participant Job as kimi-inference-compat-vitest job
participant Test as kimi-inference-compat.test.ts
participant CompatHelpers as kimi-inference-compat-helpers.ts
participant KimiMock as KimiMock
participant OpenShell as openshell inference get
Job->>Test: set NEMOCLAW_E2E_INFERENCE_MODE and NVIDIA_API_KEY
Test->>CompatHelpers: resolveKimiInferenceMode()
CompatHelpers-->>Test: mock or public-nvidia
Test->>CompatHelpers: kimiOnboardEnv(...), env(..., { mode })
alt mock
Test->>KimiMock: record upstream requests
Test->>CompatHelpers: assertKimiUpstreamTraffic({ fake })
CompatHelpers-->>Test: validate fake.requests
else public-nvidia
Test->>CompatHelpers: assertKimiUpstreamTraffic({ host, apiKey })
CompatHelpers->>OpenShell: openshell inference get
OpenShell-->>CompatHelpers: route output
CompatHelpers-->>Test: verify output
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in the Show a code coverage summary of the most covered files.
TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most covered files.
Updated |
Contributor
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Contributor
Vitest E2E Scenario RecommendationRequired Vitest E2E scenarios: Dispatch required Vitest E2E scenarios:
Full Vitest E2E advisor summaryVitest E2E Scenario AdvisorBase: Required Vitest E2E scenarios
Optional Vitest E2E scenarios
Relevant changed files
|
Contributor
PR Review Advisor — No blocking findingsMerge posture: No blocking advisor findings Action checklist
Test follow-ups to resolve or justifyIf these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.
This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision. |
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 28186561267
|
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 28186903410
|
Contributor
Vitest E2E Scenario Results — ✅ All requested jobs passedRun: 28188326056
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e-scenario/live/kimi-inference-compat-helpers.ts`:
- Around line 32-36: The mode resolution in resolveKimiInferenceMode should
honor NEMOCLAW_E2E_INFERENCE_MODE before falling back to the legacy
NEMOCLAW_KIMI_USE_MOCK flag, so an explicit "mock" selection is not overridden
by "0". Update the conditional logic in resolveKimiInferenceMode to recognize
both supported explicit values from NEMOCLAW_E2E_INFERENCE_MODE (including
"mock" and "public-nvidia") and only consult NEMOCLAW_KIMI_USE_MOCK when the
explicit mode is unset or unrecognized.
In `@test/e2e-scenario/support-tests/kimi-inference-compat-helpers.test.ts`:
- Around line 49-55: Add a precedence test for resolveKimiInferenceMode to cover
the explicit-vs-legacy conflict: verify that NEMOCLAW_E2E_INFERENCE_MODE="mock"
wins over a stale NEMOCLAW_KIMI_USE_MOCK="0". Update the existing
kimi-inference-compat-helpers test suite by extending the
resolveKimiInferenceMode cases so the resolver behavior is locked down when both
env vars are present.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 03e1d08e-def2-4cb3-89bd-a987a57b3253
📒 Files selected for processing (6)
.github/workflows/e2e-vitest-scenarios.yamltest/e2e-scenario/live/kimi-inference-compat-helpers.tstest/e2e-scenario/live/kimi-inference-compat.test.tstest/e2e-scenario/support-tests/kimi-inference-compat-helpers.test.tstest/e2e-script-workflow.test.tstest/helpers/e2e-workflow-contract.ts
Contributor
Vitest E2E Scenario Results — ✅ All requested jobs passedRun: 28188683830
|
Contributor
Author
|
P0-D loop update: addressed advisor scope concern by narrowing changed acceptance to Kimi D2/D3 and documenting D1/D4 as existing rows revalidated by selective run 28186561267. Latest Kimi selective validation passed in run 28188683830 on head 5d68e8d. |
Contributor
Vitest E2E Scenario Results — ✅ All requested jobs passedRun: 28190216767
|
Contributor
Author
|
P0-D loop update: addressed PR Review Advisor Kimi follow-ups on head f36fef6.
Local validation passed: helper support Vitest, workflow contract Vitest, |
Contributor
Vitest E2E Scenario Results — ✅ All requested jobs passedRun: 28193896380
|
Contributor
Author
|
P0-D validation update: selective |
wscurran
added
area: e2e
Contributor
|
✨
Related open issues: |
5 tasks
4 tasks
This was referenced Jun 26, 2026
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
## Summary Restore the Kimi-specific issue NVIDIA#5800 parity work for package `P0-D`; existing recovery and scope-upgrade package rows are explicitly mapped as pre-existing coverage and revalidated context, but not changed acceptance scope in this PR. ## Related Issues Refs NVIDIA#5800 Refs NVIDIA#5098 Refs NVIDIA#5342 Refs NVIDIA#5401 Refs NVIDIA#5406 Refs NVIDIA#5412 Refs NVIDIA#5413 Refs NVIDIA#5625 Refs NVIDIA#5760 ## Scope gate - Package: `P0-D — Recovery, Kimi, and scope-upgrade parity` - Included PRs all merged and touched `test/e2e`: yes - Changed acceptance scope in this PR: Kimi public-NVIDIA/mock parity (`D2`, `D3`) - Existing package rows revalidated without diff changes: recovery (`D1`) and scope-upgrade (`D4`) - Out of scope: unmerged/non-bash PRs; shell lane retirement / PR NVIDIA#5756 cleanup ## Parity map | ID | Source PR | Contract | Inference classification | Vitest assertion / waiver | Status | | --- | --- | --- | --- | --- | --- | | D1 | NVIDIA#5342, NVIDIA#5401 | Recovery proxy env sourcing, missing proxy-env warning, guard retention, ciao/networkInterfaces preload, and crash-loop stability are pre-existing package coverage. | `hermetic-default` | Existing `test/e2e-scenario/live/issue-2478-crash-loop-recovery.test.ts`, `test/e2e-scenario/support-tests/e2e-recovery-helpers.test.ts`; selective run `28186561267` job `issue-2478-crash-loop-recovery-vitest` passed. No diff changes here. | existing / revalidated context | | D2 | NVIDIA#5401 | Kimi remains a public-NVIDIA model/provider contract when run in trusted selective CI, while retaining mock fallback for local/untrusted validation. | `public-nvidia required` | `.github/workflows/e2e-vitest-scenarios.yaml`, `test/e2e-scenario/live/kimi-inference-compat.test.ts`, `test/e2e-scenario/support-tests/kimi-inference-compat-helpers.test.ts`, `test/e2e-script-workflow.test.ts` | covered / changed | | D3 | NVIDIA#5413, NVIDIA#5625 | Kimi multiturn tool calls split `hostname; date; uptime`, preserve tool-result flow, reject abandoned/continue traces, and normalize final punctuation. | `public-nvidia required` with mock fallback | `test/e2e-scenario/live/kimi-inference-compat-helpers.ts` trajectory assertions; selective run `28190216767` job `kimi-inference-compat-vitest` passed on the previous head; latest run `28193896380` passed on `f36fef6da`. | covered / changed | | D4 | NVIDIA#5406, NVIDIA#5412, NVIDIA#5760 | Scope-upgrade approval tolerates preapproved / not-reproduced states, denies `operator.admin` leakage, stays on gateway/no embedded fallback, and accepts whitespace-normalized `42`; this is pre-existing package coverage. | `hosted-compatible capable` | Existing `test/e2e-scenario/live/issue-4462-scope-upgrade-approval.test.ts`; selective run `28186561267` job `issue-4462-scope-upgrade-approval-vitest` passed. No diff changes here. | existing / revalidated context | ## Inference mode support - Default mode for touched live target: Kimi `mock` unless workflow selects `public-nvidia`. - Real inference support preserved: yes for Kimi public NVIDIA; yes for existing scope-upgrade hosted-compatible; not required for recovery. - Modes validated in this PR: Kimi public NVIDIA via selective workflows `28188683830`, `28190216767`; latest follow-up validation `28193896380` is running for head `f36fef6da`. Kimi helper/mock behavior via local support tests. - Source-of-truth contract: `NEMOCLAW_E2E_INFERENCE_MODE` is the canonical selector; absent selector defaults to mock for local/untrusted validation; unknown explicit values now fail closed; legacy `NEMOCLAW_KIMI_USE_MOCK=0` remains only as a temporary shell-lane compatibility alias until shell retirement. - Secret boundary: public Kimi workflow passes only `NVIDIA_API_KEY`; helper probe envs are secret-free by default; raw public NVIDIA key handoff is limited to onboard; sandbox `openclaw agent` now runs with a secret-free env and uses the configured `nvidia-prod` route. ## Validation - [x] `npx vitest run --project e2e-vitest-support test/e2e-scenario/support-tests/kimi-inference-compat-helpers.test.ts` - [x] `npx vitest run test/e2e-script-workflow.test.ts` - [x] `npm run typecheck:cli` - [x] `npm run test-conditionals:scan -- --top 25` - [x] `npx prek run --all-files --stage pre-push --skip tsc-plugin --skip tsc-js --skip tsc-cli --skip version-tag-sync --skip test-cli --skip test-plugin --skip source-shape-test-budget --skip test-file-size-budget --skip test-skills-yaml` - [x] `git diff --check` - [x] Kimi selective E2E / Vitest Scenarios on previous head: https://github.com/NVIDIA/NemoClaw/actions/runs/28190216767 - [x] Kimi selective E2E / Vitest Scenarios after review-gap fixes: https://github.com/NVIDIA/NemoClaw/actions/runs/28193896380 - [x] Existing recovery/scope rows revalidated in selective run: https://github.com/NVIDIA/NemoClaw/actions/runs/28186561267 (`issue-2478-crash-loop-recovery-vitest` ✅, `issue-4462-scope-upgrade-approval-vitest` ✅; Kimi in that stale run was superseded) - [ ] Local live mock Kimi: attempted but blocked by local Docker daemon unavailable (`Cannot connect to the Docker daemon at unix:///Users/jyaunches/.docker/run/docker.sock`). CI selective run is the live validation path for this head. ## Follow-ups / waivers - None. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for running Kimi compatibility e2e checks in either mock or public NVIDIA mode. * The live scenario now adapts its setup, redaction, and traffic validation based on the selected mode. * **Bug Fixes** * Improved handling of API key propagation so public NVIDIA runs use the expected credentials without exposing secrets in other paths. * **Tests** * Added coverage for mode selection, API key validation, workflow environment wiring, and the new public NVIDIA Vitest lane. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Restore the Kimi-specific issue #5800 parity work for package
P0-D; existing recovery and scope-upgrade package rows are explicitly mapped as pre-existing coverage and revalidated context, but not changed acceptance scope in this PR.Related Issues
Refs #5800
Refs #5098
Refs #5342
Refs #5401
Refs #5406
Refs #5412
Refs #5413
Refs #5625
Refs #5760
Scope gate
P0-D — Recovery, Kimi, and scope-upgrade paritytest/e2e: yesD2,D3)D1) and scope-upgrade (D4)Parity map
hermetic-defaulttest/e2e-scenario/live/issue-2478-crash-loop-recovery.test.ts,test/e2e-scenario/support-tests/e2e-recovery-helpers.test.ts; selective run28186561267jobissue-2478-crash-loop-recovery-vitestpassed. No diff changes here.public-nvidia required.github/workflows/e2e-vitest-scenarios.yaml,test/e2e-scenario/live/kimi-inference-compat.test.ts,test/e2e-scenario/support-tests/kimi-inference-compat-helpers.test.ts,test/e2e-script-workflow.test.tshostname; date; uptime, preserve tool-result flow, reject abandoned/continue traces, and normalize final punctuation.public-nvidia requiredwith mock fallbacktest/e2e-scenario/live/kimi-inference-compat-helpers.tstrajectory assertions; selective run28190216767jobkimi-inference-compat-vitestpassed on the previous head; latest run28193896380passed onf36fef6da.operator.adminleakage, stays on gateway/no embedded fallback, and accepts whitespace-normalized42; this is pre-existing package coverage.hosted-compatible capabletest/e2e-scenario/live/issue-4462-scope-upgrade-approval.test.ts; selective run28186561267jobissue-4462-scope-upgrade-approval-vitestpassed. No diff changes here.Inference mode support
mockunless workflow selectspublic-nvidia.28188683830,28190216767; latest follow-up validation28193896380is running for headf36fef6da. Kimi helper/mock behavior via local support tests.NEMOCLAW_E2E_INFERENCE_MODEis the canonical selector; absent selector defaults to mock for local/untrusted validation; unknown explicit values now fail closed; legacyNEMOCLAW_KIMI_USE_MOCK=0remains only as a temporary shell-lane compatibility alias until shell retirement.NVIDIA_API_KEY; helper probe envs are secret-free by default; raw public NVIDIA key handoff is limited to onboard; sandboxopenclaw agentnow runs with a secret-free env and uses the configurednvidia-prodroute.Validation
npx vitest run --project e2e-vitest-support test/e2e-scenario/support-tests/kimi-inference-compat-helpers.test.tsnpx vitest run test/e2e-script-workflow.test.tsnpm run typecheck:clinpm run test-conditionals:scan -- --top 25npx prek run --all-files --stage pre-push --skip tsc-plugin --skip tsc-js --skip tsc-cli --skip version-tag-sync --skip test-cli --skip test-plugin --skip source-shape-test-budget --skip test-file-size-budget --skip test-skills-yamlgit diff --checkissue-2478-crash-loop-recovery-vitest✅,issue-4462-scope-upgrade-approval-vitest✅; Kimi in that stale run was superseded)Cannot connect to the Docker daemon at unix:///Users/jyaunches/.docker/run/docker.sock). CI selective run is the live validation path for this head.Follow-ups / waivers
Summary by CodeRabbit
New Features
Bug Fixes
Tests