fix(onboard): preserve hosted Inference Hub model IDs - #5757
Merged
Conversation
Contributor
|
Caution Review failedPull request was closed or merged during review No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (18)
📝 WalkthroughWalkthroughHosted inference model identifiers were updated to ChangesHosted inference model namespace update
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in the Show a code coverage summary of the most covered files.
TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most covered files.
Updated |
Contributor
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Contributor
Vitest E2E Scenario RecommendationRequired Vitest E2E scenarios: Dispatch required Vitest E2E scenarios:
Full Vitest E2E advisor summaryVitest E2E Scenario AdvisorBase: Required Vitest E2E scenarios
Optional Vitest E2E scenarios
Relevant changed files
|
Contributor
PR Review Advisor — No blocking findingsMerge posture: No blocking advisor findings Action checklist
Test follow-ups to resolve or justifyIf these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.
This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision. |
Closed
6 tasks
This was referenced Jun 25, 2026
jyaunches
added a commit
that referenced
this pull request
Jun 25, 2026
## Summary Restore issue #5800 parity package `P0-A` for merged bash-suite inference-helper deltas only. This is a focused support-test parity PR: the helper/product behavior already exists on `main`; this adds missing Vitest assertions so shell retirement keeps the hosted/hermetic inference contracts covered. ## Related Issues Refs #5800 Refs #5098 Refs #5373 Refs #5374 Refs #5385 Refs #5395 Refs #5399 Refs #5400 Refs #5411 Refs #5751 Refs #5672 Refs #5757 ## Scope gate - Package: `P0-A — Hosted/hermetic inference helper parity` - Included PRs all merged and touched `test/e2e`: yes - Out of scope: unmerged/non-bash PRs; product cleanup; shell lane retirement / PR #5756 cleanup ## Parity map | ID | Source PR | Contract | Inference classification | Vitest assertion / waiver | Status | | --- | --- | --- | --- | --- | --- | | A1 | #5373 | Fake OpenAI-compatible helper supports `/models`, chat completions, responses API, auth checking, and request capture. | `hermetic-default` | `test/e2e-scenario/support-tests/hosted-inference.test.ts` starts `startFakeOpenAiCompatibleServer` and asserts models/chat/responses/request log behavior. | covered | | A2 | #5374, #5385, #5395 | Hosted CI inference stages `NVIDIA_INFERENCE_API_KEY` as `COMPATIBLE_API_KEY`, routes as `custom`/`compatible-endpoint`, and prefers `openai-completions`. | `hosted-compatible capable` | Existing workflow/helper assertions plus new shell helper staging assertion in `hosted-inference.test.ts`. | covered | | A3 | #5399, #5751, #5672, #5757 | Hosted model default remains the Inference Hub provider/namespace/model ID `nvidia/nvidia/nemotron-3-ultra`; explicit `NEMOCLAW_MODEL` takes precedence over `NEMOCLAW_COMPAT_MODEL`, which takes precedence over helper options/default. | `hosted-compatible capable` | New `requireHostedInferenceConfig` model precedence/default assertion; existing workflow/model namespace tests remain green. | covered | | A4 | #5400, #5411 | Hosted reachability probe is bounded and low-cost: no `/models`, chat completions, auth header, or bearer token spend. | `hosted-compatible capable` | Existing probe tests retained and revalidated. | covered | | A5 | #5385 | Public NVIDIA/nvapi shell mode remains distinct from hosted-compatible mode and is not restaged as compatible inference. | `public-nvidia required` | New shell helper assertion checks `nvapi-*` + `cloud` keeps `nvidia-prod`, leaves `COMPATIBLE_API_KEY` unset. | covered | ## Inference mode support - Default mode for touched live targets: none touched; this PR only changes support tests. - Real inference support preserved: yes, by asserting hosted-compatible and public-NVIDIA helper boundaries without invoking real inference. - Modes validated in this PR: hermetic fake endpoint and shell helper mocked hosted-compatible/public boundary. - If not validated with real inference: not required; no live target or hosted secret path changed. ## Validation - [x] `npx vitest run --project e2e-vitest-support test/e2e-scenario/support-tests/hosted-inference.test.ts` - [x] `npx vitest run test/e2e-script-workflow.test.ts test/issue-5667-hosted-inference-model-namespace.test.ts src/lib/inference/onboard-probes.test.ts src/lib/onboard/providers.test.ts` - [x] `git diff --check` - [ ] hosted/public selective E2E workflow, if required by classification: not required; support-test-only PR, no live/workflow behavior changed. ## Follow-ups / waivers - Pre-push full `Test (CLI)` / `Test (plugin)` hooks were not clean on local macOS after the commit: CLI run hit existing macOS/stat/OOM-style failures; plugin run could not import package `json5`. Focused target tests above passed. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Expanded end-to-end coverage for hosted inference compatibility and model ID/model precedence behavior. * Added validations for environment variable staging rules and shell mode behavior across NVAPI key scenarios. * Introduced a fake OpenAI-compatible server and added contract checks for `/models`, auth-required flows, and streamed responses on chat/response endpoints, including cleanup after runs. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
## Summary Preserves the provider/namespace/model ID convention for the private CI-compatible `https://inference-api.nvidia.com/v1` endpoint. This follow-up keeps the public NVIDIA provider catalog separate from the custom OpenAI-compatible Inference Hub path used by CI. ## Related Issue Follow-up to NVIDIA#5672 / NVIDIA#5667. ## Changes - Restores hosted-compatible CI defaults to `nvidia/nvidia/nemotron-3-ultra` across workflow env, E2E fixtures, and shell helpers. - Documents in `src/lib/onboard/providers.ts` that the private `inference-api.nvidia.com` endpoint uses provider/namespace/model IDs and is staged as a custom compatible provider. - Updates hosted inference regression tests to assert the provider-convention ID is preserved end-to-end instead of normalized. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [x] Targeted tests pass for changed behavior - [ ] Full `npm test` passes (broad runtime changes only) - [x] Tests added or updated for new or changed behavior - [x] No secrets, API keys, or credentials committed - [ ] Docs updated for user-facing behavior changes - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Carlos Villela <cvillela@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Updated hosted inference and E2E scenarios to use the correct model identifier consistently. * Improved fallback behavior so tests and workflows resolve the same hosted model across CI, rebuild, and upgrade flows. * Added coverage for the provider-style model naming to prevent namespace mismatches. * **Documentation** * Clarified the model naming convention used for hosted inference in the onboarding flow. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
## Summary Restore issue NVIDIA#5800 parity package `P0-A` for merged bash-suite inference-helper deltas only. This is a focused support-test parity PR: the helper/product behavior already exists on `main`; this adds missing Vitest assertions so shell retirement keeps the hosted/hermetic inference contracts covered. ## Related Issues Refs NVIDIA#5800 Refs NVIDIA#5098 Refs NVIDIA#5373 Refs NVIDIA#5374 Refs NVIDIA#5385 Refs NVIDIA#5395 Refs NVIDIA#5399 Refs NVIDIA#5400 Refs NVIDIA#5411 Refs NVIDIA#5751 Refs NVIDIA#5672 Refs NVIDIA#5757 ## Scope gate - Package: `P0-A — Hosted/hermetic inference helper parity` - Included PRs all merged and touched `test/e2e`: yes - Out of scope: unmerged/non-bash PRs; product cleanup; shell lane retirement / PR NVIDIA#5756 cleanup ## Parity map | ID | Source PR | Contract | Inference classification | Vitest assertion / waiver | Status | | --- | --- | --- | --- | --- | --- | | A1 | NVIDIA#5373 | Fake OpenAI-compatible helper supports `/models`, chat completions, responses API, auth checking, and request capture. | `hermetic-default` | `test/e2e-scenario/support-tests/hosted-inference.test.ts` starts `startFakeOpenAiCompatibleServer` and asserts models/chat/responses/request log behavior. | covered | | A2 | NVIDIA#5374, NVIDIA#5385, NVIDIA#5395 | Hosted CI inference stages `NVIDIA_INFERENCE_API_KEY` as `COMPATIBLE_API_KEY`, routes as `custom`/`compatible-endpoint`, and prefers `openai-completions`. | `hosted-compatible capable` | Existing workflow/helper assertions plus new shell helper staging assertion in `hosted-inference.test.ts`. | covered | | A3 | NVIDIA#5399, NVIDIA#5751, NVIDIA#5672, NVIDIA#5757 | Hosted model default remains the Inference Hub provider/namespace/model ID `nvidia/nvidia/nemotron-3-ultra`; explicit `NEMOCLAW_MODEL` takes precedence over `NEMOCLAW_COMPAT_MODEL`, which takes precedence over helper options/default. | `hosted-compatible capable` | New `requireHostedInferenceConfig` model precedence/default assertion; existing workflow/model namespace tests remain green. | covered | | A4 | NVIDIA#5400, NVIDIA#5411 | Hosted reachability probe is bounded and low-cost: no `/models`, chat completions, auth header, or bearer token spend. | `hosted-compatible capable` | Existing probe tests retained and revalidated. | covered | | A5 | NVIDIA#5385 | Public NVIDIA/nvapi shell mode remains distinct from hosted-compatible mode and is not restaged as compatible inference. | `public-nvidia required` | New shell helper assertion checks `nvapi-*` + `cloud` keeps `nvidia-prod`, leaves `COMPATIBLE_API_KEY` unset. | covered | ## Inference mode support - Default mode for touched live targets: none touched; this PR only changes support tests. - Real inference support preserved: yes, by asserting hosted-compatible and public-NVIDIA helper boundaries without invoking real inference. - Modes validated in this PR: hermetic fake endpoint and shell helper mocked hosted-compatible/public boundary. - If not validated with real inference: not required; no live target or hosted secret path changed. ## Validation - [x] `npx vitest run --project e2e-vitest-support test/e2e-scenario/support-tests/hosted-inference.test.ts` - [x] `npx vitest run test/e2e-script-workflow.test.ts test/issue-5667-hosted-inference-model-namespace.test.ts src/lib/inference/onboard-probes.test.ts src/lib/onboard/providers.test.ts` - [x] `git diff --check` - [ ] hosted/public selective E2E workflow, if required by classification: not required; support-test-only PR, no live/workflow behavior changed. ## Follow-ups / waivers - Pre-push full `Test (CLI)` / `Test (plugin)` hooks were not clean on local macOS after the commit: CLI run hit existing macOS/stat/OOM-style failures; plugin run could not import package `json5`. Focused target tests above passed. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Expanded end-to-end coverage for hosted inference compatibility and model ID/model precedence behavior. * Added validations for environment variable staging rules and shell mode behavior across NVAPI key scenarios. * Introduced a fake OpenAI-compatible server and added contract checks for `/models`, auth-required flows, and streamed responses on chat/response endpoints, including cleanup after runs. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
wscurran
added
area: onboarding
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Preserves the provider/namespace/model ID convention for the private CI-compatible
https://inference-api.nvidia.com/v1endpoint. This follow-up keeps the public NVIDIA provider catalog separate from the custom OpenAI-compatible Inference Hub path used by CI.Related Issue
Follow-up to #5672 / #5667.
Changes
nvidia/nvidia/nemotron-3-ultraacross workflow env, E2E fixtures, and shell helpers.src/lib/onboard/providers.tsthat the privateinference-api.nvidia.comendpoint uses provider/namespace/model IDs and is staged as a custom compatible provider.Type of Change
Verification
Verifiedin GitHubnpx prek run --from-ref main --to-ref HEADpassesnpm testpasses (broad runtime changes only)npm run docsbuilds without warnings (doc changes only)Signed-off-by: Carlos Villela cvillela@nvidia.com
Summary by CodeRabbit
Bug Fixes
Documentation