fix(inference): refresh NVIDIA Endpoints featured models (Fixes #5827) - #5861
fix(inference): refresh NVIDIA Endpoints featured models (Fixes #5827)#5861deepujain wants to merge 3 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe curated NVIDIA Endpoints model list, matching docs, and related tests were updated to reflect a new featured-model ordering and labels. The onboarding selection flow now targets Kimi, and the test budget was adjusted accordingly. ChangesNVIDIA Endpoints model list refresh
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
3ede38a to
4317bb6
Compare
|
I double-checked the source JSON while working this PR. It currently lists |
wscurran
added
area: inference
|
✨ Thanks for updating the curated NVIDIA Endpoints onboarding menu to match the current featured models from the NGC API catalog. This proposes a way to refresh the CLOUD_MODEL_OPTIONS constant with the latest model set while preserving the existing default cloud model route. Related open issues: |
4317bb6 to
e6472a7
Compare
|
Rebased this on current main and kept the refreshed NVIDIA Endpoints menu aligned with the newer onboarding test shape. Build, typecheck, source-shape, and the focused inference/onboard selection Vitest run pass. |
Fixes NVIDIA#5827 Signed-off-by: Deepak Jain <deepujain@gmail.com>
e6472a7 to
f95d06a
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/onboard-selection.test.ts`:
- Line 424: The onboarding selection assertions are now expecting the wrong
preserved NVIDIA default model. Update the checks in the relevant test cases
around the default/fallback cloud-model path so they continue to assert
nvidia/nemotron-3-super-120b-a12b instead of Ultra, using the existing
payload.result.model expectations in test/onboard-selection.test.ts to keep the
suite guarding the intended default route.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 1397afc7-3aaf-4d45-98b6-9ba52b725fcc
📒 Files selected for processing (6)
ci/test-file-size-budget.jsondocs/inference/inference-options.mdxsrc/lib/inference/config.test.tssrc/lib/inference/config.tssrc/lib/inference/model-prompts.test.tstest/onboard-selection.test.ts
✅ Files skipped from review due to trivial changes (2)
- ci/test-file-size-budget.json
- docs/inference/inference-options.mdx
🚧 Files skipped from review as they are similar to previous changes (3)
- src/lib/inference/config.test.ts
- src/lib/inference/model-prompts.test.ts
- src/lib/inference/config.ts
| const payload = JSON.parse(result.stdout.trim()); | ||
| assert.equal(payload.result.provider, "nvidia-prod"); | ||
| assert.equal(payload.result.model, "nvidia/nemotron-3-super-120b-a12b"); | ||
| assert.equal(payload.result.model, "nvidia/nemotron-3-ultra-550b-a55b"); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
These assertions flip the preserved default NVIDIA model.
The PR objective and linked notes say the default cloud-model route stays on nvidia/nemotron-3-super-120b-a12b, but Lines 424 and 3319 now bless Ultra instead. Keeping these expectations would stop the suite from catching a regression in the default/fallback onboarding path.
Also applies to: 3319-3319
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test/onboard-selection.test.ts` at line 424, The onboarding selection
assertions are now expecting the wrong preserved NVIDIA default model. Update
the checks in the relevant test cases around the default/fallback cloud-model
path so they continue to assert nvidia/nemotron-3-super-120b-a12b instead of
Ultra, using the existing payload.result.model expectations in
test/onboard-selection.test.ts to keep the suite guarding the intended default
route.
Fixes NVIDIA#5827 Signed-off-by: Deepak Jain <deepujain@gmail.com>
|
Preserved |
Signed-off-by: Apurv Kumaria <36614+apurvvkumaria@users.noreply.github.com>
06d41a5 to
b027281
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/inference/inference-options.mdx`:
- Line 67: Update the NVIDIA Endpoints curated model list in the inference
options table to include GLM 5.1 so the docs match the featured set being
shipped. Edit the existing NVIDIA Endpoints row in the inference options content
and add GLM 5.1 alongside the current curated models, keeping the row format
consistent with the surrounding table entries.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 2eeac1b1-87cb-4ff7-bffc-6e31b4041910
📒 Files selected for processing (5)
ci/test-file-size-budget.jsondocs/inference/inference-options.mdxsrc/lib/inference/config.test.tssrc/lib/inference/config.tssrc/lib/inference/model-prompts.test.ts
💤 Files with no reviewable changes (3)
- src/lib/inference/model-prompts.test.ts
- src/lib/inference/config.test.ts
- src/lib/inference/config.ts
✅ Files skipped from review due to trivial changes (1)
- ci/test-file-size-budget.json
| | Option | Description | Curated models | | ||
| |--------|-------------|----------------| | ||
| | NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Super 120B, Nemotron 3 Ultra 550B, MiniMax M2.7, GPT-OSS 120B, DeepSeek V4 Pro | | ||
| | NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Ultra 550B, Nemotron 3 Super 120B, Kimi K2.6, Minimax M2.7 | |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Add GLM 5.1 to the NVIDIA Endpoints curated list.
The refreshed featured set in the PR objective includes GLM 5.1, but this row still lists only four NVIDIA Endpoints models. That leaves the onboarding docs out of sync with the menu you are shipping.
♻️ Proposed fix
-| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Ultra 550B, Nemotron 3 Super 120B, Kimi K2.6, Minimax M2.7 |
+| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Ultra 550B, Nemotron 3 Super 120B, GLM 5.1, Kimi K2.6, Minimax M2.7 |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Ultra 550B, Nemotron 3 Super 120B, Kimi K2.6, Minimax M2.7 | | |
| | NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Ultra 550B, Nemotron 3 Super 120B, GLM 5.1, Kimi K2.6, Minimax M2.7 | |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/inference/inference-options.mdx` at line 67, Update the NVIDIA Endpoints
curated model list in the inference options table to include GLM 5.1 so the docs
match the featured set being shipped. Edit the existing NVIDIA Endpoints row in
the inference options content and add GLM 5.1 alongside the current curated
models, keeping the row format consistent with the surrounding table entries.
|
Closing as superseded by #6074, which now carries the current-main implementation for #5827, including the live featured-model catalog integration, bounded fallback and sanitization behavior, retired-model handling, and current regression coverage. Thank you for the original refresh and default-model coverage; that work helped inform the maintained solution in #6074. |
|
Thank you for the focused static catalog refresh and for preserving the established Nemotron 3 Super default; that work helped clarify the expected default behavior. Closing this PR as superseded by #6074. The replacement addresses #5827 with the public NVIDIA featured catalog as the bounded, fixed-URL runtime source of truth, plus:
At final head This is a supersession close, not a merge of this branch. Thanks again for the contribution. |
4 participants
Summary
Refreshes the curated NVIDIA Endpoints onboarding menu from
https://assets.ngc.nvidia.com/products/api-catalog/featured-models.json, while preserving the GLM 5.1 retirement already merged in #6069.This keeps the fix static and small rather than adding runtime network fetching to onboarding. I preserved NemoClaw's existing
nvidia/nemotron-3-super-120b-a12bSuper model id even though the featured JSON currently omits thenvidia/prefix, so this PR does not change the established default cloud model route.The issue text mentions Minimax M3, but the source JSON used for this update lists
minimaxai/minimax-m2.7/Minimax M2.7. This PR follows that entry. GLM 5.1 is intentionally excluded from NVIDIA Endpoints because #6069 retired it ahead of provider removal.Fixes #5827
Refs #5844 (coordination only; does not fix endpoint health/latency)
Changes
CLOUD_MODEL_OPTIONSto show Nemotron 3 Ultra, Nemotron 3 Super, Kimi K2.6, and Minimax M2.7./models.test/onboard-selection.test.tsbecause this PR shrinks that file by two lines.Type of Change
Quality Gates
Verification
Verifiedin GitHubnpx prek run --from-ref main --to-ref HEADpassesnpm testpasses (broad runtime changes only)npm run docsbuilds without warnings (doc changes only)Validation performed:
curl -L https://assets.ngc.nvidia.com/products/api-catalog/featured-models.json- confirmed the current featured list is Nemotron 3 Ultra, Nemotron 3 Super, GLM 5.1, Kimi K2.6, and Minimax M2.7.npm install --ignore-scripts- installed dependencies in the isolated worktree.npm run build:cli- passed.npm run typecheck:cli- passed.npm run source-shape:check- passed when rerun outside the sandbox IPC restriction.npx vitest run --project cli src/lib/inference/config.test.ts src/lib/inference/model-prompts.test.ts test/onboard-selection.test.ts -t 'curated cloud model picker options|Minimax M2.7|selects Kimi K2.6'- passed.npx @biomejs/biome format src/lib/inference/config.ts src/lib/inference/config.test.ts src/lib/inference/model-prompts.test.ts test/onboard-selection.test.ts docs/inference/inference-options.mdx- passed.git diff --check- passed.npm run test-size:check- passed when rerun outside the sandbox IPC restriction.npm run docs:check-agent-variants- passed when rerun outside the sandbox IPC restriction.npm test- attempted; timed out after 300s with unrelated local-environment failures already present in real port/process/sandbox-oriented tests, includingsrc/lib/onboard/preflight.test.ts,test/sandbox-provisioning.test.ts,test/policy-tiers-onboard.test.ts, andtest/cli/connect-recovery.test.ts.Evidence it works
The focused config test now locks the curated cloud model menu to the current featured set and labels. The prompt helper test selects the last featured option, and the onboarding regression selects Kimi K2.6 from the NVIDIA Endpoints menu and validates it through the chat-completions path.
Signed-off-by: Deepak Jain deepujain@gmail.com
Summary by CodeRabbit
Bug Fixes
Documentation