Skip to content

fix(router): correct NVIDIA routed endpoints #3614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jyaunches merged 1 commit into from
May 15, 2026
Merged

fix(router): correct NVIDIA routed endpoints #3614

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions nemoclaw-blueprint/router/pool-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,14 @@ routing:
models:
- name: nemotron-3-nano-reasoning
display_name: "Nemotron 3 Nano (Reasoning)"
litellm_model: "openai/nvidia/nvidia/Nemotron-3-Nano-30B-A3B"
litellm_model: "openai/nvidia/nemotron-3-nano-30b-a3b"
cost_per_m_input_tokens: 0.05
cost_per_m_output_tokens: 0.20
api_base: "https://inference-api.nvidia.com"
api_base: "https://integrate.api.nvidia.com/v1"

- name: nemotron-3-super
display_name: "Nemotron 3 Super 120B"
litellm_model: "openai/nvidia/nvidia/nemotron-3-super-v3"
litellm_model: "openai/nvidia/nemotron-3-super-120b-a12b"
cost_per_m_input_tokens: 0.10
cost_per_m_output_tokens: 0.40
api_base: "https://inference-api.nvidia.com"
api_base: "https://integrate.api.nvidia.com/v1"
40 changes: 40 additions & 0 deletions test/validate-blueprint.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ import { describe, it, expect } from "vitest";
import YAML from "yaml";

const BLUEPRINT_PATH = new URL("../nemoclaw-blueprint/blueprint.yaml", import.meta.url);
const ROUTER_POOL_CONFIG_PATH = new URL(
"../nemoclaw-blueprint/router/pool-config.yaml",
import.meta.url,
);
const BASE_POLICY_PATH = new URL(
"../nemoclaw-blueprint/policies/openclaw-sandbox.yaml",
import.meta.url,
Expand Down Expand Up @@ -42,6 +46,16 @@ type Blueprint = {
};
};

type RouterPoolModel = {
name?: string;
litellm_model?: string;
api_base?: string;
};

type RouterPoolConfig = {
models?: RouterPoolModel[];
};

type Rule = { allow?: { method?: string; path?: string } };
type Endpoint = {
host?: string;
Expand Down Expand Up @@ -163,6 +177,32 @@ describe("blueprint.yaml", () => {
}
});

describe("Model Router pool config", () => {
const pool = loadYaml<RouterPoolConfig>(ROUTER_POOL_CONFIG_PATH);

it("regression #3255: routes NVIDIA API keys to the public NVIDIA Build endpoint", () => {
const apiBases = new Set((pool.models ?? []).map((model) => model.api_base));
expect(apiBases).toEqual(new Set(["https://integrate.api.nvidia.com/v1"]));
});

it("regression #3255: uses valid LiteLLM NVIDIA model identifiers", () => {
const modelsByName = new Map(
(pool.models ?? []).map((model) => [model.name, model.litellm_model]),
);
expect(modelsByName.get("nemotron-3-nano-reasoning")).toBe(
"openai/nvidia/nemotron-3-nano-30b-a3b",
);
expect(modelsByName.get("nemotron-3-super")).toBe(
"openai/nvidia/nemotron-3-super-120b-a12b",
);
for (const litellmModel of modelsByName.values()) {
expect(litellmModel).not.toMatch(/nvidia\/nvidia\//);
expect(litellmModel).not.toContain("Nemotron-3-Nano-30B-A3B");
expect(litellmModel).not.toContain("nemotron-3-super-v3");
}
Comment on lines +189 to +202

@coderabbitai coderabbitai Bot May 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Harden regression checks against duplicate/missing model entries.

Building modelsByName via Map can mask duplicate name rows, and optional fields make failures less explicit. Add a uniqueness/assertion step before the value checks so this test catches malformed pool entries deterministically.

Suggested patch 
   it("regression `#3255`: uses valid LiteLLM NVIDIA model identifiers", () => {
-    const modelsByName = new Map(
-      (pool.models ?? []).map((model) => [model.name, model.litellm_model]),
-    );
+    const models = pool.models ?? [];
+    const names = models.map((model) => model.name);
+    expect(names.every((name) => typeof name === "string" && name.length > 0)).toBe(true);
+    expect(new Set(names).size).toBe(names.length);
+
+    const modelsByName = new Map(models.map((model) => [model.name, model.litellm_model]));
@@
-    for (const litellmModel of modelsByName.values()) {
+    for (const litellmModel of modelsByName.values()) {
+      expect(typeof litellmModel).toBe("string");
       expect(litellmModel).not.toMatch(/nvidia\/nvidia\//);
       expect(litellmModel).not.toContain("Nemotron-3-Nano-30B-A3B");
       expect(litellmModel).not.toContain("nemotron-3-super-v3");
     }
   });
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/validate-blueprint.test.ts` around lines 189 - 202, The test builds
modelsByName from (pool.models ?? []) which can silently drop duplicate
model.name entries and hide missing fields; modify the test so it first
validates pool.models is present and then asserts uniqueness of model.name
(e.g., collect names and fail if any duplicate) and asserts each model has the
required litellm_model field before constructing modelsByName; after those
checks proceed with the existing expectations on modelsByName and litellmModel
to ensure duplicate/missing entries produce deterministic test failures
(referencing modelsByName, pool.models, model.name, and litellm_model).

});
});

describe("base sandbox policy", () => {
const policy = loadYaml<SandboxPolicy>(BASE_POLICY_PATH);

Expand Down
Loading