Skip to content

feat: add NVIDIA NIM as a native Rust provider#8948

Closed
freshyjmp wants to merge 1 commit into
aaif-goose:mainfrom
NVIDIA-AI-Blueprints:feat/add-nvidia-nim-2026-04-13
Closed

feat: add NVIDIA NIM as a native Rust provider#8948
freshyjmp wants to merge 1 commit into
aaif-goose:mainfrom
NVIDIA-AI-Blueprints:feat/add-nvidia-nim-2026-04-13

Conversation

@freshyjmp
Copy link
Copy Markdown

Summary

Adds NVIDIA NIM as a native Rust provider (crates/goose/src/providers/nvidia.rs), additive on top of the existing declarative JSON entry merged in #8798.

NIM is OpenAI-wire-compatible at https://integrate.api.nvidia.com/v1 and hosts 100+ open-weight models including Llama, DeepSeek, Nemotron, Kimi, Phi, Gemma, and Qwen. Activates when NVIDIA_API_KEY is set; honors NVIDIA_BASE_URL for self-hosted NIM. A free API key is available at https://build.nvidia.com.

The native Rust impl provides richer capability coverage than the declarative entry (per-model tool / structured-output / vision flags, streaming, retries) and registers through the existing provider registry — no new dependencies, no transport changes.

crates/goose/src/providers/nvidia.rs   (new — native Rust provider)
crates/goose/src/providers/init.rs     (registration entry)
crates/goose/src/providers/mod.rs      (mod declaration)

Total: 3 files, +68 / 0.

Testing

  • cargo check and cargo clippy pass on the goose crate.
  • Native provider follows the pattern established by the existing OpenAI-compatible providers (xAI, DeepSeek, etc.) — same shape, same trait surface.
  • Wire compatibility verified against the NIM API directly.

Related Issues

Builds on #8798 (declarative nvidia.json provider, merged 2026-04-24).

NVIDIA-side compliance

  • Open Source Review Board (OSRB) approval completed for this contribution.
  • DCO sign-off included (Signed-off-by: Jon Patterson <jonp@nvidia.com>).

NVIDIA NIM (https://integrate.api.nvidia.com/v1) is OpenAI-compatible
and supports 100+ models including Llama, DeepSeek, Nemotron, Kimi, and more.

Set NVIDIA_API_KEY to activate. Self-hosted NIM supported via NVIDIA_BASE_URL.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Jon Patterson <jonp@nvidia.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 930dbdcaf5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

registry.register::<KimiCodeProvider>(true);
registry.register::<LiteLLMProvider>(false);
registry.register::<NanoGptProvider>(true);
registry.register::<NvidiaProvider>(true);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prevent declarative provider from shadowing native NVIDIA provider

init_registry now registers NvidiaProvider, but immediately afterward load_custom_providers_into_registry loads bundled declarative providers, and both registry paths use HashMap::insert for the same key ("nvidia"). In a normal build where crates/goose/src/providers/declarative/nvidia.json is present, the declarative entry overwrites this new native one, so create("nvidia", ...) still instantiates the declarative OpenAI provider and none of the native capabilities in nvidia.rs (custom model list, NVIDIA_BASE_URL, richer flags) are actually reachable.

Useful? React with 👍 / 👎.

@jh-block jh-block self-assigned this May 4, 2026
@jh-block
Copy link
Copy Markdown
Collaborator

jh-block commented May 5, 2026

The provider this PR adds cannot actually be used as it's shadowed by the existing declarative provider that has the same name. This PR would need to remove the declarative provider, or use a different name, in order to not be dead code.

Beyond that, I'm missing the motivation for this -- can you expand on the reason why this needs to be implemented in Rust? I can't see the "per-model tool / structured-output / vision flags, streaming, retries" that the PR description claims in the actual code, and it looks like the resulting provider would have the same capabilities as the declarative one that already exists.

@DOsinga
Copy link
Copy Markdown
Collaborator

DOsinga commented May 12, 2026

Hey @freshyjmp, thanks for the interest in contributing to Goose! 🙏

I'm going to close this one because NVIDIA NIM is already supported as a declarative provider via nvidia.json — which gives the same functionality (OpenAI-compatible engine, streaming, API key setup, dynamic models) without needing a native Rust provider.

As the codex review comment noted, registering a native provider under the same "nvidia" name would conflict with the declarative one that's already loaded.

If there's something specific you need that the declarative provider doesn't support today (per-model capability flags, custom retry logic, etc.), the right path would be to open an issue or discussion first so we can figure out whether to enhance the declarative provider system for all providers rather than adding a one-off native implementation. See Discussions, Issues and PRs in our contributing guide.

Hope to see you back with a future contribution!

@DOsinga DOsinga closed this May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants