feat: add NVIDIA NIM as a native Rust provider#8948
Conversation
NVIDIA NIM (https://integrate.api.nvidia.com/v1) is OpenAI-compatible and supports 100+ models including Llama, DeepSeek, Nemotron, Kimi, and more. Set NVIDIA_API_KEY to activate. Self-hosted NIM supported via NVIDIA_BASE_URL. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Jon Patterson <jonp@nvidia.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 930dbdcaf5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| registry.register::<KimiCodeProvider>(true); | ||
| registry.register::<LiteLLMProvider>(false); | ||
| registry.register::<NanoGptProvider>(true); | ||
| registry.register::<NvidiaProvider>(true); |
There was a problem hiding this comment.
Prevent declarative provider from shadowing native NVIDIA provider
init_registry now registers NvidiaProvider, but immediately afterward load_custom_providers_into_registry loads bundled declarative providers, and both registry paths use HashMap::insert for the same key ("nvidia"). In a normal build where crates/goose/src/providers/declarative/nvidia.json is present, the declarative entry overwrites this new native one, so create("nvidia", ...) still instantiates the declarative OpenAI provider and none of the native capabilities in nvidia.rs (custom model list, NVIDIA_BASE_URL, richer flags) are actually reachable.
Useful? React with 👍 / 👎.
|
The provider this PR adds cannot actually be used as it's shadowed by the existing declarative provider that has the same name. This PR would need to remove the declarative provider, or use a different name, in order to not be dead code. Beyond that, I'm missing the motivation for this -- can you expand on the reason why this needs to be implemented in Rust? I can't see the "per-model tool / structured-output / vision flags, streaming, retries" that the PR description claims in the actual code, and it looks like the resulting provider would have the same capabilities as the declarative one that already exists. |
|
Hey @freshyjmp, thanks for the interest in contributing to Goose! 🙏 I'm going to close this one because NVIDIA NIM is already supported as a declarative provider via As the codex review comment noted, registering a native provider under the same If there's something specific you need that the declarative provider doesn't support today (per-model capability flags, custom retry logic, etc.), the right path would be to open an issue or discussion first so we can figure out whether to enhance the declarative provider system for all providers rather than adding a one-off native implementation. See Discussions, Issues and PRs in our contributing guide. Hope to see you back with a future contribution! |
Summary
Adds NVIDIA NIM as a native Rust provider (
crates/goose/src/providers/nvidia.rs), additive on top of the existing declarative JSON entry merged in #8798.NIM is OpenAI-wire-compatible at
https://integrate.api.nvidia.com/v1and hosts 100+ open-weight models including Llama, DeepSeek, Nemotron, Kimi, Phi, Gemma, and Qwen. Activates whenNVIDIA_API_KEYis set; honorsNVIDIA_BASE_URLfor self-hosted NIM. A free API key is available at https://build.nvidia.com.The native Rust impl provides richer capability coverage than the declarative entry (per-model tool / structured-output / vision flags, streaming, retries) and registers through the existing provider registry — no new dependencies, no transport changes.
Total: 3 files, +68 / 0.
Testing
cargo checkandcargo clippypass on the goose crate.Related Issues
Builds on #8798 (declarative
nvidia.jsonprovider, merged 2026-04-24).NVIDIA-side compliance
Signed-off-by: Jon Patterson <jonp@nvidia.com>).