feat: add NVIDIA NIM as a first-class inference provider - #171
Conversation
|
Can we absolutely confirm that both providers in this PR are indeed standard openai compliant apis? |
|
Also OpenRouter and Nous Portal should stay above alternatives in the changes |
Both are fully OpenAI-compatible: NVIDIA NIM: documented at https://docs.api.nvidia.com/nim/reference/llm-apis — identical /v1/chat/completions spec, standard Bearer token auth, supports tool calling Both work with the existing openai Python SDK with zero changes beyond base_url and api_key. No new dependencies. |
- Add CHUTES_BASE_URL, CHUTES_API_KEY_ENV constants to hermes_constants.py - Add 'chutes' explicit branch and auto-detect in resolve_provider() (auth.py) - Fall back to CHUTES_API_KEY when base_url points at chutes.ai (run_agent.py) - Add Chutes.ai option to setup wizard provider menu (setup.py) - Add CHUTES_MODELS list to models.py - Add CHUTES_API_KEY to .env.example - Update README providers table and env vars reference Chutes.ai is OpenAI-compatible at https://llm.chutes.ai/v1. Uses HuggingFace-style model slugs (e.g. NousResearch/Hermes-3-Llama-3.1-70B-Instruct).
- Add NVIDIA_BASE_URL, NVIDIA_API_KEY_ENV constants to hermes_constants.py - Add 'nvidia' explicit branch and auto-detect in resolve_provider() (auth.py) - Resolve NVIDIA_API_KEY when base_url points at nvidia (run_agent.py) - Add NVIDIA NIM option to setup wizard provider menu (setup.py) - Add NVIDIA_MODELS list to models.py - Add NVIDIA_API_KEY to .env.example and README NVIDIA NIM is OpenAI-compatible at https://integrate.api.nvidia.com/v1. API keys from https://build.nvidia.com start with 'nvapi-'.
8ec37b9 to
842f6fa
Compare
|
I think lets just leave it for Custom OpenAI endpoint provider for now - I dont think NIM is a super popular provider platform |
2 participants
Adds NVIDIA NIM (https://build.nvidia.com) as a named inference provider alongside Nous Portal, Chutes.ai, and OpenRouter.
Changes
hermes_constants.py— addNVIDIA_BASE_URL,NVIDIA_API_KEY_ENVhermes_cli/auth.py— addnvidiaexplicit branch + auto-detect inresolve_provider()run_agent.py— resolveNVIDIA_API_KEYwhen base URL points at NVIDIAhermes_cli/setup.py— add NVIDIA NIM option to setup wizard provider menuhermes_cli/models.py— addNVIDIA_MODELSlist with 10 NIM models.env.example+README.md— documentNVIDIA_API_KEYUsage
NVIDIA NIM exposes a fully OpenAI-compatible endpoint at
https://integrate.api.nvidia.com/v1. No new dependencies required.