Skip to content

fix(litellm): merge default model info with router models for NTC support #10187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mrubens merged 2 commits into from
Dec 18, 2025
Merged

fix(litellm): merge default model info with router models for NTC support #10187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0fb1c89
feat(types): add defaultToolProtocol: native to providers
daniel-lxs Dec 18, 2025
4d9964b
fix(litellm): merge only native tool defaults with router models
daniel-lxs Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions packages/types/src/providers/chutes.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "DeepSeek R1 0528 model.",
Expand All @@ -60,6 +62,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "DeepSeek R1 model.",
Expand All @@ -69,6 +73,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "DeepSeek V3 model.",
Expand All @@ -78,6 +84,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "DeepSeek V3.1 model.",
Expand All @@ -87,6 +95,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0.23,
outputPrice: 0.9,
description:
Expand All @@ -97,6 +107,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 1.0,
outputPrice: 3.0,
description:
Expand All @@ -107,6 +119,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0.25,
outputPrice: 0.35,
description:
Expand All @@ -117,6 +131,8 @@ export const chutesModels = {
contextWindow: 131072, // From Groq
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Unsloth Llama 3.3 70B Instruct model.",
Expand All @@ -126,6 +142,8 @@ export const chutesModels = {
contextWindow: 512000,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "ChutesAI Llama 4 Scout 17B Instruct model, 512K context.",
Expand All @@ -135,6 +153,8 @@ export const chutesModels = {
contextWindow: 128000,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Unsloth Mistral Nemo Instruct model.",
Expand All @@ -144,6 +164,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Unsloth Gemma 3 12B IT model.",
Expand All @@ -153,6 +175,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Nous DeepHermes 3 Llama 3 8B Preview model.",
Expand All @@ -162,6 +186,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Unsloth Gemma 3 4B IT model.",
Expand All @@ -171,6 +197,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Nvidia Llama 3.3 Nemotron Super 49B model.",
Expand All @@ -180,6 +208,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Nvidia Llama 3.1 Nemotron Ultra 253B model.",
Expand All @@ -189,6 +219,8 @@ export const chutesModels = {
contextWindow: 256000,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "ChutesAI Llama 4 Maverick 17B Instruct FP8 model.",
Expand All @@ -198,6 +230,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "DeepSeek V3 Base model.",
Expand All @@ -207,6 +241,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "DeepSeek R1 Zero model.",
Expand All @@ -216,6 +252,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "DeepSeek V3 (0324) model.",
Expand All @@ -225,6 +263,8 @@ export const chutesModels = {
contextWindow: 262144,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Qwen3 235B A22B Instruct 2507 model with 262K context window.",
Expand All @@ -234,6 +274,8 @@ export const chutesModels = {
contextWindow: 40960,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Qwen3 235B A22B model.",
Expand All @@ -243,6 +285,8 @@ export const chutesModels = {
contextWindow: 40960,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Qwen3 32B model.",
Expand All @@ -252,6 +296,8 @@ export const chutesModels = {
contextWindow: 40960,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Qwen3 30B A3B model.",
Expand All @@ -261,6 +307,8 @@ export const chutesModels = {
contextWindow: 40960,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Qwen3 14B model.",
Expand All @@ -270,6 +318,8 @@ export const chutesModels = {
contextWindow: 40960,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Qwen3 8B model.",
Expand All @@ -279,6 +329,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Microsoft MAI-DS-R1 FP8 model.",
Expand All @@ -288,6 +340,8 @@ export const chutesModels = {
contextWindow: 163840,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "TNGTech DeepSeek R1T Chimera model.",
Expand All @@ -297,6 +351,8 @@ export const chutesModels = {
contextWindow: 151329,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description:
Expand All @@ -307,6 +363,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description:
Expand All @@ -317,6 +375,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 1,
outputPrice: 3,
description: "GLM-4.5-turbo model with 128K token context window, optimized for fast inference.",
Expand All @@ -326,6 +386,8 @@ export const chutesModels = {
contextWindow: 202752,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description:
Expand All @@ -336,6 +398,8 @@ export const chutesModels = {
contextWindow: 202752,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 1.15,
outputPrice: 3.25,
description: "GLM-4.6-turbo model with 200K-token context window, optimized for fast inference.",
Expand All @@ -345,6 +409,8 @@ export const chutesModels = {
contextWindow: 128000,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description:
Expand All @@ -355,6 +421,8 @@ export const chutesModels = {
contextWindow: 262144,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description: "Qwen3 Coder 480B A35B Instruct FP8 model, optimized for coding tasks.",
Expand All @@ -364,6 +432,8 @@ export const chutesModels = {
contextWindow: 75000,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0.1481,
outputPrice: 0.5926,
description: "Moonshot AI Kimi K2 Instruct model with 75k context window.",
Expand All @@ -373,6 +443,8 @@ export const chutesModels = {
contextWindow: 262144,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0.1999,
outputPrice: 0.8001,
description: "Moonshot AI Kimi K2 Instruct 0905 model with 256k context window.",
Expand All @@ -382,6 +454,8 @@ export const chutesModels = {
contextWindow: 262144,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0.077968332,
outputPrice: 0.31202496,
description: "Qwen3 235B A22B Thinking 2507 model with 262K context window.",
Expand All @@ -391,6 +465,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description:
Expand All @@ -401,6 +477,8 @@ export const chutesModels = {
contextWindow: 131072,
supportsImages: false,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0,
outputPrice: 0,
description:
Expand All @@ -411,6 +489,8 @@ export const chutesModels = {
contextWindow: 262144,
supportsImages: true,
supportsPromptCache: false,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 0.16,
outputPrice: 0.65,
description:
Expand Down
1 change: 1 addition & 0 deletions packages/types/src/providers/lite-llm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ export const litellmDefaultModelInfo: ModelInfo = {
supportsImages: true,
supportsPromptCache: true,
supportsNativeTools: true,
defaultToolProtocol: "native",
inputPrice: 3.0,
outputPrice: 15.0,
cacheWritesPrice: 3.75,
Expand Down
2 changes: 2 additions & 0 deletions packages/types/src/providers/minimax.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export const minimaxModels = {
supportsImages: false,
supportsPromptCache: true,
supportsNativeTools: true,
defaultToolProtocol: "native",
preserveReasoning: true,
inputPrice: 0.3,
outputPrice: 1.2,
Expand All @@ -28,6 +29,7 @@ export const minimaxModels = {
supportsImages: false,
supportsPromptCache: true,
supportsNativeTools: true,
defaultToolProtocol: "native",
preserveReasoning: true,
inputPrice: 0.3,
outputPrice: 1.2,
Expand Down
Loading
Loading