Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions open-sse/config/providerRegistry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1199,9 +1199,9 @@ export const REGISTRY: Record<string, RegistryEntry> = {
authType: "apikey",
authHeader: "bearer",
models: [
{ id: "gpt-oss-120b", name: "GPT OSS 120B", toolCalling: false },
{ id: "openai/gpt-oss-120b", name: "GPT OSS 120B (OpenAI Prefix)", toolCalling: false },
{ id: "openai/gpt-oss-20b", name: "GPT OSS 20B", toolCalling: false },
{ id: "gpt-oss-120b", name: "GPT OSS 120B" },
{ id: "openai/gpt-oss-120b", name: "GPT OSS 120B (OpenAI Prefix)" },
{ id: "openai/gpt-oss-20b", name: "GPT OSS 20B" },
{ id: "meta/llama-3.3-70b-instruct", name: "Llama 3.3 70B" },
{ id: "nvidia/llama-3.3-70b-instruct", name: "Llama 3.3 70B (NVIDIA Prefix)" },
{ id: "meta/llama-4-maverick-17b-128e-instruct", name: "Llama 4 Maverick" },
Expand Down
2 changes: 1 addition & 1 deletion src/lib/modelCapabilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { parseModel, resolveCanonicalProviderModel } from "@omniroute/open-sse/s
import { MODEL_SPECS, getModelSpec, type ModelSpec } from "@/shared/constants/modelSpecs";
import { getSyncedCapability } from "@/lib/modelsDevSync";

const TOOL_CALLING_UNSUPPORTED_PATTERNS = ["gpt-oss-120b", "deepseek-reasoner"];
const TOOL_CALLING_UNSUPPORTED_PATTERNS: string[] = [];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

By emptying TOOL_CALLING_UNSUPPORTED_PATTERNS, the heuristicToolCalling function (lines 126-135) will now return true for any non-empty model string that isn't explicitly defined in MODEL_SPECS, the registry, or synced data.

While this unblocks the intended models, it also changes the default behavior for all unknown models to "supported". If there are other older or specialized models that definitely do not support tool calling and are not in the registry, they will now be incorrectly reported as supporting it. Consider if a minimal blocklist should be maintained for such cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The heuristic was already default-true for all unknown models before this PR -- the blocklist only contained these 2 entries. Emptying it removes 2 false positives; it does not change the paradigm.

Models that genuinely do not support tool calling should have toolCalling: false set explicitly in their registry entry or a ModelSpec with supportsTools: false, both of which take priority over the heuristic (resolution layers 1-3). The heuristic at layer 4 is intentionally optimistic as a fallback for models with no capability data.

No other models in the codebase relied on this blocklist for correctness -- all other tool-calling restrictions are handled by explicit registry fields or model specs.

const REASONING_UNSUPPORTED_PATTERNS = [
"antigravity/claude-sonnet-4-6",
"antigravity/claude-sonnet-4-5",
Expand Down
17 changes: 17 additions & 0 deletions tests/unit/model-capabilities-registry.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,20 @@ test("canonical model capability resolver merges models.dev data and keeps stati
32768
);
});

test("GPT OSS and DeepSeek Reasoner models support tool calling", () => {
// GPT OSS models should not be blocked by the heuristic
assert.equal(modelCapabilities.supportsToolCalling("nvidia/gpt-oss-120b"), true);
assert.equal(modelCapabilities.supportsToolCalling("gpt-oss-120b"), true);
assert.equal(modelCapabilities.supportsToolCalling("openai/gpt-oss-20b"), true);

// DeepSeek Reasoner supports tool calling
assert.equal(modelCapabilities.supportsToolCalling("deepseek-reasoner"), true);
assert.equal(modelCapabilities.supportsToolCalling("deepseek/deepseek-r1"), true);

// Full capability resolution
const gptOss = modelCapabilities.getResolvedModelCapabilities("nvidia/gpt-oss-120b");
assert.equal(gptOss.toolCalling, true);
const deepseek = modelCapabilities.getResolvedModelCapabilities("deepseek/deepseek-reasoner");
assert.equal(deepseek.toolCalling, true);
});
4 changes: 2 additions & 2 deletions tests/unit/services-branch-hardening.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ test("model capability helpers cover denylist, empty input and default-safe path
assert.equal(modelCapabilities.supportsReasoning("missing-provider/tool"), true);

assert.equal(modelCapabilities.supportsToolCalling(""), false);
assert.equal(modelCapabilities.supportsToolCalling("openai/gpt-oss-120b"), false);
assert.equal(modelCapabilities.supportsToolCalling("deepseek-reasoner"), false);
assert.equal(modelCapabilities.supportsToolCalling("openai/gpt-oss-120b"), true);
assert.equal(modelCapabilities.supportsToolCalling("deepseek-reasoner"), true);
Comment on lines +82 to +83

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This test imports modelCapabilities from ../../open-sse/services/modelCapabilities.ts (line 5), but the changes in this PR are applied to src/lib/modelCapabilities.ts.

Unless these two paths point to the same file (e.g., via a symlink), updating these assertions to true will likely cause this test to fail because the underlying file in open-sse/services/ has not been modified to remove the blocklist. Please verify if open-sse/services/modelCapabilities.ts also needs to be updated or if there is a duplicate implementation that should be unified.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open-sse/services/modelCapabilities.ts is a barrel re-export of src/lib/modelCapabilities.ts:

export {
  getResolvedModelCapabilities,
  supportsReasoning,
  supportsToolCalling,
} from "../../src/lib/modelCapabilities.ts";

Both import paths resolve to the same implementation. The test assertions are correct and pass against the modified code.

assert.equal(
modelCapabilities.supportsToolCalling("openai/nonexistent-default-safe-model"),
true
Expand Down