Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 84 additions & 12 deletions apps/gateway/src/anthropic/anthropic.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,12 +88,52 @@ const anthropicMessageSchema = z.object({
.optional(),
});

const anthropicToolSchema = z.object({
// Standard Anthropic "custom" tools: a name plus a JSON schema describing the
// parameters the model should produce.
const anthropicCustomToolSchema = z.object({
type: z.literal("custom").optional(),
name: z.string(),
description: z.string(),
description: z.string().optional(),
input_schema: z.record(z.unknown()),
cache_control: z
.object({
type: z.enum(["ephemeral"]),
ttl: z.enum(["5m", "1h"]).optional(),
})
.nullish(),
});

// Anthropic server-side tools (e.g. web_search_20250305, code_execution_*).
// These are executed by Anthropic, carry a versioned `type` instead of an
// `input_schema`, and must not be validated as custom tools.
const anthropicServerToolSchema = z.object({
type: z.string(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject explicit custom tools missing schemas

Because the server-tool schema accepts any string type, a request like tools: [{ type: "custom", name: "get_weather", description: "..." }] no longer fails validation for the missing input_schema: it misses the custom-tool union arm, matches this server-tool arm, and is then silently dropped as an unsupported server tool. This regresses malformed custom-tool validation for clients that include the explicit type: "custom" field that this same schema allows for valid custom tools.

Useful? React with 👍 / 👎.

name: z.string(),
max_uses: z.number().optional(),
allowed_domains: z.array(z.string()).optional(),
blocked_domains: z.array(z.string()).optional(),
user_location: z
.object({
type: z.literal("approximate").optional(),
city: z.string().optional(),
region: z.string().optional(),
country: z.string().optional(),
timezone: z.string().optional(),
})
.optional(),
cache_control: z
.object({
type: z.enum(["ephemeral"]),
ttl: z.enum(["5m", "1h"]).optional(),
})
.nullish(),
});

const anthropicToolSchema = z.union([
anthropicCustomToolSchema,
anthropicServerToolSchema,
]);

const anthropicRequestSchema = z.object({
model: z.string().openapi({
description: "The model to use for completion",
Expand Down Expand Up @@ -522,17 +562,49 @@ anthropic.openapi(messages, async (c) => {
}
}

// Transform tools if provided
// Transform tools if provided. Custom tools map to OpenAI function tools;
// Anthropic server-side tools (e.g. web_search_20250305) carry a versioned
// `type` and no `input_schema`, so they're translated to the internal
// `web_search` tool the chat completions endpoint understands. Server tools
// we can't represent are dropped (with a warning) rather than rejected.
let openaiTools;
if (anthropicRequest.tools) {
openaiTools = anthropicRequest.tools.map((tool) => ({
type: "function",
function: {
name: tool.name,
description: tool.description,
parameters: tool.input_schema,
},
}));
openaiTools = anthropicRequest.tools
.map((tool) => {
if ("input_schema" in tool) {
return {
type: "function",
function: {
name: tool.name,
description: tool.description,
parameters: tool.input_schema,
},
};
}

if (tool.type.startsWith("web_search")) {
return {
type: "web_search",
...(tool.max_uses !== undefined ? { max_uses: tool.max_uses } : {}),
Comment on lines +585 to +588

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve Anthropic web-search response blocks

When a native /v1/messages request reaches this branch and Claude actually performs a web search, Anthropic returns server_tool_use / web_search_tool_result content blocks and text citations in the Messages response (docs). The inner chat path converts those into OpenAI-style text plus message.annotations, but the outer /v1/messages response builder only reconstructs content from message.content and tool_calls, so citations and encrypted search-result content are dropped; Anthropic clients lose the source data needed for display and multi-turn search conversations.

Useful? React with 👍 / 👎.

...(tool.user_location
? { user_location: tool.user_location }
: {}),
...(tool.allowed_domains
? { allowed_domains: tool.allowed_domains }
: {}),
...(tool.blocked_domains
? { blocked_domains: tool.blocked_domains }
: {}),
};
Comment on lines +585 to +598

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve server web_search settings

When a native /v1/messages request includes Anthropic's web search tool with allowed_domains/blocked_domains or the newer web_search_20260209 type, this branch collapses it to the generic chat web_search object. Downstream prepareRequestBody for Anthropic always re-emits web_search_20250305 and only copies max_uses, so accepted requests can silently lose domain constraints and downgrade dynamic filtering before reaching Anthropic.

Useful? React with 👍 / 👎.

}

logger.warn("Dropping unsupported Anthropic server tool", {
type: tool.type,
name: tool.name,
});
return null;
})
.filter((tool): tool is NonNullable<typeof tool> => tool !== null);
}

// Build OpenAI request
Expand All @@ -544,7 +616,7 @@ anthropic.openapi(messages, async (c) => {
stream: anthropicRequest.stream,
};

if (openaiTools) {
if (openaiTools && openaiTools.length > 0) {
openaiRequest.tools = openaiTools;
}

Expand Down
150 changes: 150 additions & 0 deletions apps/gateway/src/api.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5065,4 +5065,154 @@ describe("api", () => {
expect(Number(logs[0].cost)).toBe(0);
});
});

describe("native /v1/messages server-side tools", () => {
// Anthropic server-side tools (e.g. web_search_20250305) carry a versioned
// `type` and no `description`/`input_schema`. They must pass validation and
// be forwarded to the provider, not rejected as malformed custom tools.
test("forwards Anthropic web_search server tool to the provider", async () => {
await db.insert(tables.apiKey).values({
id: "token-id",
token: "real-token",
projectId: "project-id",
description: "Test API Key",
createdBy: "user-id",
});
await db.insert(tables.providerKey).values({
id: "provider-key-id",
token: "sk-test-key",
provider: "anthropic",
organizationId: "org-id",
baseUrl: mockServerUrl,
});

let capturedBody: any;
const originalFetch = globalThis.fetch;
const fetchSpy = vi
.spyOn(globalThis, "fetch")
.mockImplementation(async (input, init) => {
const url =
typeof input === "string"
? input
: input instanceof URL
? input.toString()
: input.url;

if (url.includes(`${mockServerUrl}/v1/messages`)) {
capturedBody = JSON.parse(init?.body as string);
return new Response(
JSON.stringify({
id: "msg_ws",
type: "message",
role: "assistant",
model: "claude-sonnet-4-6",
content: [
{
type: "text",
text: "The latest version is web_search_20250305.",
},
],
stop_reason: "end_turn",
stop_sequence: null,
usage: { input_tokens: 50, output_tokens: 10 },
}),
{
status: 200,
headers: { "Content-Type": "application/json" },
},
);
}

return await originalFetch(input as RequestInfo | URL, init);
});

try {
const res = await app.request("/v1/messages", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: "Bearer real-token",
"x-no-fallback": "true",
},
body: JSON.stringify({
model: "anthropic/claude-sonnet-4-6",
max_tokens: 1024,
messages: [
{
role: "user",
content:
"Search the web for the latest Anthropic web search tool version.",
},
],
tools: [
{
type: "web_search_20250305",
name: "web_search",
max_uses: 3,
allowed_domains: ["anthropic.com", "docs.anthropic.com"],
user_location: {
type: "approximate",
city: "San Francisco",
country: "US",
},
},
],
}),
});

// The request must NOT be rejected with a ZodError about missing
// `description`/`input_schema` on the server tool.
expect(res.status).toBe(200);

// The server tool must reach the Anthropic provider as a native
// web_search tool, preserving its configuration (max_uses, domain
// filters, user_location).
const forwardedTools = capturedBody?.tools ?? [];
const forwardedWebSearch = forwardedTools.find(
(t: { type?: string }) => t.type === "web_search_20250305",
);
expect(forwardedWebSearch).toBeDefined();
expect(forwardedWebSearch.max_uses).toBe(3);
expect(forwardedWebSearch.allowed_domains).toEqual([
"anthropic.com",
"docs.anthropic.com",
]);
expect(forwardedWebSearch.user_location).toEqual({
type: "approximate",
city: "San Francisco",
country: "US",
});
} finally {
fetchSpy.mockRestore();
}
});

test("still rejects a custom tool missing input_schema", async () => {
await db.insert(tables.apiKey).values({
id: "token-id",
token: "real-token",
projectId: "project-id",
description: "Test API Key",
createdBy: "user-id",
});

const res = await app.request("/v1/messages", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: "Bearer real-token",
},
body: JSON.stringify({
model: "anthropic/claude-sonnet-4-6",
max_tokens: 1024,
messages: [{ role: "user", content: "hi" }],
tools: [{ name: "get_weather", description: "Get the weather" }],
}),
});

expect(res.status).toBe(400);
const json = await res.json();
expect(JSON.stringify(json)).toContain("input_schema");
});
});
});
2 changes: 2 additions & 0 deletions apps/gateway/src/chat/chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1534,6 +1534,8 @@ chat.openapi(completions, async (c) => {
user_location: foundTool.user_location,
search_context_size: foundTool.search_context_size,
max_uses: foundTool.max_uses,
allowed_domains: foundTool.allowed_domains,
blocked_domains: foundTool.blocked_domains,
};
// Remove the web_search tool from the tools array so it's not sent as a regular tool
tools.splice(webSearchToolIndex, 1);
Expand Down
2 changes: 2 additions & 0 deletions apps/gateway/src/chat/schemas/completions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,8 @@ export const completionsRequestSchema = z.object({
.optional(),
search_context_size: z.enum(["low", "medium", "high"]).optional(),
max_uses: z.number().optional(),
allowed_domains: z.array(z.string()).optional(),
blocked_domains: z.array(z.string()).optional(),
}),
]),
)
Expand Down
13 changes: 13 additions & 0 deletions packages/actions/src/prepare-request-body.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1829,6 +1829,19 @@ export async function prepareRequestBody(
if (webSearchTool.max_uses) {
webSearch.max_uses = webSearchTool.max_uses;
}
// Anthropic accepts either allowed_domains or blocked_domains, not both.
if (webSearchTool.allowed_domains?.length) {
webSearch.allowed_domains = webSearchTool.allowed_domains;
} else if (webSearchTool.blocked_domains?.length) {
webSearch.blocked_domains = webSearchTool.blocked_domains;
}
if (webSearchTool.user_location) {
// Anthropic requires the discriminating `type: "approximate"`.
webSearch.user_location = {
...webSearchTool.user_location,
type: "approximate",
};
}
requestBody.tools.push(webSearch);
}

Expand Down
13 changes: 12 additions & 1 deletion packages/models/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -519,13 +519,14 @@ export function hasMaxTokens(
export interface WebSearchTool {
type: "web_search";
/**
* User location for localized search results (OpenAI)
* User location for localized search results (OpenAI and Anthropic)
*/
user_location?: {
type: "approximate";
city?: string;
region?: string;
country?: string;
timezone?: string;
};
/**
* Controls how much context is retrieved from the web (OpenAI)
Expand All @@ -538,6 +539,16 @@ export interface WebSearchTool {
* Maximum number of web searches to perform (Anthropic)
*/
max_uses?: number;
/**
* Restrict search results to these domains (Anthropic). Mutually exclusive
* with blocked_domains.
*/
allowed_domains?: string[];
/**
* Exclude these domains from search results (Anthropic). Mutually exclusive
* with allowed_domains.
*/
blocked_domains?: string[];
}

/**
Expand Down
Loading