feat(api): add Anthropic endpoint - #749
Conversation
![terragon-labs[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
…nsformation - Introduce new /v1/anthropic/messages POST endpoint - Implement request schema validation and transformation from Anthropic to OpenAI format - Implement response transformation from OpenAI to Anthropic format - Add support for multi-modal message content and tools - Add Anthropic routing and integration in gateway - Add API documentation for Anthropic messages This enables handling Anthropic-style chat messages via the gateway with compatibility to existing OpenAI chat completions. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughAdds an Anthropic-compatible /v1/messages endpoint in the gateway that translates Anthropic requests/responses to/from OpenAI chat completions. Updates provider API to group/deduplicate tool IDs and gate OpenAI Responses API via env flag. Adds docs (feature page, API ref), UI content (blog, changelog), a new Anthropic model entry, and a .gitignore rule. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Client
participant AnthropicEndpoint as Gateway /v1/messages
participant Transformer as Anthropic↔OpenAI Mapper
participant OpenAI as OpenAI /v1/chat/completions
participant ProviderAPI as Provider API (tool IDs, Responses API gating)
Client->>AnthropicEndpoint: POST /v1/messages (Anthropic JSON)
AnthropicEndpoint->>Transformer: Validate & map to OpenAI format
Transformer-->>AnthropicEndpoint: OpenAI-compatible payload
AnthropicEndpoint->>ProviderAPI: Prepare request (consider USE_RESPONSES_API)
ProviderAPI-->>AnthropicEndpoint: Endpoint + body
AnthropicEndpoint->>OpenAI: Forward request
OpenAI-->>AnthropicEndpoint: Response (non-stream)
AnthropicEndpoint->>Transformer: Map to Anthropic content blocks
note over Transformer: Build text/tool_use blocks, stop_reason, usage
Transformer-->>AnthropicEndpoint: Anthropic-format response
AnthropicEndpoint-->>Client: 200 Anthropic message response
opt Streaming (future)
note over AnthropicEndpoint,Client: Placeholder 501 Not Implemented
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
changed the title
- Renamed `/v1/anthropic/messages` to `/v1/messages` - Adjusted route definitions in the gateway - Updated API documentation for the new endpoint structure This simplifies the routing and aligns it with the overall API design.
- Add support for system prompt to accept either a string or an array of structured objects. - Enhance schema validation for improved robustness. - Update system prompt description in OpenAPI documentation.
- Introduced Claude 3.5 Haiku (2024-10-22) model with pricing, context size, and feature details. - Added support for tools and streaming capabilities.
- Extend schema to include tool role and tool-related content types. - Implement transformations for tool messages, tool use, and tool results. - Enhance assistant message handling for multi-modal tool interactions. - Add tool_call_id support for improved traceability in tool messages.
- Extend schema to include tool role and tool-related content types. - Implement transformations for tool messages, tool use, and tool results. - Enhance assistant message handling for multi-modal tool interactions. - Add tool_call_id support for improved traceability in tool messages.
- Deleted commented-out `console.log` statements for cleaner code.
- Includes detailed logs for dev processes across multiple packages. - Captures API, UI, Docs, and Gateway startup information. - Aids in debugging by tracking runtime events and errors.
- Updates .gitignore to ignore all .txt files.
- Use `max_completion_tokens` for GPT-5 models instead of `max_tokens`. - Retain `max_tokens` for non-GPT-5 models.
- Introduced a new blog post explaining how to configure Claude Code to access diverse LLM models via LLMGateway. - Covers API setup, model selection, advanced configuration, and persistent setup. - Provides benefits like cost optimization, performance tracking, and enhanced reliability.
- Documented new feature enabling Claude Code configuration via LLMGateway. - Highlighted environment variable setup, model flexibility, and cost optimization. - Included detailed setup guide and popular model suggestions.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
left a comment
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (7)
packages/models/src/provider-api.ts (6)
53-61: btoa not available in Node; base64-encoding will throw server-sideThis uses btoa for data URLs, which is undefined in Node runtimes and will crash.
Use Buffer when available, falling back to btoa only in browsers:
-const base64Data = isBase64 ? data : btoa(data); +const base64Data = isBase64 + ? data + : (typeof Buffer !== "undefined" + ? Buffer.from(data, "utf-8").toString("base64") + : btoa(data));
116-122: Inefficient arrayBuffer ➜ base64 conversion; risks OOMBuilding a huge binary string then btoa is slow and memory-heavy for large images.
Prefer Buffer directly (works in Node and Bun) with a browser fallback:
-const uint8Array = new Uint8Array(arrayBuffer); -const binaryString = Array.from(uint8Array, (byte) => - String.fromCharCode(byte), -).join(""); -const base64 = btoa(binaryString); +const base64 = + typeof Buffer !== "undefined" + ? Buffer.from(arrayBuffer as ArrayBuffer).toString("base64") + : btoa(String.fromCharCode(...new Uint8Array(arrayBuffer)));
650-661: Anthropic API: remove unsupported penaltiesAnthropic Messages API does not accept frequency_penalty or presence_penalty; passing them can cause 400s.
- if (frequency_penalty !== undefined) { - requestBody.frequency_penalty = frequency_penalty; - } - if (presence_penalty !== undefined) { - requestBody.presence_penalty = presence_penalty; - }
415-419: Avoidanyto comply with TS guidelinesThe codebase standards disallow
any. Use a safer structural type.-const requestBody: any = { +const requestBody: Record<string, unknown> = { model: usedModel, messages: processedMessages, stream: stream, };Similarly, prefer precise types for other
anyoccurrences (e.g.,openaiRequest, arrays of messages).
355-361: Remove legacy “anthropic-beta” header – The Anthropic Messages API now only requiresx-api-keyandanthropic-version(withContent-Typeset elsewhere); drop the"anthropic-beta": "tools-2024-04-04"header unless you’re opting into a documented experimental feature. [1][2]
70-76: Block private/internal hosts before fetching images
- In packages/models/src/provider-api.ts (around lines 78–87), processImageUrl only enforces HTTPS in production but still allows SSRF to internal endpoints (localhost, 127.0.0.1, RFC1918 ranges, 169.254.169.254).
- Introduce a guard before the fetch call to detect and reject private or metadata hosts:
+ function isPrivateHost(urlStr: string): boolean { + try { + const h = new URL(urlStr).hostname.toLowerCase(); + if (["localhost","127.0.0.1","::1"].includes(h)) return true; + if (/^(10\.|192\.168\.|172\.(1[6-9]|2\d|3[0-1])\.)/.test(h)) return true; + if (h === "169.254.169.254") return true; + return false; + } catch { + return true; + } + } // Validate HTTPS URLs only in production environment if (!url.startsWith("https://") && isProd) { … } + if (isProd && isPrivateHost(url)) { + logger.warn("Blocked potential SSRF to private host", { host: new URL(url).hostname }); + throw new Error("Image URL host is not allowed"); + } const response = await fetch(url);
- For stronger protection, resolve the hostname (A/AAAA) and reject any IP in private ranges before fetch.
apps/gateway/src/index.ts (1)
61-80: Don’t attempt to JSON-serialize HTTPException.res (Response object)Including error.res in a JSON body will throw (circular structure) and can mask the original error. Return the provided Response when present; otherwise return structured JSON.
-app.onError((error, c) => { +app.onError(async (error, c) => { if (error instanceof HTTPException) { const status = error.status; @@ - return c.json( - { - error: true, - status, - message: error.message || "An error occurred", - ...(error.res ? { details: error.res } : {}), - }, - status, - ); + if (error.res) { + return error.getResponse(); + } + return c.json( + { + error: true, + status, + message: error.message || "An error occurred", + }, + status, + ); }
🧹 Nitpick comments (11)
.gitignore (1)
16-16: Scope down '*.txt' ignore to avoid hiding important docsIgnoring all .txt can accidentally exclude LICENSE/NOTICE/SECURITY/README files or content assets from PRs.
Consider adding common exceptions:
*.txt +!LICENSE.txt +!NOTICE.txt +!SECURITY.txt +!README.txt +!CHANGELOG.txt +!CONTRIBUTING.txtPlease confirm if any docs/assets under apps/ui or apps/docs rely on .txt files that should remain tracked.
packages/models/src/provider-api.ts (1)
256-263: Log sanitation: avoid printing full image URLsFull URLs can contain PII or tokens and will leak to logs.
-logger.error(`Failed to fetch image ${part.image_url.url}`, { +logger.error("Failed to fetch image", { + url: part.image_url.url?.slice(0, 50) + "...", err: error instanceof Error ? error : new Error(String(error)), });apps/docs/content/(api)/v1_messages.mdx (1)
17-17: Commit or un-ignore openapi.json for docs
.gitignore ignoresopenapi.jsonglobally (line 10), and there’s noapps/docs/openapi.json, so<APIPage …>will fail to load the schema. Add a minimalopenapi.jsonunderapps/docsor scope the ignore rule to keep it in version control.apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md (1)
38-48: Use headings instead of emphasis for section titles (markdownlint MD036)Switch bolded lines to headings for consistency and accessibility.
-**OpenAI Models** +### OpenAI Models ... -**Anthropic Models** +### Anthropic Models ... -**Cost-Effective Alternatives** +### Cost-Effective Alternativesapps/ui/src/content/blog/2025-01-15-configure-claude-code-with-llmgateway.md (1)
29-32: Environment variable example: provider prefix for GLMAlign with gateway’s convention if models require provider prefixes.
-export ANTHROPIC_MODEL=glm-4.5v # choose your model on llmgateway which supports tool calls +export ANTHROPIC_MODEL=zhipu/glm-4.5v # example: provider/model (adjust to your configured provider)Confirm the correct provider key for GLM in your registry.
apps/gateway/src/anthropic/anthropic.ts (4)
480-489: Return requested model id for Anthropic compatibilityOpenAI may echo a different model name; Anthropic clients expect the requested model id here.
- model: openaiResponse.model, + model: anthropicRequest.model,
455-462: Handle array content from OpenAI responsesIf message.content is an array (multimodal), current code drops it.
-if (openaiResponse.choices?.[0]?.message?.content) { - content.push({ type: "text", text: openaiResponse.choices[0].message.content }); -} +const cmsg = openaiResponse.choices?.[0]?.message; +if (Array.isArray(cmsg?.content)) { + const text = cmsg.content.filter((p: any) => p.type === "text").map((p: any) => p.text).join(""); + if (text) content.push({ type: "text", text }); +} else if (cmsg?.content) { + content.push({ type: "text", text: cmsg.content }); +}
493-506: Stop-reason mapping: include legacy 'function_call'For older models’ finish_reason, map function_call to tool_use.
switch (finishReason) { case "stop": return "end_turn"; case "length": return "max_tokens"; case "tool_calls": return "tool_use"; + case "function_call": + return "tool_use"; default: return "end_turn"; }
146-153: Schema validation errors: surface structured issuesConcatenated messages are hard to read. Consider returning issues as an array.
Not blocking, but switching to a JSON payload with issues[] improves DX.
Also applies to: 181-190
apps/gateway/src/index.ts (1)
169-171: Drizzle call style consistency with guidelinesGuidelines specify using db().query.
.findFirst(). This uses db.query. If your project exports a factory (db()), switch to it; otherwise please confirm the deviation is intentional.- await db.query.user.findFirst({}); + await db().query.user.findFirst();If db is an instance (not a factory), ignore this and consider adding a short comment to document the convention.
apps/gateway/src/anthropic/index.ts (1)
7-10: Avoid redundant wrapper or ensure it’s the one mountedMain router currently mounts anthropic directly, making this exposed router unused. Either mount this at /v1/anthropic in the main index (recommended for /v1/anthropic/messages) or remove this wrapper to reduce confusion.
Two options:
- Keep wrapper and mount it:
// apps/gateway/src/index.ts -import { anthropic } from "./anthropic/anthropic"; +import { exposed as anthropicRoutes } from "./anthropic"; ... -v1.route("/messages", anthropic); +v1.route("/anthropic", anthropicRoutes);
- Or drop the wrapper and mount the base router with the full prefix:
// apps/gateway/src/index.ts -import { anthropic } from "./anthropic/anthropic"; +import { anthropic } from "./anthropic/anthropic"; ... -v1.route("/messages", anthropic); +v1.route("/anthropic/messages", anthropic);Please pick one to prevent path drift and dead code.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
apps/ui/public/blog/configure-claude-code-with-llmgateway.pngis excluded by!**/*.pngapps/ui/public/changelog/claude-code-configuration-support.pngis excluded by!**/*.png📒 Files selected for processing (9)
.gitignore(1 hunks)apps/docs/content/(api)/v1_messages.mdx(1 hunks)apps/gateway/src/anthropic/anthropic.ts(1 hunks)apps/gateway/src/anthropic/index.ts(1 hunks)apps/gateway/src/index.ts(2 hunks)apps/ui/src/content/blog/2025-01-15-configure-claude-code-with-llmgateway.md(1 hunks)apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md(1 hunks)packages/models/src/models/anthropic.ts(1 hunks)packages/models/src/provider-api.ts(1 hunks)🧰 Additional context used
📓 Path-based instructions (5)
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use localStorage instead of cookies for client-side data persistence
Files:
apps/gateway/src/index.tspackages/models/src/provider-api.tsapps/gateway/src/anthropic/index.tspackages/models/src/models/anthropic.tsapps/gateway/src/anthropic/anthropic.ts**/*.{js,ts}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{js,ts}: Use drizzle with the latest object syntax for database operations
For read queries, always usedb().query.<table>.findMany()ordb().query.<table>.findFirst()Files:
apps/gateway/src/index.tspackages/models/src/provider-api.tsapps/gateway/src/anthropic/index.tspackages/models/src/models/anthropic.tsapps/gateway/src/anthropic/anthropic.ts**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Never use
as anyor: anyin TypeScript files.Files:
apps/gateway/src/index.tspackages/models/src/provider-api.tsapps/gateway/src/anthropic/index.tspackages/models/src/models/anthropic.tsapps/gateway/src/anthropic/anthropic.tsapps/{api,gateway}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
apps/{api,gateway}/**/*.{ts,tsx}: Use Drizzle with the latest object syntax for database operations
For read queries, use db().query..findMany() or db().query.
.findFirst()
apps/{api,gateway}/**/*.{ts,tsx}: Use the Hono framework for backend HTTP services in apps/api and apps/gateway
Use Zod for request/response validation in backend routes and handlers
Maintain OpenAPI/Swagger documentation for backend APIsFiles:
apps/gateway/src/index.tsapps/gateway/src/anthropic/index.tsapps/gateway/src/anthropic/anthropic.tsapps/gateway/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
In apps/gateway (Hono), always use Hono + Zod + OpenAPI for validation and typesafety
Files:
apps/gateway/src/index.tsapps/gateway/src/anthropic/index.tsapps/gateway/src/anthropic/anthropic.ts🧬 Code graph analysis (3)
apps/gateway/src/index.ts (1)
apps/gateway/src/anthropic/anthropic.ts (1)
anthropic(6-6)apps/gateway/src/anthropic/index.ts (1)
apps/gateway/src/anthropic/anthropic.ts (1)
anthropic(6-6)apps/gateway/src/anthropic/anthropic.ts (2)
packages/db/src/schema.ts (1)
message(387-403)packages/db/src/types.ts (2)
tool(18-21)toolCall(35-42)🪛 markdownlint-cli2 (0.17.2)
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md
38-38: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
42-42: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
46-46: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: build / run
- GitHub Check: e2e / run
🔇 Additional comments (3)
apps/docs/content/(api)/v1_messages.mdx (1)
6-7: Docs route/v1/messagesis correct
The gateway mounts theanthropicrouter atv1.route("/messages", anthropic)underapp.route("/v1", v1), so the documentation’s/v1/messagespath matches and requires no change.Likely an incorrect or invalid review comment.
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md (1)
21-24: Model naming consistencyUnprefixed
glm-4.5vmay differ from the gateway’s provider-prefixed model IDs (e.g.,zhipu/glm-4.5).Please verify the exact model id accepted by the gateway and align examples.
apps/ui/src/content/blog/2025-01-15-configure-claude-code-with-llmgateway.md (1)
55-60: Confirm model registry IDs
Ensure thatopenai/gpt-4o-miniandanthropic/claude-3-5-sonnet-20241022exactly match the IDs registered in the LLM Gateway.
| type: "tool_use", | ||
| id: toolCall.id, | ||
| name: toolCall.function.name, | ||
| input: JSON.parse(toolCall.function.arguments || "{}"), | ||
| }); | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Guard JSON.parse on tool call arguments
Malformed JSON from upstream will throw; return the raw string instead.
- input: JSON.parse(toolCall.function.arguments || "{}"),
+ input: (() => {
+ try { return JSON.parse(toolCall.function.arguments || "{}"); }
+ catch { return toolCall.function.arguments || "{}"; }
+ })(),📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| type: "tool_use", | |
| id: toolCall.id, | |
| name: toolCall.function.name, | |
| input: JSON.parse(toolCall.function.arguments || "{}"), | |
| }); | |
| } | |
| type: "tool_use", | |
| id: toolCall.id, | |
| name: toolCall.function.name, | |
| input: (() => { | |
| try { | |
| return JSON.parse(toolCall.function.arguments || "{}"); | |
| } catch { | |
| return toolCall.function.arguments || "{}"; | |
| } | |
| })(), | |
| }); | |
| } |
🤖 Prompt for AI Agents
In apps/gateway/src/anthropic/anthropic.ts around lines 466 to 471 the code
unconditionally calls JSON.parse on toolCall.function.arguments which will throw
on malformed JSON; wrap the parse in a try/catch (and handle null/undefined by
treating it as "{}" or empty string as appropriate), attempt to parse the string
and on any parse error fall back to using the original raw string (or an empty
object if that fits callers), then pass that safe value into the object so
malformed upstream JSON does not throw.
| import { HTTPException } from "hono/http-exception"; | ||
| import { z } from "zod"; | ||
|
|
||
| import { anthropic } from "./anthropic/anthropic"; |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Route prefix mismatch with PR intent; mount under /v1/anthropic/messages
PR summary/docs say the endpoint should be /v1/anthropic/messages, but this wires it at /v1/messages. Also, prefer importing the exposed router wrapper to avoid deep path imports and to keep prefixes consistent.
Apply:
-import { anthropic } from "./anthropic/anthropic";
+import { exposed as anthropicRoutes } from "./anthropic";
...
-v1.route("/messages", anthropic);
+// Results in: /v1/anthropic/messages
+v1.route("/anthropic", anthropicRoutes);Please confirm that the docs and any clients are aligned on /v1/anthropic/messages after this change.
Also applies to: 195-195
🏁 Script executed:
#!/bin/bash
# Display the import and route registration section from the gateway index file
sed -n '1,200p' apps/gateway/src/index.tsLength of output: 4494
🏁 Script executed:
ls -R apps/gateway/src/anthropicLength of output: 49
🏁 Script executed:
#!/bin/bash
# Show exports in apps/gateway/src/anthropic/index.ts to verify available router wrappers
sed -n '1,200p' apps/gateway/src/anthropic/index.tsLength of output: 231
Mount Anthropic routes under /v1/anthropic and use the exposed router wrapper
- In
apps/gateway/src/index.ts, replace the deep import and handler registration:-import { anthropic } from "./anthropic/anthropic"; -v1.route("/messages", anthropic); +import { exposed as anthropicRoutes } from "./anthropic"; +// mounts at /v1/anthropic/messages +v1.route("/anthropic", anthropicRoutes);
- Confirm that the API docs and any downstream clients reference
/v1/anthropic/messagesafter this change.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import { anthropic } from "./anthropic/anthropic"; | |
| import { exposed as anthropicRoutes } from "./anthropic"; | |
| // mounts at /v1/anthropic/messages | |
| v1.route("/anthropic", anthropicRoutes); |
🤖 Prompt for AI Agents
In apps/gateway/src/index.ts around line 9, the code deep-imports the handler
and mounts it directly; replace the deep import with the package's exposed
router wrapper (import from "./anthropic" or the module's main export) and mount
it under the path "/v1/anthropic" using app.use('/v1/anthropic',
<exposedRouter>); ensure the exported router is used (not individual handlers),
update the registration so requests go to /v1/anthropic/messages, and update API
docs and any downstream clients to reference /v1/anthropic/messages accordingly.
- Added the `draft: true` field to the changelog for clarity.
- Introduced `streamSSE` to handle event-streaming responses. - Enhanced response schema to include `text/event-stream` support. - Implemented streaming logic with event emission for content blocks, tool usage, and stop reasons.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (2)
apps/gateway/src/anthropic/anthropic.ts (2)
237-245: Tool message content must be a string (legacy “function” role).- content: message.content, + content: + typeof message.content === "string" + ? message.content + : JSON.stringify(message.content),
671-679: Guard JSON.parse on tool call arguments.- input: JSON.parse(toolCall.function.arguments || "{}"), + input: (() => { + try { return JSON.parse(toolCall.function.arguments || "{}"); } + catch { return toolCall.function.arguments || "{}"; } + })(),
🧹 Nitpick comments (5)
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md (2)
39-49: Replace bold pseudo-headings with proper headings (MD036).-**OpenAI Models** +### OpenAI Models -**Anthropic Models** +### Anthropic Models -**Cost-Effective Alternatives** +### Cost-Effective Alternatives
6-6: Branding consistency: “LLMGateway” vs “LLM Gateway”.Use one form consistently (site seems to prefer “LLMGateway”). Example:
-summary: "Configure Claude Code to use any LLM model through LLMGateway's unified API with simple environment variable setup." +summary: "Configure Claude Code to use any LLM model through LLMGateway’s unified API with simple environment variable setup." - alt: "Claude Code configuration support on LLM Gateway" + alt: "Claude Code configuration support on LLMGateway"Also applies to: 10-10
apps/gateway/src/anthropic/anthropic.ts (3)
427-435: Only forward headers when present (avoid empty Authorization).- headers: { - "Content-Type": "application/json", - Authorization: c.req.header("Authorization") || "", - "x-request-id": c.req.header("x-request-id") || "", - "x-source": c.req.header("x-source") || "", - "x-debug": c.req.header("x-debug") || "", - }, + headers: (() => { + const h: Record<string, string> = { "Content-Type": "application/json" }; + for (const k of ["Authorization","x-request-id","x-source","x-debug"] as const) { + const v = c.req.header(k); + if (v) h[k] = v; + } + return h; + })(),
439-444: Don’t narrow HTTPException status to specific union.- throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, { + throw new HTTPException(response.status, {
446-655: Consider tests for SSE bridging (happy path + out-of-order deltas).Happy to add integration tests that:
- Verify text-first and tool-first streams.
- Assert correct content_block indices and event order.
- Cover CRLF chunking and malformed JSON argument chunks.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
apps/gateway/src/anthropic/anthropic.ts(1 hunks)apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md(1 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use localStorage instead of cookies for client-side data persistence
Files:
apps/gateway/src/anthropic/anthropic.ts
**/*.{js,ts}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{js,ts}: Use drizzle with the latest object syntax for database operations
For read queries, always usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
apps/gateway/src/anthropic/anthropic.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Never use
as anyor: anyin TypeScript files.
Files:
apps/gateway/src/anthropic/anthropic.ts
apps/{api,gateway}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
.findMany() or db().query.
apps/{api,gateway}/**/*.{ts,tsx}: Use Drizzle with the latest object syntax for database operations
For read queries, use db().query..findFirst()
apps/{api,gateway}/**/*.{ts,tsx}: Use the Hono framework for backend HTTP services in apps/api and apps/gateway
Use Zod for request/response validation in backend routes and handlers
Maintain OpenAPI/Swagger documentation for backend APIsFiles:
apps/gateway/src/anthropic/anthropic.tsapps/gateway/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
In apps/gateway (Hono), always use Hono + Zod + OpenAPI for validation and typesafety
Files:
apps/gateway/src/anthropic/anthropic.ts🧬 Code graph analysis (1)
apps/gateway/src/anthropic/anthropic.ts (1)
packages/db/src/types.ts (2)
tool(18-21)toolCall(35-42)🪛 markdownlint-cli2 (0.17.2)
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md
39-39: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
43-43: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
47-47: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: e2e / run
- GitHub Check: build / run
🔇 Additional comments (5)
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md (2)
7-7: Confirm draft status.If this is meant to go live with the feature, set draft: false.
21-25: Use ANTHROPIC_API_KEY instead of ANTHROPIC_AUTH_TOKEN
The Anthropic/Claude Code CLI reads ANTHROPIC_API_KEY, not ANTHROPIC_AUTH_TOKEN. Update in both changelog and blog snippets:-export ANTHROPIC_AUTH_TOKEN=llmgtwy_.... # your llmgateway.io api key here +export ANTHROPIC_API_KEY=llmgtwy_.... # your llmgateway.io API key hereAlso verify whether ANTHROPIC_BASE_URL needs to include the Anthropic prefix (e.g.
/v1/anthropic).apps/gateway/src/anthropic/anthropic.ts (3)
135-141: Confirm mount path matches docs (/v1/anthropic/messages vs /v1/messages).Route is defined with path "/", so correctness depends on router mounting. Please confirm the resulting public path matches the PR objective and docs.
72-109: Schema coverage looks solid.Good Anthropic request schema surface (model, messages, max_tokens, system, temperature, tools, stream) with OpenAPI integration.
412-417: Model passthrough: verify mapping is handled upstream.You forward anthropicRequest.model directly to chat completions. Ensure upstream mapping accepts Anthropic IDs (e.g., anthropic/claude-*) or translate here.
- Fixed the broken guide link in the changelog entry. - Adjusted URL slug for consistency and clarity.
- Added `draft: true` to the blog metadata.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md (3)
6-6: Brand consistency: “LLMGateway” vs “LLM Gateway”Use one style consistently (prefer the product’s canonical spelling).
Apply:
-summary: "Configure Claude Code to use any LLM model through LLMGateway's unified API with simple environment variable setup." +summary: "Configure Claude Code to use any LLM model through LLMGateway’s unified API with simple environment variable setup." - alt: "Claude Code configuration support on LLM Gateway" + alt: "Claude Code configuration support on LLMGateway" -You can now configure **Claude Code** to work with any LLM model available through LLMGateway! +You can now configure **Claude Code** to work with any LLM model available through LLMGateway!Also applies to: 10-10, 15-15
29-36: Qualify comparative claims or link to pricing/perf source“50–70% cost savings” and “performance comparisons” should cite a source or state “as of 2025‑09‑08” to avoid misleading readers.
Apply:
-- `glm-4.5v` - Similar performance with 50-70% cost savings over Anthropic +- `glm-4.5v` - Similar performance with 50–70% lower cost vs. Anthropic (as of 2025-09-08; see pricing page)
7-7: Reminder: publish when readySet draft: false before release.
-draft: true +draft: false
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
apps/ui/src/content/blog/2025-09-08-how-configure-claude-code-with-llmgateway.md(1 hunks)apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- apps/ui/src/content/blog/2025-09-08-how-configure-claude-code-with-llmgateway.md
🧰 Additional context used
🪛 markdownlint-cli2 (0.17.2)
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md
39-39: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
43-43: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
47-47: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: build / run
- GitHub Check: e2e / run
🔇 Additional comments (1)
apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md (1)
39-49: Convert emphasis to headings-**OpenAI Models** +### OpenAI models -**Anthropic Models** +### Anthropic models -**Cost-Effective Alternatives** +### Cost-effective alternativesModel identifiers
gpt-5,claude-sonnet-4-20250514, andglm-4.5vare registered in the repository.
| ```bash | ||
| export ANTHROPIC_BASE_URL=https://api.llmgateway.io | ||
| export ANTHROPIC_AUTH_TOKEN=llmgtwy_.... # your llmgateway.io api key here | ||
| export ANTHROPIC_MODEL=glm-4.5v # choose your model with tool support | ||
| ``` |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Use ANTHROPIC_API_KEY and confirm base-path compatibility with /v1/anthropic/messages
- Env var should be ANTHROPIC_API_KEY, not ANTHROPIC_AUTH_TOKEN.
- Verify that Claude Code/Anthropic SDK will reach /v1/anthropic/messages. With ANTHROPIC_BASE_URL set to the host only, most clients append /v1/messages (missing /anthropic). Either document the exact base path required or add a compatibility alias route in the gateway.
Apply:
export ANTHROPIC_BASE_URL=https://api.llmgateway.io
-export ANTHROPIC_AUTH_TOKEN=llmgtwy_.... # your llmgateway.io api key here
+export ANTHROPIC_API_KEY=llmgtwy_.... # your llmgateway.io API key here
export ANTHROPIC_MODEL=glm-4.5v # choose your model with tool supportRun to confirm routes and env var names used in the repo:
🏁 Script executed:
#!/bin/bash
# Confirm gateway routes expose Anthropic at the expected path(s)
rg -nI -C2 -g 'apps/gateway/**' -e '/v1/anthropic/messages' -e '/v1/messages' -e '/anthropic/messages'
# Check for env var names referenced in code/docs
rg -nI -e 'ANTHROPIC_AUTH_TOKEN|ANTHROPIC_API_KEY|ANTHROPIC_BASE_URL|ANTHROPIC_API_URL'Length of output: 739
Replace ANTHROPIC_AUTH_TOKEN with ANTHROPIC_API_KEY and verify gateway route
- In apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md (lines 22–23), change the example to use ANTHROPIC_API_KEY to match the rest of the codebase.
- Confirm your gateway actually forwards to /v1/anthropic/messages – no explicit
/v1/anthropic/messagesor/anthropic/messagesroutes were found in apps/gateway; either document the exact path or add an alias.
- export ANTHROPIC_AUTH_TOKEN=llmgtwy_.... # your llmgateway.io api key here
+ export ANTHROPIC_API_KEY=llmgtwy_.... # your llmgateway.io API key here📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ```bash | |
| export ANTHROPIC_BASE_URL=https://api.llmgateway.io | |
| export ANTHROPIC_AUTH_TOKEN=llmgtwy_.... # your llmgateway.io api key here | |
| export ANTHROPIC_MODEL=glm-4.5v # choose your model with tool support | |
| ``` |
🤖 Prompt for AI Agents
In apps/ui/src/content/changelog/2025-09-08-claude-code-configuration-support.md
around lines 21 to 25, the example env var uses ANTHROPIC_AUTH_TOKEN but the
codebase expects ANTHROPIC_API_KEY—update the example to export
ANTHROPIC_API_KEY instead; additionally, verify the gateway actually forwards
requests to /v1/anthropic/messages and either update the changelog to document
the exact gateway path that forwards to the Anthropic endpoint or add an
alias/route in apps/gateway that maps the documented path (e.g.,
/v1/anthropic/messages) to the internal handler so the example env and
documented endpoint align with the gateway routing.
- Replaced `any` and `unknown` with specific type definitions. - Improved SSE event handling for content and tool use blocks. - Refined parsing and tracking of tool calls and text blocks.
- Introduce new Anthropic API compatibility feature documentation - Add detailed usage, configuration, and advanced features for /v1/anthropic/messages endpoint - Include example requests and responses in Anthropic message format - Update docs meta.json to include new feature page - Add API reference page for Anthropic messages endpoint This enables users to access any LLM model via Anthropic's API format using LLMGateway. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
steebchen
changed the title
- Add new /v1/anthropic/messages endpoint with request/response transformation - Support all LLMGateway models through Anthropic API format - Handle system messages, tools, and multi-modal content - Add comprehensive documentation page - Enable Claude Code compatibility with any model 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Update tool message grouping to process messages individually. Ensure unique IDs for duplicate tool_calls and improve tool_result mapping logic. Adds better tracking for tool usage across datasets.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
packages/models/src/provider-api.ts (2)
78-86: Add timeout and SSRF safeguards when fetching images.Remote image fetches can hang and are SSRF-prone. Add a per-request timeout and block private/loopback/link-local targets in prod. Also avoid logging full URLs.
- const response = await fetch(url); + const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), 15000); + const response = await fetch(url, { signal: controller.signal }); + clearTimeout(timeout);And (conceptually) validate the host/IP (deny 127.0.0.0/8, ::1, 10/8, 172.16/12, 192.168/16, 169.254/16, fc00::/7) before fetch. I can add a small utility if you want.
Also applies to: 106-116, 127-155
116-123: Use Node-safe base64 conversion.btoa with a char-code join is slow and memory-heavy for large payloads; prefer Buffer when available.
- const uint8Array = new Uint8Array(arrayBuffer); - const binaryString = Array.from(uint8Array, (byte) => - String.fromCharCode(byte), - ).join(""); - const base64 = btoa(binaryString); + const uint8Array = new Uint8Array(arrayBuffer); + const base64 = + typeof Buffer !== "undefined" + ? Buffer.from(uint8Array).toString("base64") + : btoa(String.fromCharCode(...uint8Array));
♻️ Duplicate comments (1)
apps/gateway/src/anthropic/anthropic.ts (1)
239-247: Ensure tool/function messages pass string content to OpenAI.OpenAI tool messages require string content. Coerce non-strings with JSON.stringify.
- if (message.role === "function") { - openaiMessages.push({ - role: "tool", - content: message.content, - tool_call_id: message.tool_call_id || message.name, - }); + if (message.role === "function") { + openaiMessages.push({ + role: "tool", + content: + typeof message.content === "string" + ? message.content + : JSON.stringify(message.content), + tool_call_id: message.tool_call_id || message.name, + });
🧹 Nitpick comments (8)
analyze_test.py (3)
1-1: Shebang mismatch: either make executable or drop the shebang.Ruff EXE001 is right: the file has a shebang but isn’t executable. Either chmod +x or remove the shebang.
Apply one of:
-#!/usr/bin/env python3 +#!/usr/bin/env python3Then:
chmod +x analyze_test.pyor
-#!/usr/bin/env python3 +
5-6: Specify file encoding.Be explicit to avoid locale-dependent decoding in CI.
-with open('./http/test.http', 'r') as f: +with open('./http/test.http', 'r', encoding='utf-8') as f:
27-35: Guard non-string content prints.Slicing non-strings will raise; stringify safely.
- else: - print(f"Content: {msg['content'][:100]}...") + elif isinstance(msg["content"], str): + print(f"Content: {msg['content'][:100]}...") + else: + preview = json.dumps(msg["content"])[:100] if msg["content"] is not None else "" + print(f"Content (non-string): {preview}...")apps/docs/content/features/anthropic-endpoint.mdx (1)
209-209: Fix API reference link formatting.MDX link shouldn’t be wrapped with angle brackets.
-For detailed API specifications, see the [Anthropic Messages API reference](</(api)/v1_anthropic_messages>) in our API documentation. +For detailed API specifications, see the [Anthropic Messages API reference](/(api)/v1_anthropic_messages) in our API documentation.Also confirm the actual doc filename/route is “v1_anthropic_messages” (PR text and AI summary mention both v1_messages and v1_anthropic_messages). Want me to search and align all references?
packages/models/src/provider-api.ts (2)
607-618: Consider gating reasoning_effort to Responses API only.Chat Completions may not accept reasoning_effort across all models; safer to include it only for responses API payloads.
- if (reasoning_effort !== undefined) { - requestBody.reasoning_effort = reasoning_effort; - } + // Avoid sending unknown params to Chat Completions + // (reasoning_effort is used when using the Responses API above)Want me to check model-specific support and gate by model id?
544-547: Use processedMessages when checking existing tool calls.After system-role transformation, use the same array for consistency.
- const hasExistingToolCalls = messages.some( + const hasExistingToolCalls = processedMessages.some( (msg: any) => msg.tool_calls || msg.role === "tool", );apps/gateway/src/anthropic/anthropic.ts (2)
428-435: Include usage in streaming.Without stream_options.include_usage, usage won’t arrive in SSE. Add it when stream is true. (help.openai.com)
const openaiRequest: Record<string, unknown> = { model: anthropicRequest.model, messages: openaiMessages, max_tokens: anthropicRequest.max_tokens, temperature: anthropicRequest.temperature, stream: anthropicRequest.stream, }; +if (anthropicRequest.stream) { + openaiRequest.stream_options = { include_usage: true }; +}
445-455: Forward minimal headers.Optional: build headers dynamically to avoid sending empty strings; keeps upstream cleaner.
- headers: { - "Content-Type": "application/json", - Authorization: c.req.header("Authorization") || "", - "x-request-id": c.req.header("x-request-id") || "", - "x-source": c.req.header("x-source") || "", - "x-debug": c.req.header("x-debug") || "", - }, + headers: (() => { + const h: Record<string, string> = { "Content-Type": "application/json" }; + for (const k of ["Authorization","x-request-id","x-source","x-debug"] as const) { + const v = c.req.header(k); + if (v) h[k] = v; + } + return h; + })(),
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
analyze_test.py(1 hunks)apps/docs/content/features/anthropic-endpoint.mdx(1 hunks)apps/docs/content/meta.json(1 hunks)apps/gateway/src/anthropic/anthropic.ts(1 hunks)apps/gateway/src/index.ts(2 hunks)apps/ui/src/content/blog/2025-09-08-how-configure-claude-code-with-llmgateway.md(1 hunks)packages/models/src/models/anthropic.ts(1 hunks)packages/models/src/provider-api.ts(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- apps/gateway/src/index.ts
- apps/ui/src/content/blog/2025-09-08-how-configure-claude-code-with-llmgateway.md
- packages/models/src/models/anthropic.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use top-level import; never use require() or dynamic import()
Always use top-level import; never use require or dynamic imports
Files:
apps/gateway/src/anthropic/anthropic.tspackages/models/src/provider-api.ts
{apps/{api,gateway}/**/*.ts,packages/db/**/*.ts}
📄 CodeRabbit inference engine (CLAUDE.md)
.findMany() or db().query.
{apps/{api,gateway}/**/*.ts,packages/db/**/*.ts}: For database reads, use Drizzle’s db().query..findFirst()
Use Drizzle ORM with the latest object syntaxFiles:
apps/gateway/src/anthropic/anthropic.tsapps/{api,gateway}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
apps/{api,gateway}/**/*.{ts,tsx}: Use Drizzle ORM with the latest object syntax for database access
For reads, use db().query..findMany() or db().query.
.findFirst()
After API route changes, run pnpm generate to update OpenAPI schemasFiles:
apps/gateway/src/anthropic/anthropic.ts🧬 Code graph analysis (2)
apps/gateway/src/anthropic/anthropic.ts (1)
packages/db/src/types.ts (3)
toolResults(45-45)tool(18-21)toolCall(35-42)packages/models/src/provider-api.ts (3)
packages/models/src/types.ts (3)
BaseMessage(61-67)ToolUseContent(30-35)ToolResultContent(37-41)packages/db/src/types.ts (1)
toolCall(35-42)packages/models/src/models.ts (1)
ProviderModelMapping(23-100)🪛 Ruff (0.12.2)
analyze_test.py
1-1: Shebang is present but file is not executable
(EXE001)
🪛 ESLint
apps/gateway/src/anthropic/anthropic.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/NifPzGSdLy'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
[error] 1-1: Resolve error: EACCES: permission denied, open '/rjDAFuPoOH'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/no-useless-path-segments.js:85:53)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/moduleVisitor.js:32:5)(import/no-useless-path-segments)
🔇 Additional comments (3)
apps/docs/content/features/anthropic-endpoint.mdx (1)
69-72: Keep base URL examples consistent.Elsewhere you show POST to /v1/anthropic/messages; the env var example sets ANTHROPIC_BASE_URL to /v1/anthropic (correct). Just confirming this is intentional and consistent with the SDKs.
packages/models/src/provider-api.ts (1)
542-556: GPT‑5 token cap params: current mapping is correct.
- Responses API: max_output_tokens
- Chat Completions (reasoning models): max_completion_tokens
Your branches follow this. Keep it gated to models/endpoints accordingly. (help.openai.com)
If desired, I can add an assertion to force Responses API for gpt‑5 unless tool-call history forces Chat Completions fallback.
Also applies to: 980-1001
apps/docs/content/meta.json (1)
15-16: Nav entry OK — target file present; confirm slug/permalinkapps/docs/content/features/anthropic-endpoint.mdx exists; frontmatter contains "title" but no explicit "slug"/"permalink" — confirm the docs build will publish this at features/anthropic-endpoint or add an explicit slug.
| if (openaiResponse.choices?.[0]?.message?.content) { | ||
| content.push({ | ||
| type: "text", | ||
| text: openaiResponse.choices[0].message.content, | ||
| }); | ||
| } | ||
|
|
||
| // Handle tool calls | ||
| if (openaiResponse.choices?.[0]?.message?.tool_calls) { | ||
| for (const toolCall of openaiResponse.choices[0].message.tool_calls) { | ||
| content.push({ | ||
| type: "tool_use", | ||
| id: toolCall.id, | ||
| name: toolCall.function.name, | ||
| input: JSON.parse(toolCall.function.arguments || "{}"), | ||
| }); | ||
| } | ||
| } |
There was a problem hiding this comment.
Handle array content and guard JSON.parse in non‑streaming path.
Chat Completions may return content as an array of parts; current code assumes string and will misformat. Also, JSON.parse on tool args can throw.
- if (openaiResponse.choices?.[0]?.message?.content) {
- content.push({
- type: "text",
- text: openaiResponse.choices[0].message.content,
- });
- }
+ const msg = openaiResponse.choices?.[0]?.message;
+ if (msg?.content) {
+ if (Array.isArray(msg.content)) {
+ const text = msg.content
+ .filter((p: any) => p?.type === "text" && typeof p.text === "string")
+ .map((p: any) => p.text)
+ .join("");
+ if (text) content.push({ type: "text", text });
+ } else if (typeof msg.content === "string") {
+ content.push({ type: "text", text: msg.content });
+ }
+ }
@@
- input: JSON.parse(toolCall.function.arguments || "{}"),
+ input: (() => {
+ try {
+ return JSON.parse(toolCall.function.arguments || "{}");
+ } catch {
+ return toolCall.function.arguments || "{}";
+ }
+ })(),📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (openaiResponse.choices?.[0]?.message?.content) { | |
| content.push({ | |
| type: "text", | |
| text: openaiResponse.choices[0].message.content, | |
| }); | |
| } | |
| // Handle tool calls | |
| if (openaiResponse.choices?.[0]?.message?.tool_calls) { | |
| for (const toolCall of openaiResponse.choices[0].message.tool_calls) { | |
| content.push({ | |
| type: "tool_use", | |
| id: toolCall.id, | |
| name: toolCall.function.name, | |
| input: JSON.parse(toolCall.function.arguments || "{}"), | |
| }); | |
| } | |
| } | |
| const msg = openaiResponse.choices?.[0]?.message; | |
| if (msg?.content) { | |
| if (Array.isArray(msg.content)) { | |
| const text = msg.content | |
| .filter((p: any) => p?.type === "text" && typeof p.text === "string") | |
| .map((p: any) => p.text) | |
| .join(""); | |
| if (text) content.push({ type: "text", text }); | |
| } else if (typeof msg.content === "string") { | |
| content.push({ type: "text", text: msg.content }); | |
| } | |
| } | |
| // Handle tool calls | |
| if (openaiResponse.choices?.[0]?.message?.tool_calls) { | |
| for (const toolCall of openaiResponse.choices[0].message.tool_calls) { | |
| content.push({ | |
| type: "tool_use", | |
| id: toolCall.id, | |
| name: toolCall.function.name, | |
| input: (() => { | |
| try { | |
| return JSON.parse(toolCall.function.arguments || "{}"); | |
| } catch { | |
| return toolCall.function.arguments || "{}"; | |
| } | |
| })(), | |
| }); | |
| } | |
| } |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Adjusted input, output, and cached input prices for the Claude-3.5 model to match the latest pricing updates from the provider.
Updated import paths in anthropic files to use the simplified base alias `@` for consistency.
Updated the function documentation to provide a concise description.
Replaced `for..of` on `Map` objects with `Array.from` to prevent unintended behavior with grouped tool messages.
Clarified endpoint URL and adjusted examples to match updated API.
Replaced existing streaming response logic with a `501 Not Implemented` placeholder response.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
apps/gateway/src/anthropic/anthropic.ts (4)
499-503: SSE parsing: make “data:” detection robust (space optional, trim first)Brittle match can miss lines without a space and with CRLF.
- for (const line of lines) { - if (line.startsWith("data: ")) { - const data = line.slice(6).trim(); + for (const line of lines) { + const trimmed = line.trim(); + if (trimmed.startsWith("data:")) { + const data = trimmed.slice(5).trim();
447-453: Don’t send empty headers; add Accept for SSEAvoid empty Authorization/x-* headers; set Accept: text/event-stream when streaming.
- headers: { - "Content-Type": "application/json", - Authorization: c.req.header("Authorization") || "", - "x-request-id": c.req.header("x-request-id") || "", - "x-source": c.req.header("x-source") || "", - "x-debug": c.req.header("x-debug") || "", - }, + headers: (() => { + const h: Record<string, string> = { "Content-Type": "application/json" }; + for (const k of ["Authorization","x-request-id","x-source","x-debug"] as const) { + const v = c.req.header(k); + if (v) h[k] = v; + } + if (anthropicRequest.stream) h["Accept"] = "text/event-stream"; + return h; + })(),
239-247: Tool/function role: ensure string content for OpenAI tool messagesOpenAI requires tool message content to be a string; passing arrays/objects will 400.
Apply:
- content: message.content, + content: + typeof message.content === "string" + ? message.content + : JSON.stringify(message.content),
716-733: Non‑streaming: handle array content and guard tool arg parsing
- Message content may be an array of parts; current code assumes string.
- JSON.parse on tool args can throw and 500 the request.
Use:
- if (openaiResponse.choices?.[0]?.message?.content) { - content.push({ - type: "text", - text: openaiResponse.choices[0].message.content, - }); - } + const msg = openaiResponse.choices?.[0]?.message; + if (msg?.content) { + if (Array.isArray(msg.content)) { + const text = msg.content + .filter((p: any) => p?.type === "text" && typeof p.text === "string") + .map((p: any) => p.text) + .join(""); + if (text) content.push({ type: "text", text }); + } else if (typeof msg.content === "string") { + content.push({ type: "text", text: msg.content }); + } + } @@ - input: JSON.parse(toolCall.function.arguments || "{}"), + input: (() => { + try { return JSON.parse(toolCall.function.arguments || "{}"); } + catch { return toolCall.function.arguments || "{}"; } + })(),
🧹 Nitpick comments (4)
apps/gateway/src/anthropic/anthropic.ts (4)
321-335: Typing: avoid any[] in Map payloadUse a precise type for tool_result aggregation.
- const toolResults = new Map<string, any[]>(); + const toolResults = new Map< + string, + Array<{ content?: unknown; is_error?: boolean }> + >();
714-714: Typing: replace any[] on response contentImproves safety and self‑documentation.
- const content: any[] = []; + const content: Array< + { type: "text"; text: string } | + { type: "tool_use"; id: string; name: string; input: unknown } + > = [];
445-455: Add timeout and abort on client disconnectProtects gateway threads and avoids hung upstream fetches.
- const response = await fetch(chatCompletionsUrl.toString(), { + const ac = new AbortController(); + const t = setTimeout(() => ac.abort(), 60_000); + const response = await fetch(chatCompletionsUrl.toString(), { method: "POST", + signal: ac.signal, @@ body: JSON.stringify(openaiRequest), }); + clearTimeout(t);Optionally, tie SSE to client abort:
- return streamSSE(c, async (stream) => { + return streamSSE(c, async (stream) => { + stream.onAbort(() => ac.abort());Also applies to: 466-471
754-767: Stop reason default: consider null when unknown/absentAnthropic allows null; returning end_turn for undefined may mislead.
- default: - return "end_turn"; + default: + return finishReason ? "end_turn" : null;
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
apps/gateway/src/anthropic/anthropic.ts(1 hunks)apps/gateway/src/anthropic/index.ts(1 hunks)packages/models/src/models/anthropic.ts(1 hunks)packages/models/src/provider-api.ts(3 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/models/src/provider-api.ts
- packages/models/src/models/anthropic.ts
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use top-level import; never use require() or dynamic imports
Files:
apps/gateway/src/anthropic/anthropic.tsapps/gateway/src/anthropic/index.ts
{apps/{api,gateway}/src,packages/db}/**/*.ts?(x)
📄 CodeRabbit inference engine (CLAUDE.md)
Use Drizzle ORM with the latest object syntax
Files:
apps/gateway/src/anthropic/anthropic.tsapps/gateway/src/anthropic/index.ts
apps/{api,gateway}/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
For read queries, use db().query.
.findMany() or db().query..findFirst() Files:
apps/gateway/src/anthropic/anthropic.tsapps/gateway/src/anthropic/index.ts{apps/api,apps/gateway,apps/ui,apps/docs,packages}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level import; never use require() or dynamic imports (e.g., import(), next/dynamic)
Files:
apps/gateway/src/anthropic/anthropic.tsapps/gateway/src/anthropic/index.ts{apps/api,apps/gateway,packages/db}/**/*.ts
📄 CodeRabbit inference engine (AGENTS.md)
{apps/api,apps/gateway,packages/db}/**/*.ts: Use Drizzle ORM with the latest object syntax for database access
For reads, use db().query..findMany() or db().query.
.findFirst() Files:
apps/gateway/src/anthropic/anthropic.tsapps/gateway/src/anthropic/index.ts🪛 ESLint
apps/gateway/src/anthropic/anthropic.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/FahwEPBLRF'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
apps/gateway/src/anthropic/index.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/ImdPxddHGu'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
[error] 1-1: Resolve error: EACCES: permission denied, open '/inrTvprbDW'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/no-useless-path-segments.js:85:53)
at checkSourceValue (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/moduleVisitor.js:32:5)(import/no-useless-path-segments)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Cursor Bugbot
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (3)
🔇 Additional comments (2)
apps/gateway/src/anthropic/index.ts (2)
7-9: Route wiring looks goodExports an OpenAPIHono and mounts /messages as expected.
9-9: Verify mount path in the main routerEnsure this module is mounted under /v1/anthropic so the full path is /v1/anthropic/messages as documented.
Would you like a quick repo scan script to confirm the mount in apps/gateway/src/index.ts?
| if (!response.ok) { | ||
| const errorData = await response.json(); | ||
| throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, { | ||
| message: errorData.error?.message || "Request failed", | ||
| }); | ||
| } |
There was a problem hiding this comment.
Error path: parsing assumes JSON; fallback to text to avoid masking upstream errors
If upstream returns non‑JSON, this throws and hides the real error.
- if (!response.ok) {
- const errorData = await response.json();
- throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, {
- message: errorData.error?.message || "Request failed",
- });
- }
+ if (!response.ok) {
+ const errText = await response.clone().text().catch(() => "");
+ let message = "Request failed";
+ try {
+ const errorData = await response.json();
+ message = errorData.error?.message || errText || message;
+ } catch {
+ message = errText || message;
+ }
+ throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, { message });
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (!response.ok) { | |
| const errorData = await response.json(); | |
| throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, { | |
| message: errorData.error?.message || "Request failed", | |
| }); | |
| } | |
| if (!response.ok) { | |
| const errText = await response.clone().text().catch(() => ""); | |
| let message = "Request failed"; | |
| try { | |
| const errorData = await response.json(); | |
| message = errorData.error?.message || errText || message; | |
| } catch { | |
| message = errText || message; | |
| } | |
| throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, { message }); | |
| } |
🤖 Prompt for AI Agents
In apps/gateway/src/anthropic/anthropic.ts around lines 457 to 462, the error
path currently assumes response.json() will succeed which can throw and hide the
upstream error when the body is non‑JSON; wrap the JSON parse in a try/catch,
and if JSON parsing fails fall back to await response.text() (or empty string)
and include that text in the HTTPException payload/message so upstream non‑JSON
responses are preserved and surfaced; keep using response.status for the status
code and ensure the thrown error contains both any parsed error.message and the
raw fallback text.
| content: toolResultContent, | ||
| } as ToolResultContent, | ||
| ]; | ||
| } |
There was a problem hiding this comment.
Bug: Tool Result Mapping Fails with Shared IDs
The transformAnthropicMessages function incorrectly handles tool_result mapping when multiple tool_use blocks share an original ID. The first tool_result message for such an ID consumes all unique mapped tool_use_ids, causing subsequent tool_result messages to generate duplicate blocks for already-handled IDs and breaking the one-to-one mapping.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (4)
apps/gateway/src/anthropic/anthropic.ts (4)
197-205: Block streaming upfront and don’t forwardstreamto upstream.Currently you still build the upstream request (and include
stream) before returning 501, which issues an SSE fetch you never consume. Guard earlier and omitstreamfrom the upstream payload.Apply this diff:
@@ - const anthropicRequest: AnthropicRequest = validation.data; + const anthropicRequest: AnthropicRequest = validation.data; + // Reject streaming until SSE bridging is implemented + if (anthropicRequest.stream) { + throw new HTTPException(501, { message: "Streaming not supported yet for Anthropic bridge" }); + } @@ - const openaiRequest: Record<string, unknown> = { + const openaiRequest: Record<string, unknown> = { model: anthropicRequest.model, messages: openaiMessages, max_tokens: anthropicRequest.max_tokens, temperature: anthropicRequest.temperature, - stream: anthropicRequest.stream, }; @@ - const response = await fetch(chatCompletionsUrl.toString(), { + const response = await fetch(chatCompletionsUrl.toString(), { method: "POST", - headers: { - "Content-Type": "application/json", - Authorization: c.req.header("Authorization") || "", - "x-request-id": c.req.header("x-request-id") || "", - "x-source": c.req.header("x-source") || "", - "x-debug": c.req.header("x-debug") || "", - }, + headers: (() => { + const h: Record<string, string> = { "Content-Type": "application/json" }; + for (const k of ["Authorization", "x-request-id", "x-source", "x-debug"] as const) { + const v = c.req.header(k); + if (v) h[k] = v; + } + return h; + })(), body: JSON.stringify(openaiRequest), }); @@ - if (anthropicRequest.stream) { - return new Response("Not implemented yet, sorry!", { status: 501 }); - } + // unreachable now (guarded above), but keep as a defensive fallback if refactored later + // if (anthropicRequest.stream) { + // return new Response("Not implemented yet, sorry!", { status: 501 }); + // }Also applies to: 427-435, 444-454, 463-466
456-461: Harden upstream error handling (fallback to text when JSON parse fails).Avoid masking upstream errors when body isn’t JSON.
- if (!response.ok) { - const errorData = await response.json(); - throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, { - message: errorData.error?.message || "Request failed", - }); - } + if (!response.ok) { + const fallback = await response.clone().text().catch(() => ""); + let message = "Request failed"; + try { + const errJson = await response.json(); + message = errJson.error?.message || fallback || message; + } catch { + message = fallback || message; + } + throw new HTTPException(response.status as 400 | 401 | 403 | 404 | 500, { message }); + }
238-246: Ensure function-role tool messages send string content.OpenAI tool messages require string content; passing arrays/objects will 400.
if (message.role === "function") { openaiMessages.push({ role: "tool", - content: message.content, + content: + typeof message.content === "string" + ? message.content + : JSON.stringify(message.content), tool_call_id: message.tool_call_id || message.name, }); continue; }
716-733: Handle array content and guard JSON.parse on tool arguments.Chat Completions may return content as an array; and tool args can be malformed JSON.
- if (openaiResponse.choices?.[0]?.message?.content) { - content.push({ - type: "text", - text: openaiResponse.choices[0].message.content, - }); - } + const msg = openaiResponse.choices?.[0]?.message; + if (msg?.content) { + if (Array.isArray(msg.content)) { + const text = msg.content + .filter((p: any) => p?.type === "text" && typeof p.text === "string") + .map((p: any) => p.text) + .join(""); + if (text) content.push({ type: "text", text }); + } else if (typeof msg.content === "string") { + content.push({ type: "text", text: msg.content }); + } + } @@ - if (openaiResponse.choices?.[0]?.message?.tool_calls) { - for (const toolCall of openaiResponse.choices[0].message.tool_calls) { + if (msg?.tool_calls) { + for (const toolCall of msg.tool_calls) { content.push({ type: "tool_use", id: toolCall.id, name: toolCall.function.name, - input: JSON.parse(toolCall.function.arguments || "{}"), + input: (() => { + try { return JSON.parse(toolCall.function.arguments || "{}"); } + catch { return toolCall.function.arguments || "{}"; } + })(), }); } }
🧹 Nitpick comments (5)
apps/gateway/src/anthropic/anthropic.ts (5)
226-235: Tool-role: fall back tool_call_id to name when missing.Prevents orphan tool messages when callers omit
tool_call_id.if (message.role === "tool") { openaiMessages.push({ role: "tool", content: typeof message.content === "string" ? message.content : JSON.stringify(message.content), - tool_call_id: message.tool_call_id, + tool_call_id: message.tool_call_id || message.name, }); continue; }
714-714: Avoidany[]on response content.Tighten types to match emitted blocks.
- const content: any[] = []; + const content: Array< + | { type: "text"; text: string } + | { type: "tool_use"; id: string; name: string; input: unknown } + > = [];
384-404: Drop unknown multimodal parts instead of passing through.Passing unknown blocks through to OpenAI may 400. Only emit known
textandimage_urlparts.- const content = message.content.map((block) => { + const content = message.content.flatMap((block) => { if (block.type === "text" && block.text) { - return { type: "text", text: block.text }; + return [{ type: "text", text: block.text }]; } if (block.type === "image" && block.source) { - return { + return [{ type: "image_url", image_url: { url: `data:${block.source.media_type};base64,${block.source.data}`, }, - }; + }]; } - return block; - }); + return []; + });
213-216: Preserve system block boundaries.Join with newline to avoid smashing paragraphs.
- systemContent = anthropicRequest.system - .map((block) => block.text) - .join(" "); + systemContent = anthropicRequest.system.map((b) => b.text).join("\n");
754-767: Considernulldefault for unknown/absent finish reasons.Closer to Anthropic semantics for ambiguous cases.
switch (finishReason) { case "stop": return "end_turn"; case "length": return "max_tokens"; case "tool_calls": return "tool_use"; default: - return "end_turn"; + return null; }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
apps/docs/content/features/anthropic-endpoint.mdx(1 hunks)apps/gateway/src/anthropic/anthropic.ts(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- apps/docs/content/features/anthropic-endpoint.mdx
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use top-level import; never use require() or dynamic imports
Files:
apps/gateway/src/anthropic/anthropic.ts
{apps/{api,gateway}/src,packages/db}/**/*.ts?(x)
📄 CodeRabbit inference engine (CLAUDE.md)
Use Drizzle ORM with the latest object syntax
Files:
apps/gateway/src/anthropic/anthropic.ts
apps/{api,gateway}/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
For read queries, use db().query.
.findMany() or db().query..findFirst() Files:
apps/gateway/src/anthropic/anthropic.ts{apps/api,apps/gateway,apps/ui,apps/docs,packages}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level import; never use require() or dynamic imports (e.g., import(), next/dynamic)
Files:
apps/gateway/src/anthropic/anthropic.ts{apps/api,apps/gateway,packages/db}/**/*.ts
📄 CodeRabbit inference engine (AGENTS.md)
{apps/api,apps/gateway,packages/db}/**/*.ts: Use Drizzle ORM with the latest object syntax for database access
For reads, use db().query..findMany() or db().query.
.findFirst() Files:
apps/gateway/src/anthropic/anthropic.ts🧬 Code graph analysis (1)
apps/gateway/src/anthropic/anthropic.ts (1)
packages/db/src/types.ts (3)
toolResults(45-45)tool(18-21)toolCall(35-42)🪛 ESLint
apps/gateway/src/anthropic/anthropic.ts
[error] 1-1: Resolve error: EACCES: permission denied, open '/LAwbBjysnZ'
at Object.writeFileSync (node:fs:2409:20)
at l (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:13685)
at createFilesMatcher (/home/jailuser/git/node_modules/.pnpm/get-tsconfig@4.10.1/node_modules/get-tsconfig/dist/index.cjs:7:14437)
at Object.resolve (/home/jailuser/git/node_modules/.pnpm/eslint-import-resolver-typescript@4.4.4_eslint-plugin-import@2.32.0_eslint@9.34.0_jiti@2.5.1_/node_modules/eslint-import-resolver-typescript/lib/index.cjs:298:107)
at withResolver (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:180:23)
at fullResolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:201:22)
at relative (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:217:10)
at resolve (/home/jailuser/git/node_modules/.pnpm/eslint-module-utils@2.12.1_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1__t_0245afd672d291ebf09c60ebb405cf51/node_modules/eslint-module-utils/resolve.js:233:12)
at resolveImportType (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/core/importType.js:126:2822)
at computeRank (/home/jailuser/git/node_modules/.pnpm/eslint-plugin-import@2.32.0_@typescript-eslint+parser@8.39.1_eslint@9.34.0_jiti@2.5.1___504bceda7a403f00735dcb51d6a813c7/node_modules/eslint-plugin-import/lib/rules/order.js:529:43)(import/order)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Cursor Bugbot
- GitHub Check: build / run
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (1)
🔇 Additional comments (2)
apps/gateway/src/anthropic/anthropic.ts (2)
417-424: Confirminput_schemais valid JSON Schema for OpenAI tools.If callers send non‑schema records, upstream may 400. Consider validating/normalizing to
{ type: "object", properties: {} }at minimum.Would you like me to add a Zod refinement to enforce a JSON Schema shape?
71-108: Nice Zod coverage and OpenAPI wiring.Schemas are comprehensive (messages, tools, response), and the route surface is clean.
If you want, I can generate sample cURL payloads to smoke‑test user/tool, legacy function_call, and multimodal image cases against this handler.
Also applies to: 118-132
2 participants
Summary
/v1/anthropic/messagesin the gatewayChanges
Anthropic API Implementation
anthropic.tswith full request and response schemas using Zodclaude-3-5-haiku-20241022to Anthropic models listRouting and Integration
/anthropicroute in the gateway's main routeranthropic/index.tsDocumentation
apps/docs/content/(api)/v1_anthropic_messages.mdxapps/docs/content/features/anthropic-endpoint.mdxTest plan
🌿 Generated by Terry
ℹ️ Tag @terragon-labs to ask questions and address PR feedback
📎 Task: https://www.terragonlabs.com/task/d9cbfc29-f487-460a-b973-d2a1f432b597
Summary by CodeRabbit