feat(api): add effort parameter support for claude-opus-4-5 - #1244
Conversation
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
WalkthroughAdded an optional top-level Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Gateway as chat.ts
participant Validator as validate-provider-key
participant PrepReq as prepare-request-body
participant Provider
participant Logger as create-log-entry
participant DB as log table
Client->>Gateway: POST /chat/completions { ..., effort: "high" }
Gateway->>Gateway: Parse & validate request (effort)
Gateway->>Validator: Check provider/model support for effort
alt supported
Gateway->>PrepReq: prepareRequestBody(..., effort: "high")
PrepReq->>Provider: Provider request (Anthropic: output_config.effort="high", header effort-2025-11-24)
Provider-->>Gateway: Provider response
Gateway->>Logger: createLogEntry(..., effort: "high")
Logger->>DB: INSERT log (effort = "high")
DB-->>Logger: OK
Gateway-->>Client: 200 + response
else unsupported
Gateway-->>Client: 400 "Model doesn't support effort"
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~30 minutes
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/models/src/models/anthropic.ts (1)
598-628: AddsupportedParameterswith"effort"to AWS Bedrock and Google Vertex provider entries.Claude Opus 4.5 is the only model that supports the effort parameter, and Opus 4.5 is available on all three major cloud platforms. However, only the
anthropicprovider entry declaressupportedParameters: ["temperature", "max_tokens", "top_p", "effort"](line 598).
- AWS Bedrock (lines 600-614): AWS Bedrock supports the effort parameter for Claude Opus 4.5 but lacks
supportedParametersdeclaration, causing validation failures foreffortrequests routed to this provider.- Google Vertex (lines 615-628): Should similarly declare
supportedParameterswith"effort"to maintain consistency and enable effort control for users routing to this provider.Add
supportedParameters: ["temperature", "max_tokens", "top_p", "effort"]to both provider entries.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
apps/gateway/src/chat/chat.ts(14 hunks)apps/gateway/src/chat/tools/create-log-entry.ts(2 hunks)packages/db/migrations/1764069190_clever_anthem.sql(1 hunks)packages/db/migrations/meta/_journal.json(1 hunks)packages/db/src/schema.ts(1 hunks)packages/models/src/models/anthropic.ts(1 hunks)packages/models/src/prepare-request-body.ts(2 hunks)packages/models/src/validate-provider-key.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Never useanyoras anytype assertions unless absolutely necessary
Always use top-levelimport, never use require or dynamic imports
Use cookies for user-settings which are not saved in the database to ensure SSR works
No unnecessary code comments
**/*.{ts,tsx}: Use Drizzle ORM with latest object syntax for database operations
For database reads: Usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
packages/db/src/schema.tsapps/gateway/src/chat/tools/create-log-entry.tspackages/models/src/models/anthropic.tspackages/models/src/validate-provider-key.tsapps/gateway/src/chat/chat.tspackages/models/src/prepare-request-body.ts
**/db/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/db/**/*.{ts,tsx}: Use Drizzle ORM with latest object syntax for database operations
For database reads, usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
For database schema changes, usepnpm run setupinstead of writing migrations which will generate .sql files
Always sync schema withpnpm run setupafter table/column changes
Files:
packages/db/src/schema.ts
**/*.{js,ts,jsx,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{js,ts,jsx,tsx}: Always use top-levelimport, never use require or dynamic imports
No unnecessary code comments
Files:
packages/db/src/schema.tsapps/gateway/src/chat/tools/create-log-entry.tspackages/models/src/models/anthropic.tspackages/models/src/validate-provider-key.tsapps/gateway/src/chat/chat.tspackages/models/src/prepare-request-body.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use cookies for user-settings which are not saved in the database to ensure SSR works
Files:
packages/db/src/schema.tsapps/gateway/src/chat/tools/create-log-entry.tspackages/models/src/models/anthropic.tspackages/models/src/validate-provider-key.tsapps/gateway/src/chat/chat.tspackages/models/src/prepare-request-body.ts
apps/{gateway,api}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
apps/{gateway,api}/**/*.{ts,tsx}: Use Hono web framework for backend services in Gateway and API applications
Use Zod schemas for validation in backend servicesUse Zod schemas for validation in Hono applications
Files:
apps/gateway/src/chat/tools/create-log-entry.tsapps/gateway/src/chat/chat.ts
🧠 Learnings (2)
📚 Learning: 2025-11-24T20:02:09.207Z
Learnt from: CR
Repo: theopenco/llmgateway PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-24T20:02:09.207Z
Learning: Applies to **/db/**/*.{ts,tsx} : Always sync schema with `pnpm run setup` after table/column changes
Applied to files:
packages/db/src/schema.ts
📚 Learning: 2025-11-24T20:02:09.207Z
Learnt from: CR
Repo: theopenco/llmgateway PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-24T20:02:09.207Z
Learning: Applies to **/db/**/*.{ts,tsx} : For database schema changes, use `pnpm run setup` instead of writing migrations which will generate .sql files
Applied to files:
packages/db/src/schema.ts
🧬 Code graph analysis (1)
apps/gateway/src/chat/chat.ts (2)
packages/db/src/schema.ts (1)
provider(611-643)packages/models/src/models.ts (1)
ProviderModelMapping(50-157)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (5)
- GitHub Check: e2e-shards (2)
- GitHub Check: generate / run
- GitHub Check: build / run
- GitHub Check: test / run
- GitHub Check: lint / run
- GitHub Check: autofix
🔇 Additional comments (10)
packages/db/migrations/1764069190_clever_anthem.sql (1)
1-1: LGTM!The migration correctly adds a nullable
efforttext column to the log table, matching the schema definition. Based on learnings, schema changes should usepnpm run setupto generate migrations, which appears to have been followed here.packages/db/src/schema.ts (1)
419-419: LGTM!The
effortcolumn is correctly added as a nullable text field, consistent with the migration and the optional nature of the parameter.packages/db/migrations/meta/_journal.json (1)
501-508: LGTM!The journal entry correctly tracks the new migration with consistent formatting and proper sequencing.
packages/models/src/validate-provider-key.ts (1)
131-136: LGTM!The additional parameters are correctly passed with appropriate default values for validation purposes. Using
undefinedforeffortensures the validation request doesn't include the parameter.apps/gateway/src/chat/tools/create-log-entry.ts (2)
26-26: LGTM!The
effortparameter is correctly added to the function signature with the proper type union matching the Zod schema.
60-60: LGTM!The
effortvalue is correctly mapped with|| nullfallback for database storage, consistent with other optional parameters in this function.packages/models/src/prepare-request-body.ts (1)
145-146: LGTM!The
effortparameter is correctly added to the function signature with the proper type.apps/gateway/src/chat/chat.ts (3)
682-700: LGTM!The validation logic correctly checks if the model supports the
effortparameter by looking for"effort"in the provider'ssupportedParametersarray. The error message is clear and suggests a supported model.
2015-2015: LGTM!The
effortparameter is correctly passed toprepareRequestBodyfor inclusion in the upstream API request.
3390-3390: LGTM!The
effortparameter is consistently passed to allcreateLogEntrycalls throughout the file for proper logging and observability.
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add output_config to AnthropicRequestBody interface to support the effort parameter properly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/models/src/types.ts (1)
216-218: Confirm effort literal values match Anthropic’s API and consider a shared type aliasThe new
output_config.effortshape looks good and keeps Anthropic-specific config nicely scoped. One thing to double-check is that the"minimum"literal exactly matches Anthropic’s documented values (note the OpenAI side uses"minimal"), to avoid subtle integration bugs. If this effort enum ends up used in multiple places acrosspackages/models, consider extracting a dedicatedAnthropicEfforttype alias to keep it consistent and easier to evolve.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
packages/models/src/prepare-request-body.ts(2 hunks)packages/models/src/types.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/models/src/prepare-request-body.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Never useanyoras anytype assertions unless absolutely necessary
Always use top-levelimport, never use require or dynamic imports
Use cookies for user-settings which are not saved in the database to ensure SSR works
No unnecessary code comments
**/*.{ts,tsx}: Use Drizzle ORM with latest object syntax for database operations
For database reads: Usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
packages/models/src/types.ts
**/*.{js,ts,jsx,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{js,ts,jsx,tsx}: Always use top-levelimport, never use require or dynamic imports
No unnecessary code comments
Files:
packages/models/src/types.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use cookies for user-settings which are not saved in the database to ensure SSR works
Files:
packages/models/src/types.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: lint / run
- GitHub Check: generate / run
- GitHub Check: autofix
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (5)
- GitHub Check: e2e-shards (2)
Add the required 'effort-2025-11-24' beta header when the effort parameter is specified for Anthropic API requests. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
apps/gateway/src/chat/chat.ts (1)
218-236: Aligneffort/reasoning_effortenums with Anthropic docs and includeeffortin cache key.Two points here:
reasoning_effortandeffortenums:
reasoning_effortuses"minimal"andeffortuses"minimum"alongside"low" | "medium" | "high".- Anthropic’s public APIs historically document
low | medium | highfor reasoning/effort;"minimal"/"minimum"are not standard and may break upstream validation unless you’re explicitly mapping them to canonical values elsewhere.- Recommend normalizing both enums to the same, documented set (likely
["low", "medium", "high"]) or adding an explicit mapping layer if you intend to support synonyms. This was already raised in a previous review; please either adjust or confirm against the latest Anthropic spec.Check the latest Anthropic API docs for the valid values of `reasoning_effort` / `effort` (output_config.effort): are `minimal` / `minimum` accepted, or only `low | medium | high`?Caching does not distinguish
effort(andreasoning_effort):
cachePayloadcurrently omitsreasoning_effortandeffort, so requests that differ only in effort can hit the same cache entry and receive mismatched responses/costs.- That’s a functional bug introduced by adding
effort(and previously forreasoning_effort), since callers expect effort changes to affect behavior.Consider updating
cachePayloadas follows:const cachePayload = { provider: usedProvider, model: usedModel, messages, temperature, max_tokens, top_p, frequency_penalty, presence_penalty,
- response_format,
- response_format,
- reasoning_effort,
- effort,
};This keeps streaming and non‑streaming cache keys consistent with the new parameter. Also applies to: 1696-1707 </blockquote></details> </blockquote></details> <details> <summary>🧹 Nitpick comments (1)</summary><blockquote> <details> <summary>apps/gateway/src/chat/chat.ts (1)</summary><blockquote> `682-700`: **Tighten `effort` support validation to account for the actual provider and auto‑routing.** The current check: - Looks at `modelInfo.providers.some(...)` and ignores `requestedProvider` / eventual `usedProvider`. - Skips validation entirely for `model="auto"` and `model="custom"`. This means: - If a model has multiple providers and only some have `"effort"` in `supportedParameters`, a request that pins a non‑supporting provider could still pass validation. - For `model="auto"` + `effort`, routing can select a model/provider that doesn’t support `effort`, and the parameter will be silently ignored upstream. Given you already use `supportedParameters` for this feature, I’d recommend: - Mirroring the `tools` / JSON logic by first narrowing to the providers relevant for this request: ```ts const providersToCheck = requestedProvider ? modelInfo.providers.filter( (p) => (p as ProviderModelMapping).providerId === requestedProvider, ) : modelInfo.providers; const supportsEffort = providersToCheck.some((provider) => { const params = (provider as ProviderModelMapping).supportedParameters; return params?.includes("effort"); });
- Optionally, after routing (when
usedProvider/finalModelInfoare known), performing a second guard whenmodelInput === "auto"so thateffortis only accepted if the final provider mapping’ssupportedParametersincludes"effort".This keeps validation aligned with the actual provider being called and prevents future surprises as more models gain or lack effort support.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
apps/gateway/src/chat/chat.ts(16 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Never useanyoras anytype assertions unless absolutely necessary
Always use top-levelimport, never use require or dynamic imports
Use cookies for user-settings which are not saved in the database to ensure SSR works
No unnecessary code comments
**/*.{ts,tsx}: Use Drizzle ORM with latest object syntax for database operations
For database reads: Usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
apps/gateway/src/chat/chat.ts
apps/{gateway,api}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
apps/{gateway,api}/**/*.{ts,tsx}: Use Hono web framework for backend services in Gateway and API applications
Use Zod schemas for validation in backend servicesUse Zod schemas for validation in Hono applications
Files:
apps/gateway/src/chat/chat.ts
**/*.{js,ts,jsx,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{js,ts,jsx,tsx}: Always use top-levelimport, never use require or dynamic imports
No unnecessary code comments
Files:
apps/gateway/src/chat/chat.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use cookies for user-settings which are not saved in the database to ensure SSR works
Files:
apps/gateway/src/chat/chat.ts
🧬 Code graph analysis (1)
apps/gateway/src/chat/chat.ts (1)
packages/models/src/models.ts (1)
ProviderModelMapping(50-157)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (5)
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (2)
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: generate / run
- GitHub Check: lint / run
- GitHub Check: autofix
🔇 Additional comments (2)
apps/gateway/src/chat/chat.ts (2)
425-426: Effort is correctly threaded through validation, request construction, and logging.
effortis:
- Extracted from the validated payload,
- Passed into
prepareRequestBodyalongsidereasoning_effort,- And propagated into all
createLogEntrycall sites (streaming/non‑streaming, cached, error, and cancellation paths).This covers both upstream behavior and DB logging for the new parameter; the wiring looks consistent and complete.
Also applies to: 1794-1795, 1900-1901, 2148-2149, 2230-2231, 2379-2380, 3610-3610, 3697-3697, 3789-3789, 4010-4010, 2015-2016
2111-2117: Beta flag and format are correct; optional de-duplication is defensive but not urgent.Verification confirms the code is correct:
- The
"effort-2025-11-24"beta flag follows Anthropic's documented format and comma-separated multiple flags syntax.- The header combination logic correctly appends the effort flag to any existing beta flags (e.g.,
"tools-2024-04-04,prompt-caching-2024-07-31,effort-2025-11-24").getProviderHeaders()currently does not set the effort flag by default, so no actual duplication exists today.The optional de-duplication refactor is a valid defensive measure if
getProviderHeadersever adds the effort flag in the future, but it is not addressing a current issue. Both code locations (lines 2111–2117 and 3550–3557) are identical and correct.
Show the effort parameter in the Model Parameters section of the log card UI when it's set. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add provider-specific validation for the effort parameter to ensure proper 4xx error when using unsupported provider (e.g., google-vertex). The validation now checks the actual provider being used after routing, rather than just checking if any provider supports it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Remove 'minimum' from effort parameter types. Valid values are: low, medium, high. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
apps/gateway/src/chat/chat.ts(16 hunks)apps/gateway/src/chat/tools/create-log-entry.ts(2 hunks)packages/models/src/prepare-request-body.ts(2 hunks)packages/models/src/types.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/models/src/types.ts
- apps/gateway/src/chat/tools/create-log-entry.ts
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Never useanyoras anytype assertions unless absolutely necessary
Always use top-levelimport, never use require or dynamic imports
Use cookies for user-settings which are not saved in the database to ensure SSR works
No unnecessary code comments
**/*.{ts,tsx}: Use Drizzle ORM with latest object syntax for database operations
For database reads: Usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
packages/models/src/prepare-request-body.tsapps/gateway/src/chat/chat.ts
**/*.{js,ts,jsx,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{js,ts,jsx,tsx}: Always use top-levelimport, never use require or dynamic imports
No unnecessary code comments
Files:
packages/models/src/prepare-request-body.tsapps/gateway/src/chat/chat.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use cookies for user-settings which are not saved in the database to ensure SSR works
Files:
packages/models/src/prepare-request-body.tsapps/gateway/src/chat/chat.ts
apps/{gateway,api}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
apps/{gateway,api}/**/*.{ts,tsx}: Use Hono web framework for backend services in Gateway and API applications
Use Zod schemas for validation in backend servicesUse Zod schemas for validation in Hono applications
Files:
apps/gateway/src/chat/chat.ts
🧬 Code graph analysis (1)
apps/gateway/src/chat/chat.ts (1)
packages/models/src/models.ts (1)
ProviderModelMapping(50-157)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: test / run
- GitHub Check: generate / run
- GitHub Check: lint / run
- GitHub Check: build / run
- GitHub Check: e2e-shards (2)
- GitHub Check: autofix
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (5)
🔇 Additional comments (5)
packages/models/src/prepare-request-body.ts (2)
145-145: LGTM! Parameter type is correct.The
effortparameter correctly uses the enum"low" | "medium" | "high", which matches Anthropic's API specification for the effort parameter.
515-520: LGTM! Effort parameter correctly nested in output_config.The implementation properly places the
effortparameter inside theoutput_configobject as required by the Anthropic API specification, addressing the previous review feedback.apps/gateway/src/chat/chat.ts (3)
227-236: LGTM! Effort enum correctly uses only Anthropic-supported values.The
effortparameter schema correctly restricts values to["low", "medium", "high"], which matches Anthropic's API specification. The enum does not include non-standard values like"minimum"or"minimal".Note: The
reasoning_effortparameter at line 219 does include"minimal", but that is used for different models (OpenAI) and may have different requirements.
1774-1774: LGTM! Effort parameter consistently threaded through all logging paths.The
effortparameter is correctly passed tocreateLogEntryacross all code paths (cached responses, errors, cancellations, streaming, and non-streaming flows), ensuring complete logging coverage.Also applies to: 1880-1880, 2145-2145, 2227-2227, 2376-2376, 3395-3395, 3606-3606, 3693-3693, 3785-3785, 4006-4006
2108-2114: No issues found — beta header implementation is correct.The Anthropic effort beta for Claude Opus 4.5 requires the beta header "effort-2025-11-24", which matches the implementation in your code at lines 2108-2114 and 3546-3553. The header format and conditional logic are correct.
| // Check if effort parameter is supported by the specific provider being used | ||
| if (effort !== undefined && finalModelInfo) { | ||
| const providerMapping = finalModelInfo.providers.find( | ||
| (p) => p.providerId === usedProvider && p.modelName === usedModel, | ||
| ); | ||
|
|
||
| if (providerMapping) { | ||
| const params = (providerMapping as ProviderModelMapping) | ||
| .supportedParameters; | ||
| if (!params?.includes("effort")) { | ||
| throw new HTTPException(400, { | ||
| message: `Model ${usedModel} with provider ${usedProvider} does not support the effort parameter. Try using provider 'anthropic' instead.`, | ||
| }); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
Consider whether auto/custom models need special handling.
The validation checks if the effort parameter is supported by verifying supportedParameters includes "effort". However, unlike similar validations for reasoning_effort (lines 652-680) and tools (lines 684-706), this validation does not skip checks for requestedModel === "auto" or requestedModel === "custom".
For auto-routed models, the validation runs after routing is complete, so finalModelInfo contains the selected model. This means the validation will correctly check the actually-selected model's support, which seems appropriate.
However, verify whether the same behavior is desired for custom models, where users might want to pass through arbitrary parameters to their custom OpenAI-compatible endpoints.
🏁 Script executed:
#!/bin/bash
# Check if other parameter validations skip auto/custom models and where they occur
rg -n "requestedModel.*auto.*custom" apps/gateway/src/chat/chat.ts -B2 -A5Length of output: 373
---
🏁 Script executed:
```shell
#!/bin/bash
# Check reasoning_effort validation pattern
echo "=== reasoning_effort validation ==="
sed -n '652,680p' apps/gateway/src/chat/chat.ts
echo ""
echo "=== tools validation ==="
sed -n '684,706p' apps/gateway/src/chat/chat.ts
Length of output: 207
🏁 Script executed:
#!/bin/bash
# Search for reasoning_effort validation pattern
echo "=== reasoning_effort validation ==="
rg -n "reasoning_effort" apps/gateway/src/chat/chat.ts -B3 -A3 | head -40
echo ""
echo "=== tools validation pattern ==="
rg -n "if.*tools" apps/gateway/src/chat/chat.ts -B2 -A8 | head -50Length of output: 2946
🏁 Script executed:
#!/bin/bash
# Check how custom models are handled and finalModelInfo setup
echo "=== custom model handling in finalModelInfo setup ==="
rg -n "custom.*finalModelInfo\|finalModelInfo.*custom" apps/gateway/src/chat/chat.ts -B3 -A3
echo ""
echo "=== where finalModelInfo is set ==="
rg -n "finalModelInfo\s*=" apps/gateway/src/chat/chat.ts -B2 -A2Length of output: 339
🏁 Script executed:
#!/bin/bash
# Get full context of custom model finalModelInfo setup
rg -n "if.*usedProvider.*custom" apps/gateway/src/chat/chat.ts -A15Length of output: 2460
Remove the effort parameter validation for "custom" models, consistent with other parameter validations.
The effort validation at lines 1971-1986 does not skip custom models, unlike the established pattern for other parameters:
reasoning_effortvalidation (lines 652-656): skips auto/customtoolsvalidation (lines 684-688): skips auto/customjson_schemavalidation (line 635): skips auto/custom
For custom models, the finalModelInfo structure (lines 1335-1344) does not include supportedParameters. When the effort check runs, it finds the provider mapping but supportedParameters is undefined, causing the validation to fail and throw a 400 error. Since custom models are user-configured OpenAI-compatible endpoints designed to accept arbitrary parameters, this validation should be skipped for custom models to match the established pattern.
Add the condition requestedModel !== "custom" to the effort validation check (similar to lines 654-655 and 686-687).
🤖 Prompt for AI Agents
In apps/gateway/src/chat/chat.ts around lines 1971-1986, the effort parameter
validation runs for "custom" models even though finalModelInfo for custom models
lacks supportedParameters and thus can falsely trigger a 400; update the
conditional to skip this check for custom models by adding the same guard used
elsewhere (requestedModel !== "custom") so the effort validation only runs when
requestedModel is not "custom" and retains the existing providerMapping and
supportedParameters checks.
1 participant
Summary
Added support for the
effortparameter for Claude Opus 4.5 (claude-opus-4-5-20251101). This parameter controls the computational effort for the model and is forwarded to the Anthropic API. The parameter is validated to ensure only supported models can use it, and all requests are logged with the effort value in the database.Changes
effortto the chat completions API request schema with validationeffortto the log table schema for trackingsupportedParametersto Claude Opus 4.5 model configurationeffortparameter to the upstream Anthropic APIeffortin supportedParameters can use it🤖 Generated with Claude Code
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.