feat(custom-models): per-key enterprise model catalog - #2698
Conversation
Add an enterprise per-provider-key custom model catalog so requests through custom providers are billed and limited from defined values. - DB: custom_model table + provider_key.custom_models_only toggle - API: enterprise-gated custom-models CRUD + provider key toggle - Gateway: thread catalog pricing into calculateCosts; enforce context/maxOutput; reject undefined models when restricted - UI: Custom Models org tab with provider-key selector, toggle and CRUD dialogs; cross-linked with Provider Keys Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds a custom model catalog feature end-to-end: a new ChangesCustom Model Catalog Feature
Sequence Diagram(s)sequenceDiagram
participant UI as CustomModelsClient
participant API as /api/custom-models
participant Gateway as Gateway chat.ts
participant DB as custom_model table
participant SWR as SWR Cache
rect rgba(100, 149, 237, 0.5)
note over UI,SWR: Model Management (UI → API)
UI->>API: POST /custom-models {providerKeyId, modelName, pricing...}
API->>DB: check enterprise plan + provider === "custom"
API->>DB: insert custom_model row
API->>SWR: invalidate custom_model cache
API-->>UI: { customModel }
end
rect rgba(144, 238, 144, 0.5)
note over Gateway,SWR: Request Routing (Gateway)
Gateway->>SWR: findCustomModel(providerKeyId, modelName)
SWR-->>Gateway: CustomModel or undefined
Gateway->>Gateway: reject if customModelsOnly and no entry
Gateway->>Gateway: enforce capability/context/output limits
Gateway->>Gateway: calculateCosts(customPricing: catalogMapping)
end
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2dc6b32e44
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const [customModel] = await db | ||
| .insert(tables.customModel) |
There was a problem hiding this comment.
When a gateway has already looked up a catalog entry (or a miss), this insert and the update/delete paths in this route use the plain db client, while findCustomModel reads through the cached cdb select path keyed on custom_model. Because these writes do not trigger RedisCache.onMutate, creating, editing, or deleting a custom model can be ignored by the gateway until the cached select expires, leaving stale pricing/limits or stale rejections in production. Use the cached client or explicitly invalidate the custom_model table after these mutations.
Useful? React with 👍 / 👎.
| (table) => [ | ||
| unique().on(table.providerKeyId, table.modelName), | ||
| index("custom_model_provider_key_id_idx").on(table.providerKeyId), |
There was a problem hiding this comment.
This unconditional unique constraint still includes rows whose status was set to deleted, but the API duplicate checks intentionally ignore deleted rows and the delete endpoint only soft-deletes. After a user deletes foo, creating foo again (or renaming another row to foo) will pass the route-level conflict check and then fail on this database constraint instead of allowing the normal recreate flow; make the uniqueness match active/non-deleted rows or rename/hard-delete tombstones.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (1)
apps/ui/src/app/dashboard/[orgId]/org/provider-keys/page.tsx (1)
16-16: ⚡ Quick winCentralize the provider-key list item type to avoid contract drift.
The sameproviderKeysitem shape is duplicated across these files, and this PR had to update all of them forcustomModelsOnly. Extract a shared type (or derive once from OpenAPIpaths) and reuse it.
apps/ui/src/app/dashboard/[orgId]/org/provider-keys/page.tsx#L16-L16: replace inline provider-key item typing with the shared alias.apps/ui/src/components/provider-keys/provider-keys-client.tsx#L25-L25: consume the same shared alias inProviderKeysClientProps.apps/ui/src/components/provider-keys/provider-keys-list.tsx#L53-L53: consume the same shared alias inProviderKeysListProps.As per coding guidelines,
**/*.{ts,tsx,js,jsx}should apply DRY principles for code reuse.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/app/dashboard/`[orgId]/org/provider-keys/page.tsx at line 16, Extract the duplicated provider-key item type definition (containing properties like customModelsOnly) into a single shared type alias that can be reused across multiple files. Create this shared type in a centralized types file or derive it from OpenAPI paths, then replace the inline type definitions at all three locations with references to this shared alias: in apps/ui/src/app/dashboard/[orgId]/org/provider-keys/page.tsx at line 16 (currently defining customModelsOnly inline), in apps/ui/src/components/provider-keys/provider-keys-client.tsx at line 25 (in ProviderKeysClientProps), and in apps/ui/src/components/provider-keys/provider-keys-list.tsx at line 53 (in ProviderKeysListProps). This will ensure the type contract is centralized and any future updates only need to be made in one place.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/api/src/routes/custom-models.ts`:
- Around line 91-94: The updateCustomModelSchema allows all fields to be
optional, which permits empty objects like {} to pass validation and trigger
no-op updates with unnecessary audit events. Add a refine() guard to the
updateCustomModelSchema object (similar to the pattern used in the provider-key
update route) that validates at least one field from the customModelFields is
provided, rejecting empty PATCH bodies.
- Around line 105-109: The db.query.providerKey.findFirst query in the provider
key lookup does not filter out soft-deleted records, which allows catalog
mutations to proceed with deleted provider keys. Add a condition to the where
clause that explicitly excludes soft-deleted provider keys by verifying the
deletion timestamp field (e.g., deletedAt) is null or not set, ensuring only
active provider keys can be used for catalog operations.
In `@apps/gateway/src/chat/chat.ts`:
- Around line 2493-2501: The catalog resolution for custom models happens after
validateModelCapabilities(...) has already run, which means catalog fields like
tools, vision, jsonOutput, audio, streaming, and supportedParameters do not gate
the request as intended. Additionally, the finalModelInfo branch rebuilds a
zero-price custom mapping instead of reusing the customPricingMapping that was
created. Move the customModelToProviderMapping call and the modelInfo update
(where customPricingMapping is applied to modelInfo.providers) to occur before
any capability or IAM validation checks, and then ensure that when
finalModelInfo is constructed, it preserves and reuses the customPricingMapping
instead of creating a new zero-price custom mapping.
- Around line 2523-2524: The calculation of requiredContextSize at line 2523
uses max_tokens ?? 0, which defaults to zero output tokens when max_tokens is
omitted, allowing the entire contextSize to be consumed by input. Replace this
default value of 0 with a proper fallback output token budget - either by using
the auto-routing fallback budget or by referencing the catalog maxOutput value -
to ensure output space is reserved for the model's completion generation even
when max_tokens is not explicitly provided.
In `@apps/ui/src/components/custom-models/custom-model-dialog.tsx`:
- Around line 153-160: The num function at line 153 does not validate that
Number(v) produces a valid finite positive integer, allowing NaN or Infinity to
serialize as null in the request body. Before constructing the body object, add
validation logic that checks both contextSize and maxOutput (after applying the
num transformation) to ensure they are finite positive integers. If either value
is invalid, display a toast error message with a descriptive error and prevent
the form submission by returning early. This ensures invalid numeric input is
rejected rather than persisted as null values in the database.
In `@packages/db/src/schema.ts`:
- Around line 1138-1145: The current schema enforces a permanent unique
constraint on the combination of providerKeyId and modelName, but this conflicts
with the soft-delete behavior where deleted rows should be ignored during
duplicate checks. In packages/db/src/schema.ts at lines 1138-1145, replace the
table-level unique() constraint with a partial unique index that only applies to
rows where status is not 'deleted', and ensure the status field is marked as
non-null with a default value of 'active'. In
packages/db/migrations/1781607059_real_morg.sql at lines 29-30, remove the
table-level unique constraint and create the corresponding partial unique index
in the SQL migration to match the schema change.
---
Nitpick comments:
In `@apps/ui/src/app/dashboard/`[orgId]/org/provider-keys/page.tsx:
- Line 16: Extract the duplicated provider-key item type definition (containing
properties like customModelsOnly) into a single shared type alias that can be
reused across multiple files. Create this shared type in a centralized types
file or derive it from OpenAPI paths, then replace the inline type definitions
at all three locations with references to this shared alias: in
apps/ui/src/app/dashboard/[orgId]/org/provider-keys/page.tsx at line 16
(currently defining customModelsOnly inline), in
apps/ui/src/components/provider-keys/provider-keys-client.tsx at line 25 (in
ProviderKeysClientProps), and in
apps/ui/src/components/provider-keys/provider-keys-list.tsx at line 53 (in
ProviderKeysListProps). This will ensure the type contract is centralized and
any future updates only need to be made in one place.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 5adf1a84-9861-40ff-b317-bc86e19e15be
📒 Files selected for processing (19)
apps/api/src/routes/custom-models.tsapps/api/src/routes/index.tsapps/api/src/routes/keys-provider.tsapps/gateway/src/chat/chat.tsapps/gateway/src/lib/cached-queries.tsapps/gateway/src/lib/costs.spec.tsapps/gateway/src/lib/costs.tsapps/ui/src/app/dashboard/[orgId]/org/custom-models/page.tsxapps/ui/src/app/dashboard/[orgId]/org/provider-keys/page.tsxapps/ui/src/components/custom-models/custom-model-dialog.tsxapps/ui/src/components/custom-models/custom-models-client.tsxapps/ui/src/components/dashboard/dashboard-sidebar.tsxapps/ui/src/components/provider-keys/provider-keys-client.tsxapps/ui/src/components/provider-keys/provider-keys-list.tsxpackages/db/migrations/1781607059_real_morg.sqlpackages/db/migrations/meta/1781607059_snapshot.jsonpackages/db/migrations/meta/_journal.jsonpackages/db/src/relations.tspackages/db/src/schema.ts
| status: text({ | ||
| enum: ["active", "inactive", "deleted"], | ||
| }).default("active"), | ||
| }, | ||
| (table) => [ | ||
| unique().on(table.providerKeyId, table.modelName), | ||
| index("custom_model_provider_key_id_idx").on(table.providerKeyId), | ||
| index("custom_model_organization_id_idx").on(table.organizationId), |
There was a problem hiding this comment.
Make custom-model uniqueness match the soft-delete contract. The schema and migration both enforce permanent uniqueness for (provider_key_id, model_name), while the API soft-deletes rows and intentionally ignores deleted rows during duplicate checks.
packages/db/src/schema.ts#L1138-L1145: replace the table-levelunique()with a partial unique index over non-deleted rows, and makestatusnon-null/default active.packages/db/migrations/1781607059_real_morg.sql#L29-L30: remove the table-level unique constraint and create the matching partial unique index in the migration SQL.
📍 Affects 2 files
packages/db/src/schema.ts#L1138-L1145(this comment)packages/db/migrations/1781607059_real_morg.sql#L29-L30
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/db/src/schema.ts` around lines 1138 - 1145, The current schema
enforces a permanent unique constraint on the combination of providerKeyId and
modelName, but this conflicts with the soft-delete behavior where deleted rows
should be ignored during duplicate checks. In packages/db/src/schema.ts at lines
1138-1145, replace the table-level unique() constraint with a partial unique
index that only applies to rows where status is not 'deleted', and ensure the
status field is marked as non-null with a default value of 'active'. In
packages/db/migrations/1781607059_real_morg.sql at lines 29-30, remove the
table-level unique constraint and create the corresponding partial unique index
in the SQL migration to match the schema change.
- Invalidate SWR cache on custom-model and provider-key-toggle writes so the gateway picks up catalog/restriction changes promptly - Partial unique index on (provider_key_id, model_name) for non-deleted rows so soft-deleted names can be recreated; status now NOT NULL - Reject empty custom-model PATCH bodies; exclude soft-deleted provider keys from catalog management - Reuse the catalog mapping in finalModelInfo instead of a zero-price mock - Reserve a completion budget in the context-size check when max_tokens is omitted - Validate context size / max output as positive integers in the UI Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2730af18bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| jsonOutput: cm.jsonOutput ?? undefined, | ||
| audio: cm.audio ?? undefined, | ||
| supportedParameters: cm.supportedParameters ?? undefined, | ||
| streaming, |
There was a problem hiding this comment.
When a catalog entry sets streaming to false or only, this value has no effect: later streaming validation calls getModelStreamingSupport(usedInternalModel, usedProvider, usedRegion), which reads only the static models registry and returns null for custom model ids. As a result streaming requests to a catalog model marked non-streaming are allowed, and models marked streaming-only are not forced to use SSE.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/api/src/routes/keys-provider.ts`:
- Around line 703-705: The cache invalidation call using
invalidateSwrByTables([providerKeyTableName]) is only present in the
PATCH/update endpoint but is missing from the POST /provider and DELETE
/provider/{id} endpoints. Since these endpoints also mutate the provider_key
table, they must also invalidate the SWR cache to prevent stale gateway
behavior. Add the same await invalidateSwrByTables([providerKeyTableName]); call
to both the POST endpoint (after key creation) and the DELETE endpoint (after
key deletion) to ensure cache consistency across all write operations on
provider keys.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 90a0cb54-f344-4f49-a4a0-221eaf6f80b4
📒 Files selected for processing (8)
apps/api/src/routes/custom-models.tsapps/api/src/routes/keys-provider.tsapps/gateway/src/chat/chat.tsapps/ui/src/components/custom-models/custom-model-dialog.tsxpackages/db/migrations/1781624949_big_vindicator.sqlpackages/db/migrations/meta/1781624949_snapshot.jsonpackages/db/migrations/meta/_journal.jsonpackages/db/src/schema.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- apps/api/src/routes/custom-models.ts
- apps/ui/src/components/custom-models/custom-model-dialog.tsx
- apps/gateway/src/chat/chat.ts
- Enforce catalog capability flags (vision, audio, tools, jsonOutput, reasoning, streaming) for custom models when explicitly disabled, since validateModelCapabilities skips custom providers - Invalidate provider-key SWR cache on create and delete too, not just the update path Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/gateway/src/chat/chat.ts`:
- Around line 2535-2538: The condition in the customModelEntry.reasoning ===
false block at line 2535-2538 incorrectly rejects requests where
reasoning_effort is set to "none". Since "none" is a valid value indicating no
reasoning is requested, update the condition to allow reasoning_effort: "none"
for non-reasoning models. Modify the check to only reject when reasoning_effort
is defined and is NOT equal to "none", or when reasoning_max_tokens is defined.
This way, explicit "none" values will pass through as valid no-reasoning
requests.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 4b7d621f-3fee-4521-b0ed-df074b6c6854
📒 Files selected for processing (2)
apps/api/src/routes/keys-provider.tsapps/gateway/src/chat/chat.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- apps/api/src/routes/keys-provider.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b96584852f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| modelName: z | ||
| .string() | ||
| .min(1, "Model name is required") | ||
| .regex(/^[\w.:/-]+$/, "Model name contains invalid characters"), |
There was a problem hiding this comment.
Allowing : in modelName creates catalog entries that the gateway cannot match: parseModelInput strips everything after the last colon as a region suffix before findCustomModel looks up the catalog entry. For a custom provider model such as anthropic.claude-v1:0, the API accepts and stores that name, but requests to provider/anthropic.claude-v1:0 look up anthropic.claude-v1; with customModelsOnly enabled the model is rejected even though it is defined.
Useful? React with 👍 / 👎.
| cacheWriteInputPrice1h: cm.cacheWriteInputPrice1h ?? undefined, | ||
| requestPrice: cm.requestPrice ?? undefined, | ||
| webSearchPrice: cm.webSearchPrice ?? undefined, | ||
| imageInputPrice: cm.imageInputPrice ?? undefined, |
There was a problem hiding this comment.
Threading imageInputPrice into the custom mapping never bills image inputs for arbitrary custom models, because inputImageCount is only set for the hard-coded Gemini image model IDs before the custom catalog lookup. A catalog model with vision plus an image input price will still call calculateCosts with inputImageCount as 0, so image requests record no imageInputCost despite the configured price.
Useful? React with 👍 / 👎.
Custom models produce text output only; multi-modal input (images, audio) is still supported and priced via input fields. Drop the image-output (generation) price field, which is too provider-specific to bill generically. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d1985bd602
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }); | ||
| } | ||
| // Restricting to a custom catalog is an enterprise feature. | ||
| if (providerKey.organization?.plan !== "enterprise") { |
There was a problem hiding this comment.
When an organization downgrades while a custom provider already has customModelsOnly enabled, this check also rejects { customModelsOnly: false }, so the customer cannot turn off the restriction even though the gateway continues honoring the stored flag and will reject any model missing from the old catalog. Only require the enterprise plan when enabling the restriction, or otherwise allow disabling it for non-enterprise orgs.
Useful? React with 👍 / 👎.
| customModelEntry.tools === false && | ||
| (tool_choice !== undefined || (tools && tools.length > 0)) |
There was a problem hiding this comment.
For a request that uses the native web_search tool, the handler removes that entry from tools before reaching this new check, so a catalog entry with tools: false still forwards web search requests instead of enforcing the explicit capability flag. Fresh evidence is that this condition now checks only the post-splice tools array; include webSearchTool (or a separate catalog capability) in the rejection path.
Useful? React with 👍 / 👎.
# Conflicts: # apps/gateway/src/chat/chat.ts # packages/db/migrations/meta/_journal.json
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 664d00170e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const customModelEntry = await findCustomModel( | ||
| customProviderKey.id, | ||
| requestedModel, | ||
| ); |
There was a problem hiding this comment.
For custom-provider catalog models, the API key IAM validation has already run above using the synthetic custom model mapping with zero prices, and the real catalog prices are only loaded here. In an org that uses IAM allow_pricing max input/output price limits, a catalog model priced above those limits will still be allowed and billed, because the price guard never sees customPricingMapping; revalidate price-based IAM rules after resolving the custom model entry or move this lookup before IAM validation.
Useful? React with 👍 / 👎.
# Conflicts: # packages/db/migrations/meta/_journal.json
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fd18709ad0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }); | ||
| // The gateway caches provider keys (incl. customModelsOnly) via SWR; | ||
| // invalidate so status/restriction changes take effect promptly. | ||
| await invalidateSwrByTables([providerKeyTableName]); |
There was a problem hiding this comment.
When a gateway has already cached this custom provider key, PATCHing customModelsOnly writes through the uncached db client and this only clears the SWR fallback mirror; it does not clear the Drizzle Redis cache used by findCustomProviderKey in apps/gateway/src/lib/cached-queries.ts. Until that cache entry expires, turning the catalog restriction on can keep allowing uncataloged models, and turning it off can keep rejecting them, so use the cached client/onMutate path or explicitly invalidate the Drizzle table cache as well.
Useful? React with 👍 / 👎.
| const priceField = z | ||
| .string() | ||
| .refine( | ||
| (v) => v.trim() !== "" && Number.isFinite(Number(v)) && Number(v) >= 0, |
There was a problem hiding this comment.
For direct API callers, values like " 0.000003 " pass this validation because Number(v) is finite, but the untrimmed string is stored and later passed to new Decimal(...) in calculateCosts, which rejects whitespace-padded values and breaks requests for that custom model. Trim the value before storing it or validate with the same Decimal-compatible format the gateway will parse.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e9721a6b92
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| reasoning: cm.reasoning ?? undefined, | ||
| jsonOutput: cm.jsonOutput ?? undefined, | ||
| audio: cm.audio ?? undefined, | ||
| supportedParameters: cm.supportedParameters ?? undefined, |
There was a problem hiding this comment.
When a catalog entry omits parameters such as tool_choice or reasoning_effort from supportedParameters, those fields can still be forwarded to the custom provider. The synthetic mapping stores the allowlist here, but the request-body preparation path re-resolves support from the static model registry, which has no custom provider mapping, so it treats the allowlist as absent for those parameters. A custom OpenAI-compatible backend that rejects tool_choice/reasoning_effort will still receive them despite the catalog configuration; strip these fields using the resolved custom mapping before calling prepareRequestBody or pass the mapping through.
Useful? React with 👍 / 👎.
1 participant
Context
Today, requests routed through a custom provider key (
mycustom/<model>) are never billed and have no limits enforced: the gateway builds a mock model withinputPrice/outputPrice: "0"andcalculateCostskeys pricing on the static catalog byproviderId, which never matches"custom". So even a known id likegpt-5.5through a custom provider yields null cost on the activity log, and context/output limits are unenforced.This adds an enterprise per-provider-key custom model catalog. Each entry optionally defines context size, max output, all token prices, and capabilities. The gateway uses those values for cost attribution and limit enforcement. A per-key switch optionally restricts a provider to only catalog-defined models so cost/limits are always known and shown.
What's included
DB (
packages/db)custom_modeltable (perprovider_key, full optional field set; prices stored as text to preserve3.0e-6format)provider_key.custom_models_onlyboolean togglecustom_modelaudit actions/resource type; migration generated via the migrations skillAPI (
apps/api)custom-modelsCRUD routesPATCHto acceptcustomModelsOnly(enterprise-gated)Gateway (
apps/gateway)findCustomModelcached querycustomPricingoverride intocalculateCosts(syntheticproviderId: "custom"mapping) so requests bill at catalog rates. Models without an entry stay null/unbilled, exactly as before.UI (
apps/ui)Behavior notes
Verification
pnpm build(full) ✅pnpm format✅pnpm test:unit✅ — 2004 passed (incl. 2 newcalculateCostscustom-pricing tests)🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
UI Enhancements