feat: major improvements, multi regions, models, fixes - #1795
Merged
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds region-aware provider support: model/provider mappings and catalog entries gain optional regions; routing, scoring, retries, endpoint resolution, validation, metrics, worker sync, DB schema, and UI provider-key dialogs are made region-aware and propagated through gateway flows. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Gateway
participant ModelCatalog
participant RegionExpander
participant Router
participant Resolver
participant EndpointResolver
participant ProviderAPI
Client->>Gateway: request(modelId[:region], ...)
Gateway->>ModelCatalog: fetch modelDef (providers)
ModelCatalog-->>Gateway: modelDef (may include regions)
Gateway->>RegionExpander: expandAllProviderRegions(providers)
RegionExpander-->>Gateway: flattened (provider,region) candidates
Gateway->>Router: filter & score candidates (mode, keys, metrics, requestedRegion)
Router-->>Gateway: selected (providerId, modelName, region)
Gateway->>Resolver: resolveProviderContext(providerId, modelName, region)
Resolver-->>Gateway: ProviderContext (usedToken, usedRegion) or HTTP 400
Gateway->>EndpointResolver: getProviderEndpoint(providerId, region, ...)
EndpointResolver-->>Gateway: region-specific URL
Gateway->>ProviderAPI: proxied request (endpoint, token)
ProviderAPI-->>Gateway: response
Gateway-->>Client: response (includes usedRegion)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
changed the title
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rcogal
force-pushed
the
feat/provider-region-support
branch
from
March 6, 2026 15:12
f5553f9 to
c950d80
Compare
- Resolve region once in resolve-provider-context and pass it to getProviderEndpoint (no dual resolution) - Models without regions skip region logic entirely and use the default endpoint (current behavior) - Generic regionBaseUrl lookup so any provider with regionConfig can use region-based routing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
rcogal
changed the title
- Reset selectedRegion state on dialog close to prevent stale values - Log error when region not found in cost pricing (upstream bug indicator) - Narrow alibaba_region type to valid union instead of string Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove duplicate top-level pricingTiers when regions define them; costs.ts now falls back to regions[0] - Replace IIFE patterns with cleaner conditionals in provider key dialog - Support region-aware tiered pricing in model cards - Relax alibaba_region type to string in schema Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Make inputPrice/outputPrice optional on ProviderRegion with fallback to parent mapping. Add discount, requestPrice, webSearchPrice, and maxOutput as optional region overrides. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement Luca's approach where each region becomes a separate row in model_provider_mapping, scored and routed independently by the existing scoring/failover pipeline. - Add region column to model_provider_mapping with updated unique constraint (modelId, providerId, region) - Add expandProviderRegions helper for runtime expansion of nested region definitions into flat provider entries - Update worker sync to create per-region DB rows - Update metrics key format to include region - Update scoring to match providers by region - Add region-specific env var support (LLM_X_API_KEY__REGION) - Region-aware retry: only skip the failed region, not all regions of the same provider - Simplify costs.ts by removing regionPricing fallback logic - Expose region in API response and UI interface Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move usedRegion declaration earlier and set it at every scoring result point. Expand finalModelInfo.providers for region support. Add region matching to all .providers.find() calls to ensure the correct region entry is matched for capabilities and pricing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove unnecessary `as ProviderModelMapping` casts across chat.ts, resolve-provider-context.ts, and get-cheapest-from-available-providers.ts - Remove unused `region` param from calculateCosts - Add region validation in resolveProviderContext - Filter region candidates by available env keys in credits/hybrid mode to avoid wasting retries - Add hasRegionSpecificEnvKey helper - Fix provider-metrics tests for new key format - Simplify sync-models region detection - Add region/stability to ModelWithPricing type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pass region to getProviderEndpoint in chat.ts - Resolve usedRegion from DB provider key in api-keys and hybrid modes - Add alibaba_region to Zod schemas for key creation - Filter validation models by selected region - Skip image/video generation models in validation - Fix validation error handling in provider key dialog - Add region-aware hybrid mode filtering - Add region debug logging Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
rcogal
changed the title
rcogal
marked this pull request as ready for review
March 24, 2026 16:59
github-actions
Bot
changed the title
github-actions
Bot
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (5)
packages/models/src/models/alibaba.ts (1)
342-356:⚠️ Potential issue | 🟡 MinorMultiple Alibaba provider mappings are missing the
regionsfield.Several
providerId: "alibaba"entries lack aregionsarray, while others include it. This creates an inconsistency in region support across Alibaba models. The following Alibaba models are missing theregionsfield:
qwen-omni-turbo(lines 342-356)qwen-vl-max(lines 923-935)qwen-vl-plus(lines 946-958)qwen3-next-80b-a3b-thinking(lines 969-984)qwen3-next-80b-a3b-instruct(lines 1009-1021)qwen3-coder-plus(lines 398-410)qwen3-vl-235b-a22b-instructqwen3-vl-235b-a22b-thinkingqwen2-5-vl-32b-instruct(lines 1910-1925)qwen-image-plus,qwen-image-max,qwen-image,qwen-image-edit-plus,qwen-image-edit-maxModels like
qwen-max,qwen-plus,qwen-turbo, andqwen3-vl-plusalready defineregions. For consistency and to enable region-aware routing, all Alibaba models should include theregionsfield.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/models/src/models/alibaba.ts` around lines 342 - 356, Several provider mappings with providerId "alibaba" (modelName values: "qwen-omni-turbo", "qwen-vl-max", "qwen-vl-plus", "qwen3-next-80b-a3b-thinking", "qwen3-next-80b-a3b-instruct", "qwen3-coder-plus", "qwen3-vl-235b-a22b-instruct", "qwen3-vl-235b-a22b-thinking", "qwen2-5-vl-32b-instruct", "qwen-image-plus", "qwen-image-max", "qwen-image", "qwen-image-edit-plus", "qwen-image-edit-max") are missing a regions field; add a regions array to each of those model objects (the same region values used by other Alibaba entries such as the "qwen-max"/"qwen-plus"/"qwen-turbo"/"qwen3-vl-plus" mappings) so all Alibaba provider entries include a regions property for consistent region-aware routing.apps/gateway/src/chat/chat.ts (4)
2738-2758:⚠️ Potential issue | 🔴 CriticalPropagate
ctx.usedRegionin the streaming retry loop.After a fallback, this branch updates the provider, model, URL, and token, but it leaves
usedRegionstale. The laterproviderRetryKey(usedProvider, usedRegion)calls then record failures against the previous region and can reselect the same regional candidate again.💡 Minimal fix
frequency_penalty = ctx.frequency_penalty; presence_penalty = ctx.presence_penalty; + usedRegion = ctx.usedRegion;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/gateway/src/chat/chat.ts` around lines 2738 - 2758, The streaming retry loop fails to propagate ctx.usedRegion into local state so providerRetryKey(usedProvider, usedRegion) uses a stale region; update the catch/fallback branch where locals like usedProvider/usedModel/etc. are assigned from ctx to also assign usedRegion = ctx.usedRegion so the retry tracking and region-based selection reflect the new candidate region (look for the block assigning usedProvider, usedModel, url, providerKey, providerRetryKey usage and add usedRegion assignment there).
1593-1606:⚠️ Potential issue | 🟠 MajorCarry
regionin synthesized provider scores.These score entries drop
p.region, andisSelectedonly comparesproviderId. With regional mappings that marks everyalibaba:*row as selected and givesselectNextProvider()no way to map a retry score back to the correct expanded row.💡 Minimal fix
- const isSelected = p.providerId === usedProvider; + const isSelected = + p.providerId === usedProvider && p.region === usedRegion; return { providerId: p.providerId, + region: p.region, score: isSelected ? 1 : 0, price, uptime: metrics?.uptime ?? 0,🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/gateway/src/chat/chat.ts` around lines 1593 - 1606, The synthesized provider score objects created from allModelProviders omit region and only compare providerId to usedProvider, which breaks regional mappings and retries in selectNextProvider; update the object returned in the allModelProviders.map (the score entries) to include p.region and change the isSelected check to compare both p.providerId and p.region (or construct usedProvider identity that includes region) so that region is carried through and selectNextProvider can map retry scores back to the correct expanded row (referencing allModelProviders, metricsMap, metricsKey, usedProvider, and selectNextProvider).
1221-1233:⚠️ Potential issue | 🟠 MajorLow-uptime fallback is still keyed by provider, not provider+region.
The metrics lookup omits
usedRegion, so regional rows are queried asregion IS NULL. Thenp.providerId !== usedProviderremoves every region of that provider, not just the unhealthy one. For expanded mappings this fallback can neither read the failing region's uptime nor try a healthier region of the same provider.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/gateway/src/chat/chat.ts` around lines 1221 - 1233, The metrics lookup is only keyed by provider (metricsKey used with { modelId: baseModelId, providerId: usedProvider }) so regional metrics are treated as region IS NULL and your fallback excludes all regions of the provider; fix by including usedRegion when calling getProviderMetricsForCombinations and when building the lookup key (use { modelId: baseModelId, providerId: usedProvider, region: usedRegion } via metricsKey/getProviderMetricsForCombinations), then when selecting fallback candidates from modelInfo.providers only filter out the exact failing provider+region pair (compare both providerId and region) so other regions for the same provider remain eligible.
1794-1797:⚠️ Potential issue | 🔴 CriticalThe first env-backed request still uses the base key for non-default regions.
In credits and hybrid-without-DB-key mode,
usedRegioncan already be a non-default region, but token resolution still callsgetProviderEnv(usedProvider)and never switches to the region-specific env var. The initial request therefore hits the regional endpoint with the wrong credential, and the retry path is the only place that knows aboutgetRegionSpecificEnvValue().Also applies to: 1854-1858
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/gateway/src/chat/chat.ts` around lines 1794 - 1797, The initial env-backed token lookup uses getProviderEnv(usedProvider) unconditionally, causing regional requests to use the base key; change this to perform a region-aware lookup first: call getRegionSpecificEnvValue(usedProvider, usedRegion) when usedRegion is a non-default region (and when credits/hybrid-without-DB-key paths may set a non-default usedRegion), and if that returns nothing fall back to getProviderEnv(usedProvider); update the same logic in both places mentioned (the block using usedToken/configIndex/envVarName and the other occurrence around lines 1854-1858) so initial requests and retries use the same region-specific env resolution.
🧹 Nitpick comments (3)
apps/api/src/routes/keys-provider.ts (1)
36-36: Consider validatingalibaba_regionagainst known valid regions.The
alibaba_regionfield accepts any string, but only specific values are valid:"singapore","us-virginia","cn-beijing"(per theregionConfiginpackages/models/src/providers.ts). Invalid values will cause runtime errors when the gateway validates the region.While this is consistent with how other free-form options like
azure_resourceare handled, validating at the API layer provides better user feedback.♻️ Suggested validation
const providerKeySchema = z.object({ // ... options: z .object({ aws_bedrock_region_prefix: z.enum(["us.", "global.", "eu."]).optional(), azure_resource: z.string().optional(), azure_api_version: z.string().optional(), azure_deployment_type: z.enum(["openai", "ai-foundry"]).optional(), azure_validation_model: z.string().optional(), - alibaba_region: z.string().optional(), + alibaba_region: z.enum(["singapore", "us-virginia", "cn-beijing"]).optional(), }) .nullable(), // ... });Apply the same change to
createProviderKeySchema.Also applies to: 64-64
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/api/src/routes/keys-provider.ts` at line 36, Add explicit validation for alibaba_region so it only accepts the allowed region strings instead of any string: update the schema property (in the schema object that defines alibaba_region and in createProviderKeySchema) to use a constrained enum/union with the permitted values "singapore", "us-virginia", and "cn-beijing" (matching regionConfig in packages/models/src/providers.ts); keep the field optional but validate its value when present and ensure both occurrences (the one at line ~36 and the one in createProviderKeySchema) are changed.docs/plans/2026-03-06-provider-region-support.md (1)
13-13: Markdown heading level skip.The document jumps from h1 (
#) to h3 (###) without an h2 (##) heading. Consider adding a section header like## Implementation Tasksbefore the first task for better document structure.📝 Suggested fix
--- +## Implementation Tasks + ### Task 1: Add `regions` field to `ProviderModelMapping`🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/plans/2026-03-06-provider-region-support.md` at line 13, The document skips heading level by jumping from h1 to h3; insert an h2 heading such as "## Implementation Tasks" immediately before the existing "### Task 1: Add `regions` field to `ProviderModelMapping`" header so the structure follows h1 → h2 → h3 and improves accessibility and outline consistency.apps/worker/src/services/sync-models.ts (1)
104-116: Refactor to usedb.query.modelProviderMapping.findFirst()for consistency.This single-row lookup should use the repository's standard Drizzle query API instead of
.select().from(...).limit(1). Refactor to:const mappings = await database.query.modelProviderMapping.findFirst({ where: and( eq(modelProviderMapping.modelId, modelDef.id), eq(modelProviderMapping.providerId, mapping.providerId), mappingRegion ? eq(modelProviderMapping.region, mappingRegion) : isNull(modelProviderMapping.region), ), });This keeps the codebase consistent with the established pattern used throughout the repository.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/worker/src/services/sync-models.ts` around lines 104 - 116, Replace the raw select/limit pattern with the repository-style findFirst query: instead of calling database.select().from(modelProviderMapping).where(...).limit(1), call database.query.modelProviderMapping.findFirst({ where: and(...) }) using the same predicates (eq(modelProviderMapping.modelId, modelDef.id), eq(modelProviderMapping.providerId, mapping.providerId), and mappingRegion ? eq(modelProviderMapping.region, mappingRegion) : isNull(modelProviderMapping.region)). Ensure you import/retain references to database, modelProviderMapping, modelDef, mapping.providerId, and mappingRegion so the call matches the repo convention.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@apps/gateway/src/chat/chat.ts`:
- Around line 1012-1019: The expandedCandidateProviders computation applies
filterRegionsByAvailableKeys only for project.mode === "credits", causing
api-keys and hybrid modes to surface regions without matching API keys or
env-backed constraints; update the logic where expandedCandidateProviders is
built (the expression using expandAllProviderRegions and
filterRegionsByAvailableKeys) to apply the appropriate region filtering for all
relevant modes (including "api-keys" and "hybrid") before scoring/auto-routing
so that scoring and later resolveRegionFromProviderKey() cannot leave an invalid
usedRegion; ensure you keep the existing project.mode branching but call
filterRegionsByAvailableKeys (or an equivalent check) for api-keys/hybrid paths
so only regions with available keys or valid env-backed credentials are returned
to the scorer.
In `@apps/gateway/src/chat/tools/resolve-provider-context.ts`:
- Around line 203-208: When a regional token is chosen, envVarName is left
pointing at the base provider key so health reporting attributes failures to the
wrong env var; update the code so that when getRegionSpecificEnvValue(...)
yields a regionToken you also update envVarName to the corresponding regional
env var name. Do this by either changing getRegionSpecificEnvValue(usedProvider,
usedRegion) to return both { value, key } (or a tuple) and assign usedToken =
result.value; envVarName = result.key, or by calling/adding a helper
getRegionSpecificEnvKey(usedProvider, usedRegion) and setting envVarName when
regionToken is applied; ensure references are to getRegionSpecificEnvValue,
envVarName, usedToken, usedProvider, and usedRegion.
- Around line 183-201: The code currently validates regions against
providerDef.regionConfig (provider-wide regions) which allows a region like
"cn-beijing" even if the selected (providerId, modelName) mapping doesn't
include it; instead, validate the selected model/region pair against the
expanded mapping found in modelInfo.providers (the providerMappingForSelected
result). Replace or augment the region check to look up
providerMappingForSelected (from modelInfo.providers.find) and if a usedRegion
is present but not listed on that mapping’s region list (or if
providerMappingForSelected is undefined for the chosen region), throw the
HTTPException with the same 400 message listing available regions derived from
the mapping entries for that model (use modelInfo.providers.filter to collect
region ids) so requests fail locally with a clear error rather than proceeding
and erroring upstream.
In `@apps/ui/src/components/provider-keys/create-provider-key-dialog.tsx`:
- Line 75: The Select currently displays defaultRegion while selectedRegion
remains "", so the form saves an empty region; fix by deriving an
effectiveRegion (e.g., const effectiveRegion = selectedRegion || defaultRegion)
and use effectiveRegion as the Select's value and when building
payload.options.region in the submit handler (instead of selectedRegion); ensure
the onChange still updates setSelectedRegion so explicit changes override
defaultRegion; apply the same effectiveRegion pattern to the other Select usages
in this component (the other Select blocks mentioned).
In `@packages/actions/src/validate-provider-key.ts`:
- Around line 26-35: The selectedRegion calculation currently only reads the
explicit providerKeyOptions value and can be undefined; update selectedRegion to
use the provider key option OR fall back to defaultRegion (i.e., compute
selectedRegion = providerKeyOptions[regionKey] ?? defaultRegion) so that model
filtering uses the resolved region, not an undefined explicit option; apply the
same change to the other occurrence that uses
providerDef/regionKey/providerKeyOptions (the block around validationRegion
usage) so both model-selection sites consistently use the defaultRegion
fallback.
In `@packages/db/src/schema.ts`:
- Line 985: The schema added a new nullable region column in
packages/db/src/schema.ts (symbol: region) and updated the unique constraint to
unique().on(table.modelId, table.providerId, table.region) but no migration
exists; add a new migration that ALTER TABLE to add the region column (nullable
or with a default sentinel) and update the unique index to include region,
ensuring uniqueness semantics are deterministic: either create the unique index
with nullsNotDistinct: true (Postgres 15+) or migrate existing NULLs to a
non-null sentinel and make region non-null before creating the unique
constraint; also search for the consumer isNull(modelProviderMapping.region) in
apps/worker/src/services/sync-models.ts and adjust logic if you switch to a
sentinel value.
In `@packages/models/src/provider.ts`:
- Around line 133-142: hasRegionSpecificEnvKey currently only returns true for
explicit per-region vars like `${BASE}__${REGION}` which excludes the case where
only a base key exists for the provider; update hasRegionSpecificEnvKey (and use
getProviderEnvVar) to also return true when the base env var
(process.env[baseEnvVar]) exists and the queried region equals the provider's
configured/default region (obtain the configured/default region from the
Provider object — e.g. provider.region or a provider.getRegion() method), while
still returning true for explicit `${baseEnvVar}__${REGION}` entries.
---
Outside diff comments:
In `@apps/gateway/src/chat/chat.ts`:
- Around line 2738-2758: The streaming retry loop fails to propagate
ctx.usedRegion into local state so providerRetryKey(usedProvider, usedRegion)
uses a stale region; update the catch/fallback branch where locals like
usedProvider/usedModel/etc. are assigned from ctx to also assign usedRegion =
ctx.usedRegion so the retry tracking and region-based selection reflect the new
candidate region (look for the block assigning usedProvider, usedModel, url,
providerKey, providerRetryKey usage and add usedRegion assignment there).
- Around line 1593-1606: The synthesized provider score objects created from
allModelProviders omit region and only compare providerId to usedProvider, which
breaks regional mappings and retries in selectNextProvider; update the object
returned in the allModelProviders.map (the score entries) to include p.region
and change the isSelected check to compare both p.providerId and p.region (or
construct usedProvider identity that includes region) so that region is carried
through and selectNextProvider can map retry scores back to the correct expanded
row (referencing allModelProviders, metricsMap, metricsKey, usedProvider, and
selectNextProvider).
- Around line 1221-1233: The metrics lookup is only keyed by provider
(metricsKey used with { modelId: baseModelId, providerId: usedProvider }) so
regional metrics are treated as region IS NULL and your fallback excludes all
regions of the provider; fix by including usedRegion when calling
getProviderMetricsForCombinations and when building the lookup key (use {
modelId: baseModelId, providerId: usedProvider, region: usedRegion } via
metricsKey/getProviderMetricsForCombinations), then when selecting fallback
candidates from modelInfo.providers only filter out the exact failing
provider+region pair (compare both providerId and region) so other regions for
the same provider remain eligible.
- Around line 1794-1797: The initial env-backed token lookup uses
getProviderEnv(usedProvider) unconditionally, causing regional requests to use
the base key; change this to perform a region-aware lookup first: call
getRegionSpecificEnvValue(usedProvider, usedRegion) when usedRegion is a
non-default region (and when credits/hybrid-without-DB-key paths may set a
non-default usedRegion), and if that returns nothing fall back to
getProviderEnv(usedProvider); update the same logic in both places mentioned
(the block using usedToken/configIndex/envVarName and the other occurrence
around lines 1854-1858) so initial requests and retries use the same
region-specific env resolution.
In `@packages/models/src/models/alibaba.ts`:
- Around line 342-356: Several provider mappings with providerId "alibaba"
(modelName values: "qwen-omni-turbo", "qwen-vl-max", "qwen-vl-plus",
"qwen3-next-80b-a3b-thinking", "qwen3-next-80b-a3b-instruct",
"qwen3-coder-plus", "qwen3-vl-235b-a22b-instruct",
"qwen3-vl-235b-a22b-thinking", "qwen2-5-vl-32b-instruct", "qwen-image-plus",
"qwen-image-max", "qwen-image", "qwen-image-edit-plus", "qwen-image-edit-max")
are missing a regions field; add a regions array to each of those model objects
(the same region values used by other Alibaba entries such as the
"qwen-max"/"qwen-plus"/"qwen-turbo"/"qwen3-vl-plus" mappings) so all Alibaba
provider entries include a regions property for consistent region-aware routing.
---
Nitpick comments:
In `@apps/api/src/routes/keys-provider.ts`:
- Line 36: Add explicit validation for alibaba_region so it only accepts the
allowed region strings instead of any string: update the schema property (in the
schema object that defines alibaba_region and in createProviderKeySchema) to use
a constrained enum/union with the permitted values "singapore", "us-virginia",
and "cn-beijing" (matching regionConfig in packages/models/src/providers.ts);
keep the field optional but validate its value when present and ensure both
occurrences (the one at line ~36 and the one in createProviderKeySchema) are
changed.
In `@apps/worker/src/services/sync-models.ts`:
- Around line 104-116: Replace the raw select/limit pattern with the
repository-style findFirst query: instead of calling
database.select().from(modelProviderMapping).where(...).limit(1), call
database.query.modelProviderMapping.findFirst({ where: and(...) }) using the
same predicates (eq(modelProviderMapping.modelId, modelDef.id),
eq(modelProviderMapping.providerId, mapping.providerId), and mappingRegion ?
eq(modelProviderMapping.region, mappingRegion) :
isNull(modelProviderMapping.region)). Ensure you import/retain references to
database, modelProviderMapping, modelDef, mapping.providerId, and mappingRegion
so the call matches the repo convention.
In `@docs/plans/2026-03-06-provider-region-support.md`:
- Line 13: The document skips heading level by jumping from h1 to h3; insert an
h2 heading such as "## Implementation Tasks" immediately before the existing
"### Task 1: Add `regions` field to `ProviderModelMapping`" header so the
structure follows h1 → h2 → h3 and improves accessibility and outline
consistency.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 3ba18be9-da29-4a17-8184-b262e435d6dc
⛔ Files ignored due to path filters (4)
apps/code/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsapps/playground/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsapps/ui/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsee/admin/src/lib/api/v1.d.tsis excluded by!**/v1.d.ts
📒 Files selected for processing (26)
apps/api/src/routes/internal-models.tsapps/api/src/routes/keys-provider.tsapps/gateway/src/chat/chat.tsapps/gateway/src/chat/tools/resolve-provider-context.tsapps/gateway/src/chat/tools/retry-with-fallback.tsapps/ui/src/components/models/model-comparison.tsxapps/ui/src/components/models/model-provider-card.tsxapps/ui/src/components/provider-keys/create-provider-key-dialog.tsxapps/ui/src/components/shared/model-card.tsxapps/ui/src/lib/fetch-models.tsapps/worker/src/services/sync-models.tsdocs/plans/2026-03-06-provider-region-support-design.mddocs/plans/2026-03-06-provider-region-support.mdpackages/actions/src/get-cheapest-from-available-providers.tspackages/actions/src/get-provider-endpoint.tspackages/actions/src/validate-provider-key.tspackages/db/src/provider-metrics.spec.tspackages/db/src/provider-metrics.tspackages/db/src/schema.tspackages/models/src/index.tspackages/models/src/models.tspackages/models/src/models/alibaba.tspackages/models/src/provider.tspackages/models/src/providers.tspackages/models/src/region-helpers.tspackages/models/src/types.ts
| .notNull() | ||
| .references(() => provider.id, { onDelete: "cascade" }), | ||
| modelName: text().notNull(), | ||
| region: text(), |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect schema and migrations for model_provider_mapping uniqueness + NULL handling.
rg -n -C3 'model_provider_mapping|NULLS NOT DISTINCT|nullsNotDistinct|COALESCE|unique\('Repository: theopenco/llmgateway
Length of output: 50378
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find migration files that mention region and model_provider_mapping
echo "=== Recent migrations mentioning region ==="
fd '\.sql$' packages/db/migrations | xargs grep -l 'region' | tail -5
# Check latest migrations for region column addition
echo -e "\n=== Latest migrations in order ==="
ls -lt packages/db/migrations/*.sql | head -10 | awk '{print $NF}'Repository: theopenco/llmgateway
Length of output: 109
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read schema.ts around lines 985-1035 to see the region column and constraint definition
echo "=== schema.ts lines 980-1040 ==="
sed -n '980,1040p' packages/db/src/schema.ts
# Also check if there's a unique constraint definition for region
echo -e "\n=== Searching for region unique constraint in schema.ts ==="
rg -n 'region.*unique|unique.*region' packages/db/src/schema.ts -A2 -B2Repository: theopenco/llmgateway
Length of output: 2236
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find the migration that adds region column to model_provider_mapping
echo "=== Migration adding region to model_provider_mapping ==="
rg -n 'ALTER TABLE.*model_provider_mapping.*ADD COLUMN.*region|ADD COLUMN.*region.*ALTER TABLE.*model_provider_mapping' packages/db/migrations/ -A 1 -B 1
# Also look for migration creating new unique constraint with region
echo -e "\n=== Migrations with unique constraint on (modelId, providerId, region) ==="
rg -n 'UNIQUE.*model_id.*provider_id.*region|UNIQUE.*region.*provider_id.*model_id' packages/db/migrations/ -A 1 -B 1Repository: theopenco/llmgateway
Length of output: 121
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check sync-models.ts for the isNull usage mentioned in the review
echo "=== sync-models.ts lookup pattern for global mappings ==="
rg -n 'isNull.*region|region.*isNull' apps/worker/src/services/sync-models.ts -A 5 -B 5Repository: theopenco/llmgateway
Length of output: 503
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check if nullsNotDistinct is used anywhere in schema.ts for modelProviderMapping
echo "=== Checking nullsNotDistinct usage in schema.ts ==="
rg -n 'nullsNotDistinct' packages/db/src/schema.ts | head -20
# Check the Drizzle unique constraint syntax to see if nullsNotDistinct can be applied
echo -e "\n=== Looking at unique constraint definitions in schema.ts ==="
rg -n 'unique()' packages/db/src/schema.ts | head -20
# Check migration metadata for any nullsNotDistinct settings
echo -e "\n=== Check latest migration metadata snapshots ==="
ls -lt packages/db/migrations/meta/*.json | head -1 | awk '{print $NF}' | xargs grep -n 'model_provider_mapping.*-A100' | head -50Repository: theopenco/llmgateway
Length of output: 116
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check if there's documentation or examples of how the unique constraint with nullable region works
echo "=== Check how unique constraint is defined in Drizzle ==="
rg -B5 -A5 'unique\(\)\.on\(.*region' packages/db/src/schema.ts
# Look at other tables with nullable columns in unique constraints to see pattern
echo -e "\n=== Other nullable columns in unique constraints ==="
rg -n 'unique.*' packages/db/src/schema.ts | grep -v '//'Repository: theopenco/llmgateway
Length of output: 1368
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Get the exact Drizzle version to check if it supports nullsNotDistinct
echo "=== Drizzle ORM version ==="
grep -A 2 '"drizzle-orm"' packages/db/package.json
# Check if nullsNotDistinct is documented in the codebase
echo -e "\n=== nullsNotDistinct in package.json or docs ==="
rg -i 'nullsNotDistinct' packages/db/Repository: theopenco/llmgateway
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo zipefail
# Check the latest migration snapshot for model_provider_mapping unique constraint
echo "=== Latest snapshot with model_provider_mapping constraint details ==="
ls -t packages/db/migrations/meta/*.json | head -1 | xargs cat | jq '.schemas[] | select(.name=="model_provider_mapping") | .uniqueConstraints' 2>/dev/null || echo "Using grep approach..."
# Alternative: Check the latest snapshot for the constraint definition
echo -e "\n=== model_provider_mapping unique constraint in latest snapshot ==="
ls -t packages/db/migrations/meta/*.json | head -1 | xargs grep -A5 '"name": "model_provider_mapping_modelId_providerId_region_unique"' || echo "Constraint name not found, checking alternatives..."
# Check for the constraint more broadly
echo -e "\n=== Any unique constraints in latest snapshot with model_provider_mapping ==="
ls -t packages/db/migrations/meta/*.json | head -1 | xargs grep -B2 -A5 'modelId.*providerId.*region\|region.*provider_id.*model_id'Repository: theopenco/llmgateway
Length of output: 117
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find which migration added the region column with the new unique constraint
echo "=== Migration files and region column addition ==="
for f in $(ls -t packages/db/migrations/*.sql | head -10); do
if grep -q 'region' "$f" 2>/dev/null; then
echo "File: $f"
grep -n 'region' "$f" | head -5
fi
done
# Check if any migration explicitly sets NULLS NOT DISTINCT
echo -e "\n=== Migrations with NULLS NOT DISTINCT ==="
rg -n 'NULLS NOT DISTINCT' packages/db/migrations/ | head -20Repository: theopenco/llmgateway
Length of output: 158
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check the latest migration snapshot for model_provider_mapping constraint
echo "=== Latest migration metadata for model_provider_mapping ==="
LATEST_SNAPSHOT=$(ls -t packages/db/migrations/meta/*.json | head -1)
echo "Using: $LATEST_SNAPSHOT"
grep -A 100 '"name": "model_provider_mapping"' "$LATEST_SNAPSHOT" | grep -A 50 'uniqueConstraints' | head -60Repository: theopenco/llmgateway
Length of output: 184
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check if region column was added via ALTER TABLE or in table creation
echo "=== Check if region column appears in initial table creation ==="
grep -A 50 'CREATE TABLE "model_provider_mapping"' packages/db/migrations/1757957101_misty_whiplash.sql | grep -i region
# Check for DROP and recreate pattern
echo -e "\n=== Check ALTER TABLE for region addition ==="
rg -n 'ALTER TABLE.*model_provider_mapping' packages/db/migrations/ | grep -i regionRepository: theopenco/llmgateway
Length of output: 128
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# The region column should have been added recently, check git history or migrations in order
echo "=== All migrations affecting model_provider_mapping in chronological order ==="
for f in $(find packages/db/migrations -name '*.sql' -type f | sort); do
if grep -q 'model_provider_mapping' "$f"; then
echo "=== $(basename $f) ==="
grep -n 'model_provider_mapping\|region' "$f" 2>/dev/null | head -10
fi
done | tail -100Repository: theopenco/llmgateway
Length of output: 6328
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# List all migration files in order
echo "=== All migration files in order ==="
ls -1 packages/db/migrations/*.sql | sort -V | tail -20
# Check if there's a migration that modifies the unique constraint to add region
echo -e "\n=== Migrations mentioning unique constraint changes ==="
rg -n 'DROP.*CONSTRAINT.*model_provider_mapping.*unique|ALTER.*CONSTRAINT' packages/db/migrations/ -iRepository: theopenco/llmgateway
Length of output: 12151
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Look for any mention of region in schema.ts context around the definition
echo "=== Full modelProviderMapping table definition in schema.ts ==="
sed -n '965,1040p' packages/db/src/schema.tsRepository: theopenco/llmgateway
Length of output: 2392
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check the latest snapshot to see if region column actually exists in schema
echo "=== Check latest migration snapshot for region column in model_provider_mapping ==="
LATEST=$(ls -t packages/db/migrations/meta/*.json | head -1)
cat "$LATEST" | jq '.schemas[] | select(.name=="model_provider_mapping") | .columns[] | select(.name | contains("region"))' 2>/dev/null || echo "No region column found in schema or error parsing JSON"Repository: theopenco/llmgateway
Length of output: 202
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# The region column exists in schema.ts but appears to not be in any migration
# Let's verify this is a schema/migration mismatch issue
echo "=== Check if region column exists in any migration ==="
rg -n 'ADD COLUMN.*region|region.*ADD COLUMN' packages/db/migrations/
echo -e "\n=== Check if the new unique constraint is in any migration ==="
rg -n 'modelId.*providerId.*region|model_id.*provider_id.*region' packages/db/migrations/
echo -e "\n=== Verify schema.ts region definition one more time ==="
rg -B2 -A2 'region: text\(\)' packages/db/src/schema.tsRepository: theopenco/llmgateway
Length of output: 118
Schema and migrations are out of sync—the region column and updated unique constraint don't exist in any migration.
The region column is defined in schema.ts (line 985) and the unique constraint is updated to include it (line 1035), but neither the column addition nor the constraint change appear in any migration file. The latest migration snapshot does not include the region column, meaning the database schema doesn't match the code.
Additionally, if this migration is added, the constraint unique().on(table.modelId, table.providerId, table.region) with a nullable region column is unsafe: PostgreSQL's unique constraints treat NULL as distinct, so multiple (modelId, providerId, NULL) rows can coexist. The worker lookup at apps/worker/src/services/sync-models.ts line 113 using isNull(modelProviderMapping.region) with .limit(1) will become nondeterministic when duplicate null-region rows exist.
Create the missing migration to add the region column and either use nullsNotDistinct: true on the constraint (PostgreSQL 15+) or replace NULL with a non-null sentinel for global mappings.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/db/src/schema.ts` at line 985, The schema added a new nullable
region column in packages/db/src/schema.ts (symbol: region) and updated the
unique constraint to unique().on(table.modelId, table.providerId, table.region)
but no migration exists; add a new migration that ALTER TABLE to add the region
column (nullable or with a default sentinel) and update the unique index to
include region, ensuring uniqueness semantics are deterministic: either create
the unique index with nullsNotDistinct: true (Postgres 15+) or migrate existing
NULLs to a non-null sentinel and make region non-null before creating the unique
constraint; also search for the consumer isNull(modelProviderMapping.region) in
apps/worker/src/services/sync-models.ts and adjust logic if you switch to a
sentinel value.
- Add deepseek-v3.2 for Singapore and cn-beijing - Fix qwen-max/qwen-max-latest context sizes - Update cn-beijing tiered pricing for qwen-plus, qwen-flash, qwen3-coder-flash, qwen3-vl-plus, qwen3-vl-flash, qwen3-max-2026-01-23 - Fix qwen-coder-plus cn-beijing pricing - Remove unsupported us-virginia from qwen-plus-latest, qwen-flash, qwen3.5-397b-a17b - Add contextSize to ProviderRegion type - Add 20% discount for deepseek-v3.2 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…support # Conflicts: # apps/ui/src/components/provider-keys/create-provider-key-dialog.tsx
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Support provider/model:region syntax to pin a specific region (e.g. alibaba/qwen-plus:cn-beijing) - Expand regions in e2e test helpers so each region becomes a separate test case - Fix duplicate deepseek-v3.2 model ID by moving Alibaba provider mapping to deepseek.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
apps/gateway/src/chat-helpers.e2e.ts (1)
196-198: Avoid duplicate region expansion work intestModels.You can compute
expandedProvidersonce and reuse it for both the root-model case and provider-specific cases.Suggested refactor
.flatMap((model) => { const testCases = []; + const expandedProviders = expandAllProviderRegions( + model.providers as ProviderModelMapping[], + ); if (process.env.TEST_ALL_VARIATIONS) { // test root model without a specific provider testCases.push({ model: model.id, - providers: expandAllProviderRegions( - model.providers as ProviderModelMapping[], - ).filter((provider: ProviderModelMapping) => provider.test !== "skip"), + providers: expandedProviders.filter( + (provider: ProviderModelMapping) => provider.test !== "skip", + ), }); } // Create entries for provider-specific requests using provider/model format // Expand regions so each provider:region combo becomes a separate test case - const expandedProviders = expandAllProviderRegions( - model.providers as ProviderModelMapping[], - ); for (const provider of expandedProviders) {Also applies to: 204-206
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@apps/gateway/src/chat-helpers.e2e.ts` around lines 196 - 198, The testModels function currently calls expandAllProviderRegions twice; compute expandedProviders once by calling expandAllProviderRegions(model.providers as ProviderModelMapping[]) and store the result in a local variable (e.g., expandedProviders) and then reuse that variable for both the root-model branch and the provider-specific branch filters (instead of re-invoking expandAllProviderRegions), updating references in testModels and the adjacent provider-specific logic (the same change applies to the similar calls around the 204-206 area).packages/models/src/models/alibaba.ts (2)
1720-1805: Similar naming inconsistency in qwen3-vl-flash regions.Same pattern as qwen3-vl-plus: us-virginia uses "Over 128K" (Line 1768) while cn-beijing uses "128K-256K" (Line 1797). If the tier boundaries are actually different (us-virginia truly unlimited vs cn-beijing capped at 256K), the naming is appropriate. Otherwise, consider aligning for consistency.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/models/src/models/alibaba.ts` around lines 1720 - 1805, The regions array contains inconsistent pricing tier names between region id "us-virginia" (tier named "Over 128K" with upToTokens: Infinity) and region id "cn-beijing" (tier named "128K-256K" with upToTokens: 256000); update the tier naming or boundaries so they match their upToTokens meaning — either rename the us-virginia tier to "128K-256K" if it should be capped at 256K, or change cn-beijing's tier to "Over 128K" and set upToTokens: Infinity if it should be unlimited; edit the entries inside the regions array (look for id: "us-virginia" and id: "cn-beijing" and their pricingTiers) to make the label and upToTokens consistent.
1610-1695: Minor inconsistency in pricing tier naming for qwen3-vl-plus.The us-virginia region (Line 1658) uses "Over 128K" for the highest tier, while cn-beijing (Line 1687) uses "128K-256K". This naming inconsistency is minor but could cause confusion when comparing regional pricing.
Singapore's highest tier is also "128K-256K" (Line 1629-1630), so us-virginia's "Over 128K" with
upToTokens: Infinityis the outlier.📝 Suggested naming alignment
{ - name: "Over 128K", + name: "128K-256K", upToTokens: Infinity,Or alternatively, if us-virginia truly supports unlimited context beyond 128K, leave as-is but document why it differs.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/models/src/models/alibaba.ts` around lines 1610 - 1695, The highest pricing tier for region entries is inconsistent: in the regions array the us-virginia entry uses name "Over 128K" with upToTokens: Infinity while singapore and cn-beijing use "128K-256K"; update the us-virginia pricing tier in the regions array (look for region id "us-virginia" and its pricingTiers) to use the same naming and bounds as the others by renaming the tier to "128K-256K" and setting upToTokens: 256000 (or, if us-virginia truly supports unlimited context, keep Infinity but add a clear comment/documentation explaining why this tier differs).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@apps/gateway/src/chat-helpers.e2e.ts`:
- Around line 203-207: After expandAllProviderRegions returns provider+region
entries, the code still builds and matches model IDs using only
`${provider.providerId}/${model.id}`, collapsing distinct region cases; update
both the emitted test IDs and any TEST_MODELS/specifedModels matching logic to
include the region (e.g., incorporate provider.region or provider.regionId into
the key), so IDs become region-aware like
`${provider.providerId}/${provider.region}/${model.id}` (or similar consistent
canonical form), and apply this change for all occurrences including the loop
using expandedProviders and the later block around lines 289-293.
In `@apps/gateway/src/chat/tools/parse-model-input.ts`:
- Around line 56-61: The current colon-suffix parsing unconditionally splits
modelName (mutating modelName and setting requestedRegion) which breaks custom
provider model IDs that legitimately contain ':' and allows empty regions;
update the parsing logic so it first checks the provider (skip parsing when
provider === "custom") and only treat the trailing ":region" as a region when
the part after the last ':' is non-empty; operate on a local copy (e.g., use
originalModelName or temp variable) so you don't corrupt modelName for custom
providers, and assign requestedRegion only when the suffix is non-empty.
---
Nitpick comments:
In `@apps/gateway/src/chat-helpers.e2e.ts`:
- Around line 196-198: The testModels function currently calls
expandAllProviderRegions twice; compute expandedProviders once by calling
expandAllProviderRegions(model.providers as ProviderModelMapping[]) and store
the result in a local variable (e.g., expandedProviders) and then reuse that
variable for both the root-model branch and the provider-specific branch filters
(instead of re-invoking expandAllProviderRegions), updating references in
testModels and the adjacent provider-specific logic (the same change applies to
the similar calls around the 204-206 area).
In `@packages/models/src/models/alibaba.ts`:
- Around line 1720-1805: The regions array contains inconsistent pricing tier
names between region id "us-virginia" (tier named "Over 128K" with upToTokens:
Infinity) and region id "cn-beijing" (tier named "128K-256K" with upToTokens:
256000); update the tier naming or boundaries so they match their upToTokens
meaning — either rename the us-virginia tier to "128K-256K" if it should be
capped at 256K, or change cn-beijing's tier to "Over 128K" and set upToTokens:
Infinity if it should be unlimited; edit the entries inside the regions array
(look for id: "us-virginia" and id: "cn-beijing" and their pricingTiers) to make
the label and upToTokens consistent.
- Around line 1610-1695: The highest pricing tier for region entries is
inconsistent: in the regions array the us-virginia entry uses name "Over 128K"
with upToTokens: Infinity while singapore and cn-beijing use "128K-256K"; update
the us-virginia pricing tier in the regions array (look for region id
"us-virginia" and its pricingTiers) to use the same naming and bounds as the
others by renaming the tier to "128K-256K" and setting upToTokens: 256000 (or,
if us-virginia truly supports unlimited context, keep Infinity but add a clear
comment/documentation explaining why this tier differs).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: e322ff64-97cd-4a05-8949-827694764897
⛔ Files ignored due to path filters (4)
apps/code/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsapps/playground/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsapps/ui/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsee/admin/src/lib/api/v1.d.tsis excluded by!**/v1.d.ts
📒 Files selected for processing (5)
apps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat/chat.tsapps/gateway/src/chat/tools/parse-model-input.tspackages/models/src/models/alibaba.tspackages/models/src/models/deepseek.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- apps/gateway/src/chat/chat.ts
Comment on lines
+56
to
+61
| // Parse optional region suffix (e.g. "qwen-plus:cn-beijing") | ||
| if (modelName.includes(":")) { | ||
| const colonIdx = modelName.lastIndexOf(":"); | ||
| requestedRegion = modelName.slice(colonIdx + 1); | ||
| modelName = modelName.slice(0, colonIdx); | ||
| } |
Contributor
There was a problem hiding this comment.
Region suffix parsing can break custom provider model names with : and accepts empty region values.
Line 56 currently parses : for all providers, including custom. That can mutate valid custom model IDs (e.g., names containing :...). Also, provider/model: yields an empty region without validation.
💡 Proposed fix
// Handle model names with multiple slashes (e.g. together.ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo)
let modelName = split.slice(1).join("/");
// Parse optional region suffix (e.g. "qwen-plus:cn-beijing")
- if (modelName.includes(":")) {
+ if (requestedProvider !== "custom" && modelName.includes(":")) {
const colonIdx = modelName.lastIndexOf(":");
- requestedRegion = modelName.slice(colonIdx + 1);
+ const regionCandidate = modelName.slice(colonIdx + 1).trim();
+ if (!regionCandidate) {
+ throw new HTTPException(400, {
+ message: `Invalid region in model input ${modelInput}`,
+ });
+ }
+ requestedRegion = regionCandidate;
modelName = modelName.slice(0, colonIdx);
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@apps/gateway/src/chat/tools/parse-model-input.ts` around lines 56 - 61, The
current colon-suffix parsing unconditionally splits modelName (mutating
modelName and setting requestedRegion) which breaks custom provider model IDs
that legitimately contain ':' and allows empty regions; update the parsing logic
so it first checks the provider (skip parsing when provider === "custom") and
only treat the trailing ":region" as a region when the part after the last ':'
is non-empty; operate on a local copy (e.g., use originalModelName or temp
variable) so you don't corrupt modelName for custom providers, and assign
requestedRegion only when the suffix is non-empty.
- Propagate usedRegion in streaming retry loop - Use region-specific env vars for initial token resolution in credits/hybrid mode - Carry region in synthesized provider scores and fix isSelected to compare provider+region - Fix low-uptime fallback to exclude only the exact provider+region pair, not all regions - Update hasRegionSpecificEnvKey to accept base key for default region - Fall back to defaultRegion in validate-provider-key - Fix effectiveRegion in create-provider-key dialog - Update envVarName when regional token is resolved - Validate region against model mappings, not provider catalog - Validate alibaba_region against known enum values - Remove unused plan docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add optional :region suffix to TEST_MODELS entries for targeting specific provider regions in e2e tests. Example: TEST_MODELS=alibaba/deepseek-v3.2:cn-beijing Without region suffix, all regions of that provider/model match. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The model-level filters used raw (unexpanded) providers, so region-specific TEST_MODELS entries like alibaba/model:cn-beijing were not matching nested region arrays. Expand providers before matching in both modelMatchesAnyTestModel and the final filteredModels filter. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add used_region to response metadata and region to routing attempts array so clients can see which region was used. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix region not appearing in response metadata by: - Resolving usedRegion in the direct-provider path (when requestedProvider is set, not just auto-routing) - Expanding modelWithPricing providers so scoring can match region entries for price lookup - Adding used_region to all Alibaba and other provider-specific metadata blocks in transform-response-to-openai.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
steebchen
changed the title
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements region-aware provider/model mappings end-to-end, centered on Alibaba multi-region support, and carries the follow-up fixes needed to make regional routing work cleanly after rebases onto
main.What Changed Relative to
mainpackages/modelsand expands regional mappings into routable provider candidates.main(including Alibaba regional mappings and related provider/model follow-ups).Notes
main.provider:regionas distinct routing candidates where applicable, while DB-backed provider keys can pin a region explicitly.Validation
pnpm installpnpm formatpnpm buildpnpm lintpnpm --filter ui buildpnpm --filter ui devroute check for the activity page after the latest branch changes