feat(azure): add Grok models via Azure AI Foundry - #2128
Merged
Merged
Conversation
Adds a new `azure-ai-foundry` provider for the Azure AI Foundry Models inference endpoint (services.ai.azure.com/models/chat/completions), distinct from Azure OpenAI. Maps grok-4-1-fast-reasoning and grok-4-1-fast-non-reasoning to the new provider alongside the existing xAI direct route. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds first-class support for a new provider "azure-ai-foundry": env variables, provider registration, model/provider entries, endpoint/header resolution, DB & API schema updates for provider-key options, gateway streaming routing, tests, and CI env wiring. Changes
Sequence Diagram(s)sequenceDiagram
actor Client
participant Gateway
participant Actions as ActionsService
participant DB as Database
participant Provider as AzureAI_Foundry
Client->>Gateway: request using provider "azure-ai-foundry"
Gateway->>Actions: resolveEndpoint(providerId, providerOptions)
Actions->>DB: fetch providerKey/options
DB-->>Actions: return token + options (resource, apiVersion)
Actions-->>Gateway: endpoint URL + headers (api-key)
Gateway->>Provider: send streaming request (endpoint includes resource & api-version)
Provider-->>Gateway: streaming chunks (OpenAI-like)
Gateway->>Gateway: transformOpenaiStreaming -> normalized events
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cf6eb2cc17
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+377
to
+380
| id: "azure-ai-foundry", | ||
| name: "Azure AI Foundry", | ||
| description: | ||
| "Microsoft Azure AI Foundry - third-party models (Grok, Llama, Mistral, ...) via the Azure Models inference endpoint", |
There was a problem hiding this comment.
Registering azure-ai-foundry here makes it immediately selectable in the provider-key UI, but the create flow still only sends Azure-specific options for provider === "azure" (see apps/ui/src/components/provider-keys/create-provider-key-dialog.tsx), so it never submits azure_ai_foundry_resource. In BYOK validation, getProviderEndpoint requires that field and throws, which means users cannot add Azure AI Foundry keys from the dashboard unless a global env resource is set.
Useful? React with 👍 / 👎.
Comment on lines
+437
to
+441
| getProviderEnvValue( | ||
| "azure-ai-foundry", | ||
| "apiVersion", | ||
| configIndex, | ||
| "2024-05-01-preview", |
There was a problem hiding this comment.
This new branch always reads LLM_AZURE_AI_FOUNDRY_API_VERSION via getProviderEnvValue even when skipEnvVars is true. In BYOK contexts (for example validateProviderKey calls getProviderEndpoint(..., true)), endpoint construction can silently depend on server-wide env instead of the key's options/default, so one incompatible global API version can break validation and requests for all BYOK Azure AI Foundry keys.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
.env.example (1)
198-200: ⚡ Quick winAdd the optional Azure AI Foundry API version env var to the example.
The runtime supports
LLM_AZURE_AI_FOUNDRY_API_VERSION, but it is not surfaced here, which makes the override harder to discover.Proposed patch
# Azure AI Foundry (third-party models like Grok, Llama, Mistral) LLM_AZURE_AI_FOUNDRY_API_KEY=your_azure_ai_foundry_key_here LLM_AZURE_AI_FOUNDRY_RESOURCE=your_azure_ai_foundry_resource_here +# Optional override (leave unset to use the default in code) +LLM_AZURE_AI_FOUNDRY_API_VERSION=2024-05-01-preview🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.env.example around lines 198 - 200, Add the optional Azure AI Foundry API version environment variable to the example by documenting LLM_AZURE_AI_FOUNDRY_API_VERSION alongside LLM_AZURE_AI_FOUNDRY_API_KEY and LLM_AZURE_AI_FOUNDRY_RESOURCE in .env.example; include a short comment indicating it is optional and used to override the default API version at runtime so users can discover and set LLM_AZURE_AI_FOUNDRY_API_VERSION when needed..env.unified.example (1)
111-113: ⚡ Quick winMirror the optional Azure AI Foundry API version in the unified env example.
This keeps
.env.unified.examplealigned with supported runtime config and avoids hidden behavior.Proposed patch
# Azure AI Foundry (third-party models like Grok, Llama, Mistral) LLM_AZURE_AI_FOUNDRY_API_KEY=your_azure_ai_foundry_key_here LLM_AZURE_AI_FOUNDRY_RESOURCE=your_azure_ai_foundry_resource_here +# Optional override (leave unset to use the default in code) +LLM_AZURE_AI_FOUNDRY_API_VERSION=2024-05-01-preview🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.env.unified.example around lines 111 - 113, Add the optional Azure AI Foundry API version variable to the unified env example so it mirrors runtime config: add a new env var named LLM_AZURE_AI_FOUNDRY_API_VERSION (with a placeholder value like your_api_version_here) alongside LLM_AZURE_AI_FOUNDRY_API_KEY and LLM_AZURE_AI_FOUNDRY_RESOURCE in .env.unified.example so consumers can opt into a specific API version at runtime.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/actions/src/get-provider-endpoint.ts`:
- Around line 265-280: The code constructs the Azure Foundry base URL from
`resource` (from `providerKeyOptions?.azure_ai_foundry_resource` or
`getProviderEnvValue`) without validating its format, so add validation in the
`case "azure-ai-foundry"` block: ensure `resource` matches a safe DNS-label
pattern (e.g. only letters, numbers and hyphens, no dots, slashes or scheme, and
acceptable length), reject values containing '.' '/' ':' or '://' or other
invalid chars, and if invalid throw an Error (use
`getProviderEnvConfig("azure-ai-foundry")` to build the message similar to the
existing one). Only compose `url =
\`https://${resource}.services.ai.azure.com\`` after the check. Ensure
references: `providerKeyOptions?.azure_ai_foundry_resource`,
`getProviderEnvValue`, `getProviderEnvConfig`, and the `url` assignment are
updated.
- Around line 434-445: In the "azure-ai-foundry" branch of the
get-provider-endpoint logic, the apiVersion is currently resolved using
getProviderEnvValue even when skipEnvVars is true; update the resolution so it
first uses providerKeyOptions?.azure_ai_foundry_api_version, then only calls
getProviderEnvValue("azure-ai-foundry","apiVersion",configIndex,"2024-05-01-preview")
when skipEnvVars is false, otherwise fall back to the literal default
"2024-05-01-preview"; modify the code in the case "azure-ai-foundry" block to
reference skipEnvVars, providerKeyOptions?.azure_ai_foundry_api_version, and
getProviderEnvValue accordingly so BYOK/skip-env behavior matches other
branches.
In `@packages/models/src/models/xai.ts`:
- Around line 392-408: The azure-ai-foundry model entries (providerId
"azure-ai-foundry", modelName "grok-4-1-fast-reasoning" and the other
azure-ai-foundry model around the 450-465 region) are missing pricingTiers;
update those objects to include the same 128K split pricing tiers used by the
sibling xai entries so long-context and cached-segment billing matches xAI
mappings, ensuring you add a pricingTiers property that mirrors the xai entries'
tier definitions (the entry that uses xaiSupportedParamsNoFreqPresence can be
used as reference).
---
Nitpick comments:
In @.env.example:
- Around line 198-200: Add the optional Azure AI Foundry API version environment
variable to the example by documenting LLM_AZURE_AI_FOUNDRY_API_VERSION
alongside LLM_AZURE_AI_FOUNDRY_API_KEY and LLM_AZURE_AI_FOUNDRY_RESOURCE in
.env.example; include a short comment indicating it is optional and used to
override the default API version at runtime so users can discover and set
LLM_AZURE_AI_FOUNDRY_API_VERSION when needed.
In @.env.unified.example:
- Around line 111-113: Add the optional Azure AI Foundry API version variable to
the unified env example so it mirrors runtime config: add a new env var named
LLM_AZURE_AI_FOUNDRY_API_VERSION (with a placeholder value like
your_api_version_here) alongside LLM_AZURE_AI_FOUNDRY_API_KEY and
LLM_AZURE_AI_FOUNDRY_RESOURCE in .env.unified.example so consumers can opt into
a specific API version at runtime.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: b857ebb4-64a9-4cb6-9396-a235ffeeb9cd
⛔ Files ignored due to path filters (4)
apps/code/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsapps/playground/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsapps/ui/src/lib/api/v1.d.tsis excluded by!**/v1.d.tsee/admin/src/lib/api/v1.d.tsis excluded by!**/v1.d.ts
📒 Files selected for processing (11)
.env.example.env.unified.exampleapps/api/src/routes/keys-provider.tsapps/gateway/src/chat-helpers.e2e.tsapps/gateway/src/chat/tools/transform-streaming-to-openai.tspackages/actions/src/get-provider-endpoint.spec.tspackages/actions/src/get-provider-endpoint.tspackages/actions/src/get-provider-headers.tspackages/db/src/schema.tspackages/models/src/models/xai.tspackages/models/src/providers.ts
Comment on lines
+434
to
+445
| case "azure-ai-foundry": { | ||
| const apiVersion = | ||
| providerKeyOptions?.azure_ai_foundry_api_version ?? | ||
| getProviderEnvValue( | ||
| "azure-ai-foundry", | ||
| "apiVersion", | ||
| configIndex, | ||
| "2024-05-01-preview", | ||
| ) ?? | ||
| "2024-05-01-preview"; | ||
| return `${url}/models/chat/completions?api-version=${apiVersion}`; | ||
| } |
Contributor
There was a problem hiding this comment.
Respect skipEnvVars for Foundry apiVersion resolution
This branch reads env vars even when skipEnvVars is true. That makes BYOK mode behavior inconsistent with the helper used elsewhere in this function.
Suggested fix
case "azure-ai-foundry": {
const apiVersion =
providerKeyOptions?.azure_ai_foundry_api_version ??
- getProviderEnvValue(
- "azure-ai-foundry",
- "apiVersion",
- configIndex,
- "2024-05-01-preview",
- ) ??
+ envValueOrDefault(
+ "azure-ai-foundry",
+ "apiVersion",
+ "2024-05-01-preview",
+ ) ??
"2024-05-01-preview";
return `${url}/models/chat/completions?api-version=${apiVersion}`;
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/actions/src/get-provider-endpoint.ts` around lines 434 - 445, In the
"azure-ai-foundry" branch of the get-provider-endpoint logic, the apiVersion is
currently resolved using getProviderEnvValue even when skipEnvVars is true;
update the resolution so it first uses
providerKeyOptions?.azure_ai_foundry_api_version, then only calls
getProviderEnvValue("azure-ai-foundry","apiVersion",configIndex,"2024-05-01-preview")
when skipEnvVars is false, otherwise fall back to the literal default
"2024-05-01-preview"; modify the code in the case "azure-ai-foundry" block to
reference skipEnvVars, providerKeyOptions?.azure_ai_foundry_api_version, and
getProviderEnvValue accordingly so BYOK/skip-env behavior matches other
branches.
Comment on lines
+392
to
+408
| { | ||
| providerId: "azure-ai-foundry", | ||
| modelName: "grok-4-1-fast-reasoning", | ||
| inputPrice: 0.2 / 1e6, | ||
| outputPrice: 0.5 / 1e6, | ||
| cachedInputPrice: 0.05 / 1e6, | ||
| requestPrice: 0, | ||
| imageInputPrice: undefined, | ||
| contextSize: 2_000_000, | ||
| maxOutput: 30000, | ||
| streaming: true, | ||
| vision: true, | ||
| reasoning: true, | ||
| tools: true, | ||
| jsonOutput: true, | ||
| supportedParameters: xaiSupportedParamsNoFreqPresence, | ||
| }, |
Contributor
There was a problem hiding this comment.
Keep Azure AI Foundry pricing tiers aligned with the xAI mappings.
Both new azure-ai-foundry entries omit pricingTiers, while the sibling xai entries for the same models include 128K split pricing. This can undercharge/overcharge long-context calls and cached segments.
Proposed patch
{
providerId: "azure-ai-foundry",
modelName: "grok-4-1-fast-reasoning",
inputPrice: 0.2 / 1e6,
outputPrice: 0.5 / 1e6,
+ pricingTiers: [
+ {
+ name: "Up to 128K",
+ upToTokens: 128000,
+ inputPrice: 0.2 / 1e6,
+ outputPrice: 0.5 / 1e6,
+ cachedInputPrice: 0.05 / 1e6,
+ },
+ {
+ name: "Over 128K",
+ upToTokens: Infinity,
+ inputPrice: 0.4 / 1e6,
+ outputPrice: 1.0 / 1e6,
+ cachedInputPrice: 0,
+ },
+ ],
cachedInputPrice: 0.05 / 1e6,
requestPrice: 0,
imageInputPrice: undefined,
@@
{
providerId: "azure-ai-foundry",
modelName: "grok-4-1-fast-non-reasoning",
inputPrice: 0.2 / 1e6,
outputPrice: 0.5 / 1e6,
+ pricingTiers: [
+ {
+ name: "Up to 128K",
+ upToTokens: 128000,
+ inputPrice: 0.2 / 1e6,
+ outputPrice: 0.5 / 1e6,
+ cachedInputPrice: 0.05 / 1e6,
+ },
+ {
+ name: "Over 128K",
+ upToTokens: Infinity,
+ inputPrice: 0.4 / 1e6,
+ outputPrice: 1.0 / 1e6,
+ cachedInputPrice: 0,
+ },
+ ],
cachedInputPrice: 0.05 / 1e6,
requestPrice: 0,
imageInputPrice: undefined,Also applies to: 450-465
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/models/src/models/xai.ts` around lines 392 - 408, The
azure-ai-foundry model entries (providerId "azure-ai-foundry", modelName
"grok-4-1-fast-reasoning" and the other azure-ai-foundry model around the
450-465 region) are missing pricingTiers; update those objects to include the
same 128K split pricing tiers used by the sibling xai entries so long-context
and cached-segment billing matches xAI mappings, ensuring you add a pricingTiers
property that mirrors the xai entries' tier definitions (the entry that uses
xaiSupportedParamsNoFreqPresence can be used as reference).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reject any value with chars outside the Azure resource-name alphabet (letters, digits, hyphens; 1-64 long) so a BYOK option like "evil.com/path" can't redirect requests away from services.ai.azure.com. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
packages/actions/src/get-provider-endpoint.ts (1)
440-449:⚠️ Potential issue | 🟠 Major | ⚡ Quick winRespect
skipEnvVarswhen resolving FoundryapiVersion.Line 443 still reads env unconditionally, so BYOK mode can pick up
LLM_AZURE_AI_FOUNDRY_API_VERSIONunexpectedly. This is inconsistent with the rest of this function’s BYOK behavior.Suggested patch
case "azure-ai-foundry": { const apiVersion = providerKeyOptions?.azure_ai_foundry_api_version ?? - getProviderEnvValue( - "azure-ai-foundry", - "apiVersion", - configIndex, - "2024-05-01-preview", - ) ?? + envValueOrDefault( + "azure-ai-foundry", + "apiVersion", + "2024-05-01-preview", + ) ?? "2024-05-01-preview"; return `${url}/models/chat/completions?api-version=${apiVersion}`; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/actions/src/get-provider-endpoint.ts` around lines 440 - 449, The azure-ai-foundry branch reads env vars unconditionally when computing apiVersion; change it to respect skipEnvVars like other providers by only calling getProviderEnvValue when skipEnvVars is false. Update the apiVersion assignment in the "azure-ai-foundry" case (the variable apiVersion) to: prefer providerKeyOptions?.azure_ai_foundry_api_version, else if !skipEnvVars call getProviderEnvValue("azure-ai-foundry","apiVersion",configIndex,"2024-05-01-preview"), otherwise skip the env lookup, and finally fall back to "2024-05-01-preview".
🧹 Nitpick comments (1)
packages/actions/src/get-provider-endpoint.spec.ts (1)
189-250: ⚡ Quick winAdd a BYOK regression test for Foundry
api-versionresolution.This block should also assert that
skipEnvVars = trueignoresLLM_AZURE_AI_FOUNDRY_API_VERSIONand uses the default2024-05-01-preview. That would catch the current inconsistency in production code.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/actions/src/get-provider-endpoint.spec.ts` around lines 189 - 250, Add a regression test in the "azure-ai-foundry" describe block that sets process.env.LLM_AZURE_AI_FOUNDRY_API_VERSION to a non-default value, sets process.env.LLM_AZURE_AI_FOUNDRY_RESOURCE to a valid resource, then calls getProviderEndpoint("azure-ai-foundry", { skipEnvVars: true }, "grok-4-1-fast-non-reasoning") and asserts the returned endpoint uses the default api-version "2024-05-01-preview" (i.e., ignores the env var). Ensure you also test the normal behavior (without skipEnvVars) if desired and clean up/delete the env vars after the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/actions/src/get-provider-endpoint.ts`:
- Around line 278-283: Update the validation for the Azure Foundry `resource` in
get-provider-endpoint.ts: replace the permissive /^[a-zA-Z0-9-]{1,64}$/ check
with a regex that enforces an alphanumeric start and end with optional internal
hyphens (e.g. require first and last char to be [A-Za-z0-9] and total length
1–64), so values like "-abc" or "abc-" are rejected; keep the existing error
message and use the same
azureFoundryEnv/getProviderEnvConfig("azure-ai-foundry") context for the thrown
Error.
---
Duplicate comments:
In `@packages/actions/src/get-provider-endpoint.ts`:
- Around line 440-449: The azure-ai-foundry branch reads env vars
unconditionally when computing apiVersion; change it to respect skipEnvVars like
other providers by only calling getProviderEnvValue when skipEnvVars is false.
Update the apiVersion assignment in the "azure-ai-foundry" case (the variable
apiVersion) to: prefer providerKeyOptions?.azure_ai_foundry_api_version, else if
!skipEnvVars call
getProviderEnvValue("azure-ai-foundry","apiVersion",configIndex,"2024-05-01-preview"),
otherwise skip the env lookup, and finally fall back to "2024-05-01-preview".
---
Nitpick comments:
In `@packages/actions/src/get-provider-endpoint.spec.ts`:
- Around line 189-250: Add a regression test in the "azure-ai-foundry" describe
block that sets process.env.LLM_AZURE_AI_FOUNDRY_API_VERSION to a non-default
value, sets process.env.LLM_AZURE_AI_FOUNDRY_RESOURCE to a valid resource, then
calls getProviderEndpoint("azure-ai-foundry", { skipEnvVars: true },
"grok-4-1-fast-non-reasoning") and asserts the returned endpoint uses the
default api-version "2024-05-01-preview" (i.e., ignores the env var). Ensure you
also test the normal behavior (without skipEnvVars) if desired and clean
up/delete the env vars after the test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: bcd49697-394f-4523-ae1c-b078ef342b5b
📒 Files selected for processing (2)
packages/actions/src/get-provider-endpoint.spec.tspackages/actions/src/get-provider-endpoint.ts
Comment on lines
+278
to
+283
| if (!/^[a-zA-Z0-9-]{1,64}$/.test(resource)) { | ||
| const azureFoundryEnv = getProviderEnvConfig("azure-ai-foundry"); | ||
| throw new Error( | ||
| `Azure AI Foundry resource is invalid - must be 1-64 chars of letters, digits, or hyphens (set via provider options or ${azureFoundryEnv?.required.resource ?? "LLM_AZURE_AI_FOUNDRY_RESOURCE"} env var)`, | ||
| ); | ||
| } |
Contributor
There was a problem hiding this comment.
Tighten the Foundry resource regex to enforce valid DNS-label boundaries.
Line 278 currently allows values like -abc or abc-, which are invalid label forms and will fail later at request time. Consider requiring alphanumeric start/end with optional inner hyphens.
Suggested patch
- if (!/^[a-zA-Z0-9-]{1,64}$/.test(resource)) {
+ if (
+ !/^[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,62}[a-zA-Z0-9])?$/.test(resource)
+ ) {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if (!/^[a-zA-Z0-9-]{1,64}$/.test(resource)) { | |
| const azureFoundryEnv = getProviderEnvConfig("azure-ai-foundry"); | |
| throw new Error( | |
| `Azure AI Foundry resource is invalid - must be 1-64 chars of letters, digits, or hyphens (set via provider options or ${azureFoundryEnv?.required.resource ?? "LLM_AZURE_AI_FOUNDRY_RESOURCE"} env var)`, | |
| ); | |
| } | |
| if ( | |
| !/^[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,62}[a-zA-Z0-9])?$/.test(resource) | |
| ) { | |
| const azureFoundryEnv = getProviderEnvConfig("azure-ai-foundry"); | |
| throw new Error( | |
| `Azure AI Foundry resource is invalid - must be 1-64 chars of letters, digits, or hyphens (set via provider options or ${azureFoundryEnv?.required.resource ?? "LLM_AZURE_AI_FOUNDRY_RESOURCE"} env var)`, | |
| ); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/actions/src/get-provider-endpoint.ts` around lines 278 - 283, Update
the validation for the Azure Foundry `resource` in get-provider-endpoint.ts:
replace the permissive /^[a-zA-Z0-9-]{1,64}$/ check with a regex that enforces
an alphanumeric start and end with optional internal hyphens (e.g. require first
and last char to be [A-Za-z0-9] and total length 1–64), so values like "-abc" or
"abc-" are rejected; keep the existing error message and use the same
azureFoundryEnv/getProviderEnvConfig("azure-ai-foundry") context for the thrown
Error.
Azure AI Foundry's chat completions response counts reasoning_tokens in usage but does not include the chain-of-thought text in message.reasoning_content, unlike xAI direct. Set reasoningOutput: "omit" so the gateway and tests don't expect a reasoning string back. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
packages/models/src/models/xai.ts (1)
392-409:⚠️ Potential issue | 🟠 Major | ⚡ Quick winKeep Azure AI Foundry pricing tiers aligned with the xAI mappings.
Both new
azure-ai-foundryentries still omitpricingTiers, unlike the siblingxaientries for the same models. That means long-context requests will be billed with the flat prices here instead of the 128K split used elsewhere, which can undercharge or overcharge usage.Also applies to: 451-466
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/models/src/models/xai.ts` around lines 392 - 409, The azure-ai-foundry model entries (e.g., providerId "azure-ai-foundry", modelName "grok-4-1-fast-reasoning") are missing pricingTiers and therefore use flat pricing; update these entries to include the same pricingTiers structure used by the corresponding xai sibling models (copy the pricingTiers object from the xai mapping for the same modelName) so long-context billing uses the 128K split tiers—apply the same fix to the other azure-ai-foundry entry referenced around the second block (lines ~451-466).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@packages/models/src/models/xai.ts`:
- Around line 392-409: The azure-ai-foundry model entries (e.g., providerId
"azure-ai-foundry", modelName "grok-4-1-fast-reasoning") are missing
pricingTiers and therefore use flat pricing; update these entries to include the
same pricingTiers structure used by the corresponding xai sibling models (copy
the pricingTiers object from the xai mapping for the same modelName) so
long-context billing uses the 128K split tiers—apply the same fix to the other
azure-ai-foundry entry referenced around the second block (lines ~451-466).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 94e75806-4739-4401-80d4-994dde356d11
📒 Files selected for processing (1)
packages/models/src/models/xai.ts
There was a problem hiding this comment.
Pull request overview
Adds a new azure-ai-foundry provider to route Grok models through Azure AI Foundry’s Models inference endpoint, alongside the existing direct xAI route. This extends provider key options / API schemas to support Foundry-specific configuration and updates e2e plumbing to upsert keys and pass Foundry options.
Changes:
- Added
azure-ai-foundryprovider definition + endpoint/header handling and streaming transform support. - Mapped Grok fast reasoning/non-reasoning variants to the new provider.
- Extended provider key options + API typings and updated tests/e2e setup and CI env wiring.
Reviewed changes
Copilot reviewed 12 out of 16 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/models/src/providers.ts | Registers the new azure-ai-foundry provider and its env var configuration. |
| packages/models/src/models/xai.ts | Adds Azure AI Foundry provider mappings for Grok fast reasoning/non-reasoning. |
| packages/db/src/schema.ts | Extends ProviderKeyOptions with Foundry-specific resource and api_version. |
| packages/actions/src/get-provider-headers.ts | Uses api-key auth header for azure-ai-foundry. |
| packages/actions/src/get-provider-endpoint.ts | Builds Foundry base URL and /models/chat/completions endpoint with api-version. |
| packages/actions/src/get-provider-endpoint.spec.ts | Adds unit tests for Foundry endpoint construction and validation. |
| apps/gateway/src/chat/tools/transform-streaming-to-openai.ts | Treats Foundry streaming as OpenAI-compatible streaming. |
| apps/gateway/src/chat-helpers.e2e.ts | Upserts provider keys and threads Foundry options into BYOK-mode e2e runs. |
| apps/api/src/routes/keys-provider.ts | Accepts Foundry options in provider-key create/update payload validation. |
| apps/*/src/lib/api/v1.d.ts, ee/admin/src/lib/api/v1.d.ts | Updates generated API typings to include Foundry options. |
| .github/workflows/e2e.yml | Wires Foundry secrets into CI e2e environment variables. |
| .env.example, .env.unified.example | Adds example env vars for Foundry API key + resource. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+443
to
+448
| getProviderEnvValue( | ||
| "azure-ai-foundry", | ||
| "apiVersion", | ||
| configIndex, | ||
| "2024-05-01-preview", | ||
| ) ?? |
| "https://gkapitech.services.ai.azure.com/models/chat/completions?api-version=2025-01-01-preview", | ||
| ); | ||
| }); | ||
|
|
|
|
||
| # Azure AI Foundry (third-party models like Grok, Llama, Mistral) | ||
| LLM_AZURE_AI_FOUNDRY_API_KEY=your_azure_ai_foundry_key_here | ||
| LLM_AZURE_AI_FOUNDRY_RESOURCE=your_azure_ai_foundry_resource_here |
|
|
||
| # Azure AI Foundry (third-party models like Grok, Llama, Mistral) | ||
| LLM_AZURE_AI_FOUNDRY_API_KEY=your_azure_ai_foundry_key_here | ||
| LLM_AZURE_AI_FOUNDRY_RESOURCE=your_azure_ai_foundry_resource_here |
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts: # apps/api/src/routes/keys-provider.ts # apps/code/src/lib/api/v1.d.ts # apps/gateway/src/chat-helpers.e2e.ts # apps/playground/src/lib/api/v1.d.ts # apps/ui/src/lib/api/v1.d.ts # ee/admin/src/lib/api/v1.d.ts # packages/db/src/schema.ts
This was referenced May 4, 2026
This was referenced May 13, 2026
This was referenced Jun 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
azure-ai-foundryprovider for the Azure AI Foundry Models inference endpoint ({resource}.services.ai.azure.com/models/chat/completions), distinct from the existing Azure OpenAI provider since it uses a different hostname, URL path, and Azure resource/key. Mapsgrok-4-1-fast-reasoningandgrok-4-1-fast-non-reasoningto it (alongside the existing direct xAI route) so users with an Azure AI Foundry subscription can route Grok via Azure. Configurable throughLLM_AZURE_AI_FOUNDRY_API_KEY/LLM_AZURE_AI_FOUNDRY_RESOURCE/LLM_AZURE_AI_FOUNDRY_API_VERSIONenv vars or BYOK options. The e2e harness now upserts provider keys and threads the Azure AI Foundry resource from env into options for BYOK-mode test runs.Test plan
get-provider-endpoint.spec.tschat-basic,chat-streaming,chat-toolcallsagainst bothazure-ai-foundry/grok-4-1-fast-{reasoning,non-reasoning}andxai/grok-4-1-fast-{reasoning,non-reasoning}(22 / 22 passed)pnpm build,pnpm format,pnpm lintall clean🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Configuration
Endpoints & Headers
Tests & CI