Skip to content

fix(models): correct xAI reasoning_effort support - #3483

Merged
steebchen merged 5 commits into
mainfrom
claude/e2e-verify-pr-3438-lqfntw
Aug 8, 2026
Merged

steebchen merged 5 commits into
mainfrom
claude/e2e-verify-pr-3438-lqfntw

Conversation

@steebchen

@steebchen steebchen commented Aug 8, 2026 •

Copy link
Copy Markdown
Member

Supersedes #3438 (fixes #3436, #3403).

Problem

Six xAI mappings declare reasoning: true but omit reasoning_effort from supportedParameters, so a caller-supplied effort is dropped at the gate in packages/actions/src/prepare-request-body.ts:4077-4086. #3438 treated all six as the same oversight and declared the parameter on every one.

E2E against api.x.ai shows that premise does not hold. Three of those deployments reason unconditionally but reject the effort knob outright:

externalId reasoning_effort Result
grok-4 low 200 (aliases to grok-4.3 upstream)
grok-4.3 low 200
grok-4.20-beta-0309-reasoning low / medium / high 400 does not support parameter reasoningEffort
grok-build-0.1 low / medium / high 400 same
grok-4.20-reasoning low 400 same

Control requests with the parameter omitted return 200 for all three, so the parameter is the sole cause; streaming behaves identically. For those mappings the omission was the catalogue correctly reflecting upstream, not the bug the issues describe — declaring it turns every effort-bearing request into an upstream 400.

Change

packages/models/src/models/xai.ts — keep #3438's addition for xai/grok-4, azure-ai-foundry/grok-4-3 and azure-ai-foundry/grok-4-1-fast-reasoning; restore the shared xaiSupportedParamsNoFreqPresence list for the three rejecting mappings, each with a comment recording why the omission is load-bearing so it does not get "fixed" again:

  • xai / grok-4-20-beta-0309-reasoning
  • xai / grok-build-0-1
  • vertex-openai / grok-4-20-reasoning — same upstream model as the first

packages/actions/src/prepare-request-body.spec.ts — split #3438's six-row table into a positive table (three mappings forward the effort) and a negative one asserting the rejecting mappings are reasoning: true, do not list reasoning_effort, and drop the value from the request body. Also narrows the mapping lookup to ProviderModelMapping; reading .supportedParameters off the raw catalogue union fails tsc with TS2339, which broke pnpm build on #3438.

Verification

Ran with a live xAI key.

  • TEST_MODELS="xai/grok-4,xai/grok-4-20-beta-0309-reasoning,xai/grok-build-0-1" pnpm test:e2e — 29 passed, 2 skipped, 0 failed. On fix(models): reasoning_effort for azure grok-4.3 #3438 the same command produced 4 failures (chat-reasoning.e2e.ts > basic reasoning and chat-rs.e2e.ts > reasoning + streaming, both models, 400 vs expected 200). Those four pass on main, confirming the regression rather than a pre-existing failure.
  • prepare-request-body.spec.ts + models specs: 380 passed.
  • pnpm build, pnpm format, pnpm lint: clean.

Not verified

azure-ai-foundry/grok-4-1-fast-reasoning keeps the parameter but was not exercised — no Azure credentials here, and the id does not resolve on native xAI (grok-4.1-fast-reasoning → "Model not found"; our xai mapping for it is deactivated). Since every xAI reasoning model probed except grok-4.3 rejects reasoning_effort, this one is worth a /e2e run with the Azure key before merging. azure-ai-foundry/grok-4-3 is safer — native grok-4.3 accepts the parameter.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RbUnEaa7vrZZkD82wSStTr


Generated by Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of the reasoning_effort setting across supported xAI and Azure AI Foundry models.
    • Ensured the setting is forwarded only when supported and omitted for deployments that reject it.
    • Updated Azure AI Foundry Grok 4.3 to use a default reasoning effort of medium.

andyst-dev and others added 5 commits August 8, 2026 11:45
…ings

grok-4-3 and grok-4-1-fast-reasoning (azure-ai-foundry), grok-4-20-reasoning
(vertex-openai), grok-4, grok-4-20-beta-0309-reasoning and grok-build-0-1
(xai) all declare reasoning: true but omitted reasoning_effort from their
supportedParameters, so any effort value was silently dropped before the
request was built. Inline the shared xaiSupportedParamsNoFreqPresence list
(following #3159) and add reasoning_effort to each active reasoning mapping;
deactivated and non-reasoning mappings keep the shared constant untouched.

Fixes #3436, Fixes #3403
…or xAI mappings

Direct capability assertion on each fixed mapping's supportedParameters
array, not just the serialized request body — catches a future refactor
that stops reading supportedParameters while serialization tests keep
passing.
The reasoning_effort regression test read .supportedParameters off the
raw catalogue union, which fails tsc because not every mapping member
declares it. Narrow the lookup to ProviderModelMapping.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbUnEaa7vrZZkD82wSStTr
e2e against api.x.ai shows three of the six mappings widened in #3438
answer reasoning_effort with 400 'does not support parameter
reasoningEffort': xai/grok-4-20-beta-0309-reasoning, xai/grok-build-0-1,
and vertex-openai/grok-4-20-reasoning (same upstream model as the
former). Declaring the parameter made the gateway forward it, turning
every effort-bearing request into an upstream 400 — 4 gateway e2e tests
that pass on main fail with it.

Restore the shared parameter list for those three, with a comment on
each recording why the omission is deliberate, and split the spec's
table so the rejecting mappings assert the drop rather than the forward.

xai/grok-4 keeps the parameter (verified 200; it aliases to grok-4.3
upstream). The two azure-ai-foundry mappings keep it as well and remain
unverified here — no Azure credentials.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbUnEaa7vrZZkD82wSStTr
Copilot AI lite review requested due to automatic review settings August 8, 2026 13:39
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2271edaf-8f3a-4964-a10a-eb813202de69

📥 Commits

Reviewing files that changed from the base of the PR and between ed39fc9 and fc91bc9.

📒 Files selected for processing (2)
  • packages/actions/src/prepare-request-body.spec.ts
  • packages/models/src/models/xai.ts

Walkthrough

The change updates selected xAI and Azure AI Foundry model mappings to declare reasoning_effort support. Regression tests verify forwarding for supported deployments and omission for deployments whose APIs reject the parameter.

Changes

xAI reasoning effort support

Layer / File(s) Summary
Model capability declarations
packages/models/src/models/xai.ts
Grok 4, Azure Grok 4.1 Fast Reasoning, and Azure Grok 4.3 now declare reasoning_effort. Grok 4.20 reasoning variants and Grok Build 0.1 document that the parameter remains unsupported.
Request-body regression coverage
packages/actions/src/prepare-request-body.spec.ts
Parameterized tests verify forwarding and omission behavior. The Azure AI Foundry Grok 4.3 expectation now requires "medium" reasoning effort.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

  • theopenco/llmgateway#3403 — Covers the same reasoning_effort mapping gap for the Vertex OpenAI Grok 4.20 reasoning deployment.

Possibly related PRs

Suggested reviewers: smakosh

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely identifies the correction to xAI reasoning_effort support.
Linked Issues check ✅ Passed The changes add reasoning_effort support to both Azure AI Foundry mappings and add regression tests for forwarding and omission behavior [#3436].
Out of Scope Changes check ✅ Passed The additional model-specific mappings and exclusions support the stated reasoning_effort correction and related model behavior.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/e2e-verify-pr-3438-lqfntw

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@steebchen
steebchen merged commit 89e1010 into main Aug 8, 2026
12 checks passed
@steebchen
steebchen deleted the claude/e2e-verify-pr-3438-lqfntw branch August 8, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: reasoning_effort dropped entirely for grok-4.3 and grok-4-1-fast-reasoning on azure-ai-foundry

4 participants