Conversation
Adds configuration and parameters specific to the ZAI reasoning models, including support for optional parameters (e.g., `temperature`, `max_tokens`) and a new `thinking` field to enable reasoning. Removes "zai" from the fallback provider case.
WalkthroughSeparated ZAI-specific request body handling into its own switch case in provider-api, enabling ZAI/GLM reasoning via thinking, setting stream_options for streaming, passing response_format when provided, and applying optional sampling/penalty parameters. Removed ZAI from the mixed-provider case to avoid duplicate handling. No exported API changes. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant App
participant ProviderAPI as Provider API
participant ZAI as ZAI/GLM Endpoint
App->>ProviderAPI: generate(request, provider="zai", opts)
activate ProviderAPI
note right of ProviderAPI: prepareRequestBody(provider="zai")
ProviderAPI->>ProviderAPI: Set stream_options.include_usage if streaming
ProviderAPI->>ProviderAPI: Attach response_format if provided
ProviderAPI->>ProviderAPI: Apply temperature/max_tokens/top_p/penalties if provided
ProviderAPI->>ProviderAPI: If supportsReasoning, set thinking={ type: "enabled" }
ProviderAPI->>ZAI: POST /chat/completions with body
ZAI-->>ProviderAPI: response (stream or json)
deactivate ProviderAPI
ProviderAPI-->>App: normalized result
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/models/src/provider-api.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use top-level import; never use require() or dynamic import()
Always use top-level import; never use require or dynamic imports
Files:
packages/models/src/provider-api.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: e2e / run
- GitHub Check: build / run
🔇 Additional comments (1)
packages/models/src/provider-api.ts (1)
527-560: Good split: dedicated ZAI case avoids duplicate handling.Extracting ZAI into its own switch case prevents double-processing and keeps reasoning behavior provider-specific.
| case "zai": { | ||
| if (stream) { | ||
| requestBody.stream_options = { | ||
| include_usage: true, | ||
| }; | ||
| } | ||
| if (response_format) { | ||
| requestBody.response_format = response_format; | ||
| } | ||
|
|
||
| // Add optional parameters if they are provided | ||
| if (temperature !== undefined) { | ||
| requestBody.temperature = temperature; | ||
| } | ||
| if (max_tokens !== undefined) { | ||
| requestBody.max_tokens = max_tokens; | ||
| } | ||
| if (top_p !== undefined) { | ||
| requestBody.top_p = top_p; | ||
| } | ||
| if (frequency_penalty !== undefined) { | ||
| requestBody.frequency_penalty = frequency_penalty; | ||
| } | ||
| if (presence_penalty !== undefined) { | ||
| requestBody.presence_penalty = presence_penalty; | ||
| } | ||
| // ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort' | ||
| if (supportsReasoning) { | ||
| requestBody.thinking = { | ||
| type: "enabled", | ||
| }; | ||
| } | ||
| break; | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Guard ZAI params by supportedParameters to avoid unsupported-field 400s.
ZAI may not accept all OpenAI-shaped fields (e.g., response_format, stream_options, penalties). Use the model mapping’s supportedParameters to set only what’s allowed.
Apply this diff within the ZAI case:
case "zai": {
- if (stream) {
- requestBody.stream_options = {
- include_usage: true,
- };
- }
- if (response_format) {
- requestBody.response_format = response_format;
- }
+ const zaiProviderMapping = modelDef?.providers.find(
+ (p) => p.providerId === usedProvider,
+ );
+ const supportsParam = (param: string) =>
+ ((zaiProviderMapping as ProviderModelMapping | undefined)?.supportedParameters?.includes(param) ?? true);
+
+ if (stream && supportsParam("stream_options")) {
+ requestBody.stream_options = { include_usage: true };
+ }
+ if (response_format && supportsParam("response_format")) {
+ requestBody.response_format = response_format;
+ }
// Add optional parameters if they are provided
- if (temperature !== undefined) {
+ if (temperature !== undefined && supportsParam("temperature")) {
requestBody.temperature = temperature;
}
- if (max_tokens !== undefined) {
+ if (max_tokens !== undefined && supportsParam("max_tokens")) {
requestBody.max_tokens = max_tokens;
}
- if (top_p !== undefined) {
+ if (top_p !== undefined && supportsParam("top_p")) {
requestBody.top_p = top_p;
}
- if (frequency_penalty !== undefined) {
+ if (frequency_penalty !== undefined && supportsParam("frequency_penalty")) {
requestBody.frequency_penalty = frequency_penalty;
}
- if (presence_penalty !== undefined) {
+ if (presence_penalty !== undefined && supportsParam("presence_penalty")) {
requestBody.presence_penalty = presence_penalty;
}
// ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort'
- if (supportsReasoning) {
+ if (supportsReasoning && supportsParam("thinking")) {
requestBody.thinking = {
type: "enabled",
};
}
break;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| case "zai": { | |
| if (stream) { | |
| requestBody.stream_options = { | |
| include_usage: true, | |
| }; | |
| } | |
| if (response_format) { | |
| requestBody.response_format = response_format; | |
| } | |
| // Add optional parameters if they are provided | |
| if (temperature !== undefined) { | |
| requestBody.temperature = temperature; | |
| } | |
| if (max_tokens !== undefined) { | |
| requestBody.max_tokens = max_tokens; | |
| } | |
| if (top_p !== undefined) { | |
| requestBody.top_p = top_p; | |
| } | |
| if (frequency_penalty !== undefined) { | |
| requestBody.frequency_penalty = frequency_penalty; | |
| } | |
| if (presence_penalty !== undefined) { | |
| requestBody.presence_penalty = presence_penalty; | |
| } | |
| // ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort' | |
| if (supportsReasoning) { | |
| requestBody.thinking = { | |
| type: "enabled", | |
| }; | |
| } | |
| break; | |
| } | |
| case "zai": { | |
| const zaiProviderMapping = modelDef?.providers.find( | |
| (p) => p.providerId === usedProvider, | |
| ); | |
| const supportsParam = (param: string) => | |
| ((zaiProviderMapping as ProviderModelMapping | undefined)?.supportedParameters?.includes(param) ?? true); | |
| if (stream && supportsParam("stream_options")) { | |
| requestBody.stream_options = { include_usage: true }; | |
| } | |
| if (response_format && supportsParam("response_format")) { | |
| requestBody.response_format = response_format; | |
| } | |
| // Add optional parameters if they are provided | |
| if (temperature !== undefined && supportsParam("temperature")) { | |
| requestBody.temperature = temperature; | |
| } | |
| if (max_tokens !== undefined && supportsParam("max_tokens")) { | |
| requestBody.max_tokens = max_tokens; | |
| } | |
| if (top_p !== undefined && supportsParam("top_p")) { | |
| requestBody.top_p = top_p; | |
| } | |
| if (frequency_penalty !== undefined && supportsParam("frequency_penalty")) { | |
| requestBody.frequency_penalty = frequency_penalty; | |
| } | |
| if (presence_penalty !== undefined && supportsParam("presence_penalty")) { | |
| requestBody.presence_penalty = presence_penalty; | |
| } | |
| // ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort' | |
| if (supportsReasoning && supportsParam("thinking")) { | |
| requestBody.thinking = { | |
| type: "enabled", | |
| }; | |
| } | |
| break; | |
| } |
🤖 Prompt for AI Agents
In packages/models/src/provider-api.ts around lines 527 to 560, the ZAI branch
unconditionally sets fields that some ZAI models may not support, causing 400
errors; change each optional assignment to first check
modelMapping.supportedParameters (or supportedParameters) includes the
corresponding parameter name before adding it to requestBody. Specifically: only
set stream_options when stream is true AND supportedParameters includes the
stream-related key (e.g., "stream" or "stream_options"); only set
response_format if supportedParameters includes "response_format"; only set
temperature, max_tokens, top_p, frequency_penalty, and presence_penalty when
each respective name is present in supportedParameters; and only add thinking
when supportsReasoning is true AND supportedParameters includes "thinking" (or
the model's reasoning parameter name). Ensure you use the existing model mapping
variable (supportedParameters or modelMapping.supportedParameters) and preserve
current values/structure while gating each assignment.
💡 Verification agent
🧩 Analysis chain
Confirm ZAI ‘thinking’ payload shape and feature support.
Docs for ZAI/GLM may require a different shape (e.g., budget tokens/mode) or not support response_format/stream_options. Please verify to avoid runtime errors.
You can also confirm the local model metadata declares allowed params for ZAI:
🏁 Script executed:
#!/bin/bash
# Inspect ZAI provider entries and supportedParameters in models config
rg -nP -C3 '\bproviderId\s*:\s*["'\'']zai["'\'']' --type tsLength of output: 3433
🌐 Web query:
What is the Z AI (z.ai) chat completions API parameter for enabling reasoning? Does it use a `thinking` field (and with what shape)?
💡 Result:
Yes — Z.AI uses a "thinking" object. For GLM-4.5 (and the chat completions endpoint) you enable/disable reasoning with the thinking.type field (allowed values: "enabled" or "disabled"; default is enabled). Example shape: "thinking": { "type": "enabled" }.[1][2]
🌐 Web query:
Does the Z AI PaaS v4 chat completions endpoint support `response_format` and `stream_options.include_usage` like OpenAI?
💡 Result:
Short answer: No — Z.AI's paas/v4 chat/completions supports streaming via "stream": true, but its public API docs do not document a response_format parameter or a stream_options.include_usage flag the way OpenAI/Azure do. Z.AI shows streaming and a usage field in non-streamed responses but does not list stream_options/include_usage. [1][2]
If you need include_usage-like behavior while streaming, try contacting Z.AI support or testing experimentally (their docs/sdk don’t advertise that option). [1][2][3]
Sources:
- Z.AI chat/completions docs (paas/v4) — streaming and response schema. [1]
- Z.AI HTTP/streaming examples (guides). [2]
- Example discussion showing stream_options.include_usage used in OpenAI/Azure contexts (for comparison). [3]
ZAI: preserve thinking shape; remove response_format and stream_options.include_usage (unsupported)
- thinking is correct — Z.AI expects: { type: "enabled" | "disabled" }.
- Z.AI paas/v4 chat/completions does not document response_format or stream_options.include_usage; sending them may cause request failures. Use "stream": true for streaming and omit include_usage/response_format for provider "zai".
- Action: packages/models/src/provider-api.ts (≈ lines 527–560) — stop setting requestBody.stream_options = { include_usage: true } and requestBody.response_format for "zai", or guard/remove those fields.
🤖 Prompt for AI Agents
In packages/models/src/provider-api.ts around lines 527 to 560, the Z.AI branch
must preserve the thinking object shape but stop sending unsupported fields:
remove setting of requestBody.stream_options = { include_usage: true } and stop
assigning requestBody.response_format; instead rely on requestBody.stream = true
for streaming behavior and only set requestBody.thinking = { type: "enabled" }
when supportsReasoning is true, leaving other optional params unchanged.
1 participant
Adds configuration and parameters specific to the ZAI reasoning models, including support for optional parameters (e.g.,
temperature,max_tokens) and a newthinkingfield to enable reasoning. Removes "zai" from the fallback provider case.Summary by CodeRabbit