Skip to content

fix(providers): add support for ZAI payloads #812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
steebchen merged 2 commits into from
Sep 14, 2025
Merged

fix(providers): add support for ZAI payloads #812

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9699698
fix(providers): add support for ZAI payloads
steebchen Sep 14, 2025
ebcc8bc
Merge branch 'main' into fix/zai
steebchen Sep 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion packages/models/src/provider-api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -524,6 +524,40 @@ export async function prepareRequestBody(
}
break;
}
case "zai": {
if (stream) {
requestBody.stream_options = {
include_usage: true,
};
}
if (response_format) {
requestBody.response_format = response_format;
}

// Add optional parameters if they are provided
if (temperature !== undefined) {
requestBody.temperature = temperature;
}
if (max_tokens !== undefined) {
requestBody.max_tokens = max_tokens;
}
if (top_p !== undefined) {
requestBody.top_p = top_p;
}
if (frequency_penalty !== undefined) {
requestBody.frequency_penalty = frequency_penalty;
}
if (presence_penalty !== undefined) {
requestBody.presence_penalty = presence_penalty;
}
// ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort'
if (supportsReasoning) {
requestBody.thinking = {
type: "enabled",
};
}
break;
}
Comment on lines +527 to +560

@coderabbitai coderabbitai Bot Sep 14, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Guard ZAI params by supportedParameters to avoid unsupported-field 400s.

ZAI may not accept all OpenAI-shaped fields (e.g., response_format, stream_options, penalties). Use the model mapping’s supportedParameters to set only what’s allowed.

Apply this diff within the ZAI case:

 case "zai": {
-  if (stream) {
-    requestBody.stream_options = {
-      include_usage: true,
-    };
-  }
-  if (response_format) {
-    requestBody.response_format = response_format;
-  }
+  const zaiProviderMapping = modelDef?.providers.find(
+    (p) => p.providerId === usedProvider,
+  );
+  const supportsParam = (param: string) =>
+    ((zaiProviderMapping as ProviderModelMapping | undefined)?.supportedParameters?.includes(param) ?? true);
+
+  if (stream && supportsParam("stream_options")) {
+    requestBody.stream_options = { include_usage: true };
+  }
+  if (response_format && supportsParam("response_format")) {
+    requestBody.response_format = response_format;
+  }
 
   // Add optional parameters if they are provided
-  if (temperature !== undefined) {
+  if (temperature !== undefined && supportsParam("temperature")) {
     requestBody.temperature = temperature;
   }
-  if (max_tokens !== undefined) {
+  if (max_tokens !== undefined && supportsParam("max_tokens")) {
     requestBody.max_tokens = max_tokens;
   }
-  if (top_p !== undefined) {
+  if (top_p !== undefined && supportsParam("top_p")) {
     requestBody.top_p = top_p;
   }
-  if (frequency_penalty !== undefined) {
+  if (frequency_penalty !== undefined && supportsParam("frequency_penalty")) {
     requestBody.frequency_penalty = frequency_penalty;
   }
-  if (presence_penalty !== undefined) {
+  if (presence_penalty !== undefined && supportsParam("presence_penalty")) {
     requestBody.presence_penalty = presence_penalty;
   }
   // ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort'
-  if (supportsReasoning) {
+  if (supportsReasoning && supportsParam("thinking")) {
     requestBody.thinking = {
       type: "enabled",
     };
   }
   break;
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
case "zai": {
if (stream) {
requestBody.stream_options = {
include_usage: true,
};
}
if (response_format) {
requestBody.response_format = response_format;
}
// Add optional parameters if they are provided
if (temperature !== undefined) {
requestBody.temperature = temperature;
}
if (max_tokens !== undefined) {
requestBody.max_tokens = max_tokens;
}
if (top_p !== undefined) {
requestBody.top_p = top_p;
}
if (frequency_penalty !== undefined) {
requestBody.frequency_penalty = frequency_penalty;
}
if (presence_penalty !== undefined) {
requestBody.presence_penalty = presence_penalty;
}
// ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort'
if (supportsReasoning) {
requestBody.thinking = {
type: "enabled",
};
}
break;
}
case "zai": {
const zaiProviderMapping = modelDef?.providers.find(
(p) => p.providerId === usedProvider,
);
const supportsParam = (param: string) =>
((zaiProviderMapping as ProviderModelMapping | undefined)?.supportedParameters?.includes(param) ?? true);
if (stream && supportsParam("stream_options")) {
requestBody.stream_options = { include_usage: true };
}
if (response_format && supportsParam("response_format")) {
requestBody.response_format = response_format;
}
// Add optional parameters if they are provided
if (temperature !== undefined && supportsParam("temperature")) {
requestBody.temperature = temperature;
}
if (max_tokens !== undefined && supportsParam("max_tokens")) {
requestBody.max_tokens = max_tokens;
}
if (top_p !== undefined && supportsParam("top_p")) {
requestBody.top_p = top_p;
}
if (frequency_penalty !== undefined && supportsParam("frequency_penalty")) {
requestBody.frequency_penalty = frequency_penalty;
}
if (presence_penalty !== undefined && supportsParam("presence_penalty")) {
requestBody.presence_penalty = presence_penalty;
}
// ZAI/GLM models use 'thinking' parameter for reasoning instead of 'reasoning_effort'
if (supportsReasoning && supportsParam("thinking")) {
requestBody.thinking = {
type: "enabled",
};
}
break;
}
🤖 Prompt for AI Agents 
In packages/models/src/provider-api.ts around lines 527 to 560, the ZAI branch
unconditionally sets fields that some ZAI models may not support, causing 400
errors; change each optional assignment to first check
modelMapping.supportedParameters (or supportedParameters) includes the
corresponding parameter name before adding it to requestBody. Specifically: only
set stream_options when stream is true AND supportedParameters includes the
stream-related key (e.g., "stream" or "stream_options"); only set
response_format if supportedParameters includes "response_format"; only set
temperature, max_tokens, top_p, frequency_penalty, and presence_penalty when
each respective name is present in supportedParameters; and only add thinking
when supportsReasoning is true AND supportedParameters includes "thinking" (or
the model's reasoning parameter name). Ensure you use the existing model mapping
variable (supportedParameters or modelMapping.supportedParameters) and preserve
current values/structure while gating each assignment.

💡 Verification agent

🧩 Analysis chain

Confirm ZAI ‘thinking’ payload shape and feature support.

Docs for ZAI/GLM may require a different shape (e.g., budget tokens/mode) or not support response_format/stream_options. Please verify to avoid runtime errors.

You can also confirm the local model metadata declares allowed params for ZAI:

🏁 Script executed:

#!/bin/bash
# Inspect ZAI provider entries and supportedParameters in models config
rg -nP -C3 '\bproviderId\s*:\s*["'\'']zai["'\'']' --type ts

Length of output: 3433

🌐 Web query:

What is the Z AI (z.ai) chat completions API parameter for enabling reasoning? Does it use a `thinking` field (and with what shape)?

💡 Result:

Yes — Z.AI uses a "thinking" object. For GLM-4.5 (and the chat completions endpoint) you enable/disable reasoning with the thinking.type field (allowed values: "enabled" or "disabled"; default is enabled). Example shape: "thinking": { "type": "enabled" }.[1][2]

🌐 Web query:

Does the Z AI PaaS v4 chat completions endpoint support `response_format` and `stream_options.include_usage` like OpenAI?

💡 Result:

Short answer: No — Z.AI's paas/v4 chat/completions supports streaming via "stream": true, but its public API docs do not document a response_format parameter or a stream_options.include_usage flag the way OpenAI/Azure do. Z.AI shows streaming and a usage field in non-streamed responses but does not list stream_options/include_usage. [1][2]

If you need include_usage-like behavior while streaming, try contacting Z.AI support or testing experimentally (their docs/sdk don’t advertise that option). [1][2][3]

Sources:

  • Z.AI chat/completions docs (paas/v4) — streaming and response schema. [1]
  • Z.AI HTTP/streaming examples (guides). [2]
  • Example discussion showing stream_options.include_usage used in OpenAI/Azure contexts (for comparison). [3]

ZAI: preserve thinking shape; remove response_format and stream_options.include_usage (unsupported)

  • thinking is correct — Z.AI expects: { type: "enabled" | "disabled" }.
  • Z.AI paas/v4 chat/completions does not document response_format or stream_options.include_usage; sending them may cause request failures. Use "stream": true for streaming and omit include_usage/response_format for provider "zai".
  • Action: packages/models/src/provider-api.ts (≈ lines 527–560) — stop setting requestBody.stream_options = { include_usage: true } and requestBody.response_format for "zai", or guard/remove those fields.
🤖 Prompt for AI Agents 
In packages/models/src/provider-api.ts around lines 527 to 560, the Z.AI branch
must preserve the thinking object shape but stop sending unsupported fields:
remove setting of requestBody.stream_options = { include_usage: true } and stop
assigning requestBody.response_format; instead rely on requestBody.stream = true
for streaming behavior and only set requestBody.thinking = { type: "enabled" }
when supportsReasoning is true, leaving other optional params unchanged.

case "xai":
case "groq":
case "deepseek":
Expand All @@ -532,7 +566,6 @@ export async function prepareRequestBody(
case "moonshot":
case "alibaba":
case "nebius":
case "zai":
case "routeway":
case "custom": {
if (stream) {
Expand Down
Loading