Skip to content

feat(cerebras): add Cerebras provider and models support - #1322

Merged
steebchen merged 5 commits into
mainfrom
feat/cerebras
Dec 14, 2025
Merged

steebchen merged 5 commits into
mainfrom
feat/cerebras

Conversation

@steebchen

@steebchen steebchen commented Dec 14, 2025

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • New Features

    • Added Cerebras as a selectable AI provider with icons/branding in the UI.
    • Expanded model availability to include Cerebras variants across LLaMA 3, Qwen 3, GPT-OSS and GLM-4.6 families.
    • Enabled streaming, JSON-output, and advanced runtime options (temperature, max tokens, reasoning, strict JSON schema) for Cerebras calls.
  • Chores

    • CI updated to expose Cerebras API key for end-to-end tests.

✏️ Tip: You can customize this high-level summary in your review settings.

Copilot AI review requested due to automatic review settings December 14, 2025 13:36
@coderabbitai

coderabbitai Bot commented Dec 14, 2025

Copy link
Copy Markdown
Contributor

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Adds a new Cerebras provider integration: CI env var, provider registry entry, endpoint routing, request preparation branch, multiple model provider entries, UI provider icons, and a small testing option change.

Changes

Cohort / File(s) Change Summary
CI/CD Workflow
\.github/workflows/e2e.yml
Adds LLM_CEREBRAS_API_KEY to the e2e-shards job environment.
UI Provider Icons
Admin/Playground/UI
apps/admin/src/components/ui/providers-icons.tsx, apps/playground/src/components/provider-icons.tsx, apps/playground/src/components/ui/providers-icons.tsx, apps/ui/src/lib/components/providers-icons.tsx
Adds exported CerebrasIcon SVG component and registers it in provider icon maps (ProviderIcons / logo maps).
Provider Registry
packages/models/src/providers.ts
Adds new exported provider entry cerebras with env var LLM_CEREBRAS_API_KEY, streaming/cancellation flags, color, website, and description.
Endpoint Routing
packages/models/src/get-provider-endpoint.ts
Adds cerebras base URL mapping (https://api.cerebras.ai) and routes chat completions to ${url}/v1/chat/completions.
Request Preparation
packages/models/src/prepare-request-body.ts
Adds cerebras branch: enables stream_options.include_usage, enforces response_format.strict = true for json_schema types, and forwards optional params (temperature, max_tokens, top_p, frequency_penalty, presence_penalty, reasoning_effort).
Models — Alibaba
packages/models/src/models/alibaba.ts
Appends cerebras provider entries for qwen-3-235b-a22b-instruct-2507 and qwen-3-32b with pricing, context sizes, and capability flags.
Models — Meta
packages/models/src/models/meta.ts
Adds cerebras FP16 provider entries for llama-3.1-8b-instruct and llama-3.3-70b-instruct.
Models — OpenAI family
packages/models/src/models/openai.ts
Adds cerebras provider entry for gpt-oss-120b with pricing, contextSize, maxOutput, and capability flags.
Models — Zai
packages/models/src/models/zai.ts
Adds cerebras provider entry for glm-4.6 (zai-glm-4.6) with pricing, contextSize, and capability flags.
Testing helpers
packages/models/src/testing.ts
getTestOptions now returns combined { retry, timeout } in CI/completions branches; getConcurrentTestOptions spreads updated result.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant UI as Client (UI)
  participant GW as Gateway / Server
  participant Models as models module
  participant Cere as Cerebras API

  UI->>GW: request (provider: cerebras, model, params)
  GW->>Models: prepareRequestBody(provider:cerebras, params)
  Models-->>GW: prepared body (stream_options.include_usage, response_format(strict)?, forwarded params)
  GW->>Models: getProviderEndpoint(provider:cerebras)
  Models-->>GW: endpoint URL (https://api.cerebras.ai/v1/chat/completions)
  GW->>Cere: POST (body, Authorization: Bearer LLM_CEREBRAS_API_KEY)
  Cere-->>GW: response / stream
  GW-->>UI: deliver response/stream
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Areas to pay attention to:

  • prepare-request-body: merging/enforcement of response_format.strict for json_schema and propagation of optional params.
  • get-provider-endpoint: URL/path correctness for chat completions.
  • providers.ts and CI: consistency of LLM_CEREBRAS_API_KEY naming and required env behavior.
  • Model entries: verify pricing, contextSize, capability flags, and providerId values.
  • UI: correct exports/registrations of CerebrasIcon across packages.

Possibly related PRs

Suggested reviewers

  • smakosh
  • kmk142789

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding Cerebras provider support and associated model configurations across the codebase.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/cerebras

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bff06be and 93030ed.

📒 Files selected for processing (2)
  • packages/models/src/models/meta.ts (2 hunks)
  • packages/models/src/prepare-request-body.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/models/src/models/meta.ts
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx}: Never use any or as any unless absolutely necessary in TypeScript code
For database reads: Use db().query.<table>.findMany() or db().query.<table>.findFirst()

Files:

  • packages/models/src/prepare-request-body.ts
**/*.{ts,tsx,js,jsx,json,md}

📄 CodeRabbit inference engine (CLAUDE.md)

Always use tabs for indentation

Files:

  • packages/models/src/prepare-request-body.ts
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx,js,jsx}: Always use top-level import, never use require or dynamic imports
No unnecessary code comments

Files:

  • packages/models/src/prepare-request-body.ts
**/*.{js,ts,tsx,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Always use top-level import, never use require or dynamic imports

Files:

  • packages/models/src/prepare-request-body.ts
🧬 Code graph analysis (1)
packages/models/src/prepare-request-body.ts (1)
packages/db/src/types.ts (1)
  • tool (18-21)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: build / run
  • GitHub Check: e2e-shards (3)
  • GitHub Check: e2e-shards (4)
  • GitHub Check: lint / run
  • GitHub Check: e2e-shards (1)
  • GitHub Check: e2e-shards (2)
  • GitHub Check: generate / run
  • GitHub Check: test / run
  • GitHub Check: autofix
🔇 Additional comments (2)
packages/models/src/prepare-request-body.ts (2)

785-789: LGTM!

The stream options setup follows the established pattern used by other providers.


805-814: LGTM!

The tool strict enforcement correctly adds the required strict: true flag to each tool's function object for Cerebras compatibility.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for the Cerebras provider, a high-performance inference platform, along with integration for 5 specific models across different model families.

  • Adds Cerebras as a new provider with API configuration and endpoint setup
  • Integrates 5 Cerebras-hosted models: llama3.1-8b, llama-3.3-70b, gpt-oss-120b, qwen-3-235b-a22b-instruct-2507, qwen-3-32b, and zai-glm-4.6
  • Adds Cerebras icon component to all UI applications

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
packages/models/src/providers.ts Adds Cerebras provider configuration with API key requirements, streaming support, and branding details
packages/models/src/models/zai.ts Adds Cerebras provider variant for zai-glm-4.6 model with FP16/FP8 precision
packages/models/src/models/openai.ts Adds Cerebras provider variant for gpt-oss-120b model with FP16/FP8 precision
packages/models/src/models/meta.ts Adds Cerebras provider variants for llama3.1-8b and llama-3.3-70b models with FP16 precision
packages/models/src/models/alibaba.ts Adds Cerebras provider variants for qwen-3-235b-a22b-instruct-2507 and qwen-3-32b models
packages/models/src/get-provider-endpoint.ts Configures Cerebras API endpoint and adds it to OpenAI-compatible providers list
apps/ui/src/lib/components/providers-icons.tsx Adds CerebrasIcon SVG component and registers it in the ProviderIcons map
apps/playground/src/components/ui/providers-icons.tsx Adds CerebrasIcon SVG component and registers it in the ProviderIcons map
apps/playground/src/components/provider-icons.tsx Adds CerebrasIcon SVG component and registers it in the ProviderIcons map
apps/admin/src/components/ui/providers-icons.tsx Adds CerebrasIcon SVG component and registers it in the ProviderIcons map
.github/workflows/e2e.yml Adds LLM_CEREBRAS_API_KEY secret to e2e test workflow environment

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/models/src/providers.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/playground/src/components/provider-icons.tsx (1)

1215-1241: Add cerebras to providerLogoUrls mapping.
You added ProviderIcons.cerebras, but providerLogoUrls doesn't include it. Multiple components depend on providerLogoUrls to render provider logos (model-selector, provider-select, hero, models-supported, multi-provider-selector), so Cerebras won't display a logo in these UI surfaces without the mapping.

 export const providerLogoUrls: Partial<
 	Record<ProviderId, React.FC<React.SVGProps<SVGSVGElement>>>
 > = {
+	cerebras: ProviderIcons.cerebras,
 	openai: ProviderIcons.openai,
 	anthropic: ProviderIcons.anthropic,
 	"google-ai-studio": ProviderIcons["google-ai-studio"],
 	"google-vertex": ProviderIcons["google-vertex"],
 	"inference.net": ProviderIcons["inference.net"],
 	"together.ai": ProviderIcons["together.ai"],
 	cloudrift: ProviderIcons.cloudrift,
 	mistral: ProviderIcons.mistral,
 	groq: ProviderIcons.groq,
 	xai: ProviderIcons.xai,
 	deepseek: ProviderIcons.deepseek,
 	perplexity: ProviderIcons.perplexity,
 	moonshot: ProviderIcons.moonshot,
 	novita: ProviderIcons.novita,
 	alibaba: ProviderIcons.alibaba,
 	nebius: ProviderIcons.nebius,
 	zai: ProviderIcons.zai,
 	routeway: ProviderIcons.routeway,
 	"routeway-discount": ProviderIcons.routeway,
 	nanogpt: ProviderIcons.nanogpt,
 	"aws-bedrock": ProviderIcons["aws-bedrock"],
 	azure: ProviderIcons.azure,
 	canopywave: ProviderIcons.canopywave,
 };
🧹 Nitpick comments (2)
apps/admin/src/components/ui/providers-icons.tsx (1)

1063-1105: CerebrasIcon looks fine; consider de-duplicating this SVG across apps.
This exact SVG appears in multiple packages in the PR; a shared @/components/icons/CerebrasIcon (or similar) would reduce drift.

apps/ui/src/lib/components/providers-icons.tsx (1)

1138-1181: CerebrasIcon is OK; recommend centralizing to avoid multi-app SVG duplication.
This is identical to the admin/playground versions; a shared module would reduce future drift.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 65a8958 and 7c83dd2.

📒 Files selected for processing (11)
  • .github/workflows/e2e.yml (1 hunks)
  • apps/admin/src/components/ui/providers-icons.tsx (2 hunks)
  • apps/playground/src/components/provider-icons.tsx (2 hunks)
  • apps/playground/src/components/ui/providers-icons.tsx (2 hunks)
  • apps/ui/src/lib/components/providers-icons.tsx (2 hunks)
  • packages/models/src/get-provider-endpoint.ts (2 hunks)
  • packages/models/src/models/alibaba.ts (2 hunks)
  • packages/models/src/models/meta.ts (2 hunks)
  • packages/models/src/models/openai.ts (1 hunks)
  • packages/models/src/models/zai.ts (1 hunks)
  • packages/models/src/providers.ts (1 hunks)
🧰 Additional context used
📓 Path-based instructions (6)
**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx}: Never use any or as any unless absolutely necessary in TypeScript code
For database reads: Use db().query.<table>.findMany() or db().query.<table>.findFirst()

Files:

  • apps/ui/src/lib/components/providers-icons.tsx
  • packages/models/src/providers.ts
  • apps/admin/src/components/ui/providers-icons.tsx
  • packages/models/src/models/openai.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/get-provider-endpoint.ts
  • apps/playground/src/components/ui/providers-icons.tsx
  • packages/models/src/models/meta.ts
  • apps/playground/src/components/provider-icons.tsx
  • packages/models/src/models/alibaba.ts
**/*.{ts,tsx,js,jsx,json,md}

📄 CodeRabbit inference engine (CLAUDE.md)

Always use tabs for indentation

Files:

  • apps/ui/src/lib/components/providers-icons.tsx
  • packages/models/src/providers.ts
  • apps/admin/src/components/ui/providers-icons.tsx
  • packages/models/src/models/openai.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/get-provider-endpoint.ts
  • apps/playground/src/components/ui/providers-icons.tsx
  • packages/models/src/models/meta.ts
  • apps/playground/src/components/provider-icons.tsx
  • packages/models/src/models/alibaba.ts
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx,js,jsx}: Always use top-level import, never use require or dynamic imports
No unnecessary code comments

Files:

  • apps/ui/src/lib/components/providers-icons.tsx
  • packages/models/src/providers.ts
  • apps/admin/src/components/ui/providers-icons.tsx
  • packages/models/src/models/openai.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/get-provider-endpoint.ts
  • apps/playground/src/components/ui/providers-icons.tsx
  • packages/models/src/models/meta.ts
  • apps/playground/src/components/provider-icons.tsx
  • packages/models/src/models/alibaba.ts
apps/{ui,playground}/**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

apps/{ui,playground}/**/*.{ts,tsx}: Use cookies for user-settings which are not saved in the database to ensure SSR works
Use Next.js App Router with React Server Components for frontend development

Use cookies for user-settings which are not saved in the database to ensure SSR works

Files:

  • apps/ui/src/lib/components/providers-icons.tsx
  • apps/playground/src/components/ui/providers-icons.tsx
  • apps/playground/src/components/provider-icons.tsx
**/*.{js,ts,tsx,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Always use top-level import, never use require or dynamic imports

Files:

  • apps/ui/src/lib/components/providers-icons.tsx
  • packages/models/src/providers.ts
  • apps/admin/src/components/ui/providers-icons.tsx
  • packages/models/src/models/openai.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/get-provider-endpoint.ts
  • apps/playground/src/components/ui/providers-icons.tsx
  • packages/models/src/models/meta.ts
  • apps/playground/src/components/provider-icons.tsx
  • packages/models/src/models/alibaba.ts
apps/{ui,playground,docs}/**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Use next/link for links and next/navigation's router for programmatic navigation

Files:

  • apps/ui/src/lib/components/providers-icons.tsx
  • apps/playground/src/components/ui/providers-icons.tsx
  • apps/playground/src/components/provider-icons.tsx
🧬 Code graph analysis (1)
apps/ui/src/lib/components/providers-icons.tsx (3)
apps/admin/src/components/ui/providers-icons.tsx (1)
  • CerebrasIcon (1063-1105)
apps/playground/src/components/provider-icons.tsx (1)
  • CerebrasIcon (1140-1182)
apps/playground/src/components/ui/providers-icons.tsx (1)
  • CerebrasIcon (1063-1105)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
  • GitHub Check: Agent
  • GitHub Check: generate / run
  • GitHub Check: lint / run
  • GitHub Check: test / run
  • GitHub Check: build / run
  • GitHub Check: e2e-shards (1)
  • GitHub Check: e2e-shards (5)
  • GitHub Check: e2e-shards (4)
  • GitHub Check: e2e-shards (2)
  • GitHub Check: e2e-shards (3)
  • GitHub Check: autofix
🔇 Additional comments (10)
.github/workflows/e2e.yml (1)

52-80: Good: Cerebras key wired into e2e job env; consider guarding/skip behavior if secret is unset.
Right now this will pass an empty value if the secret isn’t configured; ensure Cerebras-dependent tests either skip explicitly or fail loudly (whichever you intend).

packages/models/src/models/alibaba.ts (2)

323-336: Same: confirm Cerebras Qwen 3 32B model id + remove inline comment.
Same concern for modelName: "qwen-3-32b"; please confirm it matches Cerebras’ expected model id, and consider removing the comment for consistency with guidelines.


236-250: Remove the inline comment; keep the modelName identifier.

The modelName: "qwen-3-235b-a22b-instruct-2507" is correct and matches the Cerebras API catalog. However, the // Cerebras: FP16/FP8 (weights only) comment violates the "No unnecessary code comments" guideline and should be removed.

apps/admin/src/components/ui/providers-icons.tsx (1)

1107-1131: ProviderIcons registration for cerebras is correct.
Lookup will work via exact key "cerebras" in getProviderIcon.

packages/models/src/get-provider-endpoint.ts (1)

279-295: Routing Cerebras to /v1/chat/completions is consistent with other OpenAI-compatible providers (assuming base URL is right).

apps/ui/src/lib/components/providers-icons.tsx (1)

1182-1208: ProviderIcons registration for cerebras is correct.

apps/playground/src/components/ui/providers-icons.tsx (1)

1130-1130: Good: cerebras is registered in ProviderIcons.
This keeps provider → icon lookup consistent with the new providerId.

packages/models/src/models/meta.ts (1)

66-79: > Likely an incorrect or invalid review comment.

packages/models/src/models/openai.ts (1)

536-550: The Cerebras modelName and maxOutput values are correct and do not require changes.

The code uses modelName: "gpt-oss-120b" (without the "openai/" prefix), which matches the official Cerebras OpenAI-compatible API specification. Other providers like Groq and NanoGPT include the "openai/" prefix because they operate in a different model namespace; Cerebras does not require this prefix. The maxOutput: 32768 corresponds to Cerebras's free tier max output tokens, and the contextSize: 131072 aligns with the paid tier context window documented in Cerebras's official specs.

packages/models/src/providers.ts (1)

135-150: No action needed. The cancellation: true flag correctly indicates support for client-side request cancellation (AbortController-based connection abort), which is standard for all OpenAI-compatible streaming APIs including Cerebras. This is implemented generically in the gateway for all providers with the flag set.

Comment on lines +1063 to +1105
export const CerebrasIcon: React.FC<React.SVGProps<SVGSVGElement>> = (
props,
) => (
<svg
width={533}
height={533}
viewBox="0 0 533 533"
fill="none"
xmlns="http://www.w3.org/2000/svg"
{...props}
>
<path
fillRule="evenodd"
clipRule="evenodd"
d="M269.588 497.912C237.2 497.912 206.526 491.428 178.519 479.795C136.604 462.251 100.786 432.884 75.4467 395.888C50.1073 358.893 35.4371 314.461 35.4371 266.405C35.4371 234.367 41.9148 204.047 53.9177 176.205C71.6362 134.633 101.358 99.3535 138.7 74.3721C176.042 49.3907 221.005 34.8977 269.588 34.8977V0C232.437 0 197 7.43721 164.611 20.9767C116.218 41.1907 75.0656 74.9442 46.1063 117.47C16.9564 159.995 0 211.293 0 266.405C0 303.209 7.62087 338.298 21.1479 370.144C41.5338 418.009 75.8277 458.628 118.695 487.423C161.753 516.219 213.575 533 269.398 533V497.912H269.588Z"
fill="#F05A28"
/>
<path
fillRule="evenodd"
clipRule="evenodd"
d="M149.56 408.474C127.65 390.168 111.265 368.047 100.215 343.828C89.1642 319.609 83.4486 293.484 83.4486 267.167C83.4486 246.191 87.0685 225.214 94.1178 205.191C101.358 185.167 112.027 166.098 126.697 148.744C144.987 127.005 167.278 110.605 191.474 99.5442C215.671 88.4837 242.153 82.9535 268.445 82.9535C289.403 82.9535 310.551 86.5768 330.555 93.6326C350.751 100.879 369.803 111.558 387.14 126.051L409.813 99.1628C389.236 82 366.374 69.0326 342.368 60.6419C318.362 52.0605 293.404 47.8651 268.445 47.8651C237.009 47.8651 205.764 54.5395 176.804 67.6977C147.845 80.8558 121.362 100.498 99.6429 126.242C82.3054 146.837 69.5405 169.53 60.967 193.367C52.3935 217.205 48.202 242.186 48.202 267.167C48.202 298.442 54.8703 329.716 68.0163 358.512C81.1623 387.307 100.977 413.814 126.888 435.363L149.56 408.474Z"
fill="#F05A28"
/>
<path
fillRule="evenodd"
clipRule="evenodd"
d="M203.858 385.781C180.805 373.577 162.706 355.842 150.322 335.056C137.938 314.27 131.46 290.242 131.46 266.023C131.46 244.665 136.414 223.116 147.273 203.093C159.467 180.019 177.376 162.093 198.333 149.888C219.291 137.493 243.296 131.009 267.683 131.009C289.022 131.009 310.741 135.967 330.937 146.647L347.321 115.563C321.982 102.214 294.547 95.7303 267.493 95.921C236.819 95.921 206.526 104.121 180.234 119.567C153.942 135.014 131.27 157.898 116.028 186.693C102.691 212.056 96.2135 239.326 96.2135 266.023C96.2135 296.535 104.406 326.665 120.029 352.791C135.652 379.107 158.514 401.419 187.474 416.674L203.858 385.781Z"
fill="#F05A28"
/>
<path
fillRule="evenodd"
clipRule="evenodd"
d="M269.779 352.219C257.776 352.219 246.345 349.74 236.057 345.354C220.434 338.87 207.288 327.809 197.952 313.888C188.617 299.967 183.092 283.377 183.092 265.451C183.092 253.437 185.568 241.995 189.95 231.698C196.428 216.251 207.478 202.902 221.386 193.558C235.295 184.214 251.87 178.684 269.779 178.684V143.595C253.013 143.595 237.009 147.028 222.339 153.13C200.429 162.474 181.948 177.73 168.802 197.181C155.466 216.823 147.845 240.47 147.845 265.642C147.845 282.423 151.274 298.442 157.371 313.126C166.707 335.056 182.139 353.554 201.572 366.712C221.005 379.679 244.44 387.307 269.779 387.307V352.219Z"
fill="#F05A28"
/>
<path
fillRule="evenodd"
clipRule="evenodd"
d="M298.357 237.8C294.356 233.605 289.974 230.172 285.592 227.693C281.21 225.214 276.638 223.879 271.875 223.879C265.397 223.879 259.872 225.023 254.728 227.312C249.774 229.6 245.392 232.651 241.772 236.656C238.152 240.47 235.485 245.047 233.58 250.005C231.675 254.963 230.913 260.302 230.913 265.642C230.913 270.981 231.865 276.321 233.58 281.279C235.485 286.237 238.152 290.814 241.772 294.628C245.392 298.442 249.584 301.684 254.728 303.972C259.681 306.26 265.397 307.405 271.875 307.405C277.209 307.405 282.353 306.26 286.926 304.163C291.498 301.874 295.499 298.633 298.738 294.437L321.982 319.419C318.553 322.851 314.552 325.902 309.979 328.381C305.407 330.861 300.834 332.958 296.262 334.484C291.689 336.009 287.116 337.154 282.925 337.726C278.734 338.488 274.923 338.679 271.875 338.679C261.396 338.679 251.489 336.963 242.344 333.53C233.008 330.098 225.006 325.14 218.338 318.656C211.479 312.363 206.145 304.544 202.144 295.581C198.143 286.619 196.238 276.512 196.238 265.642C196.238 254.581 198.143 244.665 202.144 235.702C206.145 226.74 211.479 219.112 218.338 212.628C225.197 206.335 233.199 201.377 242.344 197.754C251.679 194.321 261.587 192.605 271.875 192.605C280.829 192.605 289.784 194.321 298.738 197.754C307.693 201.186 315.695 206.526 322.363 213.772L298.357 237.8Z"
fill="black"
/>
</svg>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

CerebrasIcon: check dark-mode contrast (black fill).
The logo’s last path uses fill="black" (Line 1102), unlike other icons that adapt via currentColor + dark:text-white. If the playground uses this on dark backgrounds, it may be invisible/low-contrast.

🤖 Prompt for AI Agents
In apps/playground/src/components/ui/providers-icons.tsx around lines 1063 to
1105, the final <path> uses a hardcoded fill="black" which breaks dark-mode
contrast; change that fill to use an inheritable color (e.g.,
fill="currentColor") so the icon follows surrounding text color/dark-mode
utilities, and ensure the component still spreads props (className/style) so
consumers can control color via CSS or Tailwind (no other path fills need
change).

Comment on lines +186 to +200
{
// Cerebras: FP16/FP8 (weights only)
providerId: "cerebras",
modelName: "zai-glm-4.6",
inputPrice: 2.25 / 1e6,
outputPrice: 2.75 / 1e6,
requestPrice: 0,
contextSize: 200000,
maxOutput: undefined,
streaming: true,
reasoning: true,
vision: false,
tools: true,
jsonOutput: true,
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

Cerebras GLM-4.6 model API identifier zai-glm-4.6 pricing OpenAI compatible

💡 Result:

Short answer:

  • API model identifier: zai-glm-4.6. [1]
  • OpenAI-compatible (Cerebras exposes an OpenAI‑compatible endpoint and examples using model="zai-glm-4.6"). [2]
  • Pricing varies by host: Cerebras offers pay-as-you-go/dev tiers and subscription Code plans (examples: $10 starter, $50 Code Pro, $200 Code Max). Third‑party hosts list per‑million token rates (examples: GMI Cloud shows ZAI GLM‑4.6 input $0.60 /M tokens, output $2.00 /M tokens). Confirm current rates with your chosen provider. [4][3]

Sources:
[1] Cerebras supported models (model id zai-glm-4.6).
[2] Cerebras OpenAI‑compatibility docs (example OpenAI calls using model="zai-glm-4.6").
[3] GMI Cloud pricing page (ZAI: GLM‑4.6 per‑token rates).
[4] Cerebras blog / product pages (GLM‑4.6 availability and Cerebras subscription pricing).


Remove inline comment. The modelName "zai-glm-4.6" is correct per Cerebras documentation, but the // Cerebras: FP16/FP8 (weights only) comment violates the "No unnecessary code comments" guideline—remove it.

🤖 Prompt for AI Agents
In packages/models/src/models/zai.ts around lines 186 to 200, remove the inline
comment "// Cerebras: FP16/FP8 (weights only)" that precedes the model entry;
keep the object fields unchanged and ensure only the model object remains (no
extra comments) to comply with the "No unnecessary code comments" guideline.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7c83dd2 and fefdacc.

📒 Files selected for processing (1)
  • packages/models/src/prepare-request-body.ts (1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx}: Never use any or as any unless absolutely necessary in TypeScript code
For database reads: Use db().query.<table>.findMany() or db().query.<table>.findFirst()

Files:

  • packages/models/src/prepare-request-body.ts
**/*.{ts,tsx,js,jsx,json,md}

📄 CodeRabbit inference engine (CLAUDE.md)

Always use tabs for indentation

Files:

  • packages/models/src/prepare-request-body.ts
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx,js,jsx}: Always use top-level import, never use require or dynamic imports
No unnecessary code comments

Files:

  • packages/models/src/prepare-request-body.ts
**/*.{js,ts,tsx,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Always use top-level import, never use require or dynamic imports

Files:

  • packages/models/src/prepare-request-body.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
  • GitHub Check: test / run
  • GitHub Check: build / run
  • GitHub Check: lint / run
  • GitHub Check: autofix
  • GitHub Check: generate / run
  • GitHub Check: e2e-shards (3)
  • GitHub Check: e2e-shards (1)
  • GitHub Check: e2e-shards (4)
  • GitHub Check: e2e-shards (2)
  • GitHub Check: e2e-shards (5)
🔇 Additional comments (1)
packages/models/src/prepare-request-body.ts (1)

785-789: LGTM!

Stream options handling follows the established pattern used by other providers.

Comment on lines +790 to +803
if (response_format) {
// Cerebras requires strict: true for json_schema mode
if (response_format.type === "json_schema") {
requestBody.response_format = {
...response_format,
json_schema: {
...response_format.json_schema,
strict: true,
},
};
} else {
requestBody.response_format = response_format;
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, let's examine the prepare-request-body.ts file to understand the context
wc -l packages/models/src/prepare-request-body.ts

Repository: theopenco/llmgateway

Length of output: 111


🏁 Script executed:

# Look at the response_format handling around lines 694 (Google provider) and 790-803 (Cerebras)
sed -n '690,720p' packages/models/src/prepare-request-body.ts

Repository: theopenco/llmgateway

Length of output: 1163


🏁 Script executed:

# Get the Cerebras section in full context
sed -n '780,810p' packages/models/src/prepare-request-body.ts

Repository: theopenco/llmgateway

Length of output: 830


🏁 Script executed:

# Search for type definitions of response_format in this file
grep -n "response_format" packages/models/src/prepare-request-body.ts | head -20

Repository: theopenco/llmgateway

Length of output: 998


🏁 Script executed:

# Look for imports and type definitions at the top of the file
head -80 packages/models/src/prepare-request-body.ts

Repository: theopenco/llmgateway

Length of output: 2173


🏁 Script executed:

# Search for the actual type definition of response_format
fd -e ts -e tsx | xargs grep -l "type.*response_format" 2>/dev/null | head -5

Repository: theopenco/llmgateway

Length of output: 46


🏁 Script executed:

# Find the OpenAIRequestBody type definition
fd -e ts -e tsx | xargs grep -l "OpenAIRequestBody" | head -10

Repository: theopenco/llmgateway

Length of output: 171


🏁 Script executed:

# Look at types.ts file
fd types.ts

Repository: theopenco/llmgateway

Length of output: 281


🏁 Script executed:

# Read the types file to find response_format type definition
cat packages/models/src/types.ts | head -200

Repository: theopenco/llmgateway

Length of output: 3815


🏁 Script executed:

# Search specifically for response_format type definition
grep -n "response_format" packages/models/src/types.ts | head -20

Repository: theopenco/llmgateway

Length of output: 149


🏁 Script executed:

# Search for Cerebras-related type or API documentation in comments
grep -rn "Cerebras" packages/models/src/ --include="*.ts" --include="*.tsx"

Repository: theopenco/llmgateway

Length of output: 762


🏁 Script executed:

# Look for test files that test cerebras response_format
fd -e test.ts -e spec.ts -e test.tsx | xargs grep -l "cerebras" 2>/dev/null

Repository: theopenco/llmgateway

Length of output: 48


🏁 Script executed:

# Search for any comments or documentation about strict: true
grep -rn "strict.*true" packages/models/src/ --include="*.ts" --include="*.tsx"

Repository: theopenco/llmgateway

Length of output: 239


🏁 Script executed:

# Check the test file for cerebras response_format handling
grep -A 20 -B 5 "cerebras" packages/models/src/models.spec.ts | grep -A 20 -B 5 "response_format"

Repository: theopenco/llmgateway

Length of output: 46


🏁 Script executed:

# Look for any validation logic that might ensure json_schema exists before this code
grep -B 30 "case \"cerebras\"" packages/models/src/prepare-request-body.ts | tail -40

Repository: theopenco/llmgateway

Length of output: 814


🏁 Script executed:

# Check if there's validation in the function signature or earlier checks
sed -n '100,160p' packages/models/src/prepare-request-body.ts

Repository: theopenco/llmgateway

Length of output: 1809


🏁 Script executed:

# Look at how response_format is used in other providers to see the pattern
sed -n '250,260p' packages/models/src/prepare-request-body.ts

Repository: theopenco/llmgateway

Length of output: 333


🏁 Script executed:

# Check the anthropic provider handling of response_format
sed -n '320,345p' packages/models/src/prepare-request-body.ts

Repository: theopenco/llmgateway

Length of output: 709


🏁 Script executed:

# Search for callers of prepareRequestBody to see if json_schema is always provided
grep -rn "prepareRequestBody" packages/ --include="*.ts" --include="*.tsx" | grep -v "export" | grep -v "function prepareRequestBody" | head -10

Repository: theopenco/llmgateway

Length of output: 907


🏁 Script executed:

# Check if there's validation that ensures json_schema when type === "json_schema"
grep -B 10 "prepareRequestBody" apps/ --include="*.ts" --include="*.tsx" -r | head -40

Repository: theopenco/llmgateway

Length of output: 1710


🏁 Script executed:

# Look for where response_format is constructed to understand if json_schema is always present
grep -rn "json_schema" packages/models/src/ --include="*.ts" -B 2 -A 2 | head -50

Repository: theopenco/llmgateway

Length of output: 2497


Add defensive check for json_schema field.

The code accesses response_format.json_schema without checking if it exists. The TypeScript type definition allows json_schema to be optional even when type === "json_schema", which could cause a runtime error if the field is undefined. Add a safety check similar to the Google provider implementation (line 694) which uses optional chaining.

Apply this diff:

 if (response_format) {
 	// Cerebras requires strict: true for json_schema mode
-	if (response_format.type === "json_schema") {
+	if (response_format.type === "json_schema" && response_format.json_schema) {
 		requestBody.response_format = {
 			...response_format,
 			json_schema: {
 				...response_format.json_schema,
 				strict: true,
 			},
 		};
 	} else {
 		requestBody.response_format = response_format;
 	}
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (response_format) {
// Cerebras requires strict: true for json_schema mode
if (response_format.type === "json_schema") {
requestBody.response_format = {
...response_format,
json_schema: {
...response_format.json_schema,
strict: true,
},
};
} else {
requestBody.response_format = response_format;
}
}
if (response_format) {
// Cerebras requires strict: true for json_schema mode
if (response_format.type === "json_schema" && response_format.json_schema) {
requestBody.response_format = {
...response_format,
json_schema: {
...response_format.json_schema,
strict: true,
},
};
} else {
requestBody.response_format = response_format;
}
}
🤖 Prompt for AI Agents
In packages/models/src/prepare-request-body.ts around lines 790 to 803, the code
spreads response_format.json_schema without ensuring it exists which can throw
if json_schema is undefined; change the merge to safely handle a possibly
missing json_schema (e.g., use optional chaining/nullish coalescing so you
spread ...(response_format.json_schema ?? {}) or response_format.json_schema?.
and then add strict: true) and keep the rest of the response_format fields
unchanged so the json_schema merge never errors at runtime.

Comment on lines +806 to +823
if (temperature !== undefined) {
requestBody.temperature = temperature;
}
if (max_tokens !== undefined) {
requestBody.max_tokens = max_tokens;
}
if (top_p !== undefined) {
requestBody.top_p = top_p;
}
if (frequency_penalty !== undefined) {
requestBody.frequency_penalty = frequency_penalty;
}
if (presence_penalty !== undefined) {
requestBody.presence_penalty = presence_penalty;
}
if (reasoning_effort !== undefined) {
requestBody.reasoning_effort = reasoning_effort;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Cerebras AI API chat completions parameters documentation

💡 Result:

Here are the main chat-completions request parameters from Cerebras’ API docs (concise summary). Sources: [1][2][3].

Required

  • model — model name (string). [1]
  • messages — array of message objects ({role, content}). [1]

Generation / sampling

  • max_completion_tokens — max tokens the model may generate (counts toward context). [1]
  • temperature — sampling temperature (0–1.5). [2]
  • topP (top_p) — nucleus sampling parameter. [2]
  • stop — up to 4 stop sequences. [2]

Output options / diagnostics

  • logprobs — return token log-probabilities (bool). [1]
  • echo — echo prompt back with completion (bool). [2]
  • user — opaque end‑user id for abuse monitoring (string). [1]

Tools / function-calling

  • tools — list of tools/functions (name, description, parameters) the model may call. [1]
  • parallel_tool_calls — enable parallel tool calls (bool, default true). [1]

Prediction / performance

  • prediction — predicted-output config to speed responses (object). [1]
  • seed — deterministic sampling seed (integer). [2]

Other / compatibility notes

  • Some OpenAI fields are unsupported (e.g., frequency_penalty, logit_bias, presence_penalty). Streaming has caveats with reasoning/tool JSON mode. [2]

If you want, I can paste the exact JSON schema and example request snippets from the docs.

Sources:

  1. Cerebras Inference — Chat Completions API reference. [https://inference-docs.cerebras.ai/api-reference/chat-completions]
  2. Cerebras Inference — OpenAI compatibility & parameter notes. [https://inference-docs.cerebras.ai/resources/openai]
  3. Cerebras Inference — Quickstart (examples). [https://inference-docs.cerebras.ai/quickstart]

Remove unsupported parameters for Cerebras API.

The Cerebras API does not support frequency_penalty, presence_penalty, or reasoning_effort. These parameters should not be propagated to the request body for the Cerebras case, as they will cause API errors or be silently ignored. Additionally, use max_completion_tokens instead of max_tokens for Cerebras compatibility.

🤖 Prompt for AI Agents
In packages/models/src/prepare-request-body.ts around lines 806 to 823, the
Cerebras branch is currently adding unsupported fields (frequency_penalty,
presence_penalty, reasoning_effort) and using max_tokens; remove propagation of
frequency_penalty, presence_penalty, and reasoning_effort when building the
requestBody for Cerebras, and replace max_tokens with max_completion_tokens for
the Cerebras case (keep other providers unchanged). Ensure these changes are
applied only in the Cerebras-specific conditional path so other providers still
receive their appropriate parameters.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/models/src/testing.ts (1)

14-25: Consider preserving skip even when applying retry/timeout.
Right now skip is only returned in the : { skip: ... } branch, so callers with default completions: true will never see skip even if hasTestOnly / TEST_MODELS is set. If the intent is “apply retry/timeout and still skip when requested”, you can include skip in both branches.

 	const hasTestOnly = models.some((model) =>
 		model.providers.some(
 			(provider: ProviderModelMapping) => provider.test === "only",
 		),
 	);
-	return process.env.CI || opts?.completions
+	const skip = hasTestOnly || !!process.env.TEST_MODELS;
+	return process.env.CI || opts?.completions
 		? {
+				skip,
 				retry: process.env.CI ? 3 : 0,
 				timeout: process.env.CI ? 180000 : 60000,
 			}
-		: { skip: hasTestOnly || !!process.env.TEST_MODELS };
+		: { skip };
 }
♻️ Duplicate comments (3)
packages/models/src/providers.ts (1)

135-150: Tweak provider description grammar for consistency.
This reads a bit off compared to “Groq’s …”. Consider “Cerebras’s …” or “Cerebras provides …”.

packages/models/src/models/zai.ts (1)

186-200: Remove the inline comment to match “no unnecessary code comments” guideline.
Keep the object as-is, but drop // Cerebras: FP16/FP8 (weights only). Also worth re-confirming modelName: "zai-glm-4.6" is still the correct Cerebras identifier.

Cerebras supported model identifier for Zhipu GLM-4.6 in their OpenAI-compatible API (is it "zai-glm-4.6")?
packages/models/src/prepare-request-body.ts (1)

784-825: Fix potential runtime error when response_format.json_schema is missing.
...response_format.json_schema can throw if json_schema is undefined.

 			if (response_format) {
 				// Cerebras requires strict: true for json_schema mode
-				if (response_format.type === "json_schema") {
+				if (response_format.type === "json_schema" && response_format.json_schema) {
 					requestBody.response_format = {
 						...response_format,
 						json_schema: {
 							...response_format.json_schema,
 							strict: true,
 						},
 					};
 				} else {
 					requestBody.response_format = response_format;
 				}
 			}

Also please re-check Cerebras’ current OpenAI-compat parameter support (e.g., whether max_tokens vs max_completion_tokens is expected, and whether frequency_penalty / presence_penalty / reasoning_effort are accepted).

Cerebras Inference OpenAI-compatible chat completions: supported request parameters (max_tokens vs max_completion_tokens; frequency_penalty; presence_penalty; reasoning_effort).
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fefdacc and f50812d.

📒 Files selected for processing (13)
  • .github/workflows/e2e.yml (1 hunks)
  • apps/admin/src/components/ui/providers-icons.tsx (2 hunks)
  • apps/playground/src/components/provider-icons.tsx (2 hunks)
  • apps/playground/src/components/ui/providers-icons.tsx (2 hunks)
  • apps/ui/src/lib/components/providers-icons.tsx (2 hunks)
  • packages/models/src/get-provider-endpoint.ts (2 hunks)
  • packages/models/src/models/alibaba.ts (2 hunks)
  • packages/models/src/models/meta.ts (2 hunks)
  • packages/models/src/models/openai.ts (1 hunks)
  • packages/models/src/models/zai.ts (1 hunks)
  • packages/models/src/prepare-request-body.ts (1 hunks)
  • packages/models/src/providers.ts (1 hunks)
  • packages/models/src/testing.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
  • packages/models/src/models/openai.ts
  • apps/ui/src/lib/components/providers-icons.tsx
  • apps/playground/src/components/ui/providers-icons.tsx
  • apps/admin/src/components/ui/providers-icons.tsx
  • apps/playground/src/components/provider-icons.tsx
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx}: Never use any or as any unless absolutely necessary in TypeScript code
For database reads: Use db().query.<table>.findMany() or db().query.<table>.findFirst()

Files:

  • packages/models/src/get-provider-endpoint.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/providers.ts
  • packages/models/src/testing.ts
  • packages/models/src/models/alibaba.ts
  • packages/models/src/prepare-request-body.ts
  • packages/models/src/models/meta.ts
**/*.{ts,tsx,js,jsx,json,md}

📄 CodeRabbit inference engine (CLAUDE.md)

Always use tabs for indentation

Files:

  • packages/models/src/get-provider-endpoint.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/providers.ts
  • packages/models/src/testing.ts
  • packages/models/src/models/alibaba.ts
  • packages/models/src/prepare-request-body.ts
  • packages/models/src/models/meta.ts
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.{ts,tsx,js,jsx}: Always use top-level import, never use require or dynamic imports
No unnecessary code comments

Files:

  • packages/models/src/get-provider-endpoint.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/providers.ts
  • packages/models/src/testing.ts
  • packages/models/src/models/alibaba.ts
  • packages/models/src/prepare-request-body.ts
  • packages/models/src/models/meta.ts
**/*.{js,ts,tsx,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Always use top-level import, never use require or dynamic imports

Files:

  • packages/models/src/get-provider-endpoint.ts
  • packages/models/src/models/zai.ts
  • packages/models/src/providers.ts
  • packages/models/src/testing.ts
  • packages/models/src/models/alibaba.ts
  • packages/models/src/prepare-request-body.ts
  • packages/models/src/models/meta.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
  • GitHub Check: generate / run
  • GitHub Check: test / run
  • GitHub Check: build / run
  • GitHub Check: lint / run
  • GitHub Check: e2e-shards (4)
  • GitHub Check: autofix
  • GitHub Check: e2e-shards (5)
  • GitHub Check: e2e-shards (2)
  • GitHub Check: e2e-shards (3)
  • GitHub Check: e2e-shards (1)
🔇 Additional comments (3)
.github/workflows/e2e.yml (1)

52-80: E2E secret wiring looks fine; ensure the repo secret exists.
LLM_CEREBRAS_API_KEY is correctly exposed to the shard run; just double-check secrets.CEREBRAS_API_KEY is set in the repo/org secrets so CI doesn’t fail on Cerebras tests.

packages/models/src/get-provider-endpoint.ts (1)

77-79: No action needed — Cerebras endpoint is correctly implemented.

The base URL https://api.cerebras.ai combined with the standard /v1/chat/completions path produces the correct OpenAI-compatible endpoint. Cerebras does not require additional path prefixes like Groq's /openai.

packages/models/src/models/meta.ts (1)

196-209: All Cerebras model details are accurate per official documentation.

Web search confirms the entry is correct:

  • Model name llama-3.3-70b
  • Pricing: $0.85/M input, $1.20/M output tokens ✓
  • Context size: 128k tokens (paid tier) ✓
  • Capabilities: streaming, tool calling, and structured outputs supported ✓
  • Vision: correctly set to false

No changes needed.

Comment on lines +236 to +250
{
// Cerebras: FP16/FP8 (weights only)
providerId: "cerebras",
modelName: "qwen-3-235b-a22b-instruct-2507",
inputPrice: 0.6 / 1e6,
outputPrice: 1.2 / 1e6,
requestPrice: 0,
contextSize: 262000,
maxOutput: 8192,
reasoning: false,
streaming: true,
vision: false,
tools: true,
jsonOutput: true,
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🌐 Web query:

Cerebras qwen-3-235b-a22b-instruct pricing documentation

💡 Result:

Cerebras pricing for Qwen model (model ID qwen-3-235b-a22b-instruct-2507):

  • Input: $0.60 per 1M tokens. [1]
  • Output: $1.20 per 1M tokens. [1]
  • Context (paid tier): 131K tokens (free tier: 64–65K; max output paid tier 40K). [2]
  • Model name to use: qwen-3-235b-a22b-instruct-2507 (non-thinking mode). [2]

Sources: Cerebras blog announcement and Cerebras Inference docs. [1][2]

References
[1] Cerebras blog — "Qwen3 235B 2507 Instruct Now Available on Cerebras."
[2] Cerebras Inference documentation — "Qwen 3 235B Instruct" model card.


Correct context size and max output for Cerebras Qwen model.

The model name and pricing are accurate, but context size and max output are incorrect:

  • Context size should be 131,072 tokens (not 262,000)
  • Max output should be 40,000 tokens (not 8,192)

Update lines 239 and 241 accordingly.

🤖 Prompt for AI Agents
In packages/models/src/models/alibaba.ts around lines 236 to 250, the Cerebras
Qwen model entry has incorrect contextSize and maxOutput values; update the
contextSize on line ~239 to 131072 and the maxOutput on line ~241 to 40000 so
the entry reads the correct context size (131,072 tokens) and max output (40,000
tokens).

Comment on lines +323 to +336
{
// Cerebras: FP16
providerId: "cerebras",
modelName: "qwen-3-32b",
inputPrice: 0.4 / 1e6,
outputPrice: 0.8 / 1e6,
requestPrice: 0,
contextSize: 32768,
maxOutput: 8192,
streaming: true,
vision: false,
tools: true,
jsonOutput: true,
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Cerebras qwen-3-32b model pricing capabilities documentation

💡 Result:

Summary for Cerebras qwen-3-32b (qwen-3-32b)

Pricing

  • $0.40 per million input tokens; $0.80 per million output tokens. Developers receive 1M free tokens/day on the free tier; volume/enterprise pricing and monthly growth tiers available. [1][2]

Capabilities / model card highlights

  • Reasoning model (default reasoning mode).
  • Streaming, structured outputs, tool-calling supported.
  • Context length: free tier ~65K (paid up to ~131K per docs). Max output ~8K (paid/free tiers).
  • Endpoints: OpenAI/Claude-compatible Chat Completions and Completions.
  • Typical throughput/speed: ~2,400–2,600 tokens/sec on Cerebras infrastructure; first-token latency ≈1–1.2s.
  • Recommended decoding when using reasoning: temperature ~0.6, top_p ~0.95; append “/no_think” to disable reasoning mode for specific prompts. [2][3]

Rate limits / tiers (examples)

  • Free: 30 req/min, 60k input tokens/min, 1M daily tokens.
  • Developer / Growth tiers: higher RPM and token budgets (example growth-tier monthly prices and rate limits available in docs). For large-volume or custom SLA needs, contact sales for Enterprise quotes. [2]

Sources

  1. Cerebras inference pricing & model listing (Qwen 3 32B pricing & token rates). [inference docs & pricing]. [2]
  2. Qwen-3-32B model page (capabilities, context lengths, endpoints, rate limits). [3]
  3. Cerebras press/blog announcement describing speed, latency and pricing. [1]

Would you like direct links to any of those pages or a copy of the model card text?


Correct the context size for the Cerebras qwen-3-32b model.

The model name, pricing ($0.4/M input, $0.8/M output), and capabilities (streaming, tools, JSON output support, 8192 max output) are verified against Cerebras documentation. However, the contextSize is incorrect: Cerebras documentation indicates the free tier supports ~65K tokens, with paid tiers supporting up to ~131K tokens. The current value of 32768 (32K) significantly understates the actual context window. Update contextSize accordingly based on the intended tier.

🤖 Prompt for AI Agents
In packages/models/src/models/alibaba.ts around lines 323 to 336, the Cerebras
qwen-3-32b entry incorrectly sets contextSize to 32768; update contextSize to
match Cerebras docs (use 65536 for the free-tier ~65K window or 131072 for the
paid-tier ~131K window) — replace the 32768 literal with the appropriate integer
for the intended tier and ensure any surrounding comments/metadata reflect the
chosen tier.

Comment on lines +66 to +79
{
// Cerebras: FP16
providerId: "cerebras",
modelName: "llama3.1-8b",
inputPrice: 0.1 / 1e6,
outputPrice: 0.1 / 1e6,
requestPrice: 0,
contextSize: 128000,
maxOutput: undefined,
streaming: true,
vision: false,
tools: true,
jsonOutput: true,
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

Cerebras llama 3.1 8b model pricing documentation

💡 Result:

Cerebras Inference lists Llama 3.1 8B pricing as $0.10 per million input tokens and $0.10 per million output tokens (exploration/developer tier); the model card and pricing page include rate limits and speed (~2,200 tokens/s). (inference-docs.cerebras.ai)

Citations:


Correct model name format to match Cerebras documentation: use llama-31-8b instead of llama3.1-8b.

The official Cerebras documentation identifies this model as llama-31-8b (with dashes). Update the modelName to match. The pricing ($0.1/M tokens for input and output) and context size (128k tokens) are confirmed as accurate.

🤖 Prompt for AI Agents
In packages/models/src/models/meta.ts around lines 66 to 79, the Cerebras model
entry uses an incorrect modelName "llama3.1-8b"; update the modelName value to
the documented format "llama-31-8b" while leaving the rest of the object
(pricing, contextSize, flags) unchanged.

- Disable tool calling for llama3.1-8b due to unreliability.
- Ensure strict mode is applied to tools in request body.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants