feat(cerebras): add Cerebras provider and models support - #1322
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughAdds a new Cerebras provider integration: CI env var, provider registry entry, endpoint routing, request preparation branch, multiple model provider entries, UI provider icons, and a small testing option change. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant UI as Client (UI)
participant GW as Gateway / Server
participant Models as models module
participant Cere as Cerebras API
UI->>GW: request (provider: cerebras, model, params)
GW->>Models: prepareRequestBody(provider:cerebras, params)
Models-->>GW: prepared body (stream_options.include_usage, response_format(strict)?, forwarded params)
GW->>Models: getProviderEndpoint(provider:cerebras)
Models-->>GW: endpoint URL (https://api.cerebras.ai/v1/chat/completions)
GW->>Cere: POST (body, Authorization: Bearer LLM_CEREBRAS_API_KEY)
Cere-->>GW: response / stream
GW-->>UI: deliver response/stream
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Areas to pay attention to:
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
🧰 Additional context used📓 Path-based instructions (4)**/*.{ts,tsx}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
**/*.{ts,tsx,js,jsx,json,md}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
**/*.{ts,tsx,js,jsx}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
**/*.{js,ts,tsx,jsx}📄 CodeRabbit inference engine (AGENTS.md)
Files:
🧬 Code graph analysis (1)packages/models/src/prepare-request-body.ts (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
🔇 Additional comments (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR adds support for the Cerebras provider, a high-performance inference platform, along with integration for 5 specific models across different model families.
- Adds Cerebras as a new provider with API configuration and endpoint setup
- Integrates 5 Cerebras-hosted models: llama3.1-8b, llama-3.3-70b, gpt-oss-120b, qwen-3-235b-a22b-instruct-2507, qwen-3-32b, and zai-glm-4.6
- Adds Cerebras icon component to all UI applications
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/models/src/providers.ts | Adds Cerebras provider configuration with API key requirements, streaming support, and branding details |
| packages/models/src/models/zai.ts | Adds Cerebras provider variant for zai-glm-4.6 model with FP16/FP8 precision |
| packages/models/src/models/openai.ts | Adds Cerebras provider variant for gpt-oss-120b model with FP16/FP8 precision |
| packages/models/src/models/meta.ts | Adds Cerebras provider variants for llama3.1-8b and llama-3.3-70b models with FP16 precision |
| packages/models/src/models/alibaba.ts | Adds Cerebras provider variants for qwen-3-235b-a22b-instruct-2507 and qwen-3-32b models |
| packages/models/src/get-provider-endpoint.ts | Configures Cerebras API endpoint and adds it to OpenAI-compatible providers list |
| apps/ui/src/lib/components/providers-icons.tsx | Adds CerebrasIcon SVG component and registers it in the ProviderIcons map |
| apps/playground/src/components/ui/providers-icons.tsx | Adds CerebrasIcon SVG component and registers it in the ProviderIcons map |
| apps/playground/src/components/provider-icons.tsx | Adds CerebrasIcon SVG component and registers it in the ProviderIcons map |
| apps/admin/src/components/ui/providers-icons.tsx | Adds CerebrasIcon SVG component and registers it in the ProviderIcons map |
| .github/workflows/e2e.yml | Adds LLM_CEREBRAS_API_KEY secret to e2e test workflow environment |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/playground/src/components/provider-icons.tsx (1)
1215-1241: AddcerebrastoproviderLogoUrlsmapping.
You addedProviderIcons.cerebras, butproviderLogoUrlsdoesn't include it. Multiple components depend onproviderLogoUrlsto render provider logos (model-selector, provider-select, hero, models-supported, multi-provider-selector), so Cerebras won't display a logo in these UI surfaces without the mapping.export const providerLogoUrls: Partial< Record<ProviderId, React.FC<React.SVGProps<SVGSVGElement>>> > = { + cerebras: ProviderIcons.cerebras, openai: ProviderIcons.openai, anthropic: ProviderIcons.anthropic, "google-ai-studio": ProviderIcons["google-ai-studio"], "google-vertex": ProviderIcons["google-vertex"], "inference.net": ProviderIcons["inference.net"], "together.ai": ProviderIcons["together.ai"], cloudrift: ProviderIcons.cloudrift, mistral: ProviderIcons.mistral, groq: ProviderIcons.groq, xai: ProviderIcons.xai, deepseek: ProviderIcons.deepseek, perplexity: ProviderIcons.perplexity, moonshot: ProviderIcons.moonshot, novita: ProviderIcons.novita, alibaba: ProviderIcons.alibaba, nebius: ProviderIcons.nebius, zai: ProviderIcons.zai, routeway: ProviderIcons.routeway, "routeway-discount": ProviderIcons.routeway, nanogpt: ProviderIcons.nanogpt, "aws-bedrock": ProviderIcons["aws-bedrock"], azure: ProviderIcons.azure, canopywave: ProviderIcons.canopywave, };
🧹 Nitpick comments (2)
apps/admin/src/components/ui/providers-icons.tsx (1)
1063-1105: CerebrasIcon looks fine; consider de-duplicating this SVG across apps.
This exact SVG appears in multiple packages in the PR; a shared@/components/icons/CerebrasIcon(or similar) would reduce drift.apps/ui/src/lib/components/providers-icons.tsx (1)
1138-1181: CerebrasIcon is OK; recommend centralizing to avoid multi-app SVG duplication.
This is identical to the admin/playground versions; a shared module would reduce future drift.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
.github/workflows/e2e.yml(1 hunks)apps/admin/src/components/ui/providers-icons.tsx(2 hunks)apps/playground/src/components/provider-icons.tsx(2 hunks)apps/playground/src/components/ui/providers-icons.tsx(2 hunks)apps/ui/src/lib/components/providers-icons.tsx(2 hunks)packages/models/src/get-provider-endpoint.ts(2 hunks)packages/models/src/models/alibaba.ts(2 hunks)packages/models/src/models/meta.ts(2 hunks)packages/models/src/models/openai.ts(1 hunks)packages/models/src/models/zai.ts(1 hunks)packages/models/src/providers.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (6)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Never useanyoras anyunless absolutely necessary in TypeScript code
For database reads: Usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
apps/ui/src/lib/components/providers-icons.tsxpackages/models/src/providers.tsapps/admin/src/components/ui/providers-icons.tsxpackages/models/src/models/openai.tspackages/models/src/models/zai.tspackages/models/src/get-provider-endpoint.tsapps/playground/src/components/ui/providers-icons.tsxpackages/models/src/models/meta.tsapps/playground/src/components/provider-icons.tsxpackages/models/src/models/alibaba.ts
**/*.{ts,tsx,js,jsx,json,md}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use tabs for indentation
Files:
apps/ui/src/lib/components/providers-icons.tsxpackages/models/src/providers.tsapps/admin/src/components/ui/providers-icons.tsxpackages/models/src/models/openai.tspackages/models/src/models/zai.tspackages/models/src/get-provider-endpoint.tsapps/playground/src/components/ui/providers-icons.tsxpackages/models/src/models/meta.tsapps/playground/src/components/provider-icons.tsxpackages/models/src/models/alibaba.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx,js,jsx}: Always use top-levelimport, never use require or dynamic imports
No unnecessary code comments
Files:
apps/ui/src/lib/components/providers-icons.tsxpackages/models/src/providers.tsapps/admin/src/components/ui/providers-icons.tsxpackages/models/src/models/openai.tspackages/models/src/models/zai.tspackages/models/src/get-provider-endpoint.tsapps/playground/src/components/ui/providers-icons.tsxpackages/models/src/models/meta.tsapps/playground/src/components/provider-icons.tsxpackages/models/src/models/alibaba.ts
apps/{ui,playground}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
apps/{ui,playground}/**/*.{ts,tsx}: Use cookies for user-settings which are not saved in the database to ensure SSR works
Use Next.js App Router with React Server Components for frontend developmentUse cookies for user-settings which are not saved in the database to ensure SSR works
Files:
apps/ui/src/lib/components/providers-icons.tsxapps/playground/src/components/ui/providers-icons.tsxapps/playground/src/components/provider-icons.tsx
**/*.{js,ts,tsx,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level
import, never use require or dynamic imports
Files:
apps/ui/src/lib/components/providers-icons.tsxpackages/models/src/providers.tsapps/admin/src/components/ui/providers-icons.tsxpackages/models/src/models/openai.tspackages/models/src/models/zai.tspackages/models/src/get-provider-endpoint.tsapps/playground/src/components/ui/providers-icons.tsxpackages/models/src/models/meta.tsapps/playground/src/components/provider-icons.tsxpackages/models/src/models/alibaba.ts
apps/{ui,playground,docs}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use
next/linkfor links andnext/navigation's router for programmatic navigation
Files:
apps/ui/src/lib/components/providers-icons.tsxapps/playground/src/components/ui/providers-icons.tsxapps/playground/src/components/provider-icons.tsx
🧬 Code graph analysis (1)
apps/ui/src/lib/components/providers-icons.tsx (3)
apps/admin/src/components/ui/providers-icons.tsx (1)
CerebrasIcon(1063-1105)apps/playground/src/components/provider-icons.tsx (1)
CerebrasIcon(1140-1182)apps/playground/src/components/ui/providers-icons.tsx (1)
CerebrasIcon(1063-1105)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: Agent
- GitHub Check: generate / run
- GitHub Check: lint / run
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (5)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (3)
- GitHub Check: autofix
🔇 Additional comments (10)
.github/workflows/e2e.yml (1)
52-80: Good: Cerebras key wired into e2e job env; consider guarding/skip behavior if secret is unset.
Right now this will pass an empty value if the secret isn’t configured; ensure Cerebras-dependent tests either skip explicitly or fail loudly (whichever you intend).packages/models/src/models/alibaba.ts (2)
323-336: Same: confirm Cerebras Qwen 3 32B model id + remove inline comment.
Same concern formodelName: "qwen-3-32b"; please confirm it matches Cerebras’ expected model id, and consider removing the comment for consistency with guidelines.
236-250: Remove the inline comment; keep themodelNameidentifier.The
modelName: "qwen-3-235b-a22b-instruct-2507"is correct and matches the Cerebras API catalog. However, the// Cerebras: FP16/FP8 (weights only)comment violates the "No unnecessary code comments" guideline and should be removed.apps/admin/src/components/ui/providers-icons.tsx (1)
1107-1131: ProviderIcons registration forcerebrasis correct.
Lookup will work via exact key"cerebras"ingetProviderIcon.packages/models/src/get-provider-endpoint.ts (1)
279-295: Routing Cerebras to/v1/chat/completionsis consistent with other OpenAI-compatible providers (assuming base URL is right).apps/ui/src/lib/components/providers-icons.tsx (1)
1182-1208: ProviderIcons registration forcerebrasis correct.apps/playground/src/components/ui/providers-icons.tsx (1)
1130-1130: Good:cerebrasis registered in ProviderIcons.
This keeps provider → icon lookup consistent with the new providerId.packages/models/src/models/meta.ts (1)
66-79: > Likely an incorrect or invalid review comment.packages/models/src/models/openai.ts (1)
536-550: The Cerebras modelName and maxOutput values are correct and do not require changes.The code uses
modelName: "gpt-oss-120b"(without the "openai/" prefix), which matches the official Cerebras OpenAI-compatible API specification. Other providers like Groq and NanoGPT include the "openai/" prefix because they operate in a different model namespace; Cerebras does not require this prefix. ThemaxOutput: 32768corresponds to Cerebras's free tier max output tokens, and thecontextSize: 131072aligns with the paid tier context window documented in Cerebras's official specs.packages/models/src/providers.ts (1)
135-150: No action needed. Thecancellation: trueflag correctly indicates support for client-side request cancellation (AbortController-based connection abort), which is standard for all OpenAI-compatible streaming APIs including Cerebras. This is implemented generically in the gateway for all providers with the flag set.
| export const CerebrasIcon: React.FC<React.SVGProps<SVGSVGElement>> = ( | ||
| props, | ||
| ) => ( | ||
| <svg | ||
| width={533} | ||
| height={533} | ||
| viewBox="0 0 533 533" | ||
| fill="none" | ||
| xmlns="http://www.w3.org/2000/svg" | ||
| {...props} | ||
| > | ||
| <path | ||
| fillRule="evenodd" | ||
| clipRule="evenodd" | ||
| d="M269.588 497.912C237.2 497.912 206.526 491.428 178.519 479.795C136.604 462.251 100.786 432.884 75.4467 395.888C50.1073 358.893 35.4371 314.461 35.4371 266.405C35.4371 234.367 41.9148 204.047 53.9177 176.205C71.6362 134.633 101.358 99.3535 138.7 74.3721C176.042 49.3907 221.005 34.8977 269.588 34.8977V0C232.437 0 197 7.43721 164.611 20.9767C116.218 41.1907 75.0656 74.9442 46.1063 117.47C16.9564 159.995 0 211.293 0 266.405C0 303.209 7.62087 338.298 21.1479 370.144C41.5338 418.009 75.8277 458.628 118.695 487.423C161.753 516.219 213.575 533 269.398 533V497.912H269.588Z" | ||
| fill="#F05A28" | ||
| /> | ||
| <path | ||
| fillRule="evenodd" | ||
| clipRule="evenodd" | ||
| d="M149.56 408.474C127.65 390.168 111.265 368.047 100.215 343.828C89.1642 319.609 83.4486 293.484 83.4486 267.167C83.4486 246.191 87.0685 225.214 94.1178 205.191C101.358 185.167 112.027 166.098 126.697 148.744C144.987 127.005 167.278 110.605 191.474 99.5442C215.671 88.4837 242.153 82.9535 268.445 82.9535C289.403 82.9535 310.551 86.5768 330.555 93.6326C350.751 100.879 369.803 111.558 387.14 126.051L409.813 99.1628C389.236 82 366.374 69.0326 342.368 60.6419C318.362 52.0605 293.404 47.8651 268.445 47.8651C237.009 47.8651 205.764 54.5395 176.804 67.6977C147.845 80.8558 121.362 100.498 99.6429 126.242C82.3054 146.837 69.5405 169.53 60.967 193.367C52.3935 217.205 48.202 242.186 48.202 267.167C48.202 298.442 54.8703 329.716 68.0163 358.512C81.1623 387.307 100.977 413.814 126.888 435.363L149.56 408.474Z" | ||
| fill="#F05A28" | ||
| /> | ||
| <path | ||
| fillRule="evenodd" | ||
| clipRule="evenodd" | ||
| d="M203.858 385.781C180.805 373.577 162.706 355.842 150.322 335.056C137.938 314.27 131.46 290.242 131.46 266.023C131.46 244.665 136.414 223.116 147.273 203.093C159.467 180.019 177.376 162.093 198.333 149.888C219.291 137.493 243.296 131.009 267.683 131.009C289.022 131.009 310.741 135.967 330.937 146.647L347.321 115.563C321.982 102.214 294.547 95.7303 267.493 95.921C236.819 95.921 206.526 104.121 180.234 119.567C153.942 135.014 131.27 157.898 116.028 186.693C102.691 212.056 96.2135 239.326 96.2135 266.023C96.2135 296.535 104.406 326.665 120.029 352.791C135.652 379.107 158.514 401.419 187.474 416.674L203.858 385.781Z" | ||
| fill="#F05A28" | ||
| /> | ||
| <path | ||
| fillRule="evenodd" | ||
| clipRule="evenodd" | ||
| d="M269.779 352.219C257.776 352.219 246.345 349.74 236.057 345.354C220.434 338.87 207.288 327.809 197.952 313.888C188.617 299.967 183.092 283.377 183.092 265.451C183.092 253.437 185.568 241.995 189.95 231.698C196.428 216.251 207.478 202.902 221.386 193.558C235.295 184.214 251.87 178.684 269.779 178.684V143.595C253.013 143.595 237.009 147.028 222.339 153.13C200.429 162.474 181.948 177.73 168.802 197.181C155.466 216.823 147.845 240.47 147.845 265.642C147.845 282.423 151.274 298.442 157.371 313.126C166.707 335.056 182.139 353.554 201.572 366.712C221.005 379.679 244.44 387.307 269.779 387.307V352.219Z" | ||
| fill="#F05A28" | ||
| /> | ||
| <path | ||
| fillRule="evenodd" | ||
| clipRule="evenodd" | ||
| d="M298.357 237.8C294.356 233.605 289.974 230.172 285.592 227.693C281.21 225.214 276.638 223.879 271.875 223.879C265.397 223.879 259.872 225.023 254.728 227.312C249.774 229.6 245.392 232.651 241.772 236.656C238.152 240.47 235.485 245.047 233.58 250.005C231.675 254.963 230.913 260.302 230.913 265.642C230.913 270.981 231.865 276.321 233.58 281.279C235.485 286.237 238.152 290.814 241.772 294.628C245.392 298.442 249.584 301.684 254.728 303.972C259.681 306.26 265.397 307.405 271.875 307.405C277.209 307.405 282.353 306.26 286.926 304.163C291.498 301.874 295.499 298.633 298.738 294.437L321.982 319.419C318.553 322.851 314.552 325.902 309.979 328.381C305.407 330.861 300.834 332.958 296.262 334.484C291.689 336.009 287.116 337.154 282.925 337.726C278.734 338.488 274.923 338.679 271.875 338.679C261.396 338.679 251.489 336.963 242.344 333.53C233.008 330.098 225.006 325.14 218.338 318.656C211.479 312.363 206.145 304.544 202.144 295.581C198.143 286.619 196.238 276.512 196.238 265.642C196.238 254.581 198.143 244.665 202.144 235.702C206.145 226.74 211.479 219.112 218.338 212.628C225.197 206.335 233.199 201.377 242.344 197.754C251.679 194.321 261.587 192.605 271.875 192.605C280.829 192.605 289.784 194.321 298.738 197.754C307.693 201.186 315.695 206.526 322.363 213.772L298.357 237.8Z" | ||
| fill="black" | ||
| /> | ||
| </svg> | ||
| ); |
There was a problem hiding this comment.
CerebrasIcon: check dark-mode contrast (black fill).
The logo’s last path uses fill="black" (Line 1102), unlike other icons that adapt via currentColor + dark:text-white. If the playground uses this on dark backgrounds, it may be invisible/low-contrast.
🤖 Prompt for AI Agents
In apps/playground/src/components/ui/providers-icons.tsx around lines 1063 to
1105, the final <path> uses a hardcoded fill="black" which breaks dark-mode
contrast; change that fill to use an inheritable color (e.g.,
fill="currentColor") so the icon follows surrounding text color/dark-mode
utilities, and ensure the component still spreads props (className/style) so
consumers can control color via CSS or Tailwind (no other path fills need
change).
| { | ||
| // Cerebras: FP16/FP8 (weights only) | ||
| providerId: "cerebras", | ||
| modelName: "zai-glm-4.6", | ||
| inputPrice: 2.25 / 1e6, | ||
| outputPrice: 2.75 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 200000, | ||
| maxOutput: undefined, | ||
| streaming: true, | ||
| reasoning: true, | ||
| vision: false, | ||
| tools: true, | ||
| jsonOutput: true, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Cerebras GLM-4.6 model API identifier zai-glm-4.6 pricing OpenAI compatible
💡 Result:
Short answer:
- API model identifier: zai-glm-4.6. [1]
- OpenAI-compatible (Cerebras exposes an OpenAI‑compatible endpoint and examples using model="zai-glm-4.6"). [2]
- Pricing varies by host: Cerebras offers pay-as-you-go/dev tiers and subscription Code plans (examples: $10 starter, $50 Code Pro, $200 Code Max). Third‑party hosts list per‑million token rates (examples: GMI Cloud shows ZAI GLM‑4.6 input $0.60 /M tokens, output $2.00 /M tokens). Confirm current rates with your chosen provider. [4][3]
Sources:
[1] Cerebras supported models (model id zai-glm-4.6).
[2] Cerebras OpenAI‑compatibility docs (example OpenAI calls using model="zai-glm-4.6").
[3] GMI Cloud pricing page (ZAI: GLM‑4.6 per‑token rates).
[4] Cerebras blog / product pages (GLM‑4.6 availability and Cerebras subscription pricing).
Remove inline comment. The modelName "zai-glm-4.6" is correct per Cerebras documentation, but the // Cerebras: FP16/FP8 (weights only) comment violates the "No unnecessary code comments" guideline—remove it.
🤖 Prompt for AI Agents
In packages/models/src/models/zai.ts around lines 186 to 200, remove the inline
comment "// Cerebras: FP16/FP8 (weights only)" that precedes the model entry;
keep the object fields unchanged and ensure only the model object remains (no
extra comments) to comply with the "No unnecessary code comments" guideline.
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/models/src/prepare-request-body.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Never useanyoras anyunless absolutely necessary in TypeScript code
For database reads: Usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
packages/models/src/prepare-request-body.ts
**/*.{ts,tsx,js,jsx,json,md}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use tabs for indentation
Files:
packages/models/src/prepare-request-body.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx,js,jsx}: Always use top-levelimport, never use require or dynamic imports
No unnecessary code comments
Files:
packages/models/src/prepare-request-body.ts
**/*.{js,ts,tsx,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level
import, never use require or dynamic imports
Files:
packages/models/src/prepare-request-body.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: lint / run
- GitHub Check: autofix
- GitHub Check: generate / run
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (5)
🔇 Additional comments (1)
packages/models/src/prepare-request-body.ts (1)
785-789: LGTM!Stream options handling follows the established pattern used by other providers.
| if (response_format) { | ||
| // Cerebras requires strict: true for json_schema mode | ||
| if (response_format.type === "json_schema") { | ||
| requestBody.response_format = { | ||
| ...response_format, | ||
| json_schema: { | ||
| ...response_format.json_schema, | ||
| strict: true, | ||
| }, | ||
| }; | ||
| } else { | ||
| requestBody.response_format = response_format; | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's examine the prepare-request-body.ts file to understand the context
wc -l packages/models/src/prepare-request-body.tsRepository: theopenco/llmgateway
Length of output: 111
🏁 Script executed:
# Look at the response_format handling around lines 694 (Google provider) and 790-803 (Cerebras)
sed -n '690,720p' packages/models/src/prepare-request-body.tsRepository: theopenco/llmgateway
Length of output: 1163
🏁 Script executed:
# Get the Cerebras section in full context
sed -n '780,810p' packages/models/src/prepare-request-body.tsRepository: theopenco/llmgateway
Length of output: 830
🏁 Script executed:
# Search for type definitions of response_format in this file
grep -n "response_format" packages/models/src/prepare-request-body.ts | head -20Repository: theopenco/llmgateway
Length of output: 998
🏁 Script executed:
# Look for imports and type definitions at the top of the file
head -80 packages/models/src/prepare-request-body.tsRepository: theopenco/llmgateway
Length of output: 2173
🏁 Script executed:
# Search for the actual type definition of response_format
fd -e ts -e tsx | xargs grep -l "type.*response_format" 2>/dev/null | head -5Repository: theopenco/llmgateway
Length of output: 46
🏁 Script executed:
# Find the OpenAIRequestBody type definition
fd -e ts -e tsx | xargs grep -l "OpenAIRequestBody" | head -10Repository: theopenco/llmgateway
Length of output: 171
🏁 Script executed:
# Look at types.ts file
fd types.tsRepository: theopenco/llmgateway
Length of output: 281
🏁 Script executed:
# Read the types file to find response_format type definition
cat packages/models/src/types.ts | head -200Repository: theopenco/llmgateway
Length of output: 3815
🏁 Script executed:
# Search specifically for response_format type definition
grep -n "response_format" packages/models/src/types.ts | head -20Repository: theopenco/llmgateway
Length of output: 149
🏁 Script executed:
# Search for Cerebras-related type or API documentation in comments
grep -rn "Cerebras" packages/models/src/ --include="*.ts" --include="*.tsx"Repository: theopenco/llmgateway
Length of output: 762
🏁 Script executed:
# Look for test files that test cerebras response_format
fd -e test.ts -e spec.ts -e test.tsx | xargs grep -l "cerebras" 2>/dev/nullRepository: theopenco/llmgateway
Length of output: 48
🏁 Script executed:
# Search for any comments or documentation about strict: true
grep -rn "strict.*true" packages/models/src/ --include="*.ts" --include="*.tsx"Repository: theopenco/llmgateway
Length of output: 239
🏁 Script executed:
# Check the test file for cerebras response_format handling
grep -A 20 -B 5 "cerebras" packages/models/src/models.spec.ts | grep -A 20 -B 5 "response_format"Repository: theopenco/llmgateway
Length of output: 46
🏁 Script executed:
# Look for any validation logic that might ensure json_schema exists before this code
grep -B 30 "case \"cerebras\"" packages/models/src/prepare-request-body.ts | tail -40Repository: theopenco/llmgateway
Length of output: 814
🏁 Script executed:
# Check if there's validation in the function signature or earlier checks
sed -n '100,160p' packages/models/src/prepare-request-body.tsRepository: theopenco/llmgateway
Length of output: 1809
🏁 Script executed:
# Look at how response_format is used in other providers to see the pattern
sed -n '250,260p' packages/models/src/prepare-request-body.tsRepository: theopenco/llmgateway
Length of output: 333
🏁 Script executed:
# Check the anthropic provider handling of response_format
sed -n '320,345p' packages/models/src/prepare-request-body.tsRepository: theopenco/llmgateway
Length of output: 709
🏁 Script executed:
# Search for callers of prepareRequestBody to see if json_schema is always provided
grep -rn "prepareRequestBody" packages/ --include="*.ts" --include="*.tsx" | grep -v "export" | grep -v "function prepareRequestBody" | head -10Repository: theopenco/llmgateway
Length of output: 907
🏁 Script executed:
# Check if there's validation that ensures json_schema when type === "json_schema"
grep -B 10 "prepareRequestBody" apps/ --include="*.ts" --include="*.tsx" -r | head -40Repository: theopenco/llmgateway
Length of output: 1710
🏁 Script executed:
# Look for where response_format is constructed to understand if json_schema is always present
grep -rn "json_schema" packages/models/src/ --include="*.ts" -B 2 -A 2 | head -50Repository: theopenco/llmgateway
Length of output: 2497
Add defensive check for json_schema field.
The code accesses response_format.json_schema without checking if it exists. The TypeScript type definition allows json_schema to be optional even when type === "json_schema", which could cause a runtime error if the field is undefined. Add a safety check similar to the Google provider implementation (line 694) which uses optional chaining.
Apply this diff:
if (response_format) {
// Cerebras requires strict: true for json_schema mode
- if (response_format.type === "json_schema") {
+ if (response_format.type === "json_schema" && response_format.json_schema) {
requestBody.response_format = {
...response_format,
json_schema: {
...response_format.json_schema,
strict: true,
},
};
} else {
requestBody.response_format = response_format;
}
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (response_format) { | |
| // Cerebras requires strict: true for json_schema mode | |
| if (response_format.type === "json_schema") { | |
| requestBody.response_format = { | |
| ...response_format, | |
| json_schema: { | |
| ...response_format.json_schema, | |
| strict: true, | |
| }, | |
| }; | |
| } else { | |
| requestBody.response_format = response_format; | |
| } | |
| } | |
| if (response_format) { | |
| // Cerebras requires strict: true for json_schema mode | |
| if (response_format.type === "json_schema" && response_format.json_schema) { | |
| requestBody.response_format = { | |
| ...response_format, | |
| json_schema: { | |
| ...response_format.json_schema, | |
| strict: true, | |
| }, | |
| }; | |
| } else { | |
| requestBody.response_format = response_format; | |
| } | |
| } |
🤖 Prompt for AI Agents
In packages/models/src/prepare-request-body.ts around lines 790 to 803, the code
spreads response_format.json_schema without ensuring it exists which can throw
if json_schema is undefined; change the merge to safely handle a possibly
missing json_schema (e.g., use optional chaining/nullish coalescing so you
spread ...(response_format.json_schema ?? {}) or response_format.json_schema?.
and then add strict: true) and keep the rest of the response_format fields
unchanged so the json_schema merge never errors at runtime.
| if (temperature !== undefined) { | ||
| requestBody.temperature = temperature; | ||
| } | ||
| if (max_tokens !== undefined) { | ||
| requestBody.max_tokens = max_tokens; | ||
| } | ||
| if (top_p !== undefined) { | ||
| requestBody.top_p = top_p; | ||
| } | ||
| if (frequency_penalty !== undefined) { | ||
| requestBody.frequency_penalty = frequency_penalty; | ||
| } | ||
| if (presence_penalty !== undefined) { | ||
| requestBody.presence_penalty = presence_penalty; | ||
| } | ||
| if (reasoning_effort !== undefined) { | ||
| requestBody.reasoning_effort = reasoning_effort; | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Cerebras AI API chat completions parameters documentation
💡 Result:
Here are the main chat-completions request parameters from Cerebras’ API docs (concise summary). Sources: [1][2][3].
Required
- model — model name (string). [1]
- messages — array of message objects ({role, content}). [1]
Generation / sampling
- max_completion_tokens — max tokens the model may generate (counts toward context). [1]
- temperature — sampling temperature (0–1.5). [2]
- topP (top_p) — nucleus sampling parameter. [2]
- stop — up to 4 stop sequences. [2]
Output options / diagnostics
- logprobs — return token log-probabilities (bool). [1]
- echo — echo prompt back with completion (bool). [2]
- user — opaque end‑user id for abuse monitoring (string). [1]
Tools / function-calling
- tools — list of tools/functions (name, description, parameters) the model may call. [1]
- parallel_tool_calls — enable parallel tool calls (bool, default true). [1]
Prediction / performance
- prediction — predicted-output config to speed responses (object). [1]
- seed — deterministic sampling seed (integer). [2]
Other / compatibility notes
- Some OpenAI fields are unsupported (e.g., frequency_penalty, logit_bias, presence_penalty). Streaming has caveats with reasoning/tool JSON mode. [2]
If you want, I can paste the exact JSON schema and example request snippets from the docs.
Sources:
- Cerebras Inference — Chat Completions API reference. [https://inference-docs.cerebras.ai/api-reference/chat-completions]
- Cerebras Inference — OpenAI compatibility & parameter notes. [https://inference-docs.cerebras.ai/resources/openai]
- Cerebras Inference — Quickstart (examples). [https://inference-docs.cerebras.ai/quickstart]
Remove unsupported parameters for Cerebras API.
The Cerebras API does not support frequency_penalty, presence_penalty, or reasoning_effort. These parameters should not be propagated to the request body for the Cerebras case, as they will cause API errors or be silently ignored. Additionally, use max_completion_tokens instead of max_tokens for Cerebras compatibility.
🤖 Prompt for AI Agents
In packages/models/src/prepare-request-body.ts around lines 806 to 823, the
Cerebras branch is currently adding unsupported fields (frequency_penalty,
presence_penalty, reasoning_effort) and using max_tokens; remove propagation of
frequency_penalty, presence_penalty, and reasoning_effort when building the
requestBody for Cerebras, and replace max_tokens with max_completion_tokens for
the Cerebras case (keep other providers unchanged). Ensure these changes are
applied only in the Cerebras-specific conditional path so other providers still
receive their appropriate parameters.
fefdacc to
f50812d
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/models/src/testing.ts (1)
14-25: Consider preservingskipeven when applying retry/timeout.
Right nowskipis only returned in the: { skip: ... }branch, so callers with defaultcompletions: truewill never seeskipeven ifhasTestOnly/TEST_MODELSis set. If the intent is “apply retry/timeout and still skip when requested”, you can includeskipin both branches.const hasTestOnly = models.some((model) => model.providers.some( (provider: ProviderModelMapping) => provider.test === "only", ), ); - return process.env.CI || opts?.completions + const skip = hasTestOnly || !!process.env.TEST_MODELS; + return process.env.CI || opts?.completions ? { + skip, retry: process.env.CI ? 3 : 0, timeout: process.env.CI ? 180000 : 60000, } - : { skip: hasTestOnly || !!process.env.TEST_MODELS }; + : { skip }; }
♻️ Duplicate comments (3)
packages/models/src/providers.ts (1)
135-150: Tweak provider description grammar for consistency.
This reads a bit off compared to “Groq’s …”. Consider “Cerebras’s …” or “Cerebras provides …”.packages/models/src/models/zai.ts (1)
186-200: Remove the inline comment to match “no unnecessary code comments” guideline.
Keep the object as-is, but drop// Cerebras: FP16/FP8 (weights only). Also worth re-confirmingmodelName: "zai-glm-4.6"is still the correct Cerebras identifier.Cerebras supported model identifier for Zhipu GLM-4.6 in their OpenAI-compatible API (is it "zai-glm-4.6")?packages/models/src/prepare-request-body.ts (1)
784-825: Fix potential runtime error whenresponse_format.json_schemais missing.
...response_format.json_schemacan throw ifjson_schemais undefined.if (response_format) { // Cerebras requires strict: true for json_schema mode - if (response_format.type === "json_schema") { + if (response_format.type === "json_schema" && response_format.json_schema) { requestBody.response_format = { ...response_format, json_schema: { ...response_format.json_schema, strict: true, }, }; } else { requestBody.response_format = response_format; } }Also please re-check Cerebras’ current OpenAI-compat parameter support (e.g., whether
max_tokensvsmax_completion_tokensis expected, and whetherfrequency_penalty/presence_penalty/reasoning_effortare accepted).Cerebras Inference OpenAI-compatible chat completions: supported request parameters (max_tokens vs max_completion_tokens; frequency_penalty; presence_penalty; reasoning_effort).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (13)
.github/workflows/e2e.yml(1 hunks)apps/admin/src/components/ui/providers-icons.tsx(2 hunks)apps/playground/src/components/provider-icons.tsx(2 hunks)apps/playground/src/components/ui/providers-icons.tsx(2 hunks)apps/ui/src/lib/components/providers-icons.tsx(2 hunks)packages/models/src/get-provider-endpoint.ts(2 hunks)packages/models/src/models/alibaba.ts(2 hunks)packages/models/src/models/meta.ts(2 hunks)packages/models/src/models/openai.ts(1 hunks)packages/models/src/models/zai.ts(1 hunks)packages/models/src/prepare-request-body.ts(1 hunks)packages/models/src/providers.ts(1 hunks)packages/models/src/testing.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
- packages/models/src/models/openai.ts
- apps/ui/src/lib/components/providers-icons.tsx
- apps/playground/src/components/ui/providers-icons.tsx
- apps/admin/src/components/ui/providers-icons.tsx
- apps/playground/src/components/provider-icons.tsx
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Never useanyoras anyunless absolutely necessary in TypeScript code
For database reads: Usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
packages/models/src/get-provider-endpoint.tspackages/models/src/models/zai.tspackages/models/src/providers.tspackages/models/src/testing.tspackages/models/src/models/alibaba.tspackages/models/src/prepare-request-body.tspackages/models/src/models/meta.ts
**/*.{ts,tsx,js,jsx,json,md}
📄 CodeRabbit inference engine (CLAUDE.md)
Always use tabs for indentation
Files:
packages/models/src/get-provider-endpoint.tspackages/models/src/models/zai.tspackages/models/src/providers.tspackages/models/src/testing.tspackages/models/src/models/alibaba.tspackages/models/src/prepare-request-body.tspackages/models/src/models/meta.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx,js,jsx}: Always use top-levelimport, never use require or dynamic imports
No unnecessary code comments
Files:
packages/models/src/get-provider-endpoint.tspackages/models/src/models/zai.tspackages/models/src/providers.tspackages/models/src/testing.tspackages/models/src/models/alibaba.tspackages/models/src/prepare-request-body.tspackages/models/src/models/meta.ts
**/*.{js,ts,tsx,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level
import, never use require or dynamic imports
Files:
packages/models/src/get-provider-endpoint.tspackages/models/src/models/zai.tspackages/models/src/providers.tspackages/models/src/testing.tspackages/models/src/models/alibaba.tspackages/models/src/prepare-request-body.tspackages/models/src/models/meta.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: generate / run
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: lint / run
- GitHub Check: e2e-shards (4)
- GitHub Check: autofix
- GitHub Check: e2e-shards (5)
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (1)
🔇 Additional comments (3)
.github/workflows/e2e.yml (1)
52-80: E2E secret wiring looks fine; ensure the repo secret exists.
LLM_CEREBRAS_API_KEYis correctly exposed to the shard run; just double-checksecrets.CEREBRAS_API_KEYis set in the repo/org secrets so CI doesn’t fail on Cerebras tests.packages/models/src/get-provider-endpoint.ts (1)
77-79: No action needed — Cerebras endpoint is correctly implemented.The base URL
https://api.cerebras.aicombined with the standard/v1/chat/completionspath produces the correct OpenAI-compatible endpoint. Cerebras does not require additional path prefixes like Groq's/openai.packages/models/src/models/meta.ts (1)
196-209: All Cerebras model details are accurate per official documentation.Web search confirms the entry is correct:
- Model name
llama-3.3-70b✓- Pricing: $0.85/M input, $1.20/M output tokens ✓
- Context size: 128k tokens (paid tier) ✓
- Capabilities: streaming, tool calling, and structured outputs supported ✓
- Vision: correctly set to false
No changes needed.
| { | ||
| // Cerebras: FP16/FP8 (weights only) | ||
| providerId: "cerebras", | ||
| modelName: "qwen-3-235b-a22b-instruct-2507", | ||
| inputPrice: 0.6 / 1e6, | ||
| outputPrice: 1.2 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 262000, | ||
| maxOutput: 8192, | ||
| reasoning: false, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: true, | ||
| jsonOutput: true, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Cerebras qwen-3-235b-a22b-instruct pricing documentation
💡 Result:
Cerebras pricing for Qwen model (model ID qwen-3-235b-a22b-instruct-2507):
- Input: $0.60 per 1M tokens. [1]
- Output: $1.20 per 1M tokens. [1]
- Context (paid tier): 131K tokens (free tier: 64–65K; max output paid tier 40K). [2]
- Model name to use: qwen-3-235b-a22b-instruct-2507 (non-thinking mode). [2]
Sources: Cerebras blog announcement and Cerebras Inference docs. [1][2]
References
[1] Cerebras blog — "Qwen3 235B 2507 Instruct Now Available on Cerebras."
[2] Cerebras Inference documentation — "Qwen 3 235B Instruct" model card.
Correct context size and max output for Cerebras Qwen model.
The model name and pricing are accurate, but context size and max output are incorrect:
- Context size should be 131,072 tokens (not 262,000)
- Max output should be 40,000 tokens (not 8,192)
Update lines 239 and 241 accordingly.
🤖 Prompt for AI Agents
In packages/models/src/models/alibaba.ts around lines 236 to 250, the Cerebras
Qwen model entry has incorrect contextSize and maxOutput values; update the
contextSize on line ~239 to 131072 and the maxOutput on line ~241 to 40000 so
the entry reads the correct context size (131,072 tokens) and max output (40,000
tokens).
| { | ||
| // Cerebras: FP16 | ||
| providerId: "cerebras", | ||
| modelName: "qwen-3-32b", | ||
| inputPrice: 0.4 / 1e6, | ||
| outputPrice: 0.8 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 32768, | ||
| maxOutput: 8192, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: true, | ||
| jsonOutput: true, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Cerebras qwen-3-32b model pricing capabilities documentation
💡 Result:
Summary for Cerebras qwen-3-32b (qwen-3-32b)
Pricing
- $0.40 per million input tokens; $0.80 per million output tokens. Developers receive 1M free tokens/day on the free tier; volume/enterprise pricing and monthly growth tiers available. [1][2]
Capabilities / model card highlights
- Reasoning model (default reasoning mode).
- Streaming, structured outputs, tool-calling supported.
- Context length: free tier ~65K (paid up to ~131K per docs). Max output ~8K (paid/free tiers).
- Endpoints: OpenAI/Claude-compatible Chat Completions and Completions.
- Typical throughput/speed: ~2,400–2,600 tokens/sec on Cerebras infrastructure; first-token latency ≈1–1.2s.
- Recommended decoding when using reasoning: temperature ~0.6, top_p ~0.95; append “/no_think” to disable reasoning mode for specific prompts. [2][3]
Rate limits / tiers (examples)
- Free: 30 req/min, 60k input tokens/min, 1M daily tokens.
- Developer / Growth tiers: higher RPM and token budgets (example growth-tier monthly prices and rate limits available in docs). For large-volume or custom SLA needs, contact sales for Enterprise quotes. [2]
Sources
- Cerebras inference pricing & model listing (Qwen 3 32B pricing & token rates). [inference docs & pricing]. [2]
- Qwen-3-32B model page (capabilities, context lengths, endpoints, rate limits). [3]
- Cerebras press/blog announcement describing speed, latency and pricing. [1]
Would you like direct links to any of those pages or a copy of the model card text?
Correct the context size for the Cerebras qwen-3-32b model.
The model name, pricing ($0.4/M input, $0.8/M output), and capabilities (streaming, tools, JSON output support, 8192 max output) are verified against Cerebras documentation. However, the contextSize is incorrect: Cerebras documentation indicates the free tier supports ~65K tokens, with paid tiers supporting up to ~131K tokens. The current value of 32768 (32K) significantly understates the actual context window. Update contextSize accordingly based on the intended tier.
🤖 Prompt for AI Agents
In packages/models/src/models/alibaba.ts around lines 323 to 336, the Cerebras
qwen-3-32b entry incorrectly sets contextSize to 32768; update contextSize to
match Cerebras docs (use 65536 for the free-tier ~65K window or 131072 for the
paid-tier ~131K window) — replace the 32768 literal with the appropriate integer
for the intended tier and ensure any surrounding comments/metadata reflect the
chosen tier.
| { | ||
| // Cerebras: FP16 | ||
| providerId: "cerebras", | ||
| modelName: "llama3.1-8b", | ||
| inputPrice: 0.1 / 1e6, | ||
| outputPrice: 0.1 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 128000, | ||
| maxOutput: undefined, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: true, | ||
| jsonOutput: true, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Cerebras llama 3.1 8b model pricing documentation
💡 Result:
Cerebras Inference lists Llama 3.1 8B pricing as $0.10 per million input tokens and $0.10 per million output tokens (exploration/developer tier); the model card and pricing page include rate limits and speed (~2,200 tokens/s). (inference-docs.cerebras.ai)
Citations:
Correct model name format to match Cerebras documentation: use llama-31-8b instead of llama3.1-8b.
The official Cerebras documentation identifies this model as llama-31-8b (with dashes). Update the modelName to match. The pricing ($0.1/M tokens for input and output) and context size (128k tokens) are confirmed as accurate.
🤖 Prompt for AI Agents
In packages/models/src/models/meta.ts around lines 66 to 79, the Cerebras model
entry uses an incorrect modelName "llama3.1-8b"; update the modelName value to
the documented format "llama-31-8b" while leaving the rest of the object
(pricing, contextSize, flags) unchanged.
- Disable tool calling for llama3.1-8b due to unreliability. - Ensure strict mode is applied to tools in request body.
Summary by CodeRabbit
New Features
Chores
✏️ Tip: You can customize this high-level summary in your review settings.