feat(chat): enhance chat functionality with image configuration and reasoning effort options - #1211
Conversation
…easoning effort options - Added support for customizable image aspect ratios and sizes in chat messages. - Introduced reasoning effort selection to improve AI response quality. - Updated error handling to provide more informative messages. - Refactored model mapping to ensure accurate capabilities detection. This update enhances user experience by allowing more control over image generation and reasoning parameters during chat interactions.
WalkthroughUpdates playground dependency, adds image generation and reasoning fields to chat API and client/UI, introduces root (auto-routing) model entries and filters, centralizes error-message extraction, refactors model mapping, and updates navbar resources and layout. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant ChatPageClient
participant ChatAPI as /api/chat
participant Provider
participant MCP as GitHub_MCP
User->>ChatPageClient: select image/reasoning options + send message
ChatPageClient->>ChatPageClient: build body with image_config & reasoning_effort
ChatPageClient->>ChatAPI: POST /api/chat (body includes image_config, reasoning_effort)
activate ChatAPI
ChatAPI->>ChatAPI: extract image_config & reasoning_effort
alt GitHub MCP path
ChatAPI->>MCP: call MCP with providerOptions { image_config, reasoning_effort }
MCP-->>ChatAPI: stream response
else Default provider path
ChatAPI->>Provider: stream with providerOptions { image_config, reasoning_effort }
Provider-->>ChatAPI: stream response
end
ChatAPI-->>ChatPageClient: stream messages (including image blocks)
deactivate ChatAPI
ChatPageClient->>User: render messages, images, reasoning UI
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
🔇 Additional comments (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (10)
apps/playground/src/components/ai-elements/reasoning.tsx (1)
129-136: Minor copy nit: handle singular “second” when duration === 1Right now
duration = 1produces “Thought for 1 seconds”. Consider a tiny tweak for nicer UX:} else { const unit = duration === 1 ? "second" : "seconds"; return <p>Thought for {duration} {unit}</p>; }apps/playground/src/lib/mapmodels.ts (1)
10-31: Tidy up root mapping: unused data and outdated commentFunctionally this mapping looks fine, but a few cleanups would reduce confusion:
Outdated comment vs. implementation (Lines 34‑43, 45)
The comment discusses possibly keepingproviderId/p.modelName, but theidis now${p.providerId}/${m.id}. That mismatch is likely to confuse future readers. I’d update the comment to describe the current contract (“IDs are providerId/m.id to match ModelSelector”) or remove the speculative bits.Unused
providerInfoinrootProviders(Lines 11‑14, 17‑18)
rootProvidersspreadsproviderInfo, but onlyp.visionandp.toolsare read. You can simplify to just usem.providersdirectly, e.g.:const hasVision = m.providers.some((p) => p.vision); const hasTools = m.providers.some((p) => p.tools);That avoids an extra map and unused property.
Reuse
hasImageGenfor provider entries (Lines 17‑20, 55)
SincehasImageGenalready capturesm.output?.includes("image"), you can reuse it in the per‑provider entries instead of repeating the expression:const hasImageGen = m.output?.includes("image"); // later imageGen: hasImageGen,These are non‑blocking but will make the function easier to maintain.
Also applies to: 32-56
apps/playground/src/components/playground/chat-ui.tsx (2)
350-374: Image rendering height is fixed at 400px; consider making it responsiveThe new image class
className="h-[400px] aspect-auto border rounded-lg object-cover"gives a nice uniform grid but forces every image to 400px height, which may be tall on small screens and can over‑crop very wide images with
object-cover. If you want a bit more responsiveness, consider using a max height instead:className="max-h-[400px] w-full border rounded-lg object-cover"so large images are constrained but smaller ones aren’t upscaled as aggressively.
The added streaming loader fallback for the last assistant message when there are no images (
isLastMessage && status === "streaming") is a nice UX touch.
72-88: Verification confirms: values are correctly wired into requestsThe implementation properly handles the new props:
- Line 581 passes
sendMessageWithHeadersto ChatUI as thesendMessageprop- The wrapper (lines 220–253) correctly merges
reasoningEffortasreasoning_effortandimageConfiginto the request body- Dependencies are tracked in the callback's dependency array
- Combined with the ai SDK's documented behavior of merging request-level body fields with hook-level configuration, the values will reach the API call correctly.
The literal union suggestion (duplicated in props type and casts) remains valid but optional—centralizing
type ReasoningEffort = "" | "minimal" | ...would improve maintainability without being essential.apps/ui/src/components/landing/navbar.tsx (1)
84-91: Align mobile Resources menu with Next.js 15 best practices for external andmailto:linksThe mobile Resources menu (lines 287-297) currently renders all
resourcesItemsthrough Next's<Link>component, including the external Docs URL and themailto:Contact Us link. This violates Next.js 15 best practices and creates UX/type-safety issues:
- External URLs and
mailto:links should use regular anchor tags, notLink- Docs (external) will behave as an internal route unlike the desktop version
mailto:forced throughLinkconflicts with typed routesUpdate the mobile rendering to conditionally use
<a>for external entries andmailto:links, matching desktop behavior:{resourcesItems.map((item, index) => ( <li key={index}> {item.external || item.href.startsWith("mailto:") ? ( <a href={item.href} target={item.href.startsWith("http") ? "_blank" : undefined} rel={item.href.startsWith("http") ? "noopener noreferrer" : undefined} className="text-muted-foreground hover:text-accent-foreground block duration-150" > {item.name} </a> ) : ( <Link href={item.href as Route} className="text-muted-foreground hover:text-accent-foreground block duration-150" prefetch={true} > {item.name} </Link> )} </li> ))}Also consider adding
external: trueto the Contact Usmailto:entry for consistency.apps/playground/src/lib/utils.ts (1)
8-53: Centralized error normalization is solid; consider one more fallback pathThe branching covers the common shapes (string, Zod/OpenAPI error envelope, and plain
{ message }), and it’s safe from runtime exceptions. To make it a bit more robust, you could also look for amessageon the candidate objects you iterate (e.g.,candidate.message) before falling back to the rooterror.message, so cases like{ error: { message: "..." } }withoutsuccess === falseare handled as well.apps/playground/src/app/api/chat/route.ts (2)
18-38: AlignChatRequestBodytyping with actual payload (reasoning_effort)You’re reading
reasoning_effortfrom the body viaconst reasoningEffort = (body as any)?.reasoning_effort, but it isn’t declared onChatRequestBody. Typing it there will keep the route and callers in sync and remove theanyescape hatch.For example:
interface ChatRequestBody { messages: UIMessage[]; model?: LLMGatewayChatModelId; apiKey?: string; provider?: string; // optional provider override mode?: "image" | "chat"; // optional hint to force image generation path image_config?: { aspect_ratio?: | "auto" | "1:1" | "9:16" | "3:4" | "4:3" | "3:2" | "2:3" | "5:4" | "4:5" | "21:9"; image_size?: "1K" | "2K" | "4K"; }; + reasoning_effort?: "minimal" | "low" | "medium" | "high"; } @@ - const { messages, model, apiKey, provider, image_config }: ChatRequestBody = - body; + const { + messages, + model, + apiKey, + provider, + image_config, + reasoning_effort, + }: ChatRequestBody = body; @@ - const reasoningEffort = (body as any)?.reasoning_effort || undefined; + const reasoningEffort = reasoning_effort;Also applies to: 49-52, 63-64
97-121: Reasoning/image providerOptions wiring is consistent; minor optional tighteningThe way
reasoningEffortandimage_configare threaded intoproviderOptionsfor both the MCP and default paths is coherent, and the conditional spread in the default path avoids sending providerOptions when nothing is set.If you want symmetry and slightly cleaner payloads, you could mirror that pattern in the MCP branch so
providerOptionsis only sent when at least one ofreasoningEffortorimage_configis defined (and omitreasoning_effortwhen undefined), but the current logic is functionally fine.Also applies to: 128-144
apps/playground/src/components/playground/chat-page-client.tsx (1)
6-7: Error handling and new‑chat cleanup flow are well thought out, with a tiny state nitUsing
getErrorMessageplustoast.erroracrossuseChat’sonError, the image/tool JSON parse blocks, andhandleUserMessagegives you consistent, user-friendly errors. The logic to delete a just-created chat when the first response or first user message fails (usingisNewChatRefandchatIdRef) is also a good safeguard against orphaned chats.One minor nit: if
ensureCurrentChatthrows before a chat is created,isNewChatRef.currentstaystruefromhandleUserMessage, even though there is no active chat. It doesn’t break anything because all cleanup paths also checkchatIdRef.current, but you could reset it in that catch to keep the ref strictly accurate.Also applies to: 21-22, 26-27, 102-126, 315-328, 386-410, 416-448
apps/playground/src/components/model-selector.tsx (1)
60-66: Coerce checkboxcheckedstate to boolean when updating filters
FilterState.hideUnstableandFilterState.showOnlyRootare declared as booleans, butonCheckedChangefrom the Checkbox component typically receives aCheckedState(boolean | "indeterminate"). Passingcheckedthrough directly intoupdateFiltermeans those fields can end up with"indeterminate"at runtime.It would be a small type/semantics improvement to coerce explicitly, e.g.:
<Checkbox id="show-root" checked={filters.showOnlyRoot} - onCheckedChange={(checked) => - updateFilter("showOnlyRoot", checked) - } + onCheckedChange={(checked) => + updateFilter("showOnlyRoot", checked === true) + } /> @@ <Checkbox id="hide-unstable" checked={filters.hideUnstable} - onCheckedChange={(checked) => - updateFilter("hideUnstable", checked) - } + onCheckedChange={(checked) => + updateFilter("hideUnstable", checked === true) + } />This keeps
FilterStatestrictly boolean while preserving current UX.Also applies to: 295-307, 319-327, 503-516, 644-649
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (10)
apps/playground/package.json(1 hunks)apps/playground/src/app/api/chat/route.ts(5 hunks)apps/playground/src/components/ai-elements/reasoning.tsx(1 hunks)apps/playground/src/components/model-selector.tsx(19 hunks)apps/playground/src/components/playground/chat-page-client.tsx(13 hunks)apps/playground/src/components/playground/chat-ui.tsx(6 hunks)apps/playground/src/hooks/useChats.ts(5 hunks)apps/playground/src/lib/mapmodels.ts(1 hunks)apps/playground/src/lib/utils.ts(1 hunks)apps/ui/src/components/landing/navbar.tsx(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (7)
apps/playground/src/lib/utils.ts (1)
packages/logger/src/index.ts (1)
error(153-160)
apps/playground/src/lib/mapmodels.ts (1)
packages/models/src/providers.ts (1)
providers(21-302)
apps/playground/src/app/api/chat/route.ts (1)
packages/db/src/schema.ts (1)
message(538-560)
apps/playground/src/hooks/useChats.ts (1)
apps/playground/src/lib/utils.ts (1)
getErrorMessage(8-54)
apps/playground/src/components/model-selector.tsx (5)
packages/models/src/models.ts (2)
ProviderModelMapping(51-158)ModelDefinition(162-214)packages/models/src/providers.ts (2)
ProviderDefinition(1-19)providers(21-302)apps/ui/src/lib/model-utils.ts (3)
getProviderForModel(29-35)formatPrice(3-14)formatContextSize(16-27)apps/playground/src/lib/utils.ts (1)
cn(4-6)apps/ui/src/lib/components/providers-icons.tsx (1)
getProviderIcon(1171-1184)
apps/playground/src/components/playground/chat-ui.tsx (2)
apps/playground/src/components/ui/image-zoom.tsx (1)
ImageZoom(16-52)apps/playground/src/components/ai-elements/loader.tsx (1)
Loader(87-97)
apps/playground/src/components/playground/chat-page-client.tsx (2)
apps/playground/src/lib/utils.ts (1)
getErrorMessage(8-54)apps/playground/src/hooks/useChats.ts (3)
useCreateChat(50-64)useAddMessage(98-111)useDeleteChat(82-96)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
- GitHub Check: build-split (docs, linux/amd64)
- GitHub Check: build-split (ui, linux/amd64)
- GitHub Check: build-split (worker, linux/amd64)
- GitHub Check: build-split (playground, linux/amd64)
- GitHub Check: build-split (gateway, linux/amd64)
- GitHub Check: build-split (api, linux/amd64)
- GitHub Check: build-unified (linux/amd64)
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: generate / run
- GitHub Check: lint / run
- GitHub Check: autofix
🔇 Additional comments (6)
apps/playground/package.json (1)
18-24: Confirm @llmgateway/ai-sdk-provider 2.4.0 compatibilityThe version bump looks reasonable, but please double‑check that existing usages of the provider (chat route, streaming, providerOptions, image/reasoning options) match any API changes in 2.4.0 (e.g., required fields, option names).
apps/ui/src/components/landing/navbar.tsx (1)
147-150: Navbar container width/padding change looks reasonableThe updated base container (
max-w-7xl px-6) plus tighter scrolled state (max-w-6xl lg:px-5) should be fine; just sanity‑check in design review that the width jump on scroll is the desired effect and doesn’t feel jittery.apps/playground/src/components/playground/chat-ui.tsx (1)
429-429: Footer container shrink behavior change is fineSwitching from
flex-shrink-0toshrink-0keeps the prompt area fixed-height at the bottom in modern Tailwind and should behave identically; no issues here.apps/playground/src/hooks/useChats.ts (1)
5-6: Consistent error handling viagetErrorMessagelooks goodUsing
toast.error(getErrorMessage(error))across all mutations gives you uniform, more resilient error messaging regardless of the error shape, and relaxing the onError parameter type aligns with that utility.Also applies to: 54-63, 70-79, 86-95, 102-111
apps/playground/src/components/playground/chat-page-client.tsx (1)
64-81: Reasoning and image configuration wiring matches model capabilities nicelyThe new
reasoningEffort,imageAspectRatio, andimageSizestate, plus thesupportsImages,supportsImageGen, andsupportsReasoningselectors, correctly derive capabilities fromComboboxModel/ModelDefinitionand keep the UI in sync. TheuseEffectthat clearsreasoningEffortwhen switching to a non‑reasoning model is a good safety net.
sendMessageWithHeadersonly sendsreasoning_effortandimage_configwhen applicable and when image generation is supported, which keeps the API payload lean and avoids hitting models with unsupported options. Passing all of these props through toChatUIcompletes the circuit cleanly.Also applies to: 85-86, 186-201, 220-253, 493-498, 588-595
apps/playground/src/components/model-selector.tsx (1)
60-66: Root (auto‑routing) model support and preview UX are well integratedThe approach of modeling root entries via
isRootwithvalue = model.id, alongside provider‑specific entries withvalue = providerId/model.id, is consistent with howselectedModelId/selectedProviderIdare parsed. The filters and search use normalizedsearchText, root entries respectshowOnlyRootand stability rules, and the desktop preview + details dialog cleanly distinguish root (sparkles + auto‑select copy) from provider‑backed mappings, including pricing and capabilities.This aligns nicely with the rest of the model mapping/types and should make the root vs provider experience clear to users.
Also applies to: 128-140, 161-207, 337-378, 682-743, 825-974, 987-1041
- Simplified the construction of provider options by directly spreading reasoning effort and image configuration into the request. - Removed unnecessary nesting of provider options, enhancing code readability and maintainability. This change improves the clarity of the chat API's request structure while maintaining existing functionality.
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
apps/playground/src/app/api/chat/route.ts (1)
133-138: Do not return rawerrorobjects to clients; sanitize and guardstatusReturning
details: errorrisks leaking internal metadata (SDK response objects, config, etc.) and may break if the error carries non‑serializable fields. Usingerror.statusdirectly can also produce invalid HTTP codes.A safer pattern is to log the full error server‑side and expose only a minimal, sanitized shape:
- } catch (error: any) { - const message = error.message || "LLM Gateway request failed"; - const status = error.status || 500; - return new Response(JSON.stringify({ error: message, details: error }), { - status, - }); - } + } catch (error: any) { + console.error("LLM Gateway chat route error", error); + + const message = + typeof error?.message === "string" + ? error.message + : "LLM Gateway request failed"; + + const status = + typeof error?.status === "number" && + error.status >= 400 && + error.status <= 599 + ? error.status + : 500; + + const safeDetails = + process.env.NODE_ENV === "development" + ? { + code: error?.code, + status: error?.status, + type: error?.type, + } + : undefined; + + return new Response( + JSON.stringify({ + error: message, + ...(safeDetails && { details: safeDetails }), + }), + { status }, + ); + }This keeps logs useful while avoiding accidental data leakage in responses.
🧹 Nitpick comments (1)
apps/playground/src/app/api/chat/route.ts (1)
50-51: Avoidanyfor request body; typereasoning_effort(and githubToken) inChatRequestBodyRight now
image_configis typed, butreasoning_effort(andgithubToken) are pulled via(body as any). That undermines the benefit ofChatRequestBodyand makes field name typos harder to catch.Consider extending
ChatRequestBodywith these fields and reading them via destructuring instead ofany, e.g.:interface ChatRequestBody { messages: UIMessage[]; model?: LLMGatewayChatModelId; apiKey?: string; provider?: string; mode?: "image" | "chat"; image_config?: { /* ... */ }; githubToken?: string; reasoning_effort?: string; // or a narrower union if you have one } // ... const { messages, model, apiKey, provider, image_config, githubToken, reasoning_effort, }: ChatRequestBody = body; const tokenForMcp = githubTokenHeader || githubToken; const reasoningEffort = reasoning_effort;This keeps the server route aligned with the client payload and avoids silent shape drift.
Also applies to: 63-63
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
apps/playground/src/app/api/chat/route.ts(5 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
apps/playground/src/app/api/chat/route.ts (1)
packages/db/src/schema.ts (1)
message(538-560)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: build-split (playground, linux/amd64)
- GitHub Check: build-split (docs, linux/amd64)
- GitHub Check: build-split (api, linux/amd64)
- GitHub Check: build-split (ui, linux/amd64)
- GitHub Check: build-split (gateway, linux/amd64)
- GitHub Check: build-unified (linux/amd64)
- GitHub Check: build / run
- GitHub Check: test / run
- GitHub Check: lint / run
- GitHub Check: autofix
1 participant
This update enhances user experience by allowing more control over image generation and reasoning parameters during chat interactions.
Fixes
Summary by CodeRabbit
New Features
Chores
✏️ Tip: You can customize this high-level summary in your review settings.