feat(models): add metadata - #716
Conversation
- Add metadata object with requested_model, requested_provider, used_model, used_provider, and underlying_used_model fields - Update model field format to use provider/model pattern (e.g. openai/gpt-4o-mini) - Implement metadata for all provider response transformations - All unit tests passing with new metadata structure 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
WalkthroughAdds requestedModel/requestedProvider/baseModelName to transformToOpenAIFormat and call site. Normalizes model string to provider/baseModelName for non-OpenAI providers. Injects metadata (requested/used models/providers and underlying_used_model) into all OpenAI-format responses. Updates OpenAPI 200 response schema to include metadata. Applies changes across multiple provider branches. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant C as Client
participant G as Gateway Chat Handler
participant P as Provider Adapter (OpenAI/Anthropic/Google/... )
participant T as transformToOpenAIFormat
C->>G: POST /chat/completions (modelInput, provider, ...)
G->>P: Execute request (usedProvider, usedModel, ...)
P-->>G: Provider response (json, tokens, ...)
Note over G: Compute baseModelName
G->>T: Transform(json, content, tokens,<br/>requestedModel, requestedProvider, baseModelName)
T-->>G: OpenAI-format response<br/>- model: provider/baseModelName (non-OpenAI)<br/>- metadata{requested_*, used_*, underlying_used_model}
G-->>C: 200 OK OpenAI-format response (+metadata)
rect rgba(230,245,255,0.5)
Note over T: New/changed behavior:<br/>model normalization + metadata emission
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45–60 minutes Possibly related PRs
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
github-actions
Bot
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
apps/gateway/src/chat/chat.ts (4)
831-833: Tighten the type of requestedProviderUse the Provider union instead of a plain string for better type-safety.
function transformToOpenAIFormat( usedProvider: Provider, usedModel: string, @@ - requestedModel: string, - requestedProvider: string | null, + requestedModel: string, + requestedProvider: Provider | null, baseModelName: string, )
844-844: Normalize streaming model too for consistencyNon-streaming responses set model to
${usedProvider}/${baseModelName}, but streaming chunks still emitusedModel/upstream values. This can surprise clients switching between stream=false/true.Minimal approach:
- Compute once:
const normalizedModel =${usedProvider}/${baseModelName};- Pass it to the streaming transformer and use it for the
modelfield in emitted chunks.Example changes:
@@ - function transformStreamingChunkToOpenAIFormat( - usedProvider: Provider, - usedModel: string, - data: any, - messages: any[], - ): any { + function transformStreamingChunkToOpenAIFormat( + usedProvider: Provider, + usedModel: string, + data: any, + messages: any[], + normalizedModel: string, + ): any { @@ - model: data.model || usedModel, + model: normalizedModel,And at the call site:
- const transformedData = transformStreamingChunkToOpenAIFormat( - usedProvider, usedModel, data, messages - ); + const normalizedModel = `${usedProvider}/${baseModelName}`; + const transformedData = transformStreamingChunkToOpenAIFormat( + usedProvider, usedModel, data, messages, normalizedModel + );Also applies to: 896-896, 950-950, 994-994, 1013-1013, 1055-1055, 1070-1070
881-887: Great: unified metadata across providers; consider parity for streamingThe new metadata object looks solid and consistent across providers for non-streaming. For parity, emit a one-time SSE “metadata” event at stream start so streaming consumers can access the same fields.
Add after obtaining the reader in the streaming branch:
@@ - const reader = res.body.getReader(); + const reader = res.body.getReader(); + // Emit metadata once so streaming clients get the same context as non-streaming + await writeSSEAndCache({ + event: "metadata", + data: JSON.stringify({ + requested_model: modelInput, + requested_provider: requestedProvider ?? null, + used_model: baseModelName, + used_provider: usedProvider, + underlying_used_model: usedModel, + }), + id: String(eventId++), + });Also applies to: 932-938, 975-981, 995-1001, 1044-1051, 1055-1063, 1070-1076
1897-1903: Schema update LGTM; add example for better docsThe metadata schema addition is correct. Consider adding an OpenAPI example to improve generated docs.
- metadata: z.object({ + metadata: z.object({ requested_model: z.string(), requested_provider: z.string().nullable(), used_model: z.string(), used_provider: z.string(), underlying_used_model: z.string(), - }), + }).openapi({ + example: { + requested_model: "openai/gpt-4o", + requested_provider: "openai", + used_model: "gpt-4o", + used_provider: "openai", + underlying_used_model: "gpt-4o-2024-08-06", + }, + }),
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
apps/gateway/src/chat/chat.ts(12 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use localStorage instead of cookies for client-side data persistence
Files:
apps/gateway/src/chat/chat.ts
**/*.{js,ts}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{js,ts}: Use drizzle with the latest object syntax for database operations
For read queries, always usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
apps/gateway/src/chat/chat.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Never use
as anyor: anyin TypeScript files.
Files:
apps/gateway/src/chat/chat.ts
apps/{api,gateway}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
.findMany() or db().query.
apps/{api,gateway}/**/*.{ts,tsx}: Use Drizzle with the latest object syntax for database operations
For read queries, use db().query..findFirst()
apps/{api,gateway}/**/*.{ts,tsx}: Use the Hono framework for backend HTTP services in apps/api and apps/gateway
Use Zod for request/response validation in backend routes and handlers
Maintain OpenAPI/Swagger documentation for backend APIsFiles:
apps/gateway/src/chat/chat.tsapps/gateway/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
In apps/gateway (Hono), always use Hono + Zod + OpenAPI for validation and typesafety
Files:
apps/gateway/src/chat/chat.ts⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: e2e / run
- GitHub Check: build / run
🔇 Additional comments (1)
apps/gateway/src/chat/chat.ts (1)
4400-4403: Passing raw requested inputs is correctForwarding the user’s raw model/provider/baseModelName into the transformer matches the new metadata contract.
1 participant
Summary
transformToOpenAIFormatfunction to append metadata fields such asrequested_model,requested_provider,used_model,used_provider, andunderlying_used_model.${usedProvider}/${baseModelName}for clarity.Changes
Core Functionality
requestedModel,requestedProvider, andbaseModelNameto thetransformToOpenAIFormatfunction.modelfield in responses to a combined string format reflecting provider and base model.Code Quality
Test plan
modelfield is correctly formatted as${usedProvider}/${baseModelName}.🌿 Generated by Terry
ℹ️ Tag @terragon-labs to ask questions and address PR feedback
📎 Task: https://www.terragonlabs.com/task/3bcaaf42-29c4-4bd9-a837-6a97ee302ddf
Summary by CodeRabbit
New Features
Documentation
Refactor