feat(logging): add usedModelMapping - #769
Conversation
![terragon-labs[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
- Introduced usedModelMapping field in log schema to store original provider model name. - Updated chat logging to include usedModelMapping alongside usedModelFormatted. - Enables better tracking and differentiation of models used in requests. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
WalkthroughAdds recording of both gateway-formatted and provider-original model identifiers across chat logging: chat handler computes and passes Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant C as Client
participant G as Gateway Chat Handler
participant L as Logger
participant DB as Database
C->>G: Send chat request (provider, baseModelName)
G->>G: derive usedModelMapping (original) and usedModelFormatted (provider/baseModelName)
G->>L: createLogEntry(requestId, project, apiKey, providerKey?.id, usedModelFormatted, usedModelMapping, usedProvider, ...)
L->>DB: insert log (includes used_model_mapping)
G-->>C: return response / stream
alt error or cancel
G->>L: createLogEntry(..., status=error/cancel)
L->>DB: insert/update log
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Pre-merge checks (3 passed)✅ Passed checks (3 passed)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
packages/db/src/schema.ts (1)
265-265: Add a brief field comment for clarity.Consider documenting that usedModelMapping stores the original provider model name; usedModel holds the LLMGateway-formatted value. Helps avoid future confusion.
- usedModelMapping: text(), + // Original provider model name (e.g., "gpt-4o-mini-2025-07-18") + usedModelMapping: text(),apps/gateway/src/chat/chat.ts (2)
228-236: Name the parameter to reflect its semantics (formatted vs mapping).createLogEntry now stores the formatted model string in usedModel and the provider original in usedModelMapping. Rename the parameter for precision and coerce mapping to null to avoid undefined flowing into inserts.
-function createLogEntry( +function createLogEntry( requestId: string, project: Project, apiKey: ApiKey, providerKeyId: string | undefined, - usedModel: string, - usedModelMapping: string | undefined, + usedModelFormatted: string, + usedModelMapping: string | undefined, usedProvider: string, @@ - usedModel, - usedModelMapping, + usedModel: usedModelFormatted, + usedModelMapping: usedModelMapping ?? null,
2921-2946: All createLogEntry calls correctly match the updated signature. Consider refactoring createLogEntry to accept a single object parameter (e.g.{ requestId, project, apiKey, providerKeyId, usedModel, … }) to eliminate positional-ordering risks as parameters grow.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
apps/gateway/src/chat/chat.ts(11 hunks)packages/db/migrations/1757376520_brief_roughhouse.sql(1 hunks)packages/db/migrations/meta/1757376520_snapshot.json(1 hunks)packages/db/migrations/meta/_journal.json(1 hunks)packages/db/src/schema.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (7)
**/migrations/*.{js,ts,sql}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
For DB changes, do not write manual migration files
Files:
packages/db/migrations/1757376520_brief_roughhouse.sql
**/{drizzle,migrations}/**/*.sql
📄 CodeRabbit inference engine (AGENTS.md)
Do not write manual migration files for DB changes
Files:
packages/db/migrations/1757376520_brief_roughhouse.sql
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use localStorage instead of cookies for client-side data persistence
Files:
packages/db/src/schema.tsapps/gateway/src/chat/chat.ts
**/*.{js,ts}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{js,ts}: Use drizzle with the latest object syntax for database operations
For read queries, always usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
packages/db/src/schema.tsapps/gateway/src/chat/chat.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Never use
as anyor: anyin TypeScript files.
Files:
packages/db/src/schema.tsapps/gateway/src/chat/chat.ts
apps/{api,gateway}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
.findMany() or db().query.
apps/{api,gateway}/**/*.{ts,tsx}: Use Drizzle with the latest object syntax for database operations
For read queries, use db().query..findFirst()
apps/{api,gateway}/**/*.{ts,tsx}: Use the Hono framework for backend HTTP services in apps/api and apps/gateway
Use Zod for request/response validation in backend routes and handlers
Maintain OpenAPI/Swagger documentation for backend APIsFiles:
apps/gateway/src/chat/chat.tsapps/gateway/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
In apps/gateway (Hono), always use Hono + Zod + OpenAPI for validation and typesafety
Files:
apps/gateway/src/chat/chat.ts⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: e2e / run
🔇 Additional comments (4)
packages/db/migrations/1757376520_brief_roughhouse.sql (1)
1-1: LGTM: nullable column added without blocking legacy rows. Confirmed this migration was auto-generated by Luca Steeb (bot) in commit 6287e11, satisfying the “no manual migrations” rule.packages/db/migrations/meta/1757376520_snapshot.json (1)
451-456: Snapshot includes used_model_mapping with correct nullability.The snapshot reflects the new column on table log and keeps it nullable—good for back-compat.
apps/gateway/src/chat/chat.ts (2)
2629-2632: LGTM: clear separation between formatted and provider-native names.usedModelFormatted and usedModelMapping values align with the new schema and response metadata.
4517-4537: API metadata keeps both representations—nice.used_model (formatted) and underlying_used_model (provider native) in metadata mirror the DB fields; this improves traceability across logs and client responses.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
apps/gateway/src/api.e2e.ts (1)
326-346: Extend validateLogs() to cover usedModelMapping presence.Lightweight sanity checks keep all tests aligned with the new schema while avoiding provider-specific coupling.
const log = logs[0]; expect(log.usedProvider).toBeTruthy(); expect(log.errorDetails).toBeNull(); expect(log.finishReason).not.toBeNull(); expect(log.unifiedFinishReason).not.toBeNull(); expect(log.unifiedFinishReason).toBeTruthy(); expect(log.usedModel).toBeTruthy(); expect(log.requestedModel).toBeTruthy(); + // New field added by this PR + expect(log).toHaveProperty("usedModelMapping"); + expect(log.usedModelMapping === null || typeof log.usedModelMapping === "string").toBe(true); return log;
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
apps/gateway/src/api.e2e.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (7)
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use localStorage instead of cookies for client-side data persistence
Files:
apps/gateway/src/api.e2e.ts
**/*.{js,ts}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{js,ts}: Use drizzle with the latest object syntax for database operations
For read queries, always usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
apps/gateway/src/api.e2e.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Never use
as anyor: anyin TypeScript files.
Files:
apps/gateway/src/api.e2e.ts
apps/{api,gateway}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
.findMany() or db().query.
apps/{api,gateway}/**/*.{ts,tsx}: Use Drizzle with the latest object syntax for database operations
For read queries, use db().query..findFirst()
apps/{api,gateway}/**/*.{ts,tsx}: Use the Hono framework for backend HTTP services in apps/api and apps/gateway
Use Zod for request/response validation in backend routes and handlers
Maintain OpenAPI/Swagger documentation for backend APIsFiles:
apps/gateway/src/api.e2e.tsapps/gateway/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
In apps/gateway (Hono), always use Hono + Zod + OpenAPI for validation and typesafety
Files:
apps/gateway/src/api.e2e.ts**/*.e2e.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Name end-to-end tests with the .e2e.ts suffix (run by pnpm test:e2e)
Files:
apps/gateway/src/api.e2e.ts**/*.{spec,e2e}.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Use Vitest as the test framework for unit and E2E tests
Files:
apps/gateway/src/api.e2e.ts🧬 Code graph analysis (1)
apps/gateway/src/api.e2e.ts (1)
packages/db/src/schema.ts (1)
log(244-311)⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: build / run
- GitHub Check: e2e / run
Added check for `usedModelMapping` in API e2e test to ensure proper validation of the field.
1 participant
Summary
usedModelMappingfield in the log schema to capture the original provider model nameusedModelFormatted) and the original model mapping (usedModelMapping)Changes
Database Schema
usedModelMappingas an optional text field in thelogtable schemaChat Application
createLogEntryfunction to accept and storeusedModelMappingusedModelFormatted(LLMGateway format) andusedModelMapping(original provider model name)usedModelMappingas the original provider model name andusedModelFormattedas a combination ofusedProviderand base model nameTest plan
usedModelFormattedandusedModelMappingfields with correct values🌿 Generated by Terry
ℹ️ Tag @terragon-labs to ask questions and address PR feedback
📎 Task: https://www.terragonlabs.com/task/1bd45e34-dd4b-4c83-9057-aa96f8191a87
Summary by CodeRabbit
Chores
Tests