feat(chat): log and display chat completion tools - #526
Conversation
…ty logs - Add tools and toolChoice columns to log table schema - Update chat completion logging to capture tools and tool_choice parameters - Add database migration for new columns - Update API log schema to include new fields - Enhance LogCard UI to display tool information in activity logs - Update TypeScript interfaces for new fields - Display Available Tools, Tool Choice, and Tool Calls in separate sections - Maintain backwards compatibility with existing logs
|
Warning Rate limit exceeded@steebchen has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 6 minutes and 58 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (2)
""" WalkthroughThis change replaces the previous Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Frontend
participant API
participant DB
User->>Frontend: View recent logs
Frontend->>API: GET /logs
API->>DB: Query logs (includes tools/toolChoice)
DB-->>API: Return logs with tools/toolChoice
API-->>Frontend: Respond with logs (tools/toolChoice)
Frontend->>User: Display logs (show Tool Information if present)
sequenceDiagram
participant ChatHandler
participant DB
ChatHandler->>DB: Insert log (includes tools, toolChoice)
Note over ChatHandler,DB: No toolCalls extraction/accumulation logic
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~15–20 minutes Possibly related PRs
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
steebchen
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
apps/api/src/routes/logs.ts (1)
40-42: Consider more specific typing for tool-related fields.While
z.any().nullable()works for dynamic JSON content, consider whether these fields could benefit from more specific Zod schemas based on the expected structure of tools, toolChoice, and toolCalls data. This would provide better type safety and API documentation.If the structure of these fields is well-defined, you could create more specific schemas like:
const toolSchema = z.object({ type: z.string(), function: z.object({ name: z.string(), description: z.string().optional(), parameters: z.record(z.unknown()).optional(), }).optional(), }).nullable(); const toolChoiceSchema = z.union([ z.literal("auto"), z.literal("none"), z.object({ type: z.literal("function"), function: z.object({ name: z.string(), }), }), ]).nullable();apps/next/src/types/activity.ts (1)
91-93: Good use ofunknownfor type safety, consider more specific typing.The use of
unknown | nullis better thananyfor type safety. However, if the structure of these tool-related fields is well-defined, consider creating more specific TypeScript interfaces to improve type safety and developer experience.For example, you could define:
interface ToolDefinition { type: string; function?: { name: string; description?: string; parameters?: Record<string, unknown>; }; } interface ToolChoice { type: "function"; function: { name: string; }; } | "auto" | "none"; interface ToolCall { id: string; type: "function"; function: { name: string; arguments: string; }; }Then use:
tools: ToolDefinition[] | null; toolChoice: ToolChoice | null; toolCalls: ToolCall[] | null;apps/gateway/src/chat/chat.ts (1)
81-82: Consider using more specific types instead ofany.The
toolsandtoolChoiceparameters are typed asany[] | undefinedandany | undefinedrespectively. Consider using more specific types to improve type safety and maintainability.Based on the OpenAPI schema defined elsewhere in the file, you could use more specific types:
- tools: any[] | undefined, - toolChoice: any | undefined, + tools: Array<{ + type: "function"; + function: { + name: string; + description?: string; + parameters?: Record<string, any>; + }; + }> | undefined, + toolChoice: "auto" | "none" | { + type: "function"; + function: { + name: string; + }; + } | undefined,packages/db/migrations/meta/1753466042_snapshot.json (1)
462-479: Preferjsonboverjsonfor the new columns
tools,tool_choice, and the pre-existingtool_callsare typed asjson.
In PostgreSQL,jsonbis almost always the better default because it:
- stores data in a decomposed binary format (smaller & faster to parse),
- supports GIN / GiST indexing for efficient containment queries,
- enables many more operators/functions.
Unless you have a compelling reason to preserve insertion order or exact whitespace, flip these three columns to
jsonbnow—migrating later is painful.- "tool_calls": { - "type": "json", + "tool_calls": { + "type": "jsonb", ... - "tools": { - "type": "json", + "tools": { + "type": "jsonb", ... - "tool_choice": { - "type": "json", + "tool_choice": { + "type": "jsonb",Follow-up: consider a GIN index on
tool_callsif you’ll filter by contained keys or paths.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (13)
apps/api/src/routes/logs.ts(1 hunks)apps/gateway/src/chat/chat.ts(9 hunks)apps/next/src/components/activity/recent-logs.tsx(2 hunks)apps/next/src/components/dashboard/log-card.tsx(1 hunks)apps/next/src/lib/api/v1.d.ts(1 hunks)apps/next/src/types/activity.ts(1 hunks)apps/ui/src/components/activity/recent-logs.tsx(1 hunks)apps/ui/src/components/dashboard/log-card.tsx(1 hunks)apps/ui/src/lib/api/v1.d.ts(1 hunks)packages/db/migrations/1753466042_motionless_cardiac.sql(1 hunks)packages/db/migrations/meta/1753466042_snapshot.json(1 hunks)packages/db/migrations/meta/_journal.json(1 hunks)packages/db/src/schema.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (6)
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit Inference Engine (.github/copilot-instructions.md)
Use localStorage instead of cookies for client-side data persistence
Files:
apps/next/src/types/activity.tsapps/api/src/routes/logs.tsapps/ui/src/lib/api/v1.d.tspackages/db/src/schema.tsapps/next/src/components/dashboard/log-card.tsxapps/ui/src/components/dashboard/log-card.tsxapps/next/src/lib/api/v1.d.tsapps/gateway/src/chat/chat.tsapps/next/src/components/activity/recent-logs.tsxapps/ui/src/components/activity/recent-logs.tsx
**/*.{js,ts}
📄 CodeRabbit Inference Engine (.github/copilot-instructions.md)
**/*.{js,ts}: Use drizzle with the latest object syntax for database operations
For read queries, always usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
apps/next/src/types/activity.tsapps/api/src/routes/logs.tsapps/ui/src/lib/api/v1.d.tspackages/db/src/schema.tsapps/next/src/lib/api/v1.d.tsapps/gateway/src/chat/chat.ts
**/*.{ts,tsx}
📄 CodeRabbit Inference Engine (.cursor/rules/general.mdc)
Never use
as anyor: anyin TypeScript files.
Files:
apps/next/src/types/activity.tsapps/api/src/routes/logs.tsapps/ui/src/lib/api/v1.d.tspackages/db/src/schema.tsapps/next/src/components/dashboard/log-card.tsxapps/ui/src/components/dashboard/log-card.tsxapps/next/src/lib/api/v1.d.tsapps/gateway/src/chat/chat.tsapps/next/src/components/activity/recent-logs.tsxapps/ui/src/components/activity/recent-logs.tsx
{apps/api,apps/gateway,packages/db}/**/*.ts
📄 CodeRabbit Inference Engine (CLAUDE.md)
{apps/api,apps/gateway,packages/db}/**/*.ts: Use Drizzle ORM with latest object syntax for database operations
For reads, usedb().query.<table>.findMany()ordb().query.<table>.findFirst()
Files:
apps/api/src/routes/logs.tspackages/db/src/schema.tsapps/gateway/src/chat/chat.ts
apps/ui/**/*.{js,jsx,ts,tsx}
📄 CodeRabbit Inference Engine (.github/copilot-instructions.md)
In apps/ui (a tanstack router project), always use navigate() for navigation
Use localStorage instead of cookies for client-side data persistence
Files:
apps/ui/src/lib/api/v1.d.tsapps/ui/src/components/dashboard/log-card.tsxapps/ui/src/components/activity/recent-logs.tsx
**/migrations/*.{js,ts,sql}
📄 CodeRabbit Inference Engine (.github/copilot-instructions.md)
For DB changes, do not write manual migration files
Files:
packages/db/migrations/1753466042_motionless_cardiac.sql
🧠 Learnings (4)
packages/db/migrations/meta/_journal.json (1)
Learnt from: CR
PR: theopenco/llmgateway#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-23T19:33:40.602Z
Learning: Applies to **/migrations/*.{js,ts,sql} : For DB changes, do not write manual migration files
packages/db/src/schema.ts (5)
Learnt from: CR
PR: theopenco/llmgateway#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-23T19:33:55.674Z
Learning: Applies to {apps/api,apps/gateway,packages/db}/**/*.ts : Use Drizzle ORM with latest object syntax for database operations
Learnt from: CR
PR: theopenco/llmgateway#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-23T19:33:55.674Z
Learning: Always sync schema with pnpm push after table/column changes
Learnt from: CR
PR: theopenco/llmgateway#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-23T19:33:40.602Z
Learning: Applies to **/*.{js,ts} : Use drizzle with the latest object syntax for database operations
Learnt from: CR
PR: theopenco/llmgateway#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-23T19:33:40.602Z
Learning: If any tables or columns do not exist, run pnpm push to sync the schema to the database
Learnt from: CR
PR: theopenco/llmgateway#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-23T19:33:40.602Z
Learning: Applies to **/migrations/*.{js,ts,sql} : For DB changes, do not write manual migration files
packages/db/migrations/1753466042_motionless_cardiac.sql (1)
Learnt from: CR
PR: theopenco/llmgateway#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-23T19:33:40.602Z
Learning: Applies to **/migrations/*.{js,ts,sql} : For DB changes, do not write manual migration files
packages/db/migrations/meta/1753466042_snapshot.json (1)
Learnt from: CR
PR: theopenco/llmgateway#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-23T19:33:40.602Z
Learning: If any tables or columns do not exist, run pnpm push to sync the schema to the database
🧬 Code Graph Analysis (3)
apps/next/src/types/activity.ts (1)
apps/next/src/components/activity/activity-client.tsx (1)
LogsData(20-72)
apps/ui/src/lib/api/v1.d.ts (1)
apps/next/src/components/activity/activity-client.tsx (1)
LogsData(20-72)
apps/ui/src/components/activity/recent-logs.tsx (1)
packages/db/src/schema.ts (1)
log(240-298)
🔇 Additional comments (17)
packages/db/migrations/meta/_journal.json (1)
215-221: LGTM! Properly generated migration journal entry.The new migration journal entry follows the established pattern and appears to be automatically generated by Drizzle ORM, which aligns with the coding guidelines that specify not writing manual migration files.
packages/db/src/schema.ts (1)
266-267: LGTM! Well-structured schema additions for tool logging.The new
toolsandtoolChoiceJSON columns are appropriately placed near the existingtoolCallsfield and use the correctjson()type for storing tool-related metadata. This supports the main feature of logging and displaying chat completion tools.packages/db/migrations/1753466042_motionless_cardiac.sql (1)
1-2: LGTM! Auto-generated migration adds required columns.The migration correctly adds the
toolsandtool_choiceJSON columns to the log table. This appears to be automatically generated by Drizzle ORM, which aligns with the coding guidelines.apps/ui/src/lib/api/v1.d.ts (1)
525-527: LGTM: Type definitions correctly represent the new tool-related fields.The optional
unknowntyping is appropriate for JSON data that can have varying structures. These definitions align well with the database schema's JSON columns fortools,toolChoice, andtoolCalls.Since this is an auto-generated file, ensure these changes originated from the OpenAPI schema rather than manual edits.
apps/next/src/lib/api/v1.d.ts (1)
525-527: LGTM: Consistent type definitions across applications.The type definitions are identical to those in
apps/ui/src/lib/api/v1.d.ts, ensuring consistency between the UI applications. The optionalunknowntyping remains appropriate for the JSON-based tool metadata.apps/ui/src/components/dashboard/log-card.tsx (1)
315-359: EnsureLogtype includes the new tool properties and remove allas anyassertions.I wasn’t able to locate the
Loginterface in the codebase, so please:
- Find and update the
Logtype (in the@llmgateway/dbpackage) to include:
tools?: /* appropriate type */toolChoice?: /* appropriate type */toolCalls?: /* appropriate type */- In
apps/ui/src/components/dashboard/log-card.tsx(lines 315–359), remove every(log as any)assertion and reference the properties directly:
- Conditional check:
{(log.tools || log.toolChoice || log.toolCalls) && (…)}- Each JSON output:
JSON.stringify(log.tools, …),JSON.stringify(log.toolChoice, …),JSON.stringify(log.toolCalls, …)Once the
Logtype is updated, these casts can be deleted and the code will regain full type safety.apps/next/src/components/dashboard/log-card.tsx (1)
315-357: Excellent TypeScript implementation!This implementation correctly follows the coding guidelines by avoiding
as anytype assertions and using proper TypeScript typing throughout. The tool information display logic is well-structured and maintains consistency with the existing component design.apps/next/src/components/activity/recent-logs.tsx (2)
62-64: Good interface extension with proper typing.The addition of the tool-related properties with
unknown | nulltyping is appropriate for JSON fields and follows TypeScript best practices.
296-298: Clean property forwarding implementation.The new tool-related properties are properly passed through to the LogCard component using direct property access, maintaining consistency with the existing code pattern.
apps/gateway/src/chat/chat.ts (8)
1675-1676: LGTM!The cached response logging correctly includes the new
toolsandtool_choiceparameters.
1811-1812: LGTM!The streaming cancellation logging correctly includes the new tool-related parameters.
1896-1897: LGTM!The streaming error logging correctly includes the new tool-related parameters.
2564-2565: LGTM!The streaming completion logging correctly includes the new tool-related parameters.
2651-2652: LGTM!The non-streaming cancellation logging correctly includes the new tool-related parameters.
2714-2715: LGTM!The non-streaming error logging correctly includes the new tool-related parameters.
2822-2823: LGTM!The non-streaming success logging correctly includes the new tool-related parameters.
100-101: Excellent consistency across all logging call sites.All seven invocations of
createLogEntrythroughout the file have been consistently updated to include the newtoolsandtoolChoiceparameters. This ensures comprehensive logging of tool-related metadata across all execution paths (cached responses, streaming/non-streaming, success/error cases).
| tools: (log as any).tools, | ||
| toolChoice: (log as any).toolChoice, | ||
| toolCalls: (log as any).toolCalls, |
There was a problem hiding this comment.
Remove unnecessary type assertions that violate coding guidelines.
The as any type assertions are unnecessary and violate the coding guideline "Never use as any or : any in TypeScript files." The API types already define these fields as optional unknown, so the type assertions can be safely removed.
Apply this diff to remove the type assertions:
- tools: (log as any).tools,
- toolChoice: (log as any).toolChoice,
- toolCalls: (log as any).toolCalls,
+ tools: log.tools,
+ toolChoice: log.toolChoice,
+ toolCalls: log.toolCalls,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| tools: (log as any).tools, | |
| toolChoice: (log as any).toolChoice, | |
| toolCalls: (log as any).toolCalls, | |
| tools: log.tools, | |
| toolChoice: log.toolChoice, | |
| toolCalls: log.toolCalls, |
🤖 Prompt for AI Agents
In apps/ui/src/components/activity/recent-logs.tsx around lines 160 to 162,
remove the unnecessary `as any` type assertions from the properties `tools`,
`toolChoice`, and `toolCalls`. Since the API types already define these fields
as optional `unknown`, you can directly access them without casting. Simply
delete the `as any` casts to comply with the coding guideline prohibiting `as
any` usage.
Replaced `toolCalls` with `tools` and `toolChoice` for better clarity and usability in chat response schema. Updated downstream response handling accordingly.
Eliminated the `toolCalls` property from v1 API schemas across UI, Next, and API layers as it is no longer required.
Eliminated the unused `toolCalls` property from the schema to maintain consistency and simplify the structure.
Introduced a new snapshot file for the database schema, reflecting the current structure of tables, columns, indexes, and constraints. Allows for better tracking of schema changes over time.
Updated the chat response schema to remove the unused `toolCalls` property and replace it with `tools` and `toolChoice` fields for improved clarity and functionality.
Deleted the `extractToolCallsFromProvider` function and related logic from the chat module as it is no longer required. This simplifies the codebase and eliminates redundant functionality.
Removed `toolCalls` property and related logic from UI components and type definitions to simplify the structure and maintain consistency. Retained `tools` and `toolChoice` fields for better clarity.
…y-chat-completion-tools-d8a4
Added new sections in the dashboard log card to display `tools` and `toolChoice` if available. Updated layout and styles to incorporate the changes. Removed obsolete `toolCalls` logic across related components.
2 participants
Log and display chat completion
toolsandtool_choiceparameters in the activity log UI.This provides a comprehensive view of tool interaction, showing the tools provided to the model, the model's choice of tools, and the actual tool calls made.
Open in Web • Open in Cursor
Learn more about Background Agents
Summary by CodeRabbit
New Features
Bug Fixes
Chores