Skip to content

Add seperate chat model#577

Merged
elie222 merged 5 commits intomainfrom
feat/chat-model
Jul 18, 2025
Merged

Add seperate chat model#577
elie222 merged 5 commits intomainfrom
feat/chat-model

Conversation

@elie222
Copy link
Owner

@elie222 elie222 commented Jul 17, 2025

Summary by CodeRabbit

  • New Features

    • Added support for a new chat-optimized model type, allowing selection of dedicated chat models via environment settings.
    • Introduced support for the "moonshotai/kimi-k2" model across the platform.
  • Improvements

    • Enhanced environment variable flexibility for selecting LLM providers and models, including new options for chat and economy models.
    • Refined model selection logic to use descriptive model types ("default", "economy", "chat") for improved clarity and configurability.
    • Added support for configuring OpenRouter provider options for default, economy, and chat models.
  • Tests

    • Expanded test coverage for model selection based on the new model type parameter.
  • Chores

    • Updated version to v1.9.18.

@vercel
Copy link

vercel bot commented Jul 17, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
inbox-zero ✅ Ready (Inspect) Visit Preview Jul 18, 2025 5:20pm

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jul 17, 2025

Walkthrough

The changes refactor model selection logic by introducing a modelType parameter (with values "default", "economy", or "chat") in place of the previous boolean flag. Environment variable schemas are unified with a reusable LLM provider enum, and chat-specific provider/model variables are added. A new model identifier and cost entry for "moonshotai/kimi-k2" are also introduced.

Changes

File(s) Change Summary
apps/web/env.ts Introduced llmProviderEnum, unified provider validation, added chat provider/model env vars, refactored others.
apps/web/utils/llms/model.ts, apps/web/utils/llms/index.ts Refactored to use modelType parameter, centralized model selection, added chat model support.
apps/web/utils/ai/assistant/chat.ts, apps/web/utils/ai/assistant/process-user-request.ts Updated function calls to use modelType: "chat" instead of boolean flag.
apps/web/utils/ai/knowledge/extract-from-email-history.ts, apps/web/utils/ai/knowledge/extract.ts Updated function calls to use modelType: "economy" instead of boolean flag.
apps/web/utils/llms/config.ts Added KIMI_K2_OPENROUTER model identifier.
apps/web/utils/llms/model.test.ts Extended tests for getModel to cover new modelType logic and chat/economy env vars.
apps/web/utils/usage.ts Added cost entry for "moonshotai/kimi-k2".
version.txt Bumped version from v1.9.17 to v1.9.18.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant AI_Module
    participant ModelSelector
    participant EnvConfig

    User->>AI_Module: Request (chat, economy, or default)
    AI_Module->>ModelSelector: getModel(user, modelType)
    ModelSelector->>EnvConfig: Fetch provider/model for modelType
    EnvConfig-->>ModelSelector: Return provider/model (chat/economy/default)
    ModelSelector-->>AI_Module: Selected provider/model
    AI_Module-->>User: Processed response
Loading

Possibly related PRs

  • Add Ollama support #269: Adds Ollama support and related environment/config changes, overlapping with this PR's provider enum and env var updates.
  • Custom model routing #503: Refines provider/model selection logic and introduces a "custom" provider, directly related to this PR's environment and model routing enhancements.
  • Default llm model env var #500: Modifies default LLM provider environment variables, related to this PR's broader environment variable schema refactoring.

Poem

🐇
A hop, a skip, a model type anew—
"Chat" and "economy" join the queue!
Providers enumed, configs refined,
Kimi K2’s cost now well-defined.
With every version, leaps we make,
More clever chats for inbox’s sake!


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2546dc4 and d0f1b08.

📒 Files selected for processing (1)
  • apps/web/utils/llms/model.ts (6 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • apps/web/utils/llms/model.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: claude-review
  • GitHub Check: Analyze (javascript-typescript)
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@claude
Copy link

claude bot commented Jul 17, 2025

Claude finished @elie222's task —— View job


Pull Request Review

Reviewing PR #577: "Add seperate chat model"

Review Checklist:

  • Read repository CLAUDE.md for context
  • Review environment variable changes (env.ts)
  • Analyze model configuration changes (model.ts)
  • Review test coverage (model.test.ts)
  • Check LLM utility changes (index.ts)
  • Review AI assistant integration changes
  • Assess security implications
  • Evaluate performance considerations
  • Provide comprehensive feedback

Code Review Feedback

Positive Aspects

  1. Well-structured separation of concerns: The addition of separate chat and economy models allows for optimized model selection based on use case.

  2. Comprehensive test coverage: The test suite in model.test.ts properly covers the new model type functionality with good edge case handling.

  3. Proper environment variable validation: The new environment variables are properly validated using Zod schema.

  4. Consistent API design: The modelType parameter is consistently used throughout the codebase.

⚠️ Areas for Improvement

  1. Typo in PR title: "seperate" should be "separate" - consider updating the PR title for consistency.

  2. Missing environment variable documentation: The new environment variables (CHAT_LLM_PROVIDER, CHAT_LLM_MODEL, ECONOMY_LLM_PROVIDER, ECONOMY_LLM_MODEL) lack inline documentation explaining their purpose and when to use them.

  3. Incomplete error handling: In selectChatModel and selectEconomyModel functions (model.ts:193-207), when the API key is not found, the function logs a warning but falls back to the default model. Consider whether this is the desired behavior or if it should throw an error.

  4. Potential performance impact: The getProviderApiKey function is called for every chat/economy model request. Consider caching these values or restructuring to avoid repeated lookups.

🔒 Security Considerations

  1. API key handling: The code properly handles API keys by checking for their existence before using them. Good practice.

  2. Environment variable exposure: All sensitive configuration is properly kept server-side in the env.ts file.

🏃 Performance Considerations

  1. Model selection logic: The three-tier model selection (chat, economy, default) is efficient and allows for cost optimization.

  2. Usage tracking: The changes to usage.ts properly track costs for the new models, including the Kimi K2 model.

🐛 Potential Issues

  1. Inconsistent model fallback: In model.ts:174 and model.ts:197, when a configured model provider doesn't have an API key, the code falls back to the default model. This might mask configuration issues. Consider making this behavior more explicit.

  2. Test environment mocking: The test file mocks all environment variables, which is good, but ensure that production environment variables are properly validated.

💡 Suggestions

  1. Add JSDoc comments: Consider adding JSDoc comments to the new functions explaining when to use each model type:

    /**
     * Select chat model for fast conversational tasks
     * Falls back to default model if chat model is not configured
     */
    function selectChatModel(userAi: UserAIFields) { ... }
  2. Consider adding model type validation: Add runtime validation to ensure the modelType parameter is one of the expected values.

  3. Documentation: Consider adding examples in the environment file or README showing how to configure the new model types.

Overall Assessment

This is a solid implementation that adds important functionality for model optimization. The code is well-structured, tested, and follows existing patterns. The main concerns are around error handling clarity and documentation. The changes should improve both performance (with economy models) and user experience (with chat models).

Recommendation: ✅ Approve with minor suggestions

The code is ready for merge with the minor improvements suggested above being addressed in follow-up PRs if needed.


Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
apps/web/utils/usage.ts (1)

129-133: Model cost entry added correctly, but clarify provider information.

The cost entry for "moonshotai/kimi-k2" is properly structured and follows the established pattern. However, the comment mentions "Kimi K2 Groq via OpenRouter" while the model identifier suggests MoonShot AI. Consider clarifying whether this is a Groq model or MoonShot AI model accessed through OpenRouter.

apps/web/utils/llms/model.ts (1)

187-208: Consider enhancing the JSDoc comment for better documentation.

The implementation is solid and follows the established pattern. However, the JSDoc comment could be more descriptive about specific use cases for the chat model, similar to the detailed comment for selectEconomyModel.

/**
- * Selects the appropriate chat model for fast conversational tasks
+ * Selects the appropriate chat model for fast conversational tasks
+ * Optimized for interactive chat experiences and real-time responses
+ *
+ * Use cases:
+ * - AI assistant conversations
+ * - Real-time chat interactions
+ * - Tasks requiring fast response times
+ * - Interactive user assistance
 */
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b8dcc85 and dfe92cc.

📒 Files selected for processing (11)
  • apps/web/env.ts (2 hunks)
  • apps/web/utils/ai/assistant/chat.ts (1 hunks)
  • apps/web/utils/ai/assistant/process-user-request.ts (1 hunks)
  • apps/web/utils/ai/knowledge/extract-from-email-history.ts (1 hunks)
  • apps/web/utils/ai/knowledge/extract.ts (1 hunks)
  • apps/web/utils/llms/config.ts (1 hunks)
  • apps/web/utils/llms/index.ts (15 hunks)
  • apps/web/utils/llms/model.test.ts (2 hunks)
  • apps/web/utils/llms/model.ts (4 hunks)
  • apps/web/utils/usage.ts (1 hunks)
  • version.txt (1 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
apps/web/**/*.{ts,tsx}

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • apps/web/CLAUDE.md
**/*.{ts,tsx}

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • .cursor/rules/form-handling.mdc
apps/web/**/{.env.example,env.ts,turbo.json}

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • apps/web/CLAUDE.md
apps/web/**/{.env.example,env.ts}

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • apps/web/CLAUDE.md
apps/web/env.ts

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • .cursor/rules/environment-variables.mdc
🧠 Learnings (3)
apps/web/utils/llms/model.test.ts (7)
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: When writing tests for LLM-related functionality in TypeScript (e.g., in 'apps/web/__tests__/*.test.ts'), always create helper functions for common test data to ensure consistency and reduce duplication.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Test both AI and non-AI code paths in LLM-related functionality to ensure comprehensive coverage.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Standard test cases for LLM-related functionality should include: happy path with expected input, error handling, edge cases (such as empty input or null values), different user configurations, and various input formats.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Use descriptive 'console.debug' statements to log generated content in LLM-related tests for easier debugging and traceability.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-23T12:27:30.570Z
Learning: When mocking Prisma in Vitest, import the Prisma mock from '@/utils/__mocks__/prisma', mock '@/utils/prisma', and clear all mocks in a beforeEach hook to ensure test isolation.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Do not mock the LLM call in LLM-related tests; always call the actual LLM to ensure test reliability and real-world coverage.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/posthog-feature-flags.mdc:0-0
Timestamp: 2025-06-23T12:26:51.119Z
Learning: For A/B test feature flags, always define a TypeScript type for the possible variants and ensure the hook returns a default fallback value (such as 'control') if the flag is not set.
apps/web/utils/llms/index.ts (5)
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: When writing tests for LLM-related functionality in TypeScript (e.g., in 'apps/web/__tests__/*.test.ts'), always create helper functions for common test data to ensure consistency and reduce duplication.
Learnt from: CR
PR: elie222/inbox-zero#0
File: apps/web/CLAUDE.md:0-0
Timestamp: 2025-07-03T12:02:16.137Z
Learning: Applies to apps/web/**/types/**/*.{ts,d.ts} : Centralize types in dedicated type files when shared
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm.mdc:0-0
Timestamp: 2025-06-23T12:26:36.694Z
Learning: Use TypeScript types for all parameters and return values, and define clear interfaces for complex input and output structures to ensure type safety.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm.mdc:0-0
Timestamp: 2025-06-23T12:26:36.694Z
Learning: When implementing LLM-related functions in TypeScript, follow a standard structure: create a scoped logger, define a Zod schema for output validation, perform early input validation and returns, separate system and user prompts, log inputs and outputs, call the LLM with proper configuration, and return the validated result.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/logging.mdc:0-0
Timestamp: 2025-06-23T12:26:43.177Z
Learning: In TypeScript backend code, use the `createScopedLogger` function from `@/utils/logger` to perform logging. Typically, instantiate the logger at the top of the file with a relevant scope string.
apps/web/env.ts (7)
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/environment-variables.mdc:0-0
Timestamp: 2025-07-03T12:02:45.107Z
Learning: Applies to apps/web/env.ts : For server-only environment variables, add them to the `server` section in `apps/web/env.ts`
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm.mdc:0-0
Timestamp: 2025-06-23T12:26:36.694Z
Learning: When implementing LLM-related functions in TypeScript, follow a standard structure: create a scoped logger, define a Zod schema for output validation, perform early input validation and returns, separate system and user prompts, log inputs and outputs, call the LLM with proper configuration, and return the validated result.
Learnt from: CR
PR: elie222/inbox-zero#0
File: apps/web/CLAUDE.md:0-0
Timestamp: 2025-07-03T12:02:16.137Z
Learning: Applies to apps/web/**/{.env.example,env.ts,turbo.json} : Add environment variables to `.env.example`, `env.ts`, and `turbo.json`
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/environment-variables.mdc:0-0
Timestamp: 2025-07-03T12:02:45.107Z
Learning: Applies to apps/web/env.ts : For client-side environment variables, prefix them with `NEXT_PUBLIC_` and add them to both the `client` and `experimental__runtimeEnv` sections in `apps/web/env.ts`
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: When writing tests for LLM-related functionality in TypeScript (e.g., in 'apps/web/__tests__/*.test.ts'), always create helper functions for common test data to ensure consistency and reduce duplication.
Learnt from: CR
PR: elie222/inbox-zero#0
File: apps/web/CLAUDE.md:0-0
Timestamp: 2025-07-03T12:02:16.137Z
Learning: Applies to apps/web/**/{.env.example,env.ts} : Client-side environment variables must be prefixed with `NEXT_PUBLIC_`
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/environment-variables.mdc:0-0
Timestamp: 2025-07-03T12:02:45.107Z
Learning: Applies to apps/web/.env.example : Add new environment variables to `.env.example` with an example value
🧬 Code Graph Analysis (2)
apps/web/utils/llms/index.ts (2)
apps/web/utils/llms/types.ts (1)
  • UserAIFields (3-9)
apps/web/utils/llms/model.ts (1)
  • ModelType (16-16)
apps/web/utils/llms/model.ts (2)
apps/web/utils/llms/types.ts (1)
  • UserAIFields (3-9)
apps/web/env.ts (1)
  • env (15-206)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Jit Security
  • GitHub Check: claude-review
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (17)
version.txt (1)

1-1: Version bump looks appropriate for the feature additions.

The increment from v1.9.17 to v1.9.18 correctly reflects the scope of changes for adding separate chat model functionality and refactoring model selection logic.

apps/web/utils/ai/assistant/process-user-request.ts (1)

203-203: Correctly implements the new modelType parameter.

The addition of modelType: "chat" aligns with the refactoring from boolean flags to typed parameters, appropriately specifying chat model usage for this assistant function.

apps/web/utils/ai/assistant/chat.ts (1)

435-435: Correct implementation of modelType parameter for chat functionality.

The addition of modelType: "chat" is appropriately placed in the chatCompletionStream call within the aiProcessAssistantChat function, following the established pattern for the model type refactoring.

apps/web/utils/llms/config.ts (1)

35-35: New model identifier added correctly.

The addition of KIMI_K2_OPENROUTER: "moonshotai/kimi-k2" follows the established naming convention and is consistent with the corresponding cost entry in usage.ts.

apps/web/utils/ai/knowledge/extract-from-email-history.ts (1)

106-106: LGTM - Model type refactoring applied correctly

The change from useEconomyModel: true to modelType: "economy" is consistent with the codebase refactoring and appropriate for this use case.

apps/web/utils/ai/knowledge/extract.ts (1)

108-108: LGTM - Consistent model type refactoring

The update to modelType: "economy" aligns with the broader refactoring and is appropriate for knowledge extraction tasks.

apps/web/utils/llms/model.test.ts (2)

41-44: Good test coverage for new environment variables

The mock environment correctly includes all the new LLM provider and model variables needed to test the expanded model type functionality.


219-262: Excellent test coverage for the new modelType parameter

The test cases comprehensively verify the behavior for each model type ("chat", "economy", "default") and properly use null UserAIFields to ensure selection is based solely on the modelType parameter and environment configuration.

apps/web/env.ts (2)

5-13: Excellent use of reusable enum for consistency

The llmProviderEnum provides a centralized definition of valid LLM providers, improving maintainability and ensuring consistency across all provider-related environment variables.


30-35: Well-structured environment variables for new model types

The addition of CHAT_LLM_PROVIDER and CHAT_LLM_MODEL with helpful comments follows the coding guidelines and provides clear configuration options for the new chat model type.

apps/web/utils/llms/index.ts (3)

31-31: Good import addition for type safety

The import of ModelType ensures type safety for the new parameter across all function signatures.


44-44: Consistent refactoring of function signatures

The systematic replacement of useEconomyModel?: boolean with modelType?: ModelType with a default value of "default" is well-executed and maintains backward compatibility.

Also applies to: 51-51, 90-90, 115-115, 158-158, 170-170, 232-232, 256-256, 303-303, 313-313


59-61: Correct parameter passing to getModel function

All calls to getModel have been consistently updated to pass the modelType parameter instead of the previous boolean flag.

Also applies to: 125-127, 186-188, 267-269, 323-325

apps/web/utils/llms/model.ts (4)

16-21: LGTM! Good refactoring for type safety and extensibility.

The new ModelType union type provides better type safety compared to the previous boolean flag, and the default parameter ensures backward compatibility.


34-43: Clean implementation with proper enum handling.

The switch statement correctly handles all ModelType values with a sensible default fallback.


297-307: Good generalization that improves reusability.

The function signature change from a union type to string makes it more flexible for new providers while maintaining proper fallback behavior (returning undefined for unknown providers).


25-25: Logging update correctly reflects the refactored parameter.

The logging statement properly uses the new modelType parameter instead of the previous boolean flag.

Comment on lines 26 to +32
DEFAULT_LLM_PROVIDER: z
.enum([
"anthropic",
// "bedrock",
"google",
"openai",
"openrouter",
"groq",
"ollama",
"custom",
])
.enum([...llmProviderEnum.options, "custom"])
.default("anthropic"),
DEFAULT_LLM_MODEL: z.string().optional(),
// Economy LLM configuration (for large context windows where cost efficiency matters)
ECONOMY_LLM_PROVIDER: z
.enum([
"anthropic",
"google",
"openai",
"bedrock",
"openrouter",
"groq",
"ollama",
])
.optional()
.default("openrouter"),
ECONOMY_LLM_MODEL: z
.string()
.optional()
.default("google/gemini-2.5-flash-preview-05-20"),
// Set this to a cheaper model like Gemini Flash
ECONOMY_LLM_PROVIDER: llmProviderEnum.optional(),
ECONOMY_LLM_MODEL: z.string().optional(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Request verification of environment variable additions

Please ensure these new environment variables are also added to .env.example and turbo.json as specified in the coding guidelines.


🏁 Script executed:

#!/bin/bash
# Description: Check if new environment variables are properly added to required files
# Expected: Variables should be present in both .env.example and turbo.json

echo "Checking .env.example for new environment variables..."
fd -t f "\.env\.example$" --exec grep -l "ECONOMY_LLM_PROVIDER\|ECONOMY_LLM_MODEL\|CHAT_LLM_PROVIDER\|CHAT_LLM_MODEL" {} \;

echo -e "\nChecking turbo.json for new environment variables..."
fd -t f "turbo\.json$" --exec grep -l "ECONOMY_LLM_PROVIDER\|ECONOMY_LLM_MODEL\|CHAT_LLM_PROVIDER\|CHAT_LLM_MODEL" {} \;

Length of output: 507


Add missing environment variables to .env.example

The new variables are present in turbo.json but aren’t in .env.example. Please add them with example values to satisfy our environment-variable guidelines.

• File: .env.example
– Add entries for:
• ECONOMY_LLM_PROVIDER=
• ECONOMY_LLM_MODEL=
• CHAT_LLM_PROVIDER=
• CHAT_LLM_MODEL=

🤖 Prompt for AI Agents
In apps/web/env.ts around lines 26 to 32, new environment variables
ECONOMY_LLM_PROVIDER, ECONOMY_LLM_MODEL, CHAT_LLM_PROVIDER, and CHAT_LLM_MODEL
are defined but missing from the .env.example file. To fix this, open the
.env.example file and add these variables with appropriate example values
following the existing format and environment-variable guidelines to ensure
consistency and clarity for users.

@claude
Copy link

claude bot commented Jul 17, 2025

Claude finished @elie222's task —— View job


Pull Request Review

Reviewing PR #577: "Add seperate chat model"

Review Checklist:

  • Read repository CLAUDE.md for context
  • Review environment variable changes (env.ts)
  • Analyze model configuration changes (model.ts)
  • Review test coverage (model.test.ts)
  • Check LLM utility changes (index.ts)
  • Review AI assistant integration changes
  • Assess security implications
  • Evaluate performance considerations
  • Provide comprehensive feedback

Code Review Feedback

Positive Aspects

  1. Well-structured separation of concerns: The addition of separate chat and economy models allows for optimized model selection based on use case.

  2. Comprehensive test coverage: The test suite in model.test.ts properly covers the new model type functionality with good edge case handling.

  3. Proper environment variable validation: The new environment variables are properly validated using Zod schema.

  4. Consistent API design: The modelType parameter is consistently used throughout the codebase.

⚠️ Areas for Improvement

  1. Typo in PR title: "seperate" should be "separate" - consider updating the PR title for consistency.

  2. Missing environment variable documentation: The new environment variables (CHAT_LLM_PROVIDER, CHAT_LLM_MODEL, ECONOMY_LLM_PROVIDER, ECONOMY_LLM_MODEL) lack inline documentation explaining their purpose and when to use them.

  3. Incomplete error handling: In selectChatModel and selectEconomyModel functions (model.ts:193-207), when the API key is not found, the function logs a warning but falls back to the default model. Consider whether this is the desired behavior or if it should throw an error.

  4. Potential performance impact: The getProviderApiKey function is called for every chat/economy model request. Consider caching these values or restructuring to avoid repeated lookups.

🔒 Security Considerations

  1. API key handling: The code properly handles API keys by checking for their existence before using them. Good practice.

  2. Environment variable exposure: All sensitive configuration is properly kept server-side in the env.ts file.

🏃 Performance Considerations

  1. Model selection logic: The three-tier model selection (chat, economy, default) is efficient and allows for cost optimization.

  2. Usage tracking: The changes to usage.ts properly track costs for the new models, including the Kimi K2 model.

🐛 Potential Issues

  1. Inconsistent model fallback: In model.ts:174 and model.ts:197, when a configured model provider doesn't have an API key, the code falls back to the default model. This might mask configuration issues. Consider making this behavior more explicit.

  2. Test environment mocking: The test file mocks all environment variables, which is good, but ensure that production environment variables are properly validated.

💡 Suggestions

  1. Add JSDoc comments: Consider adding JSDoc comments to the new functions explaining when to use each model type:

    /**
     * Select chat model for fast conversational tasks
     * Falls back to default model if chat model is not configured
     */
    function selectChatModel(userAi: UserAIFields) { ... }
  2. Consider adding model type validation: Add runtime validation to ensure the modelType parameter is one of the expected values.

  3. Documentation: Consider adding examples in the environment file or README showing how to configure the new model types.

Overall Assessment

This is a solid implementation that adds important functionality for model optimization. The code is well-structured, tested, and follows existing patterns. The main concerns are around error handling clarity and documentation. The changes should improve both performance (with economy models) and user experience (with chat models).

Recommendation: ✅ Approve with minor suggestions

The code is ready for merge with the minor improvements suggested above being addressed in follow-up PRs if needed.


Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
apps/web/utils/llms/model.test.ts (1)

222-354: Consider creating helper functions for test data consistency.

Based on the retrieved learnings, you should create helper functions for common test data to ensure consistency and reduce duplication. The new tests repeat similar patterns of setting up userAi objects and environment variables.

Consider creating helper functions to reduce duplication:

// Helper function for creating test user AI data
const createTestUserAi = (overrides: Partial<UserAIFields> = {}): UserAIFields => ({
  aiApiKey: null,
  aiProvider: null,
  aiModel: null,
  ...overrides,
});

// Helper function for setting up environment variables
const setupEnvForModelType = (modelType: string, provider: string, model: string) => {
  const providerVar = `${modelType.toUpperCase()}_LLM_PROVIDER`;
  const modelVar = `${modelType.toUpperCase()}_LLM_MODEL`;
  vi.mocked(env)[providerVar] = provider;
  vi.mocked(env)[modelVar] = model;
};

This would align with the learning about creating helper functions for common test data.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dfe92cc and 2546dc4.

📒 Files selected for processing (3)
  • apps/web/env.ts (2 hunks)
  • apps/web/utils/llms/model.test.ts (2 hunks)
  • apps/web/utils/llms/model.ts (6 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • apps/web/utils/llms/model.ts
  • apps/web/env.ts
🧰 Additional context used
📓 Path-based instructions (2)
apps/web/**/*.{ts,tsx}

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • apps/web/CLAUDE.md
**/*.{ts,tsx}

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • .cursor/rules/form-handling.mdc
🧠 Learnings (1)
apps/web/utils/llms/model.test.ts (7)
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: When writing tests for LLM-related functionality in TypeScript (e.g., in 'apps/web/__tests__/*.test.ts'), always create helper functions for common test data to ensure consistency and reduce duplication.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Test both AI and non-AI code paths in LLM-related functionality to ensure comprehensive coverage.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-06-23T12:27:30.570Z
Learning: When mocking Prisma in Vitest, import the Prisma mock from '@/utils/__mocks__/prisma', mock '@/utils/prisma', and clear all mocks in a beforeEach hook to ensure test isolation.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Standard test cases for LLM-related functionality should include: happy path with expected input, error handling, edge cases (such as empty input or null values), different user configurations, and various input formats.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Use descriptive 'console.debug' statements to log generated content in LLM-related tests for easier debugging and traceability.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-06-23T12:26:27.693Z
Learning: Do not mock the LLM call in LLM-related tests; always call the actual LLM to ensure test reliability and real-world coverage.
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/posthog-feature-flags.mdc:0-0
Timestamp: 2025-06-23T12:26:51.119Z
Learning: For A/B test feature flags, always define a TypeScript type for the possible variants and ensure the hook returns a default fallback value (such as 'control') if the flag is not set.
🧬 Code Graph Analysis (1)
apps/web/utils/llms/model.test.ts (4)
apps/web/utils/llms/types.ts (1)
  • UserAIFields (3-9)
apps/web/env.ts (1)
  • env (15-209)
apps/web/utils/llms/model.ts (1)
  • getModel (18-32)
apps/web/utils/llms/config.ts (1)
  • Provider (7-15)
🪛 GitHub Check: test
apps/web/utils/llms/model.test.ts

[failure] 325-325: utils/llms/model.test.ts > Models > getModel > should use OpenRouter with provider options for default model
AssertionError: expected undefined to deeply equal [ 'Google Vertex', 'Anthropic' ]

  • Expected:
    [
    "Google Vertex",
    "Anthropic",
    ]
  • Received:
    undefined

❯ utils/llms/model.test.ts:325:67

🪛 GitHub Actions: Run Tests
apps/web/utils/llms/model.test.ts

[error] 325-325: Test failure in 'should use OpenRouter with provider options for default model': AssertionError - expected undefined to deeply equal [ 'Google Vertex', 'Anthropic' ]. The test expected the provider options order array to be defined but received undefined.

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: claude-review
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (8)
apps/web/utils/llms/model.test.ts (8)

41-47: Environment variables correctly updated for new modelType functionality.

The mock environment variables properly reflect the new schema with separate provider/model configurations for chat, economy, and default model types, including OpenRouter provider options.


222-236: Test correctly validates chat model selection.

The test properly verifies that when modelType is "chat", the function selects the configured chat provider and model from environment variables.


238-257: Test correctly validates OpenRouter provider options for chat.

The test properly verifies that OpenRouter provider options are correctly parsed and applied when using the chat model type.


259-274: Test correctly validates economy model selection.

The test properly verifies that when modelType is "economy", the function selects the configured economy provider and model from environment variables.


276-296: Test correctly validates OpenRouter provider options for economy.

The test properly verifies that OpenRouter provider options are correctly parsed and applied when using the economy model type.


298-308: Test correctly validates default model selection.

The test properly verifies that when modelType is "default", the function falls back to the default provider and model configuration.


331-354: Test correctly validates custom provider logic preservation.

The test properly verifies that when using a custom provider, the default OpenRouter provider options don't override the custom logic. This ensures the custom provider behavior is preserved.


310-329: Incorrect assumption about DEFAULT_OPENROUTER_PROVIDERS handling

The model.ts implementation already injects env.DEFAULT_OPENROUTER_PROVIDERS when aiProvider === Provider.OPENROUTER and no providerOptions.openrouter exists:

if (
  aiProvider === Provider.OPENROUTER &&
  env.DEFAULT_OPENROUTER_PROVIDERS &&
  !providerOptions.openrouter
) {
  providerOptions.openrouter = createOpenRouterProviderOptions(
    env.DEFAULT_OPENROUTER_PROVIDERS,
  );
}

The failing test isn’t due to missing logic in selectModelByType or getModel, but rather to how the Vitest mock of @/env is applied after the module has already been imported. To fix the test:

• In apps/web/utils/llms/model.test.ts, move or wrap your vi.mock("@/env", …) call so it runs before any import of getModel.

  • Either place the mock at the very top of the file
  • Or call vi.resetModules() followed by your mock, then dynamically import the module under test within each it block.

Example adjustment:

// At the very top before any imports
vi.mock("@/env", () => ({
  env: {
    DEFAULT_LLM_PROVIDER: "openrouter",
    DEFAULT_LLM_MODEL: "anthropic/claude-3.5-sonnet",
    DEFAULT_OPENROUTER_PROVIDERS: "Google Vertex,Anthropic",
    OPENROUTER_API_KEY: "test-openrouter-key",
    // …other defaults
  },
}));

import { getModel } from "@/utils/llms/model";

Or per-test:

it("should use OpenRouter …", async () => {
  vi.resetModules();
  vi.mock("@/env", () => ({
    env: {
      DEFAULT_LLM_PROVIDER: "openrouter",
      DEFAULT_LLM_MODEL: "anthropic/claude-3.5-sonnet",
      DEFAULT_OPENROUTER_PROVIDERS: "Google Vertex,Anthropic",
      OPENROUTER_API_KEY: "test-openrouter-key",
    },
  }));
  const { getModel } = await import("@/utils/llms/model");
  // …test logic…
});

With the mock in place before module initialization, providerOptions.openrouter will correctly be populated and the test will pass.

Likely an incorrect or invalid review comment.

@claude
Copy link

claude bot commented Jul 18, 2025

Claude finished @elie222's task —— View job


Pull Request Review

Reviewing PR #577: "Add seperate chat model"

Review Checklist:

  • Read repository CLAUDE.md for context
  • Review environment variable changes (env.ts)
  • Analyze model configuration changes (model.ts)
  • Review test coverage (model.test.ts)
  • Check LLM utility changes (index.ts)
  • Review AI assistant integration changes
  • Assess security implications
  • Evaluate performance considerations
  • Provide comprehensive feedback

Code Review Feedback

Positive Aspects

  1. Well-structured separation of concerns: The addition of separate chat and economy models allows for optimized model selection based on use case.

  2. Comprehensive test coverage: The test suite in model.test.ts properly covers the new model type functionality with good edge case handling.

  3. Proper environment variable validation: The new environment variables are properly validated using Zod schema.

  4. Consistent API design: The modelType parameter is consistently used throughout the codebase.

⚠️ Areas for Improvement

  1. Typo in PR title: "seperate" should be "separate" - consider updating the PR title for consistency.

  2. Missing environment variable documentation: The new environment variables (CHAT_LLM_PROVIDER, CHAT_LLM_MODEL, ECONOMY_LLM_PROVIDER, ECONOMY_LLM_MODEL) lack inline documentation explaining their purpose and when to use them.

  3. Incomplete error handling: In selectChatModel and selectEconomyModel functions (model.ts:193-207), when the API key is not found, the function logs a warning but falls back to the default model. Consider whether this is the desired behavior or if it should throw an error.

  4. Potential performance impact: The getProviderApiKey function is called for every chat/economy model request. Consider caching these values or restructuring to avoid repeated lookups.

🔒 Security Considerations

  1. API key handling: The code properly handles API keys by checking for their existence before using them. Good practice.

  2. Environment variable exposure: All sensitive configuration is properly kept server-side in the env.ts file.

🏃 Performance Considerations

  1. Model selection logic: The three-tier model selection (chat, economy, default) is efficient and allows for cost optimization.

  2. Usage tracking: The changes to usage.ts properly track costs for the new models, including the Kimi K2 model.

🐛 Potential Issues

  1. Inconsistent model fallback: In model.ts:174 and model.ts:197, when a configured model provider doesn't have an API key, the code falls back to the default model. This might mask configuration issues. Consider making this behavior more explicit.

  2. Test environment mocking: The test file mocks all environment variables, which is good, but ensure that production environment variables are properly validated.

💡 Suggestions

  1. Add JSDoc comments: Consider adding JSDoc comments to the new functions explaining when to use each model type:

    /**
     * Select chat model for fast conversational tasks
     * Falls back to default model if chat model is not configured
     */
    function selectChatModel(userAi: UserAIFields) { ... }
  2. Consider adding model type validation: Add runtime validation to ensure the modelType parameter is one of the expected values.

  3. Documentation: Consider adding examples in the environment file or README showing how to configure the new model types.

Overall Assessment

This is a solid implementation that adds important functionality for model optimization. The code is well-structured, tested, and follows existing patterns. The main concerns are around error handling clarity and documentation. The changes should improve both performance (with economy models) and user experience (with chat models).

Recommendation: ✅ Approve with minor suggestions

The code is ready for merge with the minor improvements suggested above being addressed in follow-up PRs if needed.


@claude
Copy link

claude bot commented Jul 18, 2025

Claude finished @elie222's task —— View job


Pull Request Review

Reviewing PR #577: "Add seperate chat model"

Review Checklist:

  • Read repository CLAUDE.md for context
  • Review environment variable changes (env.ts)
  • Analyze model configuration changes (model.ts)
  • Review test coverage (model.test.ts)
  • Check LLM utility changes (index.ts)
  • Review AI assistant integration changes
  • Assess security implications
  • Evaluate performance considerations
  • Provide comprehensive feedback

Code Review Feedback

Positive Aspects

  1. Well-structured separation of concerns: The addition of separate chat and economy models allows for optimized model selection based on use case.

  2. Comprehensive test coverage: The test suite in model.test.ts properly covers the new model type functionality with good edge case handling.

  3. Proper environment variable validation: The new environment variables are properly validated using Zod schema.

  4. Consistent API design: The modelType parameter is consistently used throughout the codebase.

⚠️ Areas for Improvement

  1. Typo in PR title: "seperate" should be "separate" - consider updating the PR title for consistency.

  2. Missing environment variable documentation: The new environment variables (CHAT_LLM_PROVIDER, CHAT_LLM_MODEL, ECONOMY_LLM_PROVIDER, ECONOMY_LLM_MODEL) lack inline documentation explaining their purpose and when to use them.

  3. Incomplete error handling: In selectChatModel and selectEconomyModel functions (model.ts:193-207), when the API key is not found, the function logs a warning but falls back to the default model. Consider whether this is the desired behavior or if it should throw an error.

  4. Potential performance impact: The getProviderApiKey function is called for every chat/economy model request. Consider caching these values or restructuring to avoid repeated lookups.

🔒 Security Considerations

  1. API key handling: The code properly handles API keys by checking for their existence before using them. Good practice.

  2. Environment variable exposure: All sensitive configuration is properly kept server-side in the env.ts file.

🏃 Performance Considerations

  1. Model selection logic: The three-tier model selection (chat, economy, default) is efficient and allows for cost optimization.

  2. Usage tracking: The changes to usage.ts properly track costs for the new models, including the Kimi K2 model.

🐛 Potential Issues

  1. Inconsistent model fallback: In model.ts:174 and model.ts:197, when a configured model provider doesn't have an API key, the code falls back to the default model. This might mask configuration issues. Consider making this behavior more explicit.

  2. Test environment mocking: The test file mocks all environment variables, which is good, but ensure that production environment variables are properly validated.

💡 Suggestions

  1. Add JSDoc comments: Consider adding JSDoc comments to the new functions explaining when to use each model type:

    /**
     * Select chat model for fast conversational tasks
     * Falls back to default model if chat model is not configured
     */
    function selectChatModel(userAi: UserAIFields) { ... }
  2. Consider adding model type validation: Add runtime validation to ensure the modelType parameter is one of the expected values.

  3. Documentation: Consider adding examples in the environment file or README showing how to configure the new model types.

Overall Assessment

This is a solid implementation that adds important functionality for model optimization. The code is well-structured, tested, and follows existing patterns. The main concerns are around error handling clarity and documentation. The changes should improve both performance (with economy models) and user experience (with chat models).

Recommendation: ✅ Approve with minor suggestions

The code is ready for merge with the minor improvements suggested above being addressed in follow-up PRs if needed.


@elie222 elie222 merged commit 737a4a9 into main Jul 18, 2025
13 checks passed
@elie222 elie222 deleted the feat/chat-model branch July 18, 2025 17:26
@coderabbitai coderabbitai bot mentioned this pull request Aug 6, 2025
@coderabbitai coderabbitai bot mentioned this pull request Sep 16, 2025
This was referenced Dec 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant