Fix prompt for conversation classification#870
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughThis pull request refactors thread message sorting from reverse-chronological to chronological order (oldest to newest) across the reply-tracker system, enhances thread status determination criteria with detailed rules on promise-handling perspectives, and updates corresponding tests to use SystemType enums and dated test data. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes The changes involve moderate logic density with a new comparator function and refined criteria text, applied consistently across multiple related files (reply-tracker utilities). The modifications follow a coherent pattern (sorting refactor + prompt enhancement) rather than disparate changes, reducing per-file reasoning overhead. Test updates are straightforward enum and date-handling adjustments. Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/web/utils/reply-tracker/handle-conversation-status.ts (1)
60-71: Add error handling around AI call and reduce PII in logs.
- Wrap aiDetermineThreadStatus in try/catch to avoid bubbling failures.
- Log rationale at debug (or truncate) to limit PII at info level.
Apply:
- const { status, rationale } = await aiDetermineThreadStatus({ - emailAccount, - threadMessages: threadMessagesForLLM, - modelType, - }); - - logger.info("AI determined thread status", { - status, - rationale, - messageId: message.id, - }); + let status: SystemType; + let rationale: string; + try { + ({ status, rationale } = await aiDetermineThreadStatus({ + emailAccount, + threadMessages: threadMessagesForLLM, + modelType, + })); + } catch (err) { + logger.error("Failed to determine thread status", { + messageId: message.id, + threadId: message.threadId, + error: (err as Error)?.message, + }); + return { + rule: null, + reason: "AI status determination failed", + }; + } + + logger.info("AI determined thread status", { + status, + messageId: message.id, + }); + logger.debug("AI rationale", { rationale });As per coding guidelines
🧹 Nitpick comments (5)
apps/web/__tests__/helpers.ts (1)
26-43: Avoid DST skew in sequential dates.Using setHours can drift across DST. Prefer millisecond arithmetic.
Apply:
-export function generateSequentialDates( - count: number, - hoursApart = 1, - startDate = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000), -): Date[] { - return Array.from({ length: count }, (_, i) => { - const date = new Date(startDate); - date.setHours(date.getHours() + i * hoursApart); - return date; - }); -} +export function generateSequentialDates( + count: number, + hoursApart = 1, + startDate = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000), +): Date[] { + const stepMs = hoursApart * 3_600_000; + return Array.from({ length: count }, (_, i) => new Date(startDate.getTime() + i * stepMs)); +}apps/web/utils/ai/reply/determine-thread-status.ts (1)
91-99: Use z.nativeEnum(SystemType) to tighten validation.This avoids tuple literal typing quirks and stays in sync with Prisma enum.
Apply:
- const schema = z.object({ - status: z.enum([ - SystemType.TO_REPLY, - SystemType.FYI, - SystemType.AWAITING_REPLY, - SystemType.ACTIONED, - ]), - rationale: z.string(), - }); + const schema = z.object({ + status: z.nativeEnum(SystemType), + rationale: z.string(), + });apps/web/__tests__/determine-thread-status.test.ts (2)
24-71: Stabilize test dates for determinism.Use a fixed startDate to eliminate “now” variance and DST effects.
Apply:
- const getProjectThread = () => { + const getProjectThread = () => { + const base = new Date("2024-01-01T09:00:00.000Z"); const emailData = [ // ... ]; - const dates = generateSequentialDates(emailData.length, 2); // 2 hours apart + const dates = generateSequentialDates(emailData.length, 2, base); // 2 hours apart return emailData.map((email, index) => getEmail({ ...email, date: dates[index] }), ); };
388-443: Apply fixed base date to lunch thread helper too.Same determinism rationale as above.
Apply:
- const getLunchSchedulingThread = ( + const getLunchSchedulingThread = ( person1Email: string, person2Email: string, ) => { + const base = new Date("2024-01-02T09:00:00.000Z"); const emailData = [ /* ... */ ]; - const dates = generateSequentialDates(emailData.length, 3); // 3 hours apart + const dates = generateSequentialDates(emailData.length, 3, base); // 3 hours apart return emailData.map((email, index) => getEmail({ ...email, date: dates[index] }), ); };apps/web/utils/reply-tracker/handle-conversation-status.ts (1)
50-55: Sorting + last-message weighting LGTM.Oldest→newest via sortByInternalDate() and giving the last message higher maxLength is correct. Consider stabilizing ties.
Tie-break equal timestamps to keep order stable:
- const sortedMessages = [...threadMessages].sort(sortByInternalDate()); + const sortedMessages = [...threadMessages].sort((a, b) => { + const cmp = sortByInternalDate()(a, b); + if (cmp !== 0) return cmp; + return a.id.localeCompare(b.id); + });
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
apps/web/__tests__/determine-thread-status.test.ts(21 hunks)apps/web/__tests__/helpers.ts(3 hunks)apps/web/utils/ai/reply/determine-thread-status.ts(3 hunks)apps/web/utils/date.ts(1 hunks)apps/web/utils/reply-tracker/handle-conversation-status.ts(2 hunks)apps/web/utils/reply-tracker/outbound.ts(3 hunks)version.txt(1 hunks)
🧰 Additional context used
📓 Path-based instructions (16)
apps/web/**/*.{ts,tsx}
📄 CodeRabbit inference engine (apps/web/CLAUDE.md)
apps/web/**/*.{ts,tsx}: Use TypeScript with strict null checks
Path aliases: Use@/for imports from project root
Use proper error handling with try/catch blocks
Format code with Prettier
Leverage TypeScript inference for better DX
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsapps/web/utils/ai/reply/determine-thread-status.tsapps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.tsapps/web/utils/reply-tracker/outbound.ts
!{.cursor/rules/*.mdc}
📄 CodeRabbit inference engine (.cursor/rules/cursor-rules.mdc)
Never place rule files in the project root, in subdirectories outside .cursor/rules, or in any other location
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsversion.txtapps/web/utils/ai/reply/determine-thread-status.tsapps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.tsapps/web/utils/reply-tracker/outbound.ts
**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/form-handling.mdc)
**/*.ts: The same validation should be done in the server action too
Define validation schemas using Zod
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsapps/web/utils/ai/reply/determine-thread-status.tsapps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.tsapps/web/utils/reply-tracker/outbound.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/logging.mdc)
**/*.{ts,tsx}: UsecreateScopedLoggerfor logging in backend TypeScript files
Typically add the logger initialization at the top of the file when usingcreateScopedLogger
Only use.with()on a logger instance within a specific function, not for a global loggerImport Prisma in the project using
import prisma from "@/utils/prisma";
**/*.{ts,tsx}: Don't use TypeScript enums.
Don't use TypeScript const enum.
Don't use the TypeScript directive @ts-ignore.
Don't use primitive type aliases or misleading types.
Don't use empty type parameters in type aliases and interfaces.
Don't use any or unknown as type constraints.
Don't use implicit any type on variable declarations.
Don't let variables evolve into any type through reassignments.
Don't use non-null assertions with the ! postfix operator.
Don't misuse the non-null assertion operator (!) in TypeScript files.
Don't use user-defined types.
Use as const instead of literal types and type annotations.
Use export type for types.
Use import type for types.
Don't declare empty interfaces.
Don't merge interfaces and classes unsafely.
Don't use overload signatures that aren't next to each other.
Use the namespace keyword instead of the module keyword to declare TypeScript namespaces.
Don't use TypeScript namespaces.
Don't export imported variables.
Don't add type annotations to variables, parameters, and class properties that are initialized with literal expressions.
Don't use parameter properties in class constructors.
Use either T[] or Array consistently.
Initialize each enum member value explicitly.
Make sure all enum members are literal values.
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsapps/web/utils/ai/reply/determine-thread-status.tsapps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.tsapps/web/utils/reply-tracker/outbound.ts
apps/web/utils/**
📄 CodeRabbit inference engine (.cursor/rules/project-structure.mdc)
Create utility functions in
utils/folder for reusable logic
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsapps/web/utils/ai/reply/determine-thread-status.tsapps/web/utils/reply-tracker/outbound.ts
apps/web/utils/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/project-structure.mdc)
apps/web/utils/**/*.ts: Use lodash utilities for common operations (arrays, objects, strings)
Import specific lodash functions to minimize bundle size
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsapps/web/utils/ai/reply/determine-thread-status.tsapps/web/utils/reply-tracker/outbound.ts
**/*.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/ultracite.mdc)
**/*.{js,jsx,ts,tsx}: Don't useelements in Next.js projects.
Don't use elements in Next.js projects.
Don't use namespace imports.
Don't access namespace imports dynamically.
Don't use global eval().
Don't use console.
Don't use debugger.
Don't use var.
Don't use with statements in non-strict contexts.
Don't use the arguments object.
Don't use consecutive spaces in regular expression literals.
Don't use the comma operator.
Don't use unnecessary boolean casts.
Don't use unnecessary callbacks with flatMap.
Use for...of statements instead of Array.forEach.
Don't create classes that only have static members (like a static namespace).
Don't use this and super in static contexts.
Don't use unnecessary catch clauses.
Don't use unnecessary constructors.
Don't use unnecessary continue statements.
Don't export empty modules that don't change anything.
Don't use unnecessary escape sequences in regular expression literals.
Don't use unnecessary labels.
Don't use unnecessary nested block statements.
Don't rename imports, exports, and destructured assignments to the same name.
Don't use unnecessary string or template literal concatenation.
Don't use String.raw in template literals when there are no escape sequences.
Don't use useless case statements in switch statements.
Don't use ternary operators when simpler alternatives exist.
Don't use useless this aliasing.
Don't initialize variables to undefined.
Don't use the void operators (they're not familiar).
Use arrow functions instead of function expressions.
Use Date.now() to get milliseconds since the Unix Epoch.
Use .flatMap() instead of map().flat() when possible.
Use literal property access instead of computed property access.
Don't use parseInt() or Number.parseInt() when binary, octal, or hexadecimal literals work.
Use concise optional chaining instead of chained logical expressions.
Use regular expression literals instead of the RegExp constructor when possible.
Don't use number literal object member names th...
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsapps/web/utils/ai/reply/determine-thread-status.tsapps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.tsapps/web/utils/reply-tracker/outbound.ts
!pages/_document.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/ultracite.mdc)
!pages/_document.{js,jsx,ts,tsx}: Don't import next/document outside of pages/_document.jsx in Next.js projects.
Don't import next/document outside of pages/_document.jsx in Next.js projects.
Files:
apps/web/utils/date.tsapps/web/utils/reply-tracker/handle-conversation-status.tsversion.txtapps/web/utils/ai/reply/determine-thread-status.tsapps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.tsapps/web/utils/reply-tracker/outbound.ts
apps/web/utils/ai/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/llm.mdc)
apps/web/utils/ai/**/*.{ts,tsx}: Place main LLM feature implementations under apps/web/utils/ai/
LLM feature functions should follow the provided TypeScript pattern (separate system/user prompts, use createGenerateObject, Zod schema validation, early validation, return result.object)
Keep system prompts and user prompts separate
System prompt should define the LLM's role and task specifications
User prompt should contain the actual data and context
Always define a Zod schema for response validation
Make Zod schemas as specific as possible to guide LLM output
Use descriptive scoped loggers for each feature
Log inputs and outputs with appropriate log levels and include relevant context
Implement early returns for invalid inputs
Use proper error types and logging for failures
Implement fallbacks for AI failures
Add retry logic for transient failures using withRetry
Use XML-like tags to structure data in prompts
Remove excessive whitespace and truncate long inputs in prompts
Format prompt data consistently across similar functions
Use TypeScript types for all parameters and return values in LLM features
Define clear interfaces for complex input/output structures in LLM features
Files:
apps/web/utils/ai/reply/determine-thread-status.ts
apps/web/utils/{ai,llms}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/llm.mdc)
Keep related AI functions co-located and extract common patterns into utilities; document complex AI logic with clear comments
Files:
apps/web/utils/ai/reply/determine-thread-status.ts
**/*.test.{ts,js}
📄 CodeRabbit inference engine (.cursor/rules/security.mdc)
Include security tests in your test suites to verify authentication, authorization, and error handling.
Files:
apps/web/__tests__/determine-thread-status.test.ts
**/*.{test,spec}.{js,jsx,ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/ultracite.mdc)
**/*.{test,spec}.{js,jsx,ts,tsx}: Don't use export or module.exports in test files.
Don't use focused tests.
Don't use disabled tests.
Make sure the assertion function, like expect, is placed inside an it() function call.
Don't nest describe() blocks too deeply in test files.
Don't use focused tests.
Don't use disabled tests.
Don't use export or module.exports in test files.
Files:
apps/web/__tests__/determine-thread-status.test.ts
apps/web/__tests__/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/llm.mdc)
Place LLM-specific tests under apps/web/tests/
Files:
apps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/testing.mdc)
**/*.test.{ts,tsx}: Use Vitest (vitest) as the testing framework
Colocate tests next to the file under test (e.g., dir/format.ts with dir/format.test.ts)
In tests, mock theserver-onlymodule withvi.mock("server-only", () => ({}));
When testing code that uses Prisma, mock it withvi.mock("@/utils/prisma")and use the mock from@/utils/__mocks__/prisma
Use provided helpers for mocks: import{ getEmail, getEmailAccount, getRule }from@/__tests__/helpers
Each test should be independent
Use descriptive test names
Mock external dependencies in tests
Clean up mocks between tests (e.g.,vi.clearAllMocks()inbeforeEach)
Avoid testing implementation details; focus on observable behavior
Do not mock the Logger
Files:
apps/web/__tests__/determine-thread-status.test.ts
**/__tests__/**
📄 CodeRabbit inference engine (.cursor/rules/testing.mdc)
Place AI tests in the
__tests__directory and exclude them from the default test run (they use a real LLM)
Files:
apps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.ts
apps/web/__tests__/**/*.test.ts
📄 CodeRabbit inference engine (.cursor/rules/llm-test.mdc)
apps/web/__tests__/**/*.test.ts: Place all LLM-related tests under apps/web/tests/
Use Vitest in LLM tests and import { describe, expect, test, vi, beforeEach } from "vitest"
Mock the Next.js server runtime marker by adding vi.mock("server-only", () => ({})) in LLM tests
Gate LLM tests behind RUN_AI_TESTS using describe.runIf(process.env.RUN_AI_TESTS === "true")
Call vi.clearAllMocks() in a beforeEach for LLM tests
Set a TIMEOUT of 15_000ms for LLM-related tests and pass it to long-running tests/describe blocks
Create helper functions for common test data (e.g., getUser, getTestData) to reduce duplication
Include standard test cases: happy path, error handling, edge cases (empty/null), different user configurations, and various input formats
Use console.debug to log generated LLM content for inspection (e.g., console.debug("Generated content:\n", result.content))
Do not mock the actual LLM call in these tests; exercise real LLM integrations
Test both AI and non-AI paths, including cases where no AI processing is required
Prefer existing helpers from @/tests/helpers.ts (getEmailAccount, getEmail, getRule, getMockMessage, getMockExecutedRule) over custom helpers
Files:
apps/web/__tests__/determine-thread-status.test.ts
🧠 Learnings (4)
📚 Learning: 2025-10-02T23:23:48.064Z
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-10-02T23:23:48.064Z
Learning: Applies to apps/web/__tests__/**/*.test.ts : Use Vitest in LLM tests and import { describe, expect, test, vi, beforeEach } from "vitest"
Applied to files:
apps/web/__tests__/determine-thread-status.test.ts
📚 Learning: 2025-09-20T18:24:34.280Z
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-09-20T18:24:34.280Z
Learning: Applies to **/*.test.{ts,tsx} : Use provided helpers for mocks: import `{ getEmail, getEmailAccount, getRule }` from `@/__tests__/helpers`
Applied to files:
apps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.ts
📚 Learning: 2025-10-02T23:23:48.064Z
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-10-02T23:23:48.064Z
Learning: Applies to apps/web/__tests__/**/*.test.ts : Prefer existing helpers from @/__tests__/helpers.ts (getEmailAccount, getEmail, getRule, getMockMessage, getMockExecutedRule) over custom helpers
Applied to files:
apps/web/__tests__/determine-thread-status.test.tsapps/web/__tests__/helpers.ts
📚 Learning: 2025-10-02T23:23:48.064Z
Learnt from: CR
PR: elie222/inbox-zero#0
File: .cursor/rules/llm-test.mdc:0-0
Timestamp: 2025-10-02T23:23:48.064Z
Learning: Applies to apps/web/__tests__/**/*.test.ts : Gate LLM tests behind RUN_AI_TESTS using describe.runIf(process.env.RUN_AI_TESTS === "true")
Applied to files:
apps/web/__tests__/determine-thread-status.test.ts
🧬 Code graph analysis (3)
apps/web/utils/reply-tracker/handle-conversation-status.ts (2)
apps/web/utils/date.ts (1)
sortByInternalDate(94-102)apps/web/utils/get-email-from-message.ts (1)
getEmailForLLM(6-25)
apps/web/__tests__/determine-thread-status.test.ts (2)
apps/web/__tests__/helpers.ts (3)
generateSequentialDates(33-43)getEmail(45-64)getEmailAccount(6-24)apps/web/utils/ai/reply/determine-thread-status.ts (1)
aiDetermineThreadStatus(10-109)
apps/web/utils/reply-tracker/outbound.ts (2)
apps/web/utils/get-email-from-message.ts (1)
getEmailForLLM(6-25)apps/web/utils/date.ts (1)
sortByInternalDate(94-102)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: cubic · AI code reviewer
🔇 Additional comments (9)
version.txt (1)
1-1: Version bump looks good.No issues.
apps/web/__tests__/helpers.ts (2)
14-15: EmailAccount helper extension OK.multiRuleSelectionEnabled default is fine and aligns with tests.
52-63: getEmail(date) extension LGTM.Optional date passthrough matches EmailForLLM usage in tests.
apps/web/utils/ai/reply/determine-thread-status.ts (2)
23-69: Prompt clarifications and perspective rules are solid.Clearer TO_REPLY vs AWAITING_REPLY handling for promises; chronological instruction is explicit.
72-72: Verified: All production callers properly sort messages chronologically before invoking.Both
handle-conversation-status.ts(line 50) andoutbound.ts(line 117) explicitly sort usingsortByInternalDate()with the default ascending direction, ensuring oldest→newest order. The sorted messages are then mapped tothreadMessagesForLLMbefore being passed toaiDetermineThreadStatus. Test cases construct messages in chronological order. No ordering gaps found.apps/web/__tests__/determine-thread-status.test.ts (1)
3-9: Good: helpers and Prisma SystemType in tests, plus AI gating.Matches testing guidelines (Vitest, helpers, RUN_AI_TESTS, console.debug for LLM).
Based on learnings
apps/web/utils/reply-tracker/outbound.ts (3)
7-7: LGTM!The import is properly structured and follows the project's path alias conventions.
56-63: LGTM!The updated comment accurately reflects the chronological ordering, and the
maxLengthlogic correctly identifies the latest message as the last element in the chronologically sorted array.
110-131: The review comment is incorrect; no null handling issue exists.The implementation is sound. internalDateToDate handles null/undefined by returning a new Date object, so sortByInternalDate reliably converts all messages to timestamps regardless of whether internalDate is present. Messages with missing internalDate values receive the current time as a fallback and sort correctly alongside others. The function isMessageLatestInThread correctly identifies the latest message as the last element in the chronologically sorted array, which is the proper behavior for ascending-order sorting.
The code requires no changes.
Likely an incorrect or invalid review comment.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 7 files
Prompt for AI agents (all 2 issues)
Understand the root cause of the following 2 issues and fix them.
<file name="apps/web/utils/date.ts">
<violation number="1" location="apps/web/utils/date.ts:98">
`internalDateToDate` returns the current time when `internalDate` is missing or invalid, so this `|| 0` fallback never fires. Messages without an internalDate will be sorted as if they happened "now", pushing them into the middle of the timeline instead of the start/end. Guard for missing values before calling `internalDateToDate` so they fall back to epoch.</violation>
</file>
<file name="apps/web/__tests__/helpers.ts">
<violation number="1" location="apps/web/__tests__/helpers.ts:14">
Please respect the overrides parameter when setting multiRuleSelectionEnabled; otherwise tests calling getEmailAccount({ multiRuleSelectionEnabled: true }) will still receive false and cannot exercise the enabled state.</violation>
</file>
React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.
1 participant
Summary by CodeRabbit
Bug Fixes
Chores