Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
840c548
adjust button text
elie222 Dec 17, 2024
17603c3
Show onboarding dialog while waiting for rules to process
elie222 Dec 17, 2024
cac3562
Simpler check if thread or not
elie222 Dec 17, 2024
166c221
reset modal upon close
elie222 Dec 17, 2024
3cbfb4b
Merge branch 'main' into loading-onboarding-modal
elie222 Dec 17, 2024
c9cdf8f
working on onboarding loading modal
elie222 Dec 18, 2024
9f7c50d
adjust survey question
elie222 Dec 18, 2024
1f80327
Adjust sign up survey questions
elie222 Dec 18, 2024
0d80ef1
WIP on multi condition support
elie222 Dec 18, 2024
5471ae8
Save multi conditions
elie222 Dec 18, 2024
80a8d96
Fix multi condition not showing
elie222 Dec 18, 2024
d52a886
Multi condition display in table
elie222 Dec 18, 2024
6da5a8a
Fix instructions string
elie222 Dec 18, 2024
ecab8f2
Warn about duplicate conditions
elie222 Dec 18, 2024
61767ef
Stop using Rule type across app
elie222 Dec 19, 2024
10a0421
Add findPotentialMatchingRules function
elie222 Dec 19, 2024
e4bf5e1
optimise findPotentialMatchingRules
elie222 Dec 19, 2024
313e57c
handle multiple conditions. unify rule handling
elie222 Dec 19, 2024
c0f6e4d
Big clean up of condition/action logic
elie222 Dec 19, 2024
0ec71cd
Clean up migration file
elie222 Dec 19, 2024
a460c03
Add a rule for static match
elie222 Dec 19, 2024
64b2eeb
fix bug for group not matching
elie222 Dec 20, 2024
aa1a59e
fix spacing
elie222 Dec 20, 2024
33fd5a9
Fix smart category match bug
elie222 Dec 20, 2024
116429c
Add test for category exclude
elie222 Dec 20, 2024
5aa72c7
fix isCategoryRule
elie222 Dec 20, 2024
fe75996
rename
elie222 Dec 20, 2024
2c3acb6
Fix risk tests
elie222 Dec 20, 2024
1824231
skip ai tests when running regular test command
elie222 Dec 20, 2024
41e053a
Add GitHub actions to automate non ai tests on push
elie222 Dec 20, 2024
281abd2
fix run test comment
elie222 Dec 20, 2024
32e9d8c
Fix ai choose rule test
elie222 Dec 20, 2024
25dd063
fix test timeout
elie222 Dec 20, 2024
89c9518
Add match rule tests
elie222 Dec 20, 2024
2a2e249
Merge branch 'main' into loading-onboarding-modal
elie222 Dec 20, 2024
2973b3c
Don't put Rule in rule name
elie222 Dec 20, 2024
a085ab3
Make sure at least one condition for a rule
elie222 Dec 20, 2024
37e8ac7
Simplify manual prompt creation
elie222 Dec 20, 2024
e63a06a
Show spinner in progress panel
elie222 Dec 20, 2024
416a31d
adjust copy
elie222 Dec 20, 2024
67c71fc
Remove unused vars
elie222 Dec 20, 2024
37d8662
Add test for match ai + group
elie222 Dec 20, 2024
e7847c8
ai + category non match test
elie222 Dec 20, 2024
dac204e
Hide view examples. Add link to matched rule
elie222 Dec 20, 2024
3ecbf70
Add radio group match/skip for smart category
elie222 Dec 20, 2024
c037166
Fix up saving multi condition form
elie222 Dec 21, 2024
b7729a6
Remove excessive whitespace before sending to ai
elie222 Dec 21, 2024
d7c3198
show reasoning in test results
elie222 Dec 21, 2024
7a4d29b
link direct to email for test row
elie222 Dec 21, 2024
237bd00
set up smart categories link
elie222 Dec 21, 2024
1c4271d
node 22 for gh actions
elie222 Dec 21, 2024
51d019d
hide loading onboarding dialog
elie222 Dec 21, 2024
b921c2b
fix css
elie222 Dec 21, 2024
5798d61
fix warning
elie222 Dec 21, 2024
236c232
make content at least 3 rows
elie222 Dec 21, 2024
25490c4
PR review fixes
elie222 Dec 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Run Tests

on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "22"

- name: Install pnpm
uses: pnpm/action-setup@v2
with:
version: 8

- name: Get pnpm store directory
shell: bash
run: |
echo "STORE_PATH=\"$(pnpm store path --silent)\"" >> $GITHUB_ENV

- uses: actions/cache@v4
name: Setup pnpm cache
with:
path: ~/.pnpm-store
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-

- name: Install dependencies
run: pnpm install

- name: Run tests
run: pnpm -F inbox-zero-ai test
env:
Comment on lines +42 to +44
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider expanding test coverage.

The current configuration only runs tests for the inbox-zero-ai package, but the PR includes changes to web components and utilities. Consider running tests for all affected packages.

-      - name: Run tests
-        run: pnpm -F inbox-zero-ai test
+      - name: Run all tests
+        run: pnpm test
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Run tests
run: pnpm -F inbox-zero-ai test
env:
- name: Run all tests
run: pnpm test
env:

RUN_AI_TESTS: false
DATABASE_URL: "postgresql://postgres:postgres@localhost:5432/postgres"
NEXTAUTH_SECRET: "secret"
GOOGLE_CLIENT_ID: "client_id"
GOOGLE_CLIENT_SECRET: "client_secret"
GOOGLE_PUBSUB_TOPIC_NAME: "topic"
Comment on lines +42 to +50
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add PostgreSQL service container and secure credentials.

Two critical issues need to be addressed:

  1. The workflow references a PostgreSQL database but doesn't define a service container to run it.
  2. Sensitive credentials should use GitHub secrets instead of hardcoded values.

Add a PostgreSQL service container and use GitHub secrets:

 jobs:
   test:
     runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:latest
+        env:
+          POSTGRES_PASSWORD: postgres
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5

     steps:
       # ... existing steps ...
       - name: Run tests
         run: pnpm -F inbox-zero-ai test
         env:
           RUN_AI_TESTS: false
           DATABASE_URL: "postgresql://postgres:postgres@localhost:5432/postgres"
-          NEXTAUTH_SECRET: "secret"
-          GOOGLE_CLIENT_ID: "client_id"
-          GOOGLE_CLIENT_SECRET: "client_secret"
-          GOOGLE_PUBSUB_TOPIC_NAME: "topic"
+          NEXTAUTH_SECRET: ${{ secrets.NEXTAUTH_SECRET }}
+          GOOGLE_CLIENT_ID: ${{ secrets.GOOGLE_CLIENT_ID }}
+          GOOGLE_CLIENT_SECRET: ${{ secrets.GOOGLE_CLIENT_SECRET }}
+          GOOGLE_PUBSUB_TOPIC_NAME: ${{ secrets.GOOGLE_PUBSUB_TOPIC_NAME }}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Run tests
run: pnpm -F inbox-zero-ai test
env:
RUN_AI_TESTS: false
DATABASE_URL: "postgresql://postgres:postgres@localhost:5432/postgres"
NEXTAUTH_SECRET: "secret"
GOOGLE_CLIENT_ID: "client_id"
GOOGLE_CLIENT_SECRET: "client_secret"
GOOGLE_PUBSUB_TOPIC_NAME: "topic"
- name: Run tests
run: pnpm -F inbox-zero-ai test
env:
RUN_AI_TESTS: false
DATABASE_URL: "postgresql://postgres:postgres@localhost:5432/postgres"
NEXTAUTH_SECRET: ${{ secrets.NEXTAUTH_SECRET }}
GOOGLE_CLIENT_ID: ${{ secrets.GOOGLE_CLIENT_ID }}
GOOGLE_CLIENT_SECRET: ${{ secrets.GOOGLE_CLIENT_SECRET }}
GOOGLE_PUBSUB_TOPIC_NAME: ${{ secrets.GOOGLE_PUBSUB_TOPIC_NAME }}

9 changes: 6 additions & 3 deletions apps/web/__tests__/ai-categorize-senders.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@ import {
import { defaultCategory } from "@/utils/categories";
import { aiCategorizeSender } from "@/utils/ai/categorize-sender/ai-categorize-single-sender";

// pnpm test-ai ai-categorize-senders

const isAiTest = process.env.RUN_AI_TESTS === "true";

vi.mock("server-only", () => ({}));

// Test data setup
const testUser = {
email: "user@test.com",
aiProvider: null,
Expand Down Expand Up @@ -44,7 +47,7 @@ const testSenders = [
},
];

describe("AI Sender Categorization", () => {
describe.skipIf(!isAiTest)("AI Sender Categorization", () => {
describe("Bulk Categorization", () => {
it("should categorize senders with snippets using AI", async () => {
const result = await aiCategorizeSenders({
Expand Down Expand Up @@ -125,7 +128,7 @@ describe("AI Sender Categorization", () => {
expect(result?.category).toBe(expectedCategory);
}
}
}, 15_000);
}, 30_000);

it("should handle unknown sender appropriately", async () => {
const unknownSender = testSenders.find(
Expand Down
17 changes: 13 additions & 4 deletions apps/web/__tests__/ai-choose-args.test.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
import { describe, expect, test, vi } from "vitest";
import { getActionItemsWithAiArgs } from "@/utils/ai/choose-rule/ai-choose-args";
import { type Action, ActionType, RuleType } from "@prisma/client";
import { type Action, ActionType, LogicalOperator } from "@prisma/client";
import type { RuleWithActions } from "@/utils/types";

// pnpm test-ai ai-choose-args

const isAiTest = process.env.RUN_AI_TESTS === "true";

vi.mock("server-only", () => ({}));

describe("getActionItemsWithAiArgs", () => {
describe.skipIf(!isAiTest)("getActionItemsWithAiArgs", () => {
test("should return actions unchanged when no AI args needed", async () => {
const actions = [getAction({})];
const rule = getRule("Test rule", actions);
Expand Down Expand Up @@ -167,7 +172,10 @@ function getAction(action: Partial<Action> = {}): Action {
};
}

function getRule(instructions: string, actions: Action[] = []) {
function getRule(
instructions: string,
actions: Action[] = [],
): RuleWithActions {
return {
instructions,
name: "Test Rule",
Expand All @@ -183,9 +191,10 @@ function getRule(instructions: string, actions: Action[] = []) {
subject: null,
body: null,
to: null,
type: RuleType.AI,
enabled: true,
categoryFilterType: null,
conditionalOperator: LogicalOperator.AND,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Test coverage needed for OR and NOT logical operators

The current test files only include test cases for LogicalOperator.AND. Additional test coverage is needed to ensure proper functionality with:

  • LogicalOperator.OR operator
  • LogicalOperator.NOT operator
  • Combinations of multiple conditions using different operators
🔗 Analysis chain

Verify test coverage for different logical operators

While the default LogicalOperator.AND is set, consider adding test cases for:

  • Different logical operators (OR, NOT)
  • Complex conditions with multiple operators
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if there are test cases for different logical operators

# Search for test cases using different logical operators
rg -l "LogicalOperator\.(OR|NOT)" "apps/web/__tests__"

# Search for potential complex condition tests
rg -l "conditionalOperator.*LogicalOperator" "apps/web/__tests__"

Length of output: 204


Script:

#!/bin/bash
# Let's examine the actual test cases in these files
rg -B 2 -A 2 "conditionalOperator.*LogicalOperator" "apps/web/__tests__/ai-choose-rule.test.ts" "apps/web/__tests__/ai-choose-args.test.ts"

Length of output: 764

type: null,
};
}

Expand Down
165 changes: 76 additions & 89 deletions apps/web/__tests__/ai-choose-rule.test.ts
Original file line number Diff line number Diff line change
@@ -1,114 +1,101 @@
import { expect, test, vi } from "vitest";
import { chooseRule } from "@/utils/ai/choose-rule/choose";
import { type Action, ActionType, RuleType } from "@prisma/client";
import { describe, expect, test, vi } from "vitest";
import { aiChooseRule } from "@/utils/ai/choose-rule/ai-choose-rule";
import { type Action, ActionType, LogicalOperator } from "@prisma/client";

vi.mock("server-only", () => ({}));

test("Should return no rule when no rules passed", async () => {
const result = await chooseRule({
rules: [],
email: getEmail(),
user: getUser(),
});

expect(result).toEqual({ reason: "No rules" });
});
// pnpm test-ai ai-choose-rule

test("Should return correct rule when only one rule passed", async () => {
const rule = getRule(
"Match emails that have the word 'test' in the subject line",
);
const isAiTest = process.env.RUN_AI_TESTS === "true";

const result = await chooseRule({
email: getEmail({ subject: "test" }),
rules: [rule],
user: getUser(),
});
vi.mock("server-only", () => ({}));

expect(result).toEqual({ rule, reason: expect.any(String), actionItems: [] });
});
describe.skipIf(!isAiTest)("aiChooseRule", () => {
test("Should return no rule when no rules passed", async () => {
const result = await aiChooseRule({
rules: [],
email: getEmail(),
user: getUser(),
});

test("Should return correct rule when multiple rules passed", async () => {
const rule1 = getRule(
"Match emails that have the word 'test' in the subject line",
);
const rule2 = getRule(
"Match emails that have the word 'remember' in the subject line",
);

const result = await chooseRule({
rules: [rule1, rule2],
email: getEmail({ subject: "remember that call" }),
user: getUser(),
expect(result).toEqual({ reason: "No rules" });
});

expect(result).toEqual({
rule: rule2,
reason: expect.any(String),
actionItems: [],
test("Should return correct rule when only one rule passed", async () => {
const rule = getRule(
"Match emails that have the word 'test' in the subject line",
);

const result = await aiChooseRule({
email: getEmail({ subject: "test" }),
rules: [rule],
user: getUser(),
});

expect(result).toEqual({
rule,
reason: expect.any(String),
});
});
});

test("Should generate action arguments", async () => {
const rule1 = getRule(
"Match emails that have the word 'question' in the subject line",
);
const rule2 = getRule("Match emails asking for a joke", [
{
id: "id",
createdAt: new Date(),
updatedAt: new Date(),
type: ActionType.REPLY,
ruleId: "ruleId",
label: null,
subject: null,
content: "{{Write a joke}}",
to: null,
cc: null,
bcc: null,

labelPrompt: null,
subjectPrompt: null,
contentPrompt: null,
toPrompt: null,
ccPrompt: null,
bccPrompt: null,
},
]);

const result = await chooseRule({
rules: [rule1, rule2],
email: getEmail({ subject: "Joke", content: "Tell me a joke about sheep" }),
user: getUser(),
test("Should return correct rule when multiple rules passed", async () => {
const rule1 = getRule(
"Match emails that have the word 'test' in the subject line",
);
const rule2 = getRule(
"Match emails that have the word 'remember' in the subject line",
);

const result = await aiChooseRule({
rules: [rule1, rule2],
email: getEmail({ subject: "remember that call" }),
user: getUser(),
});

expect(result).toEqual({
rule: rule2,
reason: expect.any(String),
});
});

Comment on lines +39 to 58
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add test cases for different LogicalOperator values

The test suite only tests AND operator. Consider adding test cases for other LogicalOperator values.

test("Should handle OR operator correctly", async () => {
  const rule = getRule("Match emails with 'test' or 'remember'");
  rule.conditionalOperator = LogicalOperator.OR;
  
  const result = await aiChooseRule({
    rules: [rule],
    email: getEmail({ subject: "remember that call" }),
    user: getUser(),
  });

  expect(result.rule).toBe(rule);
});

console.debug("Generated content:\n", result.actionItems?.[0].content);

expect(result).toEqual({
rule: rule2,
reason: expect.any(String),
actionItems: [
test("Should generate action arguments", async () => {
const rule1 = getRule(
"Match emails that have the word 'question' in the subject line",
);
const rule2 = getRule("Match emails asking for a joke", [
{
bcc: null,
cc: null,
content: expect.any(String),
id: "id",
createdAt: new Date(),
updatedAt: new Date(),
type: ActionType.REPLY,
ruleId: "ruleId",
label: null,
subject: null,
content: "{{Write a joke}}",
to: null,
type: "REPLY",
cc: null,
bcc: null,

labelPrompt: null,
subjectPrompt: null,
contentPrompt: null,
toPrompt: null,
ccPrompt: null,
bccPrompt: null,
createdAt: expect.any(Date),
updatedAt: expect.any(Date),
id: "id",
ruleId: "ruleId",
},
],
]);

const result = await aiChooseRule({
rules: [rule1, rule2],
email: getEmail({
subject: "Joke",
content: "Tell me a joke about sheep",
}),
user: getUser(),
});

expect(result).toEqual({
rule: rule2,
reason: expect.any(String),
});
});
});

Expand All @@ -129,9 +116,9 @@ function getRule(instructions: string, actions: Action[] = []) {
subject: null,
body: null,
to: null,
type: RuleType.AI,
enabled: true,
categoryFilterType: null,
conditionalOperator: LogicalOperator.AND,
};
}

Expand Down
6 changes: 5 additions & 1 deletion apps/web/__tests__/ai-create-group.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,16 @@ import { aiGenerateGroupItems } from "@/utils/ai/group/create-group";
import { queryBatchMessages } from "@/utils/gmail/message";
import type { ParsedMessage } from "@/utils/types";

// pnpm test-ai ai-create-group

const isAiTest = process.env.RUN_AI_TESTS === "true";

vi.mock("server-only", () => ({}));
vi.mock("@/utils/gmail/message", () => ({
queryBatchMessages: vi.fn(),
}));

describe("aiGenerateGroupItems", () => {
describe.skipIf(!isAiTest)("aiGenerateGroupItems", () => {
it("should generate group items based on user prompt", async () => {
const user = {
email: "user@test.com",
Expand Down
6 changes: 5 additions & 1 deletion apps/web/__tests__/ai-diff-rules.test.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
import { describe, it, expect, vi } from "vitest";
import { aiDiffRules } from "@/utils/ai/rule/diff-rules";

// pnpm test-ai ai-diff-rules

const isAiTest = process.env.RUN_AI_TESTS === "true";

vi.mock("server-only", () => ({}));

describe("aiDiffRules", () => {
describe.skipIf(!isAiTest)("aiDiffRules", () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Inconsistent timeout values and missing documentation found across AI test files

  • Most tests use 15_000 ms timeout, but ai-categorize-senders.test.ts also uses 30_000 ms in some cases
  • Only ai-example-matches.test.ts and ai-create-group.test.ts have comments explaining the timeout reason
  • ai-example-matches.test.ts uses 15000 (without underscore) unlike other files using 15_000
🔗 Analysis chain

Document the timeout rationale.

The 15-second timeout is used for AI operations. Consider adding a comment explaining why this specific duration was chosen and if it should be consistent across all AI-related tests.

Also applies to: 52-52

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for consistency in timeout values across AI test files
rg "(\d+_000)" "apps/web/__tests__/ai-*.test.ts"

Length of output: 122


Script:

#!/bin/bash
# Let's first find the correct test files
fd "ai.*test" apps/web/__tests__

# Then search for timeout values in these files
rg "timeout|(\d+_000)" apps/web/__tests__

Length of output: 1148

it("should correctly identify added, edited, and removed rules", async () => {
const user = {
email: "user@test.com",
Expand Down
6 changes: 5 additions & 1 deletion apps/web/__tests__/ai-example-matches.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,16 @@ import { queryBatchMessages } from "@/utils/gmail/message";
import type { ParsedMessage } from "@/utils/types";
import { findExampleMatchesSchema } from "@/utils/ai/example-matches/find-example-matches";

// pnpm test-ai ai-find-example-matches

const isAiTest = process.env.RUN_AI_TESTS === "true";

vi.mock("server-only", () => ({}));
vi.mock("@/utils/gmail/message", () => ({
queryBatchMessages: vi.fn(),
}));

describe("aiFindExampleMatches", () => {
describe.skipIf(!isAiTest)("aiFindExampleMatches", () => {
it("should find example matches based on user prompt", async () => {
const user = {
email: "user@test.com",
Expand Down
6 changes: 4 additions & 2 deletions apps/web/__tests__/ai-find-snippets.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@ import { describe, expect, test, vi } from "vitest";
import { aiFindSnippets } from "@/utils/ai/snippets/find-snippets";
import type { EmailForLLM } from "@/utils/ai/choose-rule/stringify-email";

// pnpm test ai-find-snippets
// pnpm test-ai ai-find-snippets

const isAiTest = process.env.RUN_AI_TESTS === "true";

vi.mock("server-only", () => ({}));

describe("aiFindSnippets", () => {
describe.skipIf(!isAiTest)("aiFindSnippets", () => {
test("should find snippets in similar emails", async () => {
const emails = [
getEmail({
Expand Down
Loading
Loading