-
Notifications
You must be signed in to change notification settings - Fork 1.2k
AI template responses #263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
4d3e45d
Add AI template highlighting to UI
elie222 2a243f3
Adjust ai choose args to use template format
elie222 8788d87
Update logger
elie222 65038ad
Create template AI rules from prompt file
elie222 14206e2
Allow empty string in prompt. Don't allow placeholders
elie222 2049106
improve prompt
elie222 92c043c
adjust test result styling
elie222 d23e8b0
Adjust placeholder
elie222 387c9d8
remove old prompt values. allow dynamic labels
elie222 ab3ab58
Remove unused package
elie222 48a96d5
Adjust risk for templates and add tests
elie222 dd177b1
Force correct format better
elie222 b3bc1f6
adjust error log webhook
elie222 8926c75
More resilient test
elie222 f138b0d
Push AI to give grammatically correct response
elie222 c77e363
Remove only for test
elie222 0830bbb
Extract hasVariables function
elie222 ab72d0b
hasVariables support for null
elie222 23330aa
Adjust error handling
elie222 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,212 @@ | ||
| import { describe, expect, test, vi } from "vitest"; | ||
| import { getActionItemsWithAiArgs } from "@/utils/ai/choose-rule/ai-choose-args"; | ||
| import { type Action, ActionType, RuleType } from "@prisma/client"; | ||
|
|
||
| vi.mock("server-only", () => ({})); | ||
|
|
||
| describe("getActionItemsWithAiArgs", () => { | ||
| test("should return actions unchanged when no AI args needed", async () => { | ||
| const actions = [getAction({})]; | ||
| const rule = getRule("Test rule", actions); | ||
|
|
||
| const result = await getActionItemsWithAiArgs({ | ||
| email: getEmail(), | ||
| user: getUser(), | ||
| selectedRule: rule, | ||
| }); | ||
|
|
||
| expect(result).toEqual(actions); | ||
| }); | ||
|
|
||
| test("should return actions unchanged when no variables to fill", async () => { | ||
| const actions = [ | ||
| getAction({ | ||
| type: ActionType.REPLY, | ||
| content: "You can set a meeting with me here: https://cal.com/alice", | ||
| }), | ||
| ]; | ||
| const rule = getRule("Choose this rule for meeting requests", actions); | ||
|
|
||
| const result = await getActionItemsWithAiArgs({ | ||
| email: getEmail({ | ||
| subject: "Quick question", | ||
| content: "When is the meeting tomorrow?", | ||
| }), | ||
| user: getUser(), | ||
| selectedRule: rule, | ||
| }); | ||
|
|
||
| expect(result).toHaveLength(1); | ||
| expect(result[0]).toMatchObject(actions[0]); | ||
| }); | ||
|
|
||
| test("should generate AI content for actions that need it", async () => { | ||
| const actions = [ | ||
| getAction({ | ||
| type: ActionType.REPLY, | ||
| content: | ||
| "The price of pears is: {{the price with the dollar sign - pears are $1.99, apples are $2.99}}", | ||
| }), | ||
| ]; | ||
| const rule = getRule( | ||
| "Choose this when the price of an items is asked for", | ||
| actions, | ||
| ); | ||
|
|
||
| const result = await getActionItemsWithAiArgs({ | ||
| email: getEmail({ | ||
| subject: "Quick question", | ||
| content: "How much are pears?", | ||
| }), | ||
| user: getUser(), | ||
| selectedRule: rule, | ||
| }); | ||
|
|
||
| expect(result).toHaveLength(1); | ||
| expect(result[0]).toMatchObject({ | ||
| ...actions[0], | ||
| content: "The price of pears is: $1.99", | ||
| }); | ||
| console.debug("Generated content:\n", result[0].content); | ||
| }); | ||
|
|
||
| test("should handle multiple actions with mixed AI needs", async () => { | ||
| const actions = [ | ||
| getAction({ | ||
| content: "Write a professional response", | ||
| }), | ||
| getAction({}), | ||
| ]; | ||
| const rule = getRule("Test rule", actions); | ||
|
|
||
| const result = await getActionItemsWithAiArgs({ | ||
| email: getEmail({ | ||
| subject: "Project status", | ||
| content: "Can you update me on the project status?", | ||
| }), | ||
| user: getUser(), | ||
| selectedRule: rule, | ||
| }); | ||
|
|
||
| expect(result).toHaveLength(2); | ||
| expect(result[0].content).toBeTruthy(); | ||
| expect(result[1]).toEqual(actions[1]); | ||
| }); | ||
|
|
||
| test("should handle multiple variables with specific formatting", async () => { | ||
| const actions = [ | ||
| getAction({ | ||
| type: ActionType.LABEL, | ||
| label: "{{fruit}}", | ||
| }), | ||
| getAction({ | ||
| type: ActionType.REPLY, | ||
| content: `Hey {{name}}, | ||
|
|
||
| {{$10 for apples, $20 for pears}} | ||
|
|
||
| Best, | ||
| Matt`, | ||
| }), | ||
| ]; | ||
| const rule = getRule( | ||
| "Use this when someone asks about the price of fruits", | ||
| actions, | ||
| ); | ||
|
|
||
| const result = await getActionItemsWithAiArgs({ | ||
| email: getEmail({ | ||
| from: "jill@example.com", | ||
| subject: "fruits", | ||
| content: "how much do apples cost?", | ||
| }), | ||
| user: getUser(), | ||
| selectedRule: rule, | ||
| }); | ||
|
|
||
| expect(result).toHaveLength(2); | ||
|
|
||
| // Check label action | ||
| expect(result[0].label).toBeTruthy(); | ||
| expect(result[0].label).not.toContain("{{"); | ||
| expect(result[0].label).toMatch(/apple(s)?/i); | ||
|
|
||
| // Check reply action | ||
| expect(result[1].content).toMatch(/^Hey [Jj]ill,/); // Match "Hey Jill," or "Hey jill," | ||
| expect(result[1].content).toContain("$10"); | ||
| expect(result[1].content).toContain("Best,\nMatt"); | ||
| expect(result[1].content).not.toContain("{{"); | ||
| expect(result[1].content).not.toContain("}}"); | ||
|
|
||
| console.debug("Generated label:\n", result[0].label); | ||
| console.debug("Generated content:\n", result[1].content); | ||
| }); | ||
| }); | ||
|
|
||
| // helpers | ||
| function getAction(action: Partial<Action> = {}): Action { | ||
| return { | ||
| id: "a123", | ||
| createdAt: new Date(), | ||
| updatedAt: new Date(), | ||
| type: ActionType.REPLY, | ||
| ruleId: "ruleId", | ||
| label: null, | ||
| subject: null, | ||
| content: null, | ||
| to: null, | ||
| cc: null, | ||
| bcc: null, | ||
| labelPrompt: null, | ||
| subjectPrompt: null, | ||
| contentPrompt: null, | ||
| toPrompt: null, | ||
| ccPrompt: null, | ||
| bccPrompt: null, | ||
| ...action, | ||
| }; | ||
| } | ||
|
|
||
| function getRule(instructions: string, actions: Action[] = []) { | ||
| return { | ||
| instructions, | ||
| name: "Test Rule", | ||
| actions, | ||
| id: "r123", | ||
| userId: "userId", | ||
| createdAt: new Date(), | ||
| updatedAt: new Date(), | ||
| automate: false, | ||
| runOnThreads: false, | ||
| groupId: null, | ||
| from: null, | ||
| subject: null, | ||
| body: null, | ||
| to: null, | ||
| type: RuleType.AI, | ||
| enabled: true, | ||
| categoryFilterType: null, | ||
| }; | ||
| } | ||
|
|
||
| function getEmail({ | ||
| from = "from@test.com", | ||
| subject = "subject", | ||
| content = "content", | ||
| }: { from?: string; subject?: string; content?: string } = {}) { | ||
| return { | ||
| from, | ||
| subject, | ||
| content, | ||
| }; | ||
| } | ||
|
|
||
| function getUser() { | ||
| return { | ||
| aiModel: null, | ||
| aiProvider: null, | ||
| email: "user@test.com", | ||
| aiApiKey: null, | ||
| about: null, | ||
| }; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codebase verification
Add test coverage for error handling scenarios
The implementation includes error handling for
InvalidToolArgumentsErrorin thewithRetryfunction, but there are no corresponding test cases. Consider adding tests for:InvalidToolArgumentsError🔗 Analysis chain
Verify test coverage of the implementation.
Let's ensure all code paths in the implementation are covered by these tests.
Let's analyze the implementation file we found. I notice several code paths that need verification in the test coverage:
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
Length of output: 12003
Script:
Length of output: 517
Script:
Length of output: 410
Script:
Length of output: 398