Skip to content

Conversation

louis-menlo
Copy link
Contributor

@louis-menlo louis-menlo commented Jul 23, 2025

Summary

This PR replaces Jan's hardcoded tool-use preferred assistant prompt with a general assistant instruction that works better across all model types and sizes.

CleanShot 2025-07-23 at 12 15 19@2x CleanShot 2025-07-23 at 12 15 24@2x

Problem

The previous default assistant prompt was optimized specifically for tool usage, which caused issues:

  • Overwhelmed smaller models with complex tool-specific instructions
  • Created inconsistent behavior across different model capabilities
  • Forced all models to use the same tool-focused approach

Important

Replaces Jan's tool-focused prompt with a general instruction to improve assistant performance across models.

  • Behavior:
    • Replaces Jan's tool-focused prompt with a general assistant instruction in index.ts and useAssistant.ts.
    • New prompt emphasizes direct knowledge, conciseness, and tool use only when necessary.
  • Tests:
    • Updates useAssistant.test.ts to reflect new prompt instructions.
    • Ensures assistant structure and behavior align with updated instructions.
  • Misc:
    • Minor formatting changes in useAssistant.test.ts.

This description was created by Ellipsis for 02afdaf. You can customize this summary. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed everything up to 613c53c in 54 seconds. Click for details.
  • Reviewed 26 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. extensions/assistant-extension/src/index.ts:77
  • Draft comment:
    The default assistant instructions have been updated to a more general and user-friendly version. This improves compatibility across model types. Consider extracting this common text into a shared constant or configuration file if it is to be reused elsewhere to avoid duplication in the future.
  • Reason this comment was not posted:
    Confidence changes required: 0% <= threshold 50% None
2. web-app/src/hooks/useAssistant.ts:23
  • Draft comment:
    The assistant's instructions here are updated to the same general prompt, which is clear and concise. It might be beneficial to centralize these instructions to reduce duplication and ease future maintenance.
  • Reason this comment was not posted:
    Confidence changes required: 0% <= threshold 50% None

Workflow ID: wflow_TNTJvFzo8ciqHuuW

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed 02afdaf in 1 minute and 15 seconds. Click for details.
  • Reviewed 205 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. web-app/src/hooks/__tests__/useAssistant.test.ts:156
  • Draft comment:
    The expected instructions string now reflects the new generic prompt ('Only use tools when they add real value to your response'). Ensure that the default assistant configuration is updated accordingly.
  • Reason this comment was not posted:
    Comment did not seem useful. Confidence is useful = 0% <= threshold 50% This comment is purely informative and asks the PR author to ensure that the default assistant configuration is updated. It doesn't provide a specific code suggestion or ask for a specific test to be written. It violates the rule against asking the author to ensure something is updated.
2. web-app/src/hooks/__tests__/useAssistant.test.ts:22
  • Draft comment:
    There are multiple whitespace and trailing comma changes which are non-functional. Ensure these formatting changes are intentional and consistent with the project’s style guidelines.
  • Reason this comment was not posted:
    Confidence changes required: 33% <= threshold 50% None

Workflow ID: wflow_I4lmX4PzpfseUNXf

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

Copy link
Contributor

Barecheck - Code coverage report

Total: 36.95%

Your code coverage diff: -0.01% ▾

Uncovered files and lines
FileLines
web-app/src/hooks/useAssistant.ts33, 46, 50, 57

@louis-menlo louis-menlo merged commit af116dd into release/v0.6.6 Jul 23, 2025
54 of 62 checks passed
@louis-menlo louis-menlo deleted the fix/jan-default-assistant-prompt branch July 23, 2025 06:55
@github-project-automation github-project-automation bot moved this to QA in Jan Jul 23, 2025
@github-actions github-actions bot added this to the v0.6.6 milestone Jul 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

1 participant