-
-
Notifications
You must be signed in to change notification settings - Fork 7.1k
Improved preprompts for improve flow #1045
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1045 +/- ##
==========================================
- Coverage 83.41% 82.96% -0.45%
==========================================
Files 25 25
Lines 1272 1274 +2
==========================================
- Hits 1061 1057 -4
- Misses 211 217 +6 ☔ View full report in Codecov by Sentry. |
@ErikBjare is there a quick explanation why the codecov workflow fails? |
@ATheorell codecov.io has these configurable thresholds for what constitutes an acceptable code coverage. Since we only use it for informational purposes, and not as a blocker for merging PRs, we don't want the checks to fail. I was supposed to configure this, but I cannot since I don't have org-level maintainer access. Here are the steps:
coverage:
status:
patch:
default:
informational: true
project:
default:
informational: true |
Thanks for instructions. I have now updated the global yaml. Tried to rerun the jobs, but maybe I need to make a new commit for this to come into effect. Anyway, we can still merge if we want. |
Hi @ATheorell , this looks fantastic! I've noticed instances where the LLMs yield no differences, leading to hit 'no diff' exceptions for users. My workaround has been advising users to reselect files and modify their prompts. However, ensuring that the LLMs consistently offer guidance on changes would enhance user experience, as that's the primary reason users engage with this feature. |
I have noticed that the current preprompts for the improve flow sometimes leads to lazy behavior, in the sense that no or little code is written. In this PR, I have aligned the preprompts as much as possible with the generate workflow, which we know is not lazy. I hope this will lead to tangible improvements and hopefully we can soon test the actual improvement using the apps benchmarks which is about to be supported (#1025).