Skip to content

Improved preprompts for improve flow #1045

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 9, 2024
Merged

Conversation

ATheorell
Copy link
Collaborator

I have noticed that the current preprompts for the improve flow sometimes leads to lazy behavior, in the sense that no or little code is written. In this PR, I have aligned the preprompts as much as possible with the generate workflow, which we know is not lazy. I hope this will lead to tangible improvements and hopefully we can soon test the actual improvement using the apps benchmarks which is about to be supported (#1025).

Copy link

codecov bot commented Mar 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.96%. Comparing base (b16eef1) to head (fc12cdc).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1045      +/-   ##
==========================================
- Coverage   83.41%   82.96%   -0.45%     
==========================================
  Files          25       25              
  Lines        1272     1274       +2     
==========================================
- Hits         1061     1057       -4     
- Misses        211      217       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ATheorell
Copy link
Collaborator Author

@ErikBjare is there a quick explanation why the codecov workflow fails?

@ErikBjare
Copy link
Collaborator

ErikBjare commented Mar 4, 2024

@ATheorell codecov.io has these configurable thresholds for what constitutes an acceptable code coverage.

Since we only use it for informational purposes, and not as a blocker for merging PRs, we don't want the checks to fail.

I was supposed to configure this, but I cannot since I don't have org-level maintainer access.

Here are the steps:

  1. Go here: https://app.codecov.io/gh/gpt-engineer-org
  2. Go to settings -> Global YAML
    Screenshotimage
  3. Paste this:
coverage:
  status:
    patch:
      default:
        informational: true
    project:
      default:
        informational: true

@ATheorell
Copy link
Collaborator Author

Thanks for instructions. I have now updated the global yaml. Tried to rerun the jobs, but maybe I need to make a new commit for this to come into effect. Anyway, we can still merge if we want.

@similato87
Copy link
Collaborator

Hi @ATheorell , this looks fantastic! I've noticed instances where the LLMs yield no differences, leading to hit 'no diff' exceptions for users. My workaround has been advising users to reselect files and modify their prompts. However, ensuring that the LLMs consistently offer guidance on changes would enhance user experience, as that's the primary reason users engage with this feature.
Additionally, introducing an option for users to specify their preference for the extent of changes—via a flag—could align well with the new configuration feature from @ErikBjare. This approach could make the tool more user-centric and adaptable to individual user needs.

@ATheorell ATheorell merged commit 178d17d into AntonOsika:main Mar 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants