Feature: Added Self Feedback #3013
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@merwanehamadi Here is the Adverse/Feedback |
|
If it's merged I can add it to the usage.md |
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
k-boikov
added
Needs Benchmark
Can you add it in the PR so the docs get updated instantly? |
You are right I'll do it now |
github-actions
bot
removed
the
conflicts
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #3013 +/- ##
==========================================
- Coverage 49.52% 41.14% -8.38%
==========================================
Files 64 63 -1
Lines 2946 3028 +82
Branches 497 498 +1
==========================================
- Hits 1459 1246 -213
- Misses 1365 1717 +352
+ Partials 122 65 -57
... and 26 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
|
LGTM - Didn't test though |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #3013 +/- ##
==========================================
- Coverage 49.93% 41.07% -8.86%
==========================================
Files 64 64
Lines 2900 2924 +24
Branches 488 492 +4
==========================================
- Hits 1448 1201 -247
- Misses 1333 1667 +334
+ Partials 119 56 -63
... and 14 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
richbeales
approved these changes
Apr 25, 2023
|
Tested locally, LGTM |
5 tasks
misterak8
added a commit
to misterak8/Auto-GPT
that referenced
this pull request
Apr 26, 2023
* Clean up GitHub Workflows (Significant-Gravitas#3059) * initial cleanup of github workflows * only run pr-label workflow on push to master * move docker ci/release summaries to scripts * add XS label for PR's under 2 lines * draft test job for Docker CI * fix & activate Docker CI test job * add debug step to docker CI * fix Docker CI test container env * Docker CI build matrix * fixup build summaries * fix pipes in summary * optimize Dockerfile for layer caching * more markdown escaping * add gha cache scopes * add Docker CI cache clean workflow * More robust null byte checking * Adjust test_json_parser file (Significant-Gravitas#1935) Co-authored-by: Reinier van der Leer <[email protected]> * Add indirection layer between cli and application start * Agent and agent manager tests (Significant-Gravitas#3116) * Update Python version and benchmark file in benchmark.yml * Refactor main function and imports in cli.py * Update import statement in ai_config.py * Add set_temperature and set_memory_backend methods in config.py * Remove unused import in prompt.py * Add goal oriented tasks workflow * Added agent_utils to create agent * added pytest and vcrpy * added write file cassette * created goal oriented task write file with cassettes to not pay openai tokens * solve conflicts * add ability set azure because github workflow needs it off * solve conflicts in cli.py * black because linter fails * solve conflict * setup github action to v3 Signed-off-by: Merwane Hamadi <[email protected]> * fix conflicts Signed-off-by: Merwane Hamadi <[email protected]> * Plugins: debug line always printed in plugin load * add decorator to tests Signed-off-by: Merwane Hamadi <[email protected]> * move decorator higher up Signed-off-by: Merwane Hamadi <[email protected]> * init * more tests * passing tests * skip gitbranch decorator on ci * decorator skiponci * black * Update tests/utils.py decorator of skipping ci Co-authored-by: Nicholas Tindle <[email protected]> * black * I oopsed the name * black * finally * simple tests for agent and manager * ísort --------- Signed-off-by: Merwane Hamadi <[email protected]> Co-authored-by: Merwane Hamadi <[email protected]> Co-authored-by: Merwane Hamadi <[email protected]> Co-authored-by: Richard Beales <[email protected]> Co-authored-by: Nicholas Tindle <[email protected]> Co-authored-by: BillSchumacher <[email protected]> Co-authored-by: Nicholas Tindle <[email protected]> * anontation fix (Significant-Gravitas#3018) * anontation fix * fix param name and type --------- Co-authored-by: Richard Beales <[email protected]> * Hotfix/global agent manager workaround (Significant-Gravitas#3157) * Add indirection layer to entry point * Get around singleton pattern for AgentManager to fix tests * Remove dead permanent memory module (Significant-Gravitas#3145) * Remove dead permanent memory module * Delete sqlite db that snuck in * Refactor/move singleton out of config module (Significant-Gravitas#3161) * Add option to install plugin dependencies (Significant-Gravitas#3068) Co-authored-by: Nicholas Tindle <[email protected]> * Re-add install-plugin-deps to CLI (Significant-Gravitas#3170) * Update ci.yml (Significant-Gravitas#3179) * Update bulletin to warn about deprication (Significant-Gravitas#3181) * Ensure Fore.RED is followed by Fore.RESET (Significant-Gravitas#3182) This properly resets the terminal, ensuring that the red text is red and the normal text remains unaffected. Co-authored-by: Richard Beales <[email protected]> * Add a little more descriptive installation instruction (Significant-Gravitas#3180) * add Dev Container installation instruction to installation.md * add Dev Container installation instruction to installation.md * Update installation.md --------- Co-authored-by: Richard Beales <[email protected]> * Update installation.md (Significant-Gravitas#3166) * Refactor/decouple logger from global configuration (Significant-Gravitas#3171) * Decouple logging from the global configuration * Configure logging first * Clean up global voice engine creation * Remove class vars from logger * Remove duplicate implementation of --------- Co-authored-by: Richard Beales <[email protected]> * docs fix to image generation (Significant-Gravitas#3186) * Update README.md * Feature: Added Self Feedback (Significant-Gravitas#3013) * Feature: Added Self Feedback * minor fix: complied to flake8 * Add: Self Feedback To Usage.md * Add: role/goal allignment * Added: warning to usage.md * fix: Formatted with black --------- Co-authored-by: Richard Beales <[email protected]> * Update README.md * this changes it so the file from config is used, rather than a hardcoded name that might not exist (Significant-Gravitas#3189) * Test `Agent.create_agent_feedback` (Significant-Gravitas#3209) * remove plugin notice from CONTRIBUTING.md (Significant-Gravitas#3227) * Extract OpenAI API retry handler and unify ADA embeddings calls. (Significant-Gravitas#3191) * Extract retry logic, unify embedding functions * Add some docstrings * Remove embedding creation from API manager * Add test suite for retry handler * Make api manager fixture * Fix typing * Streamline tests * Use explicit API keys when querying openai rather than import time manipulation of the package attributes (Significant-Gravitas#3241) * Fix readme centering (Significant-Gravitas#3243) * Remove import time loading of config from llm_utils (Significant-Gravitas#3245) * adding probably erroneously removed return value from execut_shell, giving 'None' in return always otherise - not ideal (Significant-Gravitas#3212) Co-authored-by: James Collins <[email protected]> * Load .env in package init (Significant-Gravitas#3251) * Fix PR size autolabeler message (Significant-Gravitas#3194) * Fix `get_ada_embedding` return type (Significant-Gravitas#3263) --------- Signed-off-by: Merwane Hamadi <[email protected]> Co-authored-by: Reinier van der Leer <[email protected]> Co-authored-by: James Collins <[email protected]> Co-authored-by: YOUNESS ZEMZGUI <[email protected]> Co-authored-by: Nicholas Tindle <[email protected]> Co-authored-by: Media <[email protected]> Co-authored-by: Merwane Hamadi <[email protected]> Co-authored-by: Merwane Hamadi <[email protected]> Co-authored-by: Richard Beales <[email protected]> Co-authored-by: BillSchumacher <[email protected]> Co-authored-by: Nicholas Tindle <[email protected]> Co-authored-by: Lei Zhang <[email protected]> Co-authored-by: Daniel Chen <[email protected]> Co-authored-by: Deso <[email protected]> Co-authored-by: Lawrence Neal <[email protected]> Co-authored-by: Duong HD <[email protected]> Co-authored-by: ✔️ITtechtor <[email protected]> Co-authored-by: Toran Bruce Richards <[email protected]> Co-authored-by: AbTrax <[email protected]> Co-authored-by: Peter Petermann <[email protected]> Co-authored-by: Peter Svensson <[email protected]> Co-authored-by: Robin Richtsfeld <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
The rationale behind adding the self-feedback feature in the Agent class is to enhance the AI's ability to verify and validate its own thoughts, reasoning, plans, and criticism. This will help improve the AI's decision-making process and allow it to autonomously generate better responses over time. The self-feedback feature is aligned with the project's overall direction of creating an efficient and intelligent AI agent.
Changes
The specific changes made in this pull request involve adding a new method called get_self_feedback() to the Agent class. This method takes in a dictionary of thoughts containing keys such as 'reasoning', 'plan', 'thoughts', and 'criticism'. It combines these elements into a single feedback message and uses the create_chat_completion() function to generate a response based on the input message. This allows the AI to verify and validate its own thoughts before executing a command.
Documentation
The changes are documented through in-code comments and this explanation. The get_self_feedback() method has a docstring that explains its purpose, input, and output. Additionally, the rationale and changes sections above provide further context and explanation.
Test Plan
Testing the self-feedback functionality can be challenging due to the open-ended nature of the responses.
PR Quality Checklist
- My pull request is atomic and focuses on a single charge.
- I have thoroughly tested my changes with multiple different prompts.
- I have considered potential risks and mitigations for my changes.
- I have documented my changes clearly and comprehensively.
- I have not snuck in any "extra" small tweaks changes
Due to the open-ended nature of the Self-feedback getting continuous exact answers and negative answers is quite hard whilst getting good open answers is not always the same and can vary differ.