feat(agent): Introduce Python code execution as prompt strategy #7142
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for auto-gpt-docs canceled.
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #7142 +/- ##
==========================================
- Coverage 53.79% 53.43% -0.36%
==========================================
Files 124 127 +3
Lines 7027 7293 +266
Branches 911 967 +56
==========================================
+ Hits 3780 3897 +117
- Misses 3114 3258 +144
- Partials 133 138 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
Should this be in forge?? |
CI Failure Feedback 🧐(Checks updated until commit e204491)
✨ CI feedback usage guide:The CI feedback tool (
In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR: where Configuration options
See more information about the |
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
github-actions
bot
removed
the
conflicts
majdyz
commented
May 15, 2024
| from .base import BaseAgent, BaseAgentActionProposal | ||
| from .prompt_strategies.one_shot import OneShotAgentActionProposal |
There was a problem hiding this comment.
OneShotAgentActionProposal is not part of .agent
majdyz
commented
May 15, 2024
| @@ -97,19 +103,20 @@ def __init__( | |||
| llm_provider: ChatModelProvider, | |||
| file_storage: FileStorage, | |||
| legacy_config: Config, | |||
| prompt_strategy_class: type[PromptStrategy] = CodeFlowAgentPromptStrategy, | |||
There was a problem hiding this comment.
decouple agent.py & OneShot. pass this class for custom prompt strategy
majdyz
commented
May 15, 2024
| if not parsed_response.python_code: | ||
| raise ValueError("python_code is empty") | ||
|
|
||
| available_functions = { |
There was a problem hiding this comment.
This is for the generated code validation.
majdyz
commented
May 15, 2024
autogpts/autogpt/autogpt/app/cli.py
Outdated
| @@ -2,18 +2,21 @@ | |||
| from logging import _nameToLevel as logLevelMap | |||
| from pathlib import Path | |||
| from typing import Optional | |||
| from dotenv import load_dotenv | |||
There was a problem hiding this comment.
Ignore this, for debugging purpose, will revert
majdyz
commented
May 15, 2024
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class CodeFlowExecutionComponent(CommandProvider): |
There was a problem hiding this comment.
So... BaseAgentActionProposal use use_tool which only executes a single tool, and it's already intertwined with the agent implementation. So my approach on having this code flow execution is by returning BaseAgentActionProposal that use this newly created tool, which is basically executing python code using already existing other tools as variables.
majdyz
commented
May 15, 2024
| @@ -158,6 +159,37 @@ def fmt_line(self) -> str: | |||
| ) | |||
| return f"{self.name}: {self.description}. Params: ({params})" | |||
|
|
|||
| def fmt_header(self, callable=None) -> str: | |||
There was a problem hiding this comment.
callable part here is unused, will revert
majdyz
commented
May 15, 2024
| @@ -443,6 +443,8 @@ async def create_chat_completion( | |||
| if not parse_errors: | |||
| try: | |||
| parsed_result = completion_parser(assistant_msg) | |||
| if isinstance(parsed_result, Coroutine): | |||
There was a problem hiding this comment.
So completion_parser needs to be async for the code validation, I did this instead of changing the whole usage of it to async.
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
github-actions
bot
added
the
conflicts
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
github-actions
bot
removed
the
conflicts
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
github-actions
bot
removed
the
conflicts
- Amend `main.py:update_user(..)` to improve displaying of existing + new prompt strategy output - Replace `"execute_code_flow"` literals with references to the command definition
github-actions
bot
added
the
conflicts
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
|
Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly. |
github-actions
bot
removed
the
conflicts
|
Do we have a status on this? |
|
This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request. |
github-actions
bot
added
the
conflicts
|
@majdyz can I close this PR, seems we're not working on it? |
|
This could be a really awesome block |
|
@aarushik93 sure I'll close it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
Add support for executing agent steps by running Python code. This allows for more complicated composite function calls within each step of agent execution.
Related:
Changes 🏗️
PromptStrategy:CodeFlowAgentPromptStrategyundercode_flow.pyutils.functionpackage that handle a python code validation that utilizeruff&pyRightas the validation engines.CodeFlowAgentPromptStrategyas the existing default strategy.PR Quality Scorecard ✨
+2 pts+5 pts+5 pts+5 pts-4 pts+4 pts+5 pts-5 ptsagbenchmarkto verify that these changes do not regress performance?+10 pts