Draft: feat: allow running multiple commands at once

[yangkyeongmo]

Background

#3945

Changes

Documentation

Test Plan

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)			May 14, 2023 1:06am

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[Boostrix]

This could be really useful and as long as this is not supported we need to prime the llm to tell it that it can by default only execute a single action/command and needs to await a response before being able to execute a new action. Actions and commands also cannot be conditional directly.

That it is running in a REPL and that it would need to use shell commands or python scripts to get around this limitation.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[Boostrix]

This looks interesting and just so that you know: I kinda implemented a similar scheme to run multiple commands at once with less code by allowing pre/post execution hooks to be registered at the Command manager layer - and then added a new command to allow a chain of commands to be registered as "one" command.

Unlike you, I didn't bother with handling async stuff - I just wanted "shell-like" behavior:

• be able to do the equivalent of command1 && command2
• ideally with the option to return not just the output, but also an error code
• currently looking at supporting the equivalent of pipes: command1 | command2

Like you, my motiviation is to set up/prepare commands locally at the agent side via the LLM, which are then implicitly executed that way.

Overall, this might be something that should better be reviewed/discussed by the folks reworking the command system currently, see @DarylRodrigo :

• Command Base Class Interface #3824

[yangkyeongmo]

@Boostrix Cool. Would be helpful if you share your result!

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[Boostrix]

@Boostrix Cool. Would be helpful if you share your result!

you can look at the diff here: #4063
This is based on the old/original plugin interface where you could register pre/post execution callbacks.

That way, we can implement all sorts of fancy stuff with very little code - including new commands that are simply lists of other commands

So, in theory we could implement shell-like behavior:

check_something && do_something

By this way of implementing things, this would not have to go through the LLM at all, much less "traffic" (API tokens!)

or register new commands:

add_command <name> <params> [list of other commands]

[Swiftyos]

Great work @yangkyeongmo

I've been considering the idea of running multiple commands at the same time. It's part of the re-arch work that is going on.

One thing I was thinking is we maybe able to construct a DAG of planned actions and then run all steps that can be ran in parallel at the same time.

[yangkyeongmo]

@Swiftyos I see, thanks for your confirmation!

I was wondering if I should close this PR after peeking the re-arch figma. I'm really looking forward to result of re-arch 😄