Re-thinking signature of commands ?

[Boostrix]

Duplicates

• I have searched the existing issues

Summary 💡

A number of issues that the agent could normally deal with automatically, it doesn't detect because the signature of commands is that these just return the result of an action - the most recent example being:

• Command google returned: [] Located In USA. Google works on browser #4120

In other words, we may need a way for commands to return error codes (or more, think exceptions) so that the agent can alternate between different options to see what works

Examples 🌈

• Command google returned: [] Located In USA. Google works on browser #4120

Motivation 🔦

commands that fail should do so clearly, so that the agent can try alternatives

[ntindle]

In progress via rearch. Please comment in discord

[Boostrix]

ideally, we would like to be able to return not just the value of a command, but also its errors code - imagine a tuple of (errorcode,result).
And in general, it would indeed be useful if we could introduce a way to "connect" commands.
It seems, we already have several PRs where people worked out a way to run multiple commands:

• Draft: feat: allow running multiple commands at once #3947

I have added a new command to register a list of commands, and I am thinking of supporting shell-like behavior like || and &&
That way we could do pretty clever stuff, like locally running a command and conditionally running another one - without any of this having to go through the agent/llm exchange.

We could do some really clever stuff this way, and free up API token use quite a bit. Imagine being able to use pipes for instance. This would also standardize on the behavior and align it more with a conventional "shell", which any LLM is likely to be familiar with.

PS: would love to use discord, but like I mentioned elsewhere talking to @anonhostpi : don't seem to have access - while I can use discord and see other "channels" (sorry, for the IRC lingo), those invite URLs don't work at all -regardless of the browser/IP I am using: #1454 (comment)

[Boostrix]

In progress via rearch. Please comment in discord

Adding to this, I am not sure whether this should be closed - I just went on to discord to look for this work, only to realize that there is a ton of work happening that the folks not on discord cannot possibly be aware of:

Now, don't get me wrong, this is not to critizse anybody or anything - it's just stating the obvious, from my perspective.

You folks seem to assume that everybody and their dog is using discord these days, and indeed there is a ton of work happening there obviously (now I know!) - but there's such a disconnect between github and discord that it isn't even funny anymore.

Let me clear about it, I consider myself to have read most relevant stuff here on github and I consider myself familiar with the 101s of the project, I am able to make heads and tails of the code base - and I am able to come up with PRs and have started making larger changes - however, this disconnect I really wasn't expecting at all.

Turns out that I spent one weekend coding up a recursive BFDS-based workspace scheme on top of json to serialize agents and their tasks to disk, so that agent sessions could be discontinued - and it's only on discord that I am running into someone who's created a full planning plugin.

And there's most than just this single example - as is exemplified by the 5+ PRs that different people have openened in some cases to implement the same idea.

I do realize that this project is facing some interesting challenges due to it having gone viral - but I didn't expect that everything would happen behind the scenes - despite the wiki stating clearly that github and the wiki were to be used for persistent information.

Quite frankly, at this point, I am going to take some time off - because even just the idea of spending another weekend coding up something only to learn that it's already been implemented a month ago, is quite frankly rather frustrating.

Now, obviously potential contributors having or not having access to discord is not your problem - and if it works well for the project, all the better. It simply isn't what I was expecting when I read that github was for persistent information. I didn't expect there to be a "separate world" of contributors and contributions - and frankly, the project needs to connect these places to communicate what's happening in each.

Seeing 5+ PRs to implement "pause + require human feedback" is one thing, since we're talking about ~40 lines of code at most - but some of these contributions are far from trivial, touching a ton of places and adding new code or shuffling modules around.

Given all that, I no longer consider myself to be in a position to contribute much at this point, out of being afraid that my contributions could be redundant or obsolete in a matter of weeks/days.

PS: Like I also said elsewhere: commands would ideally use a namespace-based notation (think categories) - that way, we can provide more context to the LLM but also offer pagination or the LLM asking for a special category of commands (think I/O, internet etc)