🚀 REST API: AGI can Create, Start and Stop multiple Autonomous Agents (Monitoring coming soon)🚀 #765

waynehamadi · 2023-04-11T03:42:51Z

Background

The goal is to build an AGI. This AGI might be one single Auto-GPT that loops against itself, but I doubt it because the LLM is super slow. We can't expect to have something that has high computing power with only one autonomous agent. And we can't have one agent per computing machine, that's stupid, because most of the time the running process will just hang and do nothing, waiting for Open AI to give back an answer. Waste of resources. Running multiple agents in different tabs is not ideal either because processes are heavier than threads.
On the current PR I was able to spawn 20 concurrent workers simultaneously without any problem.

So I think that in order to build an AGI, we need to build an entity able to:

Define autonomous agents.
Start autonomous agents
Monitor autonomous agents. This means being able to identify all the meaningful events happening to them and talk to them. (this is not part of this PR unfortunately, it's too many changes at once, but I have a branch ready for it) =>
Stop these workers when necessary.

Which programmatic language is the AGI going to use to do that? A proper, well-defined REST API is the way to go, we can only do so much with a CLI.

This AGI will talk to its workers through rest api calls, which are nothing more than JSON sent over the network.

At the beginning we will be playing the AGI, so we will use these endpoints to communicate with our agents, through a web browser UI.

Changes

Now let's come back to reality, I had to work with what the repo provides, so I made the smallest changes possible to achieve these goals. The first thing I want to say, is that my changes won't affect the users in any way. All the existing features are preserved and unchanged.

Additions to the repo that are all necessary to achieve this User Story (I insist they're all necessary to be merged together):

A well-structured web server with FastAPI.
Solid integration tests, extremely readable because all the expected requests and responses are written. These integration tests record the behavior of the autonomous agents. I will use these tests to maintain an automatically generated documentation of auto-gpt api.
The integration tests work with GPT3.5 and the cost is very small. Talking about cost, @Torantulino please create an online bank account for auto-gpt organization so we can wire money and pay for an api key that will use to call open AI within github action. You can keep the key private. I will wire 100$ to the bank account when you have it ready. Also, I know some companies interested sponsoring auto-gpt, so reach out to me on discord.
Ability to stop an autonomous agent gracefully, without forcing it. Each agent runs in separates threads.
a NoMemory class. Why is that ? Because right now memories are shared accross the same worker !!! So it means if we spawn multiple autonomous agents, they will step into each other's toes. The great news is that NoMemory is actually super performant, I have tested it and it performs oftentimes better than pinecone, mostly because we load an ugly JSON of all the past memories and it confuses the AI. Particularly with GPT 3.5. It turns out summarizing the content as we go and loosing any non essential information can be great for short tasks.
the server is currently in a folder called app with the right project structure for a project of this size. This was the easiest way to do it without breaking the whole repo. This repo has no module, and I can't put the api under a folder called "scripts", that's beyond me :)

CHANGES to the repo:

finally wrap the main.py into an "if main", otherwise nothing in this project is reusable, right now I can't import the main.py without starting the loop haha
create the autonomous agent class so that it can start the start_loop method
create attributes of the class. It's pretty much adding self everywhere.
The temperature is currently None in the repo which means 1. The values are between 0 and 2. For my tests I need a very consistent LLM, so I need 0. So I just put an environment variable. The value will still be 1 for others and nothing changes. (I recommend putting the default value to 0 by the way this is a game changer, but that's another story...)

Documentation

Look at the tests, they document clearly the following requests and responses as well as the output of the worker:
2 integration tests at the moment:

Autonomous agent defined (POST), started (PATCH) STOPPED immediately after starting (PATCH)
Autonomous agent defined (POST), started (PATCH), Writes a file and shuts down on its own. When we try to shut it down we get a 500 because it's already down.

Test Plan

The CLI has to keep working !!! So I tested it thoroughly Here is one run example.

PR Quality Checklist

My pull request is atomic and focuses on a single change.
I have thouroughly tested my changes with multiple different prompts.
I have considered potential risks and mitigations for my changes.
I have documented my changes clearly and comprehensively.
I have not snuck in any "extra" small tweaks changes

waynehamadi · 2023-04-11T03:47:12Z

Hang tight on this one, I am making sure the decorator to stop the threads is added to al the major IO methods.

I know this is going to be a long road to merge this one. Once it's ready to be merged I will create more commits, currently, it's all into one.

dhensen · 2023-04-11T09:08:54Z

.gitignore

+.idea
+ai_settings.yaml
+app/.DS_Store
+.DS_Store


Good that you added .DS_Store to gitignore, I still see lots of .DS_Store files already committed, please remove them.

dhensen · 2023-04-11T09:10:37Z

app/web/api/autonomous_agents/views.py

+            if not thread.is_alive():
+                return JSONResponse(content={"message": "The thread is already terminated"}, status_code=500)
+
+            threads_to_terminate = read_file("threads_to_terminate.json")


reading and json.loading the threads_to_terminate.json could be in a function, it's also used in the def termination_check

dhensen · 2023-04-11T09:15:12Z

requirements.txt

+uvicorn[standard]==0.21.0
+pydantic[dotenv]==1.10.6
+yarl==1.8.2
+ujson==5.7.0


Can you re-use orjson, or does it really have to be ujson?

dhensen · 2023-04-11T09:17:21Z

scripts/chat.py

@@ -63,10 +64,9 @@ def chat_with_ai(
            """
            model = cfg.fast_llm_model # TODO: Change model from hardcode to argument
            # Reserve 1000 tokens for the response
-
-            if cfg.debug:
+            if os.environ.get("DEBUG"):


I think it's better to keep using cfg.debug and then move the env var to the Config.debug code. Or call cfg.set_debug_mode(os.environ.get("DEBUG")) even though those setter/getters are not pythonic.

Wladastic · 2023-04-11T09:55:09Z

Looks intriguing, does it work with gpt3.5 at all?
I feel like it may require too many tokens at once.
Any plans to use Anaconda or Vicuna with it?
I just received a new GPU to run it locally.

dhensen · 2023-04-11T11:41:22Z

Looks intriguing, does it work with gpt3.5 at all? I feel like it may require too many tokens at once. Any plans to use Anaconda or Vicuna with it? I just received a new GPU to run it locally.

Vicuna is really impressive... I don't have it running locally yet, no time yet. Would love to see/make a Vicuna integration for AutoGPT.

waynehamadi · 2023-04-11T13:32:25Z

@Wladastic don't think tokens cost because I am telling you: in a few months you're going to have a bunch of things free, or at least very cheap, for the same quality of GPT3.5 and that will run locally (Vicuna is already great for chat and questions/answers, not so much to code, I have found). When that time comes, you don't want to have to run multiple processes to run multiple agents, you want multiple threads (threads are lighter) like my implementation.

I only develop with GPT 3.5 right now, because GPT4 will be too expansive to run CI/CD on.
The ability to change temperature and the "NoMemory" option will greatly increase the consistency of GPT3.5.
GPT 3.5 is completely overwhelmed by the memory provided in a json. It dilutes its answer, it forgets the initial goals it has. And the temperature set to 0 reduced drastically the json parsed errors.

@dhensen yes, the only problem with Vicuna is it's 20Gb. If we have a github pipeline with it, we will have to create an instance in a cloud provider and call it from there, so it might be more expensive in the long run, because on github action it's going to be too much I think. But we might try.

Honestly, GPT3.5 is great and extremely cheap. I think it's a good middle-ground for now.

waynehamadi · 2023-04-11T14:10:12Z

damn it, there is still an issue in my PR because the project is working at the scripts level.
@Torantulino we're going to need something like what @ryanpeach did tried to do. I am not going to do it here in this PR, because it's going to be too much changes at once. I will try and push something targeted.
the change is very simple: you keep everything except that scripts is now a module and you run scripts module.
Then after that we will rename scripts to auto-gpt. I would rename it directly but I know people have problems with big changes in this project... (however big changes are necessary here, scripts is not a good module name)

nponeccop · 2023-04-11T14:26:58Z

@Torantulino is the scope too large to sensibly merge?

waynehamadi · 2023-04-11T14:54:09Z

@nponeccop what do you want me to do ? create just one endpoint to create the agent and then nobody can do anything with it ? and if people start an agent they have to be able to stop it.
All these changes are minimal: create agent, start agent, stop agent + proper integration tests. There is no literally no way to make it smaller. I even added a nomemory option in order to not increase the scope, otherwise I would have had to build one memory for each agent.
This is a very solid foundation for our web server. Properly defined api endpoints.

The whole skeleton of the web server has to be pushed all at once, we can't afford to just put a little app.py in scripts folder and then "improve it later".
We have to immediately build something adapted to big projects.

If you add a web server improperly built it's going to cost a lot, like the main.py in this project that easily cost probably 100 hours of dev time wasted, if you add up all the contributors.

nponeccop · 2023-04-11T15:07:11Z

Make it a compound PR where the parts are more or less independent and can be easily reviewed. I don't want you to do anything, it's @Torantulino who decides. I'm sorry if I sounded like the official opinion of the repo. I'm not.

marksmerchek · 2023-04-11T15:17:26Z

👍 👎 💯 🔢 🥇 🍇 😁 😀 🤣 😂 🚀 🎸 😧 🥺 🙏 😟 🧇 👋 🤚 🤨 🤙 👈 👇 👉 ☝️ 👆 🤛 👊 ✊ 🤜 🤳 💪 🍄 🎹 👶 🚼
😮‍💨 😶‍🌫️ 🤕 😵‍💫 🤒 🐸 😦 🙍 🙍‍♂️ 🆓 🙍 🙇 🏹 🙇‍♂️ 🙇‍♀️ 🥣 🎳

Wladastic · 2023-04-11T16:49:06Z

@merwanehamadi Have you tried vicuna?
I am not even sure which model is best for coding help.
I want to mix gpt3 and vicuna, because gpt3 has goldfish memory and keeps repeating the same 4-5 steps for me...

nponeccop · 2023-04-11T17:34:32Z

@merwanehamadi There are conflicts. But frankly you should only resolve them when you get a @Torantulino confirmation of merge.

waynehamadi · 2023-04-12T02:22:08Z

Changes:

Renamed autonomous_agents to agents, it's more generic, some agents are autonomous, some semi, some manual.
swagger UI to document automatically the API

@Torantulino I created a new branch with just the web server and the create endpoint, very little change and gives us a solid structure for the backend API.

waynehamadi · 2023-04-12T02:53:21Z

FYI @Torantulino amongst these 56 files, a lot of them are tests.

nponeccop · 2023-04-12T06:17:04Z

It's not about tests, it's about implementing multiple features at once. A single feature is fine, we don't count lines. But your PR is clearly splittable into small improvements that can be applied before you add the REST API.

waynehamadi · 2023-04-12T13:48:56Z

OK I see, so here is my suggestion. I am making these code changes right now:

allow to set temperature to 0
have a memory that always retrieves nothing.
wrap the main.py in a "if main "
wrap the while true in a method that can be reused

add the endpoints for the lifecycle of an agent: create, start and stop agent, along with integration tests.

How about I create a PR that just does this:

add the "create agent" endpoint. This PR will setup a proper web server repo structure (with versioning of the API, which is super important) and allow the creation of an agent. And that's it. This PR will have very little change to the original code.

Signed-off-by: Merwane Hamadi <[email protected]>

waynehamadi · 2023-04-12T15:19:40Z

rebase is still ongoing. I prefer to rebase often, it's easier.
Just a little word on scripts.main.py:

the while true loop really needs to go in a method.
we need to stop using global, there is a new global logging that has been added 😄. When you need global it's the sign you need a class.

@Torantulino if I create a PR to do the items above, will you consider it ?

waynehamadi · 2023-04-13T00:34:35Z

@nponeccop @Torantulino please prioritize these very small changes in the following PRs:

this one is huge: Feature/add ability change temperature #1028 my agent is way better with low temperature
this one is great for short tasks and with GPT-3.5: Feature/ability have no memory #1032
let's do very small changes that have a lot of impact: put loop in in if main #1033
useless method in main.py, we need to trim main.py as much as we can: remove useless load_variables_method #1034

waynehamadi · 2023-04-13T15:53:14Z

I have added the smallest change possible in this PR:

web server with create agent endpoint: web server with create agent endpoint #1139

github-actions · 2023-04-17T15:57:33Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions · 2023-04-17T15:57:33Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions · 2023-04-17T15:57:34Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

p-i- · 2023-05-05T00:56:59Z

This is a mass message from the AutoGPT core team.
Our apologies for the ongoing delay in processing PRs.
This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to:
https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

Boostrix · 2023-05-05T12:12:37Z

the interesting thing here is that this sort of interface would make scaling rather straightforward - you could even hook up multiple Auto-GPT instances together, running on different computers in the same network/LAN or possibly by coming up with a clustered network where people could volunteer their Auto-GPT instances to work towards a common goal (think SETI/protein folding etc) - so this sort of feature could become the foundation for crowd computing, and the scale would be massive if each Auto-GPT has access to the same network and different LLMs - the interesting thing being that a distributed setup would add network latencies to the pool of resources to be tracked (#3466 )

Related: #2497

Boostrix · 2023-05-09T13:40:27Z

The REST API would actually be good to have, as it would make it much easier to add a simple UI/web UI on top of the whole thing, without having to dissect all the underlying code.

waynehamadi · 2023-05-17T21:19:20Z

a rest api is coming in the rearch

* Small docstring change for clarity * Added tentative changes to docs * Update website/docs/Use-Cases/Task-Oriented-AutoML.md Co-authored-by: Chi Wang <[email protected]> * Update flaml/model.py Co-authored-by: Chi Wang <[email protected]> * Updated model.py to reflect `n_jobs = None` suggestion * Updated tutorial to reflect `n_jobs=None` suggestion * Update model.py Improved string Co-authored-by: Chi Wang <[email protected]> Co-authored-by: Qingyun Wu <[email protected]>

dhensen previously approved these changes Apr 11, 2023

View reviewed changes

waynehamadi dismissed dhensen’s stale review via 777063b April 11, 2023 13:47

waynehamadi mentioned this pull request Apr 11, 2023

👋 🤩 Convert to proper python module. Working Docker image. #329

Closed

stefangrotz mentioned this pull request Apr 11, 2023

Wikidata Query support #304

Closed

1 task

jaspythonis previously approved these changes Apr 11, 2023

View reviewed changes

waynehamadi dismissed jaspythonis’s stale review via 46dcfb7 April 12, 2023 01:13

waynehamadi force-pushed the feature/api-to-create-autonomous-agents branch 2 times, most recently from 9dd0eab to 7a9977f Compare April 12, 2023 01:17

waynehamadi force-pushed the feature/api-to-create-autonomous-agents branch from 7a9977f to 4ff986f Compare April 12, 2023 02:57

waynehamadi force-pushed the feature/api-to-create-autonomous-agents branch 3 times, most recently from f7b4a68 to 141e550 Compare April 12, 2023 14:53

Changed occurances of 'cfg.debug_mode' to 'cfg.debug'

18d24c7

waynehamadi force-pushed the feature/api-to-create-autonomous-agents branch from 141e550 to 972c8cc Compare April 12, 2023 14:59

Create, Start and Stop Autonomous Agents

fe26db5

Signed-off-by: Merwane Hamadi <[email protected]>

k-boikov mentioned this pull request Apr 16, 2023

Are there any plans to provide API support? #1782

Closed

1 task

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 17, 2023

sadmuphin mentioned this pull request Apr 17, 2023

Create WebUI for AutoGPT #2255

Closed

5 tasks

Pwuts linked an issue Apr 18, 2023 that may be closed by this pull request

Support Auto-GPT delegating sub-tasks to other instances of itself #70

Closed

nponeccop added the needs discussion To be discussed among maintainers label Apr 18, 2023

This was referenced Apr 18, 2023

Auto-GPT V1 Async Agent #2442

Closed

V2 Async Modular Architecture #2458

Closed

Boostrix mentioned this pull request May 4, 2023

Auto-GPT Recursive Self Improvement #15

Closed

7 tasks

This was referenced May 5, 2023

Shared information to speed up common task and to help drive self-improvment #1749

Closed

Run AutoGPT without GUI #1670

Closed

This was referenced May 11, 2023

Add rich-click CLI #1412

Closed

[DRAFT] Wizard mode (placeholder for #3820 ) #3911

Closed

write_to_file arguments are printed on a single line, which is hard to read #2302

Closed

k-boikov mentioned this pull request May 12, 2023

Is there an api that I can integrate into my own app or other ways？？ #4132

Closed

waynehamadi closed this May 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚀 REST API: AGI can Create, Start and Stop multiple Autonomous Agents (Monitoring coming soon)🚀 #765

🚀 REST API: AGI can Create, Start and Stop multiple Autonomous Agents (Monitoring coming soon)🚀 #765

waynehamadi commented Apr 11, 2023

waynehamadi commented Apr 11, 2023

dhensen Apr 11, 2023

dhensen Apr 11, 2023

dhensen Apr 11, 2023

dhensen Apr 11, 2023

Wladastic commented Apr 11, 2023

dhensen commented Apr 11, 2023

waynehamadi commented Apr 11, 2023 •

edited

Loading

waynehamadi commented Apr 11, 2023 •

edited

Loading

nponeccop commented Apr 11, 2023

waynehamadi commented Apr 11, 2023 •

edited

Loading

nponeccop commented Apr 11, 2023

marksmerchek commented Apr 11, 2023

Wladastic commented Apr 11, 2023

nponeccop commented Apr 11, 2023

waynehamadi commented Apr 12, 2023 •

edited

Loading

waynehamadi commented Apr 12, 2023 •

edited

Loading

nponeccop commented Apr 12, 2023

waynehamadi commented Apr 12, 2023

waynehamadi commented Apr 12, 2023

waynehamadi commented Apr 13, 2023 •

edited

Loading

waynehamadi commented Apr 13, 2023 •

edited

Loading

github-actions bot commented Apr 17, 2023

github-actions bot commented Apr 17, 2023

github-actions bot commented Apr 17, 2023

p-i- commented May 5, 2023

Boostrix commented May 5, 2023 •

edited

Loading

Boostrix commented May 9, 2023

waynehamadi commented May 17, 2023

🚀 REST API: AGI can Create, Start and Stop multiple Autonomous Agents (Monitoring coming soon)🚀 #765

🚀 REST API: AGI can Create, Start and Stop multiple Autonomous Agents (Monitoring coming soon)🚀 #765

Conversation

waynehamadi commented Apr 11, 2023

Background

Changes

Documentation

Test Plan

PR Quality Checklist

waynehamadi commented Apr 11, 2023

dhensen Apr 11, 2023

Choose a reason for hiding this comment

dhensen Apr 11, 2023

Choose a reason for hiding this comment

dhensen Apr 11, 2023

Choose a reason for hiding this comment

dhensen Apr 11, 2023

Choose a reason for hiding this comment

Wladastic commented Apr 11, 2023

dhensen commented Apr 11, 2023

waynehamadi commented Apr 11, 2023 • edited Loading

waynehamadi commented Apr 11, 2023 • edited Loading

nponeccop commented Apr 11, 2023

waynehamadi commented Apr 11, 2023 • edited Loading

nponeccop commented Apr 11, 2023

marksmerchek commented Apr 11, 2023

Wladastic commented Apr 11, 2023

nponeccop commented Apr 11, 2023

waynehamadi commented Apr 12, 2023 • edited Loading

waynehamadi commented Apr 12, 2023 • edited Loading

nponeccop commented Apr 12, 2023

waynehamadi commented Apr 12, 2023

waynehamadi commented Apr 12, 2023

waynehamadi commented Apr 13, 2023 • edited Loading

waynehamadi commented Apr 13, 2023 • edited Loading

github-actions bot commented Apr 17, 2023

github-actions bot commented Apr 17, 2023

github-actions bot commented Apr 17, 2023

p-i- commented May 5, 2023

Boostrix commented May 5, 2023 • edited Loading

Boostrix commented May 9, 2023

waynehamadi commented May 17, 2023

waynehamadi commented Apr 11, 2023 •

edited

Loading

waynehamadi commented Apr 11, 2023 •

edited

Loading

waynehamadi commented Apr 11, 2023 •

edited

Loading

waynehamadi commented Apr 12, 2023 •

edited

Loading

waynehamadi commented Apr 12, 2023 •

edited

Loading

waynehamadi commented Apr 13, 2023 •

edited

Loading

waynehamadi commented Apr 13, 2023 •

edited

Loading

Boostrix commented May 5, 2023 •

edited

Loading