From 6bcb191909243d21f25111f1c8413d5e65e2c01a Mon Sep 17 00:00:00 2001 From: Qingyun Wu Date: Wed, 13 Mar 2024 20:49:39 -0400 Subject: [PATCH] add RAG under topics (#1990) * add RAG * demo * correct notebook * Update quarto installation * Update gitignore * Update format * RAG doc --------- Co-authored-by: Li Jiang --- .gitignore | 2 + notebook/agentchat_RetrieveChat.ipynb | 4 +- website/README.md | 4 +- website/docs/topics/retrieval_augmentation.md | 68 +++++++++++++++++++ 4 files changed, 74 insertions(+), 4 deletions(-) create mode 100644 website/docs/topics/retrieval_augmentation.md diff --git a/.gitignore b/.gitignore index 25e88f30c774..6bed53f7c843 100644 --- a/.gitignore +++ b/.gitignore @@ -178,5 +178,7 @@ test/agentchat/test_agent_scripts/* # test cache .cache_test +.db + notebook/result.png diff --git a/notebook/agentchat_RetrieveChat.ipynb b/notebook/agentchat_RetrieveChat.ipynb index 304cc7def74a..0ff689a8ece5 100644 --- a/notebook/agentchat_RetrieveChat.ipynb +++ b/notebook/agentchat_RetrieveChat.ipynb @@ -92,7 +92,7 @@ "\n", "## Construct agents for RetrieveChat\n", "\n", - "We start by initializing the `RetrieveAssistantAgent` and `RetrieveUserProxyAgent`. The system message needs to be set to \"You are a helpful assistant.\" for RetrieveAssistantAgent. The detailed instructions are given in the user message. Later we will use the `RetrieveUserProxyAgent.generate_init_prompt` to combine the instructions and a retrieval augmented generation task for an initial prompt to be sent to the LLM assistant." + "We start by initializing the `RetrieveAssistantAgent` and `RetrieveUserProxyAgent`. The system message needs to be set to \"You are a helpful assistant.\" for RetrieveAssistantAgent. The detailed instructions are given in the user message. Later we will use the `RetrieveUserProxyAgent.message_generator` to combine the instructions and a retrieval augmented generation task for an initial prompt to be sent to the LLM assistant." ] }, { @@ -3037,7 +3037,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.12" + "version": "3.10.13" }, "skip_test": "Requires interactive usage" }, diff --git a/website/README.md b/website/README.md index fa451489eda9..f65890830afd 100644 --- a/website/README.md +++ b/website/README.md @@ -23,9 +23,9 @@ yarn install `quarto` is used to render notebooks. -Install it [here](https://quarto.org/docs/get-started/). +Install it [here](https://github.com/quarto-dev/quarto-cli/releases). -> Note: Support for Docusaurus 3.0 in Quarto is from version `1.4`. Ensure that your `quarto` version is `1.4` or higher. +> Note: Ensure that your `quarto` version is `1.5.23` or higher. ## Local Development diff --git a/website/docs/topics/retrieval_augmentation.md b/website/docs/topics/retrieval_augmentation.md new file mode 100644 index 000000000000..7de22cce84f6 --- /dev/null +++ b/website/docs/topics/retrieval_augmentation.md @@ -0,0 +1,68 @@ +# Retrieval Augmentation + +Retrieval Augmented Generation (RAG) is a powerful technique that combines language models with external knowledge retrieval to improve the quality and relevance of generated responses. + +One way to realize RAG in AutoGen is to construct agent chats with `RetrieveAssistantAgent` and `RetrieveUserProxyAgent` classes. + +## Example Setup: RAG with Retrieval Augmented Agents +The following is an example setup demonstrating how to create retrieval augmented agents in AutoGen: + +### Step 1. Create an instance of `RetrieveAssistantAgent` and `RetrieveUserProxyAgent`. + +Here `RetrieveUserProxyAgent` instance acts as a proxy agent that retrieves relevant information based on the user's input. +```python +assistant = RetrieveAssistantAgent( + name="assistant", + system_message="You are a helpful assistant.", + llm_config={ + "timeout": 600, + "cache_seed": 42, + "config_list": config_list, + }, +) +ragproxyagent = RetrieveUserProxyAgent( + name="ragproxyagent", + human_input_mode="NEVER", + max_consecutive_auto_reply=3, + retrieve_config={ + "task": "code", + "docs_path": [ + "https://raw.githubusercontent.com/microsoft/FLAML/main/website/docs/Examples/Integrate%20-%20Spark.md", + "https://raw.githubusercontent.com/microsoft/FLAML/main/website/docs/Research.md", + os.path.join(os.path.abspath(""), "..", "website", "docs"), + ], + "custom_text_types": ["mdx"], + "chunk_token_size": 2000, + "model": config_list[0]["model"], + "client": chromadb.PersistentClient(path="/tmp/chromadb"), + "embedding_model": "all-mpnet-base-v2", + "get_or_create": True, # set to False if you don't want to reuse an existing collection, but you'll need to remove the collection manually + }, + code_execution_config=False, # set to False if you don't want to execute the code +) +``` + +### Step 2. Initiating Agent Chat with Retrieval Augmentation + +Once the retrieval augmented agents are set up, you can initiate a chat with retrieval augmentation using the following code: + +```python +code_problem = "How can I use FLAML to perform a classification task and use spark to do parallel training. Train 30 seconds and force cancel jobs if time limit is reached." +ragproxyagent.initiate_chat( + assistant, message=ragproxyagent.message_generator, problem=code_problem, search_string="spark" +) # search_string is used as an extra filter for the embeddings search, in this case, we only want to search documents that contain "spark". +``` + +## Online Demo +[Retrival-Augmented Chat Demo on Huggingface](https://huggingface.co/spaces/thinkall/autogen-demos) + +## More Examples and Notebooks +For more detailed examples and notebooks showcasing the usage of retrieval augmented agents in AutoGen, refer to the following: +- Automated Code Generation and Question Answering with Retrieval Augmented Agents - [View Notebook](/docs/notebooks/agentchat_RetrieveChat) +- Automated Code Generation and Question Answering with [Qdrant](https://qdrant.tech/) based Retrieval Augmented Agents - [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_qdrant_RetrieveChat.ipynb) +- Chat with OpenAI Assistant with Retrieval Augmentation - [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_oai_assistant_retrieval.ipynb) +- **RAG**: Group Chat with Retrieval Augmented Generation (with 5 group member agents and 1 manager agent) - [View Notebook](/docs/notebooks/agentchat_groupchat_RAG) + +## Roadmap + +Explore our detailed roadmap [here](https://github.com/microsoft/autogen/issues/1657) for further advancements plan around RAG. Your contributions, feedback, and use cases are highly appreciated! We invite you to engage with us and play a pivotal role in the development of this impactful feature.