From 8f6788be7a5e0c04a4ca8888bf6b2863b9397711 Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Mon, 29 Jan 2024 08:28:47 -0500 Subject: [PATCH 1/8] docs: jupyter support for website docs, move config docs --- .github/workflows/deploy-website.yml | 16 +++++ website/.gitignore | 8 +++ website/README.md | 9 ++- .../docs/llm_endpoint_configuration.ipynb | 61 ++++++++++++++----- website/sidebars.js | 1 + 5 files changed, 79 insertions(+), 16 deletions(-) rename notebook/oai_openai_utils.ipynb => website/docs/llm_endpoint_configuration.ipynb (93%) diff --git a/.github/workflows/deploy-website.yml b/.github/workflows/deploy-website.yml index ccc6e4b219ba..766a9a9fb540 100644 --- a/.github/workflows/deploy-website.yml +++ b/.github/workflows/deploy-website.yml @@ -41,6 +41,14 @@ jobs: - name: pydoc-markdown run run: | pydoc-markdown + - name: quarto install + run: | + wget -q https://github.com/quarto-dev/quarto-cli/releases/download/v1.4.549/quarto-1.4.549-linux-amd64.tar.gz + tar -xzf quarto-1.4.549-linux-amd64.tar.gz + echo "$(pwd)/quarto-1.4.549/bin/" >> $GITHUB_PATH + - name: quarto run + run: | + quarto render ./website/ - name: Test Build run: | if [ -e yarn.lock ]; then @@ -75,6 +83,14 @@ jobs: - name: pydoc-markdown run run: | pydoc-markdown + - name: quarto install + run: | + wget -q https://github.com/quarto-dev/quarto-cli/releases/download/v1.4.549/quarto-1.4.549-linux-amd64.tar.gz + tar -xzf quarto-1.4.549-linux-amd64.tar.gz + echo "$(pwd)/quarto-1.4.549/bin/" >> $GITHUB_PATH + - name: quarto run + run: | + quarto render ./website/ - name: Build website run: | if [ -e yarn.lock ]; then diff --git a/website/.gitignore b/website/.gitignore index b88fd58714d9..6d05c9c25ace 100644 --- a/website/.gitignore +++ b/website/.gitignore @@ -10,6 +10,12 @@ package-lock.json .cache-loader docs/reference +docs/llm_endpoint_configuration.mdx + +docs/examples/**/*.png +docs/examples/**/*.mdx +docs/examples/**/*.svg + # Misc .DS_Store .env.local @@ -20,3 +26,5 @@ docs/reference npm-debug.log* yarn-debug.log* yarn-error.log* + +/.quarto/ diff --git a/website/README.md b/website/README.md index a931c9024310..18c38df5730e 100644 --- a/website/README.md +++ b/website/README.md @@ -19,12 +19,19 @@ cd website yarn install ``` +### Install Quarto + +`quarto` is used to render notebooks. + +Install it [here](https://quarto.org/docs/get-started/). + ## Local Development -Navigate to the website folder and run: +Navigate to the `website` folder and run: ```console pydoc-markdown +quarto render ./docs yarn start ``` diff --git a/notebook/oai_openai_utils.ipynb b/website/docs/llm_endpoint_configuration.ipynb similarity index 93% rename from notebook/oai_openai_utils.ipynb rename to website/docs/llm_endpoint_configuration.ipynb index 2040f570c5ed..1f4760669653 100644 --- a/notebook/oai_openai_utils.ipynb +++ b/website/docs/llm_endpoint_configuration.ipynb @@ -4,20 +4,60 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# In-depth Guide to OpenAI Utility Functions\n", + "---\n", + "custom_edit_url: https://github.com/microsoft/autogen/edit/main/website/docs/llm_endpoint_configuration.ipynb\n", + "---\n", + "\n", + "# LLM Endpoint Conifguration\n", + "\n", + "## TL;DR\n", + "\n", + "For common use cases, the following is sufficient:\n", + "\n", + "\n", + "Create a file called `OAI_CONFIG_LIST`:\n", + "\n", + "```json\n", + "[\n", + " {\n", + " \"model\": \"gpt-4\",\n", + " \"api_key\": \"YOUR_OPENAI_API_KEY\"\n", + " },\n", + "]\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import autogen\n", + "\n", + "# This will first look for the file called `OAI_CONFIG_LIST` in the current\n", + "# directory, then it will look for the environment variable `OAI_CONFIG_LIST`\n", + "# and test if that is a file and load it or the JSON content directly.\n", + "config_list = autogen.config_list_from_json(env_or_file=\"OAI_CONFIG_LIST\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## In-depth\n", "\n", "Managing API configurations can be tricky, especially when dealing with multiple models and API versions. The provided utility functions assist users in managing these configurations effectively. Ensure your API keys and other sensitive data are stored securely. You might store keys in `.txt` or `.env` files or environment variables for local development. Never expose your API keys publicly. If you insist on storing your key files locally on your repo (you shouldn't), ensure the key file path is added to the `.gitignore` file.\n", "\n", - "#### Steps:\n", + "### Steps:\n", "1. Obtain API keys from OpenAI and optionally from Azure OpenAI (or other provider).\n", "2. Store them securely using either:\n", " - Environment Variables: `export OPENAI_API_KEY='your-key'` in your shell.\n", " - Text File: Save the key in a `key_openai.txt` file.\n", " - Env File: Save the key to a `.env` file eg: `OPENAI_API_KEY=sk-********************`\n", + "3. [Ensure `pyautogen` is installed](./installation/Installation.mdx)\n", "\n", - "---\n", - "\n", - "**TL;DR:**
\n", "There are many ways to generate a `config_list` depending on your use case:\n", "\n", "- `get_config_list`: Generates configurations for API calls, primarily from provided API keys.\n", @@ -31,7 +71,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### What is a `config_list`?\n", + "### What is a `config_list`?\n", "When instantiating an assistant, such as the example below, it is passed a `config_list`. This is used to tell the `AssistantAgent` which models or configurations it has access to:\n", "```python\n", "\n", @@ -54,15 +94,6 @@ "Different tasks may require different models, and the `config_list` aids in dynamically selecting the appropriate model configuration, managing API keys, endpoints, and versions for efficient operation of the intelligent assistant. In summary, the `config_list` helps the agents work efficiently, reliably, and optimally by managing various configurations and interactions with the OpenAI API - enhancing the adaptability and functionality of the agents." ] }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "# %pip install \"pyautogen>=0.2.3\"" - ] - }, { "cell_type": "code", "execution_count": 3, diff --git a/website/sidebars.js b/website/sidebars.js index aa50eef0ca1b..4f5e90f47a53 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -23,6 +23,7 @@ }, }, {'Use Cases': [{type: 'autogenerated', dirName: 'Use-Cases'}]}, + 'llm_endpoint_configuration', 'Contribute', 'Research', 'Migration-Guide' From 97863b03014edf6b5187dcd881ba8e83e64448d5 Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Mon, 29 Jan 2024 08:57:00 -0500 Subject: [PATCH 2/8] update devcontainer dockerfile, fix CI issues --- .devcontainer/dev/Dockerfile | 10 +++++++++- .github/workflows/deploy-website.yml | 4 ++-- website/README.md | 2 +- website/docs/llm_endpoint_configuration.ipynb | 11 +---------- 4 files changed, 13 insertions(+), 14 deletions(-) diff --git a/.devcontainer/dev/Dockerfile b/.devcontainer/dev/Dockerfile index 62b18ade3a6c..4821574371c7 100644 --- a/.devcontainer/dev/Dockerfile +++ b/.devcontainer/dev/Dockerfile @@ -4,7 +4,7 @@ FROM python:3.11-slim-bookworm # Update and install necessary packages RUN apt-get update && apt-get -y update # added vim and nano for convenience -RUN apt-get install -y sudo git npm vim nano curl +RUN apt-get install -y sudo git npm vim nano curl wget # Setup a non-root user 'autogen' with sudo access RUN adduser --disabled-password --gecos '' autogen @@ -32,6 +32,14 @@ RUN sudo pip install pydoc-markdown RUN cd website RUN yarn install --frozen-lockfile --ignore-engines +RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && \ + wget -q https://github.com/quarto-dev/quarto-cli/releases/download/v1.4.549/quarto-1.4.549-linux-${arch}.tar.gz && \ + mkdir -p /home/autogen/quarto/ && \ + tar -xzf quarto-1.4.549-linux-${arch}.tar.gz --directory /home/autogen/quarto/ && \ + rm quarto-1.4.549-linux-${arch}.tar.gz + +ENV PATH="${PATH}:/home/autogen/quarto/quarto-1.4.549/bin/" + # Exposes the Yarn port for Docusaurus EXPOSE 3000 diff --git a/.github/workflows/deploy-website.yml b/.github/workflows/deploy-website.yml index 766a9a9fb540..f1b7801a86cf 100644 --- a/.github/workflows/deploy-website.yml +++ b/.github/workflows/deploy-website.yml @@ -48,7 +48,7 @@ jobs: echo "$(pwd)/quarto-1.4.549/bin/" >> $GITHUB_PATH - name: quarto run run: | - quarto render ./website/ + quarto render ./website/docs - name: Test Build run: | if [ -e yarn.lock ]; then @@ -90,7 +90,7 @@ jobs: echo "$(pwd)/quarto-1.4.549/bin/" >> $GITHUB_PATH - name: quarto run run: | - quarto render ./website/ + quarto render ./website/docs - name: Build website run: | if [ -e yarn.lock ]; then diff --git a/website/README.md b/website/README.md index 18c38df5730e..cc52f448078a 100644 --- a/website/README.md +++ b/website/README.md @@ -21,7 +21,7 @@ yarn install ### Install Quarto -`quarto` is used to render notebooks. +`quarto` is used to render notebooks. Install it [here](https://quarto.org/docs/get-started/). diff --git a/website/docs/llm_endpoint_configuration.ipynb b/website/docs/llm_endpoint_configuration.ipynb index 1f4760669653..e3fd5159d096 100644 --- a/website/docs/llm_endpoint_configuration.ipynb +++ b/website/docs/llm_endpoint_configuration.ipynb @@ -8,7 +8,7 @@ "custom_edit_url: https://github.com/microsoft/autogen/edit/main/website/docs/llm_endpoint_configuration.ipynb\n", "---\n", "\n", - "# LLM Endpoint Conifguration\n", + "# LLM Endpoint Configuration\n", "\n", "## TL;DR\n", "\n", @@ -94,15 +94,6 @@ "Different tasks may require different models, and the `config_list` aids in dynamically selecting the appropriate model configuration, managing API keys, endpoints, and versions for efficient operation of the intelligent assistant. In summary, the `config_list` helps the agents work efficiently, reliably, and optimally by managing various configurations and interactions with the OpenAI API - enhancing the adaptability and functionality of the agents." ] }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], - "source": [ - "import autogen" - ] - }, { "cell_type": "markdown", "metadata": {}, From 1d44e08b94308f147a8de6cafc26472906e5e11d Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Mon, 29 Jan 2024 09:00:05 -0500 Subject: [PATCH 3/8] update bsed on working directory --- .github/workflows/deploy-website.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/deploy-website.yml b/.github/workflows/deploy-website.yml index f1b7801a86cf..4d304d1a6d83 100644 --- a/.github/workflows/deploy-website.yml +++ b/.github/workflows/deploy-website.yml @@ -48,7 +48,7 @@ jobs: echo "$(pwd)/quarto-1.4.549/bin/" >> $GITHUB_PATH - name: quarto run run: | - quarto render ./website/docs + quarto render . - name: Test Build run: | if [ -e yarn.lock ]; then @@ -90,7 +90,7 @@ jobs: echo "$(pwd)/quarto-1.4.549/bin/" >> $GITHUB_PATH - name: quarto run run: | - quarto render ./website/docs + quarto render . - name: Build website run: | if [ -e yarn.lock ]; then From 8a07d67fe0ec36726ee9ce5d0709d195d3c8fa04 Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Mon, 29 Jan 2024 09:16:41 -0500 Subject: [PATCH 4/8] update TLDR --- website/docs/llm_endpoint_configuration.ipynb | 31 +++++++++---------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/website/docs/llm_endpoint_configuration.ipynb b/website/docs/llm_endpoint_configuration.ipynb index e3fd5159d096..7c8f0c61875d 100644 --- a/website/docs/llm_endpoint_configuration.ipynb +++ b/website/docs/llm_endpoint_configuration.ipynb @@ -12,19 +12,7 @@ "\n", "## TL;DR\n", "\n", - "For common use cases, the following is sufficient:\n", - "\n", - "\n", - "Create a file called `OAI_CONFIG_LIST`:\n", - "\n", - "```json\n", - "[\n", - " {\n", - " \"model\": \"gpt-4\",\n", - " \"api_key\": \"YOUR_OPENAI_API_KEY\"\n", - " },\n", - "]\n", - "```" + "For just getting started with AutoGen you can use the following to define your LLM endpoint configuration:" ] }, { @@ -35,16 +23,25 @@ "source": [ "import autogen\n", "\n", - "# This will first look for the file called `OAI_CONFIG_LIST` in the current\n", - "# directory, then it will look for the environment variable `OAI_CONFIG_LIST`\n", - "# and test if that is a file and load it or the JSON content directly.\n", - "config_list = autogen.config_list_from_json(env_or_file=\"OAI_CONFIG_LIST\")" + "config_list = autogen.get_config_list([\"YOUR_OPENAI_API_KEY\"], api_type=\"openai\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\\:\\:\\:danger \n", + "\n", + "Never commit secrets into your code. Before committing, change the code to use a different way of providing your API keys as described below.\n", + "\n", + "\\:\\:\\:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ + "\n", "\n", "## In-depth\n", "\n", From b31189184f4e80b153537a2a05712f773e6e38fc Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Mon, 29 Jan 2024 09:19:02 -0500 Subject: [PATCH 5/8] install into temp --- .github/workflows/deploy-website.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/deploy-website.yml b/.github/workflows/deploy-website.yml index 4d304d1a6d83..f90a78f877f4 100644 --- a/.github/workflows/deploy-website.yml +++ b/.github/workflows/deploy-website.yml @@ -42,6 +42,7 @@ jobs: run: | pydoc-markdown - name: quarto install + working-directory: ${{ runner.temp }} run: | wget -q https://github.com/quarto-dev/quarto-cli/releases/download/v1.4.549/quarto-1.4.549-linux-amd64.tar.gz tar -xzf quarto-1.4.549-linux-amd64.tar.gz @@ -84,6 +85,7 @@ jobs: run: | pydoc-markdown - name: quarto install + working-directory: ${{ runner.temp }} run: | wget -q https://github.com/quarto-dev/quarto-cli/releases/download/v1.4.549/quarto-1.4.549-linux-amd64.tar.gz tar -xzf quarto-1.4.549-linux-amd64.tar.gz From aa2e61cdb12dacfbd9027148c1101228a0e7d1c0 Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Mon, 29 Jan 2024 09:25:56 -0500 Subject: [PATCH 6/8] remove unrelated change from diff --- website/.gitignore | 4 ---- 1 file changed, 4 deletions(-) diff --git a/website/.gitignore b/website/.gitignore index 6d05c9c25ace..495a9753d172 100644 --- a/website/.gitignore +++ b/website/.gitignore @@ -12,10 +12,6 @@ docs/reference docs/llm_endpoint_configuration.mdx -docs/examples/**/*.png -docs/examples/**/*.mdx -docs/examples/**/*.svg - # Misc .DS_Store .env.local From 3cedbd0987e21f08b26cf1e486a35319e3218818 Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Tue, 30 Jan 2024 10:53:25 -0500 Subject: [PATCH 7/8] fix spelling issue --- website/docs/llm_endpoint_configuration.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/llm_endpoint_configuration.ipynb b/website/docs/llm_endpoint_configuration.ipynb index 7628a6a75afe..2cbca3d3f5fc 100644 --- a/website/docs/llm_endpoint_configuration.ipynb +++ b/website/docs/llm_endpoint_configuration.ipynb @@ -63,7 +63,7 @@ "- `config_list_from_models`: Creates configurations based on a provided list of models, useful when targeting specific models without manually specifying each configuration.\n", "- `config_list_from_dotenv`: Constructs a configuration list from a `.env` file, offering a consolidated way to manage multiple API configurations and keys from a single file.\n", "\n", - "If mutiple models are provided, the Autogen client (`OpenAIWrapper`) and agents don't choose the \"best model\" on any criteria - inference is done through the very first model and the next one is used only if the current model fails (e.g. API throttling by the provider or a filter condition is unsatisfied)." + "If multiple models are provided, the Autogen client (`OpenAIWrapper`) and agents don't choose the \"best model\" on any criteria - inference is done through the very first model and the next one is used only if the current model fails (e.g. API throttling by the provider or a filter condition is unsatisfied)." ] }, { From d5409871bed71e9bd34dbf61f6167da934c7c806 Mon Sep 17 00:00:00 2001 From: Jack Gerrits Date: Tue, 30 Jan 2024 16:56:41 -0500 Subject: [PATCH 8/8] Update sidebars.js --- website/sidebars.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/sidebars.js b/website/sidebars.js index 4f5e90f47a53..cdc3ef6f2fe9 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -22,8 +22,8 @@ id: "installation/Installation" }, }, - {'Use Cases': [{type: 'autogenerated', dirName: 'Use-Cases'}]}, 'llm_endpoint_configuration', + {'Use Cases': [{type: 'autogenerated', dirName: 'Use-Cases'}]}, 'Contribute', 'Research', 'Migration-Guide'