-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add OpenAI Gemini Example for VertexAI Notebook (#3290)
* add openai-gemini example * fix exec numbering * improve isntructions * fix br tag * mention roles/aiplatform.user and fix markdown reference * remove mentioning the editor role, and only use the Vertex AI User role --------- Co-authored-by: Chi Wang <[email protected]>
- Loading branch information
Showing
1 changed file
with
138 additions
and
7 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -62,7 +62,12 @@ | |
| "<img src=\"https://github.com/microsoft/autogen/blob/main/website/static/img/create_gcp_svc.png?raw=true\" width=\"1000\" />\n", | ||
| "</div>\n", | ||
| "\n", | ||
| "For the sake of simplicity we will assign the Editor role to our service account for autogen on our Autogen-with-Gemini Google Cloud project.\n", | ||
| "Next we assign the [Vertex AI User](https://cloud.google.com/vertex-ai/docs/general/access-control#aiplatform.user) for the service account. This can be done in the [Google Cloud console](https://console.cloud.google.com/iam-admin/iam?project=autogen-with-gemini) in our `autogen-with-gemini` project.<br/>\n", | ||
| "Alternatively, we can also grant the [Vertex AI User](https://cloud.google.com/vertex-ai/docs/general/access-control#aiplatform.user) role by running a command using the gcloud CLI, for example in [Cloud Shell](https://shell.cloud.google.com/cloudshell):\n", | ||
| "```bash\n", | ||
| "gcloud projects add-iam-policy-binding autogen-with-gemini \\\n", | ||
| " --member=serviceAccount:[email protected] --role roles/aiplatform.user\n", | ||
| "```\n", | ||
| "\n", | ||
| "* Under IAM & Admin > Service Account select the newly created service accounts, and click the option \"Manage keys\" among the items. \n", | ||
| "* From the \"ADD KEY\" dropdown select \"Create new key\" and select the JSON format and click CREATE.\n", | ||
|
|
@@ -83,23 +88,23 @@ | |
| "Additionally, AutoGen also supports authentication using `Credentials` objects in Python with the [google-auth library](https://google-auth.readthedocs.io/), which enables even more flexibility.<br/>\n", | ||
| "For example, we can even use impersonated credentials.\n", | ||
| "\n", | ||
| "#### Use Service Account Keyfile\n", | ||
| "#### <a id='use_svc_keyfile'></a>Use Service Account Keyfile\n", | ||
| "\n", | ||
| "The Google Cloud service account can be specified by setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path to the JSON key file of the service account. <br/>\n", | ||
| "\n", | ||
| "We could even just directly set the environment variable, or we can add the `\"google_application_credentials\"` key with the respective value for our model in the OAI_CONFIG_LIST.\n", | ||
| "\n", | ||
| "#### Use the Google Default Credentials\n", | ||
| "\n", | ||
| "If you are using [Cloud Shell](https://shell.cloud.google.com/cloudshell) or [Cloud Shell editor](https://shell.cloud.google.com/cloudshell/editor) in Google Cloud, <br/> then you are already authenticated. If you have the Google Cloud SDK installed locally, <br/> then you can login by running `gcloud auth login` in the command line. \n", | ||
| "If you are using [Cloud Shell](https://shell.cloud.google.com/cloudshell) or [Cloud Shell editor](https://shell.cloud.google.com/cloudshell/editor) in Google Cloud, <br/> then you are already authenticated. If you have the Google Cloud SDK installed locally, <br/> then you can login by running `gcloud auth application-default login` in the command line. \n", | ||
| "\n", | ||
| "Detailed instructions for installing the Google Cloud SDK can be found [here](https://cloud.google.com/sdk/docs/install).\n", | ||
| "\n", | ||
| "#### Authentication with the Google Auth Library for Python\n", | ||
| "\n", | ||
| "The google-auth library supports a wide range of authentication scenarios, and you can simply pass a previously created `Credentials` object to the `llm_config`.<br/>\n", | ||
| "The [official documentation](https://google-auth.readthedocs.io/) of the Python package provides a detailed overview of the supported methods and usage examples.<br/>\n", | ||
| "If you are already authenticated, like in [Cloud Shell](https://shell.cloud.google.com/cloudshell), or after running the `gcloud auth login` command in a CLI, then the `google.auth.default()` Python method will automatically return your currently active credentials." | ||
| "If you are already authenticated, like in [Cloud Shell](https://shell.cloud.google.com/cloudshell), or after running the `gcloud auth application-default login` command in a CLI, then the `google.auth.default()` Python method will automatically return your currently active credentials." | ||
| ] | ||
| }, | ||
| { | ||
|
|
@@ -307,13 +312,13 @@ | |
| "gcloud auth application-default login\n", | ||
| "gcloud config set project autogen-with-gemini\n", | ||
| "```\n", | ||
| "The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is a path to our service account JSON keyfile, as described in the [Use Service Account Keyfile](#Use Service Account Keyfile) section above.<br/>\n", | ||
| "The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is a path to our service account JSON keyfile, as described in the [Use Service Account Keyfile](#use_svc_keyfile) section above.<br/>\n", | ||
| "We also need to set the Google cloud project, which is `autogen-with-gemini` in this example.<br/><br/>\n", | ||
| "\n", | ||
| "Note, we could also run `gcloud auth login` in case we wish to use our personal Google account instead of a service account.\n", | ||
| "Note, we could also run `gcloud auth application-default login` to use our personal Google account instead of a service account.\n", | ||
| "In this case we need to run the following commands:\n", | ||
| "```bash\n", | ||
| "gcloud auth login\n", | ||
| "gcloud gcloud auth application-default login\n", | ||
| "gcloud config set project autogen-with-gemini\n", | ||
| "```" | ||
| ] | ||
|
|
@@ -390,6 +395,132 @@ | |
| "<img https://github.com/microsoft/autogen/blob/main/website/static/img/autogen_agentchat.png?raw=true>.\"\"\",\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Use Gemini via the OpenAI Library in Autogen\n", | ||
| "Using Gemini via the OpenAI library is also possible once you are already authenticated. <br/>\n", | ||
| "Run `gcloud auth application-default login` to set up application default credentials locally for the example below.<br/>\n", | ||
| "Also set the Google cloud project on the CLI if you have not done so far: <br/>\n", | ||
| "```bash\n", | ||
| "gcloud config set project autogen-with-gemini\n", | ||
| "```\n", | ||
| "The prerequisites are essentially the same as in the example above.<br/>\n", | ||
| "\n", | ||
| "You can read more on the topic in the [official Google docs](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library).\n", | ||
| "<br/> A list of currently supported models can also be found in the [docs](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library#supported_models)\n", | ||
| "<br/>\n", | ||
| "<br/>\n", | ||
| "Note, that you will need to refresh your token regularly, by default every 1 hour." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 7, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import google.auth\n", | ||
| "\n", | ||
| "scopes = [\"https://www.googleapis.com/auth/cloud-platform\"]\n", | ||
| "creds, project = google.auth.default(scopes)\n", | ||
| "auth_req = google.auth.transport.requests.Request()\n", | ||
| "creds.refresh(auth_req)\n", | ||
| "location = \"us-west1\"\n", | ||
| "prompt_price_per_1k = (\n", | ||
| " 0.000125 # For more up-to-date prices see https://cloud.google.com/vertex-ai/generative-ai/pricing\n", | ||
| ")\n", | ||
| "completion_token_price_per_1k = (\n", | ||
| " 0.000375 # For more up-to-date prices see https://cloud.google.com/vertex-ai/generative-ai/pricing\n", | ||
| ")\n", | ||
| "\n", | ||
| "openai_gemini_config = [\n", | ||
| " {\n", | ||
| " \"model\": \"google/gemini-1.5-pro-001\",\n", | ||
| " \"api_type\": \"openai\",\n", | ||
| " \"base_url\": f\"https://{location}-aiplatform.googleapis.com/v1beta1/projects/{project}/locations/{location}/endpoints/openapi\",\n", | ||
| " \"api_key\": creds.token,\n", | ||
| " \"price\": [prompt_price_per_1k, completion_token_price_per_1k],\n", | ||
| " }\n", | ||
| "]" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 8, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "\u001b[33muser_proxy\u001b[0m (to assistant):\n", | ||
| "\n", | ||
| "\n", | ||
| " Compute the integral of the function f(x)=x^3 on the interval 0 to 10 using a Python script,\n", | ||
| " which returns the value of the definite integral.\n", | ||
| "\n", | ||
| "--------------------------------------------------------------------------------\n" | ||
| ] | ||
| }, | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "\u001b[33massistant\u001b[0m (to user_proxy):\n", | ||
| "\n", | ||
| "```python\n", | ||
| "# filename: integral.py\n", | ||
| "def integrate_x_cubed(a, b):\n", | ||
| " \"\"\"\n", | ||
| " This function calculates the definite integral of x^3 from a to b.\n", | ||
| "\n", | ||
| " Args:\n", | ||
| " a: The lower limit of integration.\n", | ||
| " b: The upper limit of integration.\n", | ||
| "\n", | ||
| " Returns:\n", | ||
| " The value of the definite integral.\n", | ||
| " \"\"\"\n", | ||
| " return (b**4 - a**4) / 4\n", | ||
| "\n", | ||
| "# Calculate the integral of x^3 from 0 to 10\n", | ||
| "result = integrate_x_cubed(0, 10)\n", | ||
| "\n", | ||
| "# Print the result\n", | ||
| "print(result)\n", | ||
| "```\n", | ||
| "\n", | ||
| "This script defines a function `integrate_x_cubed` that takes the lower and upper limits of integration as arguments and returns the definite integral of x^3 using the power rule of integration. The script then calls this function with the limits 0 and 10 and prints the result.\n", | ||
| "\n", | ||
| "Execute the script `python integral.py`, you should get the result: `2500.0`.\n", | ||
| "\n", | ||
| "TERMINATE\n", | ||
| "\n", | ||
| "\n", | ||
| "--------------------------------------------------------------------------------\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "assistant = AssistantAgent(\"assistant\", llm_config={\"config_list\": openai_gemini_config}, max_consecutive_auto_reply=3)\n", | ||
| "\n", | ||
| "user_proxy = UserProxyAgent(\n", | ||
| " \"user_proxy\",\n", | ||
| " code_execution_config={\"work_dir\": \"coding\", \"use_docker\": False},\n", | ||
| " human_input_mode=\"NEVER\",\n", | ||
| " is_termination_msg=lambda x: content_str(x.get(\"content\")).find(\"TERMINATE\") >= 0,\n", | ||
| ")\n", | ||
| "\n", | ||
| "result = user_proxy.initiate_chat(\n", | ||
| " assistant,\n", | ||
| " message=\"\"\"\n", | ||
| " Compute the integral of the function f(x)=x^3 on the interval 0 to 10 using a Python script,\n", | ||
| " which returns the value of the definite integral.\"\"\",\n", | ||
| ")" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
|
|
||