Skip to content

feat: Simplify grounding intro to focus on grounding with Google search only #1905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
diemtvu wants to merge 5 commits into from
Closed

feat: Simplify grounding intro to focus on grounding with Google search only #1905

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2a3fd88
Fix grounding call typo
diemtvu Mar 14, 2025
d5f97f3
Merge branch 'GoogleCloudPlatform:main' into main
diemtvu Mar 31, 2025
fa934bd
feat. Simplify grounding gemini intro to focus on grounding with goog…
diemtvu Mar 31, 2025
b788b3a
Update some wordings
diemtvu Mar 31, 2025
d63a10f
Merge branch 'main' into main
holtskinner Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
254 changes: 22 additions & 232 deletions gemini/grounding/intro-grounding-gemini.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,11 +117,10 @@
"- Anchors model responses to specific information, documents, and data sources\n",
"- Enhances the trustworthiness, accuracy, and applicability of the generated content\n",
"\n",
"You can configure two different sources of grounding in Vertex AI:\n",
"In Vertex AI, you can choose different types of grounding sources. This tutorial focuses on grounding with public web data via Google Search. For grounding with your own data, please see another tutorial, such as [Grounding with Vertex AI Search](./grounding_with_vais.ipynb)\n",
"\n",
"1. Google Search results for data that is publicly available and indexed.\n",
" - If you use this service in a production application, you will also need to [use a Google Search entry point](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/grounding-search-entry-points).\n",
"2. [Data stores in Vertex AI Search](https://cloud.google.com/generative-ai-app-builder/docs/create-datastore-ingest), which can include your own data in the form of website data, unstructured data, or structured data"
"\n",
"> Important: If you use this service in a production application, you will also need to [use a Google Search entry point](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/grounding-search-entry-points)."
]
},
{
Expand All @@ -136,20 +135,11 @@
"\n",
"- Generate LLM text and chat model responses grounded in Google Search results\n",
"- Compare the results of ungrounded LLM responses with grounded LLM responses\n",
"- Create and use a data store in Vertex AI Search to ground responses in custom documents and data\n",
"- Generate LLM text and chat model responses grounded in Vertex AI Search results\n",
"\n",
"This tutorial uses the following Google Cloud AI services and resources:\n",
"\n",
"- Vertex AI\n",
"- Vertex AI Search\n",
"\n",
"The steps performed include:\n",
"\n",
"- Configuring the LLM and prompt for various examples\n",
"- Sending example prompts to generative text and chat models in Vertex AI\n",
"- Setting up a data store in Vertex AI Search with your own data\n",
"- Sending example prompts with various levels of grounding (no grounding, web grounding, data store grounding)"
"- Sending example prompts to generative text and chat models in Vertex AI"
]
},
{
Expand All @@ -166,7 +156,7 @@
"\n",
"1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
"1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
"1. Enable the [Vertex AI and Vertex AI Agent Builder APIs](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com,discoveryengine.googleapis.com).\n",
"1. Enable the [Vertex AI APIs](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
"1. If you are running this notebook locally, you need to install the [Cloud SDK](https://cloud.google.com/sdk)."
]
},
Expand Down Expand Up @@ -284,9 +274,7 @@
" GenerateContentResponse,\n",
" GoogleSearch,\n",
" Part,\n",
" Retrieval,\n",
" Tool,\n",
" VertexAISearch,\n",
")"
]
},
Expand All @@ -310,12 +298,6 @@
"def print_grounding_data(response: GenerateContentResponse) -> None:\n",
" candidate = response.candidates[0]\n",
"\n",
" citation_metadata = getattr(candidate, \"citation_metadata\")\n",
" if citation_metadata:\n",
" print(\"Citations\")\n",
" for citation in citation_metadata.citations:\n",
" print(citation)\n",
"\n",
" grounding_metadata = getattr(candidate, \"grounding_metadata\")\n",
" if grounding_metadata:\n",
" print(\"\\nGrounding Chunks\")\n",
Expand Down Expand Up @@ -430,6 +412,22 @@
"print(print_grounding_data(response))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> Note: For model before Gemini 2.0, please use the `GoogleSearchRetrieval` tool instead)\n",
"\n",
"```\n",
"response = client.models.generate_content(\n",
" model=MODEL_ID,\n",
" contents=PROMPT,\n",
" config=GenerateContentConfig(tools=[GoogleSearchRetrieval()]),\n",
")\n",
"\n",
"```"
]
Comment on lines +418 to +429
Copy link
Collaborator

@holtskinner holtskinner Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're trying to remove all references to previous models across our content, so can you remove this section?

},
{
"cell_type": "markdown",
"metadata": {
Expand Down Expand Up @@ -480,163 +478,6 @@
"print_grounding_data(response)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "77f0800f8762"
},
"source": [
"## Example: Grounding with custom documents and data\n",
"\n",
"In this example, you'll compare LLM responses with no grounding with responses that are grounded in the [results of a data store in Vertex AI Search](https://cloud.google.com/generative-ai-app-builder/docs/create-datastore-ingest).\n",
"\n",
"The data store will contain internal documents from a fictional bank, Cymbal Bank. These documents aren't available on the public internet, so the Gemini model won't have any information about them by default."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1b308548c68b"
},
"source": [
"### Creating a data store in Vertex AI Search\n",
"\n",
"In this example, you'll use a Google Cloud Storage bucket with a few sample internal documents for our bank. There's some docs about booking business travel, strategic plan for this Fiscal Year and HR docs describing the different jobs available in the company.\n",
"\n",
"Follow the tutorial steps in the Vertex AI Search documentation to:\n",
"\n",
"1. [Create a data store with unstructured data](https://cloud.google.com/generative-ai-app-builder/docs/try-enterprise-search#unstructured-data) that loads in documents from the GCS folder `gs://cloud-samples-data/gen-app-builder/search/cymbal-bank-employee`.\n",
"2. [Create a search app](https://cloud.google.com/generative-ai-app-builder/docs/try-enterprise-search#create_a_search_app) that is attached to that data store. You should also enable the **Enterprise edition features** so that you can search indexed records within the data store.\n",
"\n",
"Note: The data store must be in the same project that you are using for Gemini.\n",
"\n",
"You can also follow this notebook to do it with code. [Create a Vertex AI Search Datastore and App\n",
"](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/search/create_datastore_and_search.ipynb)\n",
"\n",
"Once you've created a data store, obtain the Data Store ID and input it below.\n",
"\n",
"Note: You will need to wait for data ingestion to finish before using a data store with grounding. For more information, see [create a data store](https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fcd767476241"
},
"outputs": [],
"source": [
"DATA_STORE_PROJECT_ID = PROJECT_ID # @param {type: \"string\"}\n",
"DATA_STORE_REGION = \"global\" # @param {type: \"string\"}\n",
"# Replace this with your data store ID from Vertex AI Search\n",
"DATA_STORE_ID = \"your-data-store-id_1234567890123\" # @param {type: \"string\"}\n",
"\n",
"DATA_STORE_NAME = f\"projects/{DATA_STORE_PROJECT_ID}/locations/{DATA_STORE_REGION}/collections/default_collection/dataStores/{DATA_STORE_ID}\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ccc156676e0a"
},
"source": [
"Now you can ask a question about the company culture:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9c1e1b1743bd"
},
"outputs": [],
"source": [
"PROMPT = \"What is the company culture like?\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "f365681544bb"
},
"source": [
"### Text generation without grounding\n",
"\n",
"Make a prediction request to the LLM with no grounding:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "299818ae71e9"
},
"outputs": [],
"source": [
"response = client.models.generate_content(\n",
" model=MODEL_ID,\n",
" contents=PROMPT,\n",
")\n",
"\n",
"display(Markdown(response.text))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "073f2ec42ff6"
},
"source": [
"### Text generation grounded in Vertex AI Search results\n",
"\n",
"Now we can add the `tools` keyword arg with a grounding tool of `grounding.VertexAISearch()` to instruct the LLM to first perform a search within your custom data store, then construct an answer based on the relevant documents:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "d4c5d53a37b4"
},
"outputs": [],
"source": [
"vertex_ai_search_tool = Tool(\n",
" retrieval=Retrieval(vertex_ai_search=VertexAISearch(datastore=DATA_STORE_NAME))\n",
")\n",
"\n",
"response = client.models.generate_content(\n",
" model=MODEL_ID,\n",
" contents=\"What is the company culture like?\",\n",
" config=GenerateContentConfig(tools=[vertex_ai_search_tool]),\n",
")\n",
"\n",
"display(Markdown(response.text))\n",
"print_grounding_data(response)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e3f985c704cd"
},
"source": [
"Note that the response without grounding doesn't have any context about what company we are asking about. Whereas the response that was grounded in Vertex AI Search results contains information from the documents provided, along with citations of the information.\n",
"\n",
"<div class=\"alert alert-block alert-warning\">\n",
"<b>⚠️ Important notes:</b><br>\n",
"<br>\n",
"<b>If you get an error when running the previous cell:</b><br>\n",
"&nbsp;&nbsp;&nbsp;&nbsp;In order for this sample notebook to work with data store in Vertex AI Search,<br>\n",
"&nbsp;&nbsp;&nbsp;&nbsp;you'll need to create a <a href=\"https://cloud.google.com/generative-ai-app-builder/docs/try-enterprise-search#create_a_data_store\">data store</a> <b>and</b> a <a href=\"https://cloud.google.com/generative-ai-app-builder/docs/try-enterprise-search#create_a_search_app\">search app</a> associated with it in Vertex AI Search.<br>\n",
"&nbsp;&nbsp;&nbsp;&nbsp;If you only create a data store, the previous request will return errors when making queries against the data store.\n",
"<br><br>\n",
"<b>If you get an empty response when running the previous cell:</b><br>\n",
"&nbsp;&nbsp;&nbsp;&nbsp;You will need to wait for data ingestion to finish before using a data store with grounding.<br>\n",
"&nbsp;&nbsp;&nbsp;&nbsp;For more information, see <a href=\"https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es\">create a data store</a>.\n",
"</div>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
Expand Down Expand Up @@ -691,57 +532,6 @@
"display(Markdown(response.text))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "87be7f661f14"
},
"source": [
"### Chat session grounded in Vertex AI Search results\n",
"\n",
"Now we can add the `tools` keyword arg with a grounding tool of `grounding.VertexAISearch()` to instruct the chat model to first perform a search within your custom data store, then construct an answer based on the relevant documents:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8fdad0c3f1f3"
},
"outputs": [],
"source": [
"PROMPT = \"How do I book business travel?\"\n",
"PROMPT_FOLLOWUP = \"Give me more details.\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1a824202a8f0"
},
"outputs": [],
"source": [
"chat = client.chats.create(\n",
" model=MODEL_ID,\n",
" config=GenerateContentConfig(\n",
" tools=[\n",
" Tool(\n",
" retrieval=Retrieval(\n",
" vertex_ai_search=VertexAISearch(datastore=DATA_STORE_NAME)\n",
" )\n",
" )\n",
" ]\n",
" ),\n",
")\n",
"\n",
"response = chat.send_message(PROMPT)\n",
"display(Markdown(response.text))\n",
"\n",
"response = chat.send_message(PROMPT_FOLLOWUP)\n",
"display(Markdown(response.text))"
]
},
{
"cell_type": "markdown",
"metadata": {
Expand All @@ -754,7 +544,7 @@
"\n",
"1. To avoid unnecessary Google Cloud charges, use the [Google Cloud console](https://console.cloud.google.com/) to delete your project if you do not need it. Learn more in the Google Cloud documentation for [managing and deleting your project](https://cloud.google.com/resource-manager/docs/creating-managing-projects).\n",
"1. If you used an existing Google Cloud project, delete the resources you created to avoid incurring charges to your account. For more information, refer to the documentation to [Delete data from a data store in Vertex AI Search](https://cloud.google.com/generative-ai-app-builder/docs/delete-datastores), then delete your data store.\n",
"2. Disable the [Vertex AI Search API](https://console.cloud.google.com/apis/api/discoveryengine.googleapis.com) and [Vertex AI API](https://console.cloud.google.com/apis/api/aiplatform.googleapis.com) in the Google Cloud Console."
"2. Disable the [Vertex AI API](https://console.cloud.google.com/apis/api/aiplatform.googleapis.com) in the Google Cloud Console."
]
}
],
Expand Down