[Docs Agent] Release of Docs Agent v 0.3.5.

What's changed:

- **`agent runtask` command:** Run tasks defined in `.yaml` files, for example,
  generating release notes based on commit messages.
- **Session Management:** Start and maintain sessions using `--new` and `--cont`
  flags with the `helpme` and `tellme` commands.
- **CLI Command Refactoring:** The CLI is more modular and easier to extend
  with new commands.
- **Web App Mode Updates:** A new `widget-pro` mode is added as an
  iFrame-friendly template.
- **Bug Fixes:**  Various bug fixes related to flags, UI, and file handling.

Author: kyolee415

examples/gemini/python/docs-agent/README.md

@@ -63,6 +63,15 @@ The following list summarizes the tasks and features supported by Docs Agent:
   `gemini-1.5-pro-latest` and `text-embedding-004`. The new ["1.5"][new-15-mode] web app
   mode uses all three Gemini models to their strength: AQA (`aqa`), Gemini 1.0 Pro
   (`gemini-pro`), and Gemini 1.5 Pro (`gemini-1.5-pro-latest`).
+- **Complete a task using the Docs Agent CLI**: The `agent runtask` command allows you
+  to run pre-defined chains of prompts, which are referred to as tasks. These tasks
+  simplify complex interactions by defining a series of steps that the Docs Agent will
+  execute. The tasks are defined in .yaml files stored in the [`tasks`][tasks-dir]
+  directory of your Docs Agent project. To run a task in this directory, for example:
+
+  ```sh
+  agent runtask --task DraftReleaseNotes
+  ```
 
 For more information on Docs Agent's architecture and features,
 see the [Docs Agent concepts][docs-agent-concepts] page.
@@ -279,13 +288,17 @@ Update settings in the Docs Agent project to use your custom dataset:
 
    (**Optional**) Or if you want to use the Gemini AQA model and populate
    a corpus online via the [Semantic Retrieval API][semantic-api], use the
-   following settings:
+   following settings (and update the `corpus_name` field):
 
    ```
    models:
      - language_model: "models/aqa"
    ...
    db_type: "google_semantic_retriever"
+   db_configs:
+     ...
+     - db_type: "google_semantic_retriever"
+       corpus_name: "corpora/flutter-dev"
    ```
 
 7. Save the `config.yaml` file and exit the text editor.
@@ -416,3 +429,4 @@ Meggin Kearney (`@Meggin`), and Kyo Lee (`@kyolee415`).
 [cli-reference]: docs/cli-reference.md
 [chunking-process]: docs/chunking-process.md
 [new-15-mode]: docs/config-reference.md#app_mode
+[tasks-dir]: tasks/

examples/gemini/python/docs-agent/apps_script/helper_functions.gs

@@ -25,6 +25,17 @@ function folderExists(folderName) {
   }
 }
 
+// Checks to see if a folder already exists in the specified root folder
+function folderExistsInRoot(folderName, rootFolder) {
+  const folderIterator = rootFolder.getFoldersByName(folderName);
+  if(folderIterator.hasNext()) {
+    return true;
+  }
+  else {
+    return false;
+  }
+}
+
 // Checks to see if a folder already exists in the drive and exits if it doesn't. Useful for input directories
 function folderExistsInput(folderName){
   if (folderExists(folderName)) {
@@ -49,6 +60,20 @@ function folderExistsOrCreate(folderName){
     return true;
   }
 }
+
+// Checks to see if folder exists or creates it. Useful for output directories
+function folderExistsOrCreateSubdir(folderName, rootFolder){
+  if(folderExistsInRoot(folderName, rootFolder)) {
+    Logger.log("Folder exists: "+ folderName);
+    return true;
+  }
+  else {
+    Logger.log("Folder does not exist: "+ folderName + ". Creating the directory.");
+    rootFolder.createFolder(folderName);
+    return true;
+  }
+}
+
 // Checks to see if a file exists in a folder
 function checkFileExists(fileName,folderName){
   let folder = DriveApp.getFoldersByName(folderName);

examples/gemini/python/docs-agent/config.yaml

@@ -15,7 +15,7 @@
 #
 
 configs:
-  - product_name: "Flutter"
+  - product_name: "Fuchsia"
     models:
       - language_model: "models/aqa"
         embedding_model: "models/embedding-001"
@@ -31,7 +31,7 @@ configs:
         vector_db_dir: "vector_stores/chroma"
         collection_name: "docs_collection"
       - db_type: "google_semantic_retriever"
-        corpus_name: "corpora/flutter-dev"
+        corpus_name: "corpora/fuchsia-dev"
     output_path: "data/plain_docs"
     inputs:
       - path: "/usr/local/home/user01/website/src"

examples/gemini/python/docs-agent/docs/cli-reference.md

@@ -94,6 +94,16 @@ a widget-friendly template:
 agent chatbot --app_mode widget
 ```
 
+### Launch the Docs Agent web app in full mode
+
+The command below launches the Docs Agent web app to use
+a special template that uses three Gemini models (AQA, Gemini 1.5,
+and Gemini 1.0):
+
+```sh
+agent chatbot --app_mode full
+```
+
 ### Launch the Docs Agent web app with a log view enabled
 
 The command below launches the Docs Agent web app while enabling
@@ -172,6 +182,15 @@ absolure or relative path, for example:
 agent helpme write comments for this C++ file? --file ../my-project/test.cc
 ```
 
+### Ask for advice using RAG
+
+The command below uses a local or online vector database (specified in
+the `config.yaml` file) to retrieve relevant context for the request:
+
+```sh
+agent helpme <REQUEST> --file <PATH_TO_FILE> --rag
+```
+
 ### Ask for advice in a session
 
 The command below starts a new session (`--new`), which tracks responses,
@@ -200,15 +219,63 @@ For example:
 agent helpme write a concept doc that delves into more details of these features? --cont
 ```
 
-### Ask for advice using RAG
+### Print the context in the current session
 
-The command below uses a local or online vector database (specified in
-the `config.yaml` file) to retrieve relevant context for the request:
+The command below prints the questions, files, and responses that
+are being used as context in the current session:
 
 ```sh
-agent helpme <REQUEST> --file <PATH_TO_FILE> --rag
+agent show-session
+```
+
+### Ask the model to perform the request to each file in a directory
+
+The command below applies the request to each file found in the
+specified directory:
+
+```sh
+agent helpme <REQUEST> --perfile <PATH_TO_DIRECTORY>
+```
+
+For example:
+
+```sh
+agent helpme explain what this file does? --perfile ~/my-project --new
 ```
 
+### Ask the model to include all files in a directory as context
+
+The command below includes all files found in the specified directory
+as context to the request:
+
+```sh
+agent helpme <REQUEST> --allfiles <PATH_TO_DIRECTORY>
+```
+
+For example:
+
+```sh
+agent helpme write a concept doc covering all features in this project? --allfiles ~/my-project --new
+```
+
+### Ask the model to run a pre-defined chain of prompts
+
+The command below runs a task (a sequence of prompts) defined in
+a `.yaml` file stored in the [`tasks`][tasks-dir] directory:
+
+```sh
+agent runtask --task <TASK>
+```
+
+For example:
+
+```sh
+agent runtask --task DraftReleaseNotes
+```
+
+To see the list of all tasks available in your project, run
+`agent runtask` without any arguments.
+
 ## Managing online corpora
 
 ### List all existing online corpora
@@ -267,3 +334,4 @@ agent delete-corpus --name corpora/example01
 [benchmarks-yaml]: ../docs_agent/benchmarks/benchmarks.yaml
 [set-up-docs-agent-cli]: ../docs_agent/interfaces/README.md
 [semantic-api]: https://ai.google.dev/docs/semantic_retriever
+[tasks-dir]: ../tasks

examples/gemini/python/docs-agent/docs/config-reference.md

@@ -28,14 +28,10 @@ The options are:
   app_mode: "widget"
   ```
 
-* `1.5`: This special mode is designed to be used with the Gemini 1.5 Pro
-  model.
+* `full`: This special mode is designed to be used with Gemini 1.5 models.
 
   ```
-  models:
-    - language_model: "models/aqa"
-  ...
-  app_mode: "1.5"
+  app_mode: "full"
   ```
 
 When this field is not specified, the web app is set to use the standard mode.

examples/gemini/python/docs-agent/docs_agent/agents/docs_agent.py

@@ -49,8 +49,17 @@ def __init__(
     ):
         # Models settings
         self.config = config
+        self.language_model = str(self.config.models.language_model)
         self.embedding_model = str(self.config.models.embedding_model)
         self.api_endpoint = str(self.config.models.api_endpoint)
+
+        # Initialize the default Gemini model.
+        if self.language_model.startswith("models/gemini"):
+            self.gemini = Gemini(
+                models_config=config.models, conditions=config.conditions
+            )
+            self.context_model = self.language_model
+
         # Use the new chroma db for all queries
         # Should make a function for this or clean this behavior
         if init_chroma:
@@ -69,9 +78,28 @@ def __init__(
                     self.config.models.api_key, self.embedding_model
                 ),
             )
+
         # AQA model settings
         if init_semantic:
-            if self.config.models.language_model == "models/aqa":
+            # Except in "full" and "pro" modes, the semantic retriever option requires
+            # the AQA model. If not, exit the program.
+            if (
+                self.config.app_mode != "full"
+                and self.config.app_mode != "widget-pro"
+                and self.config.db_type == "google_semantic_retriever"
+            ):
+                if self.language_model != "models/aqa":
+                    logging.error(
+                        "The db_type `google_semnatic_retriever` option"
+                        + " requires the AQA model (`models/aqa`)."
+                    )
+                    exit(1)
+            # If the AQA model is selected or the web app is on "full" and "pro" modes.
+            if (
+                self.language_model == "models/aqa"
+                or self.config.app_mode == "full"
+                or self.config.app_mode == "widget-pro"
+            ):
                 # AQA model setup
                 self.generative_service_client = glm.GenerativeServiceClient()
                 self.retriever_service_client = glm.RetrieverServiceClient()
@@ -86,7 +114,7 @@ def __init__(
                 self.gemini = Gemini(
                     models_config=gemini_model_config, conditions=config.conditions
                 )
-            # Semantic retriever
+            # If semantic retriever is selected as the main database.
             if self.config.db_type == "google_semantic_retriever":
                 for item in self.config.db_configs:
                     if "google_semantic_retriever" in item.db_type:
@@ -99,13 +127,7 @@ def __init__(
                             )
                 self.aqa_response_buffer = ""
 
-        if self.config.models.language_model.startswith("models/gemini"):
-            self.gemini = Gemini(
-                models_config=config.models, conditions=config.conditions
-            )
-            self.context_model = self.config.models.language_model
-
-        # Always initialize the gemini-pro model for other tasks.
+        # Always initialize the Gemini 1.0 pro model for other tasks.
         gemini_pro_model_config = Models(
             language_model="models/gemini-pro",
             embedding_model=self.embedding_model,
@@ -115,10 +137,10 @@ def __init__(
             models_config=gemini_pro_model_config, conditions=config.conditions
         )
 
-        if self.config.app_mode == "1.5":
-            # Initialize the gemini-1.5.pro model for summarization.
+        if self.config.app_mode == "full" or self.config.app_mode == "widget-pro":
+            # Initialize the Gemini 1.5 model for generating main responses.
             gemini_15_model_config = Models(
-                language_model="models/gemini-1.5-pro-latest",
+                language_model=self.language_model,
                 embedding_model=self.embedding_model,
                 api_endpoint=self.api_endpoint,
             )
@@ -174,7 +196,7 @@ def ask_aqa_model_using_local_vector_store(
             )
             verbose_prompt += "\nID: " + index_id + "\n" + returned_context + "\n"
         req = glm.GenerateAnswerRequest(
-            model=self.config.models.language_model,
+            model="models/aqa",
             contents=[user_query_content],
             inline_passages=grounding_passages,
             answer_style=answer_style,
@@ -244,19 +266,14 @@ def ask_aqa_model_using_corpora(
             source=corpus_name, query=user_question_content
         )
 
-        if self.config.models.language_model == "models/aqa":
-            req = glm.GenerateAnswerRequest(
-                model=self.config.models.language_model,
-                contents=[user_question_content],
-                semantic_retriever=retriever_config,
-                answer_style=answer_style,
-            )
-        else:
-            self.aqa_response_buffer = ""
-            logging.error(
-                "This function should only be used with a setting of language_model models/aqa"
-            )
-            return self.config.conditions.model_error_message, search_result
+        # Ask the AQA model.
+        req = glm.GenerateAnswerRequest(
+            model="models/aqa",
+            contents=[user_question_content],
+            semantic_retriever=retriever_config,
+            answer_style=answer_style,
+        )
+
         try:
             aqa_response = self.generative_service_client.generate_answer(req)
             self.aqa_response_buffer = aqa_response
@@ -509,7 +526,7 @@ def ask_content_model_with_context_prompt(
             response = ""
             if model == "gemini-pro":
                 response = self.gemini_pro.generate_content(contents=new_prompt)
-            elif model == "gemini-1.5-pro":
+            elif model == "gemini-1.5":
                 response = self.gemini_15.generate_content(contents=new_prompt)
             else:
                 response = self.gemini.generate_content(contents=new_prompt)