diff --git a/docs/notebooks/0_Getting_Started_with_AIQ.ipynb b/docs/notebooks/0_Getting_Started_with_AIQ.ipynb index 0d21b6647..d789aa41b 100644 --- a/docs/notebooks/0_Getting_Started_with_AIQ.ipynb +++ b/docs/notebooks/0_Getting_Started_with_AIQ.ipynb @@ -51,6 +51,7 @@ "> [Run agent with NAT](#run-this-agent-using-the-nemo-agent-toolkit-run-command) \n", "> [Meta / Shallow / Deep examples](#meta-responses) \n", "> [Deployment using Docker Compose](#deployment-using-docker-compose) \n", + "> [Optional Guardrails and Sandbox Profiles](#optional-guardrails-and-sandbox-profiles) \n", "> [Next Steps](#next-steps)" ] }, @@ -70,11 +71,14 @@ "The following are used by this project:\n", "\n", "- [NVIDIA NeMo Agent toolkit](https://docs.nvidia.com/nemo/agent-toolkit/latest/)\n", - "- [NIM of nvidia/nemotron-3-super-120b-a12b](https://build.nvidia.com/nvidia/nemotron-3-super-120b-a12b)\n", + "- [NIM of nvidia/nemotron-3-super-120b-a12b](https://build.nvidia.com/nvidia/nemotron-3-super-120b-a12b) for intent classification, shallow research, and deep-research writing\n", + "- [NIM of nvidia/nemotron-3-ultra-550b-a55b](https://build.nvidia.com/nvidia/nemotron-3-ultra-550b-a55b) for deep-research source routing, orchestration, planning, and research\n", "- [NIM of nvidia/llama-nemotron-embed-vl-1b-v2](https://build.nvidia.com/nvidia/llama-nemotron-embed-vl-1b-v2) (Optional)\n", "- [NIM of nvidia/nemotron-nano-12b-v2-vl](https://build.nvidia.com/nvidia/nemotron-nano-12b-v2-vl) (Optional)\n", "- [NVIDIA nvidia/nemotron-mini-4b-instruct](https://build.nvidia.com/nvidia/nemotron-mini-4b-instruct) (Optional)\n", - "- [Tavily Search API](https://tavily.com/) for web search" + "- [Tavily Search API](https://tavily.com/) for web search\n", + "- [NeMo Guardrails](https://docs.nvidia.com/nemo/guardrails/latest/) through NeMo Agent Toolkit middleware (Optional)\n", + "- [NVIDIA OpenShell](https://docs.nvidia.com/openshell/latest/about/installation) (Optional)" ] }, { @@ -90,7 +94,7 @@ "If you are deploying this blueprint without the NVIDIA RAG knowledge layer backend, there are **no GPU requirements** for this blueprint on its own.\n", "\n", "- **Self-hosting NIMs:**\n", - "To deploy the NIM for `nvidia/nemotron-3-super-120b-a12b` see the model card. \n", + "To self-host the default LLM stack, see the model cards for `nvidia/nemotron-3-super-120b-a12b` and `nvidia/nemotron-3-ultra-550b-a55b`. \n", "If using NVIDIA RAG knowledge layer backend, please see [NVIDIA RAG Documentation Pages](https://docs.nvidia.com/rag/latest/support-matrix.html)\n", "\n" ] @@ -314,23 +318,23 @@ " _type: nim\n", " model_name: nvidia/nemotron-3-super-120b-a12b\n", " base_url: \"https://integrate.api.nvidia.com/v1\"\n", - " temperature: 0.1\n", - " top_p: 0.3\n", - " max_tokens: 16384\n", + " temperature: 0.7\n", + " top_p: 0.7\n", + " max_tokens: 65536\n", " num_retries: 5\n", " chat_template_kwargs:\n", " enable_thinking: true\n", "\n", - " \n", - " gpt_oss_llm:\n", + " nemotron_ultra_llm:\n", " _type: nim\n", - " model_name: openai/gpt-oss-120b\n", - " base_url: https://integrate.api.nvidia.com/v1\n", - " temperature: 1.0\n", - " top_p: 1.0\n", - " max_tokens: 256000\n", - " api_key: ${NVIDIA_API_KEY}\n", - " max_retries: 10\n", + " model_name: nvidia/nemotron-3-ultra-550b-a55b\n", + " base_url: \"https://integrate.api.nvidia.com/v1\"\n", + " temperature: 0.7\n", + " top_p: 0.7\n", + " max_tokens: 65536\n", + " num_retries: 5\n", + " chat_template_kwargs:\n", + " enable_thinking: true\n", "\n", "functions:\n", " web_search_tool:\n", @@ -357,16 +361,18 @@ "\n", " deep_research_agent:\n", " _type: deep_research_agent\n", - " orchestrator_llm: gpt_oss_llm\n", - " planner_llm: gpt_oss_llm\n", - " researcher_llm: nemotron_super_llm\n", + " orchestrator_llm: nemotron_ultra_llm\n", + " source_router_llm: nemotron_ultra_llm\n", + " researcher_llm: nemotron_ultra_llm\n", + " planner_llm: nemotron_ultra_llm\n", + " writer_llm: nemotron_super_llm\n", " tools:\n", " - advanced_web_search_tool\n", "\n", "workflow:\n", " _type: chat_deepresearcher_agent\n", " interactive_auth: false\n", - " checkpoint_db: ${AIQ_CHECKPOINT_DB:-./checkpoints.db}" + " checkpoint_db: ${AIQ_CHECKPOINT_DB:-./checkpoints.db}\n" ] }, { @@ -378,27 +384,24 @@ "\n", "This config defines a NAT workflow for the AI-Q chat researcher. \n", "\n", - "**`general`** turns on console logging. \n", + "**`general`** includes the API worker settings and telemetry configuration.\n", "\n", "**`general.front_end`** wires the Web API (AI-Q API Worker), async job store, and CORS for the chat UI.\n", "\n", - "**`llms`** wires the models for the agents: Nemotron Super for intent/orchestration/planner, GPT-OSS or similar for clarifier and deep-research orchestrator/planner, and Nemotron 3 Super for the deep research researcher, with different settings per role for best performance.\n", + "**`llms`** wires the models for the agents: Nemotron Super for intent, shallow research, and deep-research writing; Nemotron Ultra for deep-research source routing, orchestration, planning, and research.\n", "\n", - "**`functions`** register tools (Tavily web search, plus advanced search), the intent classifier, the clarifier agent, and the shallow and deep research agents with their assigned tools and LLMs.\n", + "**`functions`** register tools (Tavily web search, plus advanced search), the intent classifier, and the shallow and deep research agents with their assigned tools and LLMs.\n", " \n", - "**`workflow`** is set to `chat_deepresearcher_agent`. See source code for this full agent [here](../../src/aiq_agent/agents/chat_researcher/agent.py).\n", + "**`workflow`** is set to `chat_deepresearcher_agent`. See source code for this full agent in `src/aiq_agent/agents/chat_researcher/agent.py`.\n", "\n", "**Optional settings:** \n", "\n", - "`enable_clarifier: true` lets the agent ask clarifying questions before deep research; \n", - "When the request is vague, the clarifier may also ask you to narrow the scope or clarify the type of output requested. \n", - "\n", - ">**Note:** the above human-in-the-loop interactions may not work in a notebook environment; you can experience these using the AI-Q CLI or the web UI deployed later in this notebook.\n", + "`interactive_auth: false` disables interactive auth prompts for this local demo config.\n", "\n", - "**Main agents:** intent classifier, clarifier, shallow research, deep research. \n", - "**Tools:** web search tool for shallow researcher, advanced web search for deep research, and knowledge retrieval (using the LlamaIndex backend for user-uploaded documents).\n", + "**Main agents:** intent classifier, shallow research, deep research. \n", + "**Tools:** web search tool for shallow researcher and advanced web search for deep research.\n", "\n", - ">**Optional — Frontier model config:** For improved deep research report quality, you can use `configs/config_frontier_models.yml`, that replaces GPT-OSS with a frontier model (e.g. GPT-5.2) for orchestration and planning. When using Docker Compose or the web UI, set `BACKEND_CONFIG` to `/app/configs/config_frontier_models.yml`. Remember to set the required API key in your `.env` file.\n" + ">**Optional — Frontier model config:** For improved deep research report quality, you can use `configs/config_frontier_models.yml`, which assigns a frontier model (for example, GPT-5.2) to deep-research orchestration, planning, and writing while retaining Nemotron 3 Ultra for source routing and research. When using Docker Compose or the web UI, set `BACKEND_CONFIG` to `/app/configs/config_frontier_models.yml`. Remember to set the required API key in your `.env` file.\n" ] }, { @@ -408,7 +411,9 @@ "source": [ "#### Meta Responses \n", "\n", - ">[Documentation](../../docs/source/architecture/agents/intent-classifier.md)" + "Documentation: `docs/source/architecture/agents/intent-classifier.md`\n", + "\n", + "Meta responses handle greetings and questions about what AI-Q can do.\n" ] }, { @@ -427,7 +432,10 @@ "metadata": {}, "source": [ "#### Shallow Research\n", - ">[Documentation](../../docs/source/architecture/agents/shallow-researcher.md)" + "\n", + "Documentation: `docs/source/architecture/agents/shallow-researcher.md`\n", + "\n", + "Shallow research handles focused factual questions with a short tool-assisted lookup.\n" ] }, { @@ -447,10 +455,11 @@ "source": [ "#### Deep Research \n", "\n", - ">[Documentation](../../docs/source/architecture/agents/deep-researcher.md) \n", + "Documentation: `docs/source/architecture/agents/deep-researcher.md` \n", + "\n", "
\n", "Note: Deep research can take several minutes: multiple LLM calls and web searches run in sequence. Watch the logs to see planning and research steps.\n", - "
" + "\n" ] }, { @@ -616,13 +625,32 @@ "id": "fed113dd", "metadata": {}, "source": [ - "At this point, you should be able to access the NVIDIA AI-Q frontend web application by visiting http://localhost:3000.\n", + "At this point, you should be able to access the NVIDIA AI-Q frontend web application. On a local machine, visit http://localhost:3000. On Brev, open the forwarded/proxied port 3000 URL from the Brev environment.\n", + "\n", + ">**Warning:** This demo sets `REQUIRE_AUTH=false`. Keep the local or Brev URL private and use it only in a trusted demo environment. Anyone with access to the URL can access jobs, reports, and, when artifact capture is enabled, generated files. Enable authentication before sharing the deployment.\n", "\n", ">**Tip:** If you are running this notebook on brev, you will need to make the port for the AI-Q frontend accessible. On the settings page for your machine, navigate to \"Using Ports\", enter \"3000\", click \"Expose Port\", and then click \"I accept\".\n", "\n", "**Select data sources:** In the UI, open the **data sources** panel (e.g. from the right side or connections icon). You will see available sources (e.g. Web Search, and optionally Paper Search, Knowledge Layer, or enterprise sources if configured). Toggle **on** the sources you want the agent to use for the next query. Web Search is usually enabled by default.\n", "\n", - "**Send a query:** Type your question or research request in the chat input and send it. The agent will use only the **enabled** data sources for that query. For deep research, the UI may show clarification steps; respond as prompted. Results and the final report will appear in the chat." + "**Send a query:** Type your question or research request in the chat input and send it. The agent will use only the **enabled** data sources for that query. For deep research, the UI may show clarification steps; respond as prompted. Results and the final report will appear in the chat.\n", + "\n", + "**Follow up on a report:** After a deep research report completes, you can ask follow-up questions or request a targeted rewrite from the completed report view. AI-Q keeps the original report as context and starts a follow-up job when more research or synthesis is needed." + ] + }, + { + "cell_type": "markdown", + "id": "optional-policy-sandbox-22", + "metadata": {}, + "source": [ + "### Optional Guardrails and Sandbox Profiles\n", + "\n", + "The default UI deployment above uses `configs/config_web_default_llamaindex.yml`. AI-Q also ships focused profiles for optional capabilities:\n", + "\n", + "- **Guardrails:** `configs/config_web_default_guardrails.yml` adds NeMo Guardrails checks at selected workflow and agent boundaries. Select it by setting `BACKEND_CONFIG=/app/configs/config_web_default_guardrails.yml` in `deploy/.env` before starting Docker Compose. See `docs/source/customization/guardrails.md`.\n", + "- **OpenShell sandbox and durable artifacts:** `configs/config_openshell.yml` enables policy-bound sandbox execution, skills, and capture of generated files for the UI **Files** tab. Before using this profile, install and start the OpenShell gateway, generate or provide the policy file, and set the `AIQ_OPENSHELL_*` environment variables described in `docs/source/deployment/openshell.md`.\n", + "\n", + "These profiles are optional and are not required for the basic web-search workflow." ] }, { @@ -759,14 +787,14 @@ "\n", "You can swap the LLM provider in `config_simple_researcher.yml` with any OpenAI-compatible API.\n", "\n", - "For example, to use [Together.ai](https://www.together.ai/) with the [Nemotron model](https://www.together.ai/models/nvidia-nemotron-3-super) — sign up at [together.ai](https://www.together.ai/), generate an API key, and update the Nemotron LLM blocks (`nemotron_llm_intent`, `nemotron_super_llm`) following this pattern:\n", + "For example, to use [Together.ai](https://www.together.ai/) with the [Nemotron model](https://www.together.ai/models/nvidia-nemotron-3-super) — sign up at [together.ai](https://www.together.ai/), generate an API key, and update the relevant Nemotron LLM blocks. The default config uses `nemotron_super_llm` and `nemotron_ultra_llm`; the pattern below shows the provider change for one block:\n", "\n", "```yaml\n", "llms:\n", " nemotron_super_llm:\n", " _type: openai\n", " model_name: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16\n", - " base_url: \"https://api.together.xyz/v1\"\n", + " base_url: \"https://api.together.ai/v1\"\n", " api_key: ${TOGETHER_API_KEY}\n", " temperature: 0.1\n", " top_p: 0.3\n", @@ -792,9 +820,9 @@ "\n", "You have now deployed the NVIDIA AI-Q Blueprint and interacted with the chat-based agent. To continue this journey, checkout the [Github repo](https://github.com/NVIDIA-AI-Blueprints/aiq).\n", "\n", - "**More notebooks**: We have walkthroughs of the deep researcher agent and its customization (see [notebooks](../notebooks/)).\n", + "**More notebooks**: We have walkthroughs of the deep researcher agent and its customization in the `docs/notebooks/` directory.\n", "\n", - "**Customization Guide**: Checkout the [customization guide](../source/customization/)" + "**Customization Guide**: Check out the [customization guide](https://docs.nvidia.com/aiq-blueprint/latest/customization/index.html)." ] } ],