Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .secrets.baseline

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ limitations under the License.

## Overview

The NVIDIA AI-Q Blueprint is an enterprise-grade research agent built on the [NVIDIA NeMo Agent Toolkit](https://docs.nvidia.com/nemo/agent-toolkit/latest/) and uses [LangChain Deep Agents](https://docs.langchain.com/oss/python/deepagents/overview). It gives you both **quick, cited answers** and **in-depth, report-style research** in one system, with benchmarks and evaluation harnesses so you can measure quality and improve over time.
The NVIDIA AI-Q Blueprint is a deployable research backend built on the [NVIDIA NeMo Agent Toolkit](https://docs.nvidia.com/nemo/agent-toolkit/latest/) and [LangChain Deep Agents](https://docs.langchain.com/oss/python/deepagents/overview). Teams can self-host the application boundary and connect deployment-owned models, data sources, authentication, policy controls, storage, and observability. It provides both **quick, cited answers** and **in-depth, report-style research**, plus benchmarks and evaluation harnesses for measuring quality. AI-Q is focused on governed research workflows; it is not a general-purpose coding-agent harness.

<p align="center">
<img src="./docs/assets/AIQ-arch-light.png" alt="AI-Q Architecture" width="800">
Expand Down Expand Up @@ -116,9 +116,11 @@ The checked-in default CLI and web profiles use these core components:
- [Tavily Search API](https://tavily.com/) for web search
- Serper, SerpAPI, or SearchAPI for Google Scholar paper search

Focused profiles can instead use GPT-5.2 or other supported frontier models for selected
deep-research roles. Refer to [Configuration Files](#configuration-files); there is no single
all-features profile.
The shipped frontier profile, `configs/config_frontier_models.yml`, uses GPT-5.6 Sol for deep-research orchestration,
planning, and writing, and GPT-5.6 Luna for source routing and research. Treat any bring-your-own model or modified
profile as an experimental customization until the complete workflow is evaluated with that exact model, prompt,
hyperparameter, tool-calling, and structured-output configuration. Refer to
[Configuration Files](#configuration-files); there is no single all-features profile.

## Target Audience

Expand Down Expand Up @@ -292,7 +294,7 @@ The `configs/` directory holds YAML workflow configs that define agents, tools,
| `config_web_frag.yml` | Nemotron 3 Super/Ultra | Web/API and Helm base with Foundational RAG plus Tavily. Requires separately deployed RAG query and ingestion services. |
| `config_web_opensearch.yml` | Nemotron 3 Super/Ultra; Nemotron 3 Embed | Web/API with built-in OpenSearch knowledge retrieval plus Tavily; supports self-hosted, `es`, and `aoss` authentication modes. |
| `config_web_azure_ai_search.yml` | Nemotron 3 Super/Ultra; Nemotron 3 Embed | Web/API with Azure AI Search knowledge retrieval plus Tavily; supports API-key and Azure identity authentication. |
| `config_frontier_models.yml` | GPT Sol/Luna; Nemotron 3 Super/Ultra; Gemma 4 summary | LlamaIndex profile using GPT Sol/Luna for deep research and Super for intent/shallow. Requires `OPENAI_API_KEY` and `NVIDIA_API_KEY`. |
| `config_frontier_models.yml` | GPT Sol/Luna; Nemotron 3 Super; Gemma 4 summary | LlamaIndex profile using GPT Sol/Luna for deep research and Super for intent/shallow. Requires `OPENAI_API_KEY` and `NVIDIA_API_KEY`. |
| `config_web_default_guardrails.yml` | Nemotron 3 Super/Ultra; Gemma 4 summary | LlamaIndex profile with workflow Guardrails explicitly attached, shallow-agent Guardrails dynamically attached through `workflow_functions`, and async deep-agent Guardrails applied by the AI-Q runner from the same target configuration. |
| `config_web_frag_mcp_auth.yml` | Nemotron 3 Super/Ultra | Foundational RAG plus an opt-in protected per-user OAuth MCP source example. Requires a real MCP endpoint and shared token store. |
| `config_domain_routing_and_skills.yml` | Nemotron 3 Ultra; Gemma 4 summary | Direct deep-research profile with domain routing, DuckDuckGo news, Polymarket, enabled Serper paper search, LlamaIndex, built-in skills, and a fresh per-job Modal sandbox. |
Expand Down Expand Up @@ -445,8 +447,7 @@ If your config enables Phoenix tracing, start the Phoenix server before running
Start server (separate terminal):

```bash
source .venv/bin/activate
phoenix serve
uvx --from arize-phoenix phoenix serve
```

For detailed benchmark documentation, refer to:
Expand Down Expand Up @@ -498,7 +499,7 @@ indicate availability in a published release.
- End users are responsible for ensuring the availability of their deployment.
- End users are responsible for building, and patching, the container images to keep them up to date.
- The end users are responsible for ensuring that OSS packages used by the developer blueprint are current.
- The logs from middleware, backend, and demo app are printed to standard out. They can include input prompts and output completions for development purposes. The end users are advised to handle logging securely and avoid information leakage for production use cases.
- The built-in workflow and agent logger paths redact raw prompts, tool payloads, and model completions. Enabled source adapters, external middleware, model and tool providers, tracing exporters, and reverse proxies can have separate request-logging and retention behavior; audit and configure every enabled component independently, and keep production logs access-controlled.


## License
Expand Down
5 changes: 3 additions & 2 deletions configs/config_frontier_models.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# This is an example configuration for GPT frontier model mode.
# This is the shipped GPT frontier model profile. Validate the complete workflow
# with these exact provider endpoints and credentials before deployment.
# It has the following features:
# - GPT Sol for orchestration, planning, and report writing
# - GPT Luna for source routing and research
# - OpenAI API access through the standard OPENAI_API_KEY
# - Knowledge retrieval using LlamaIndex
# - Web search and Paper search tools by default
# - Web search by default; Paper search is available as a commented opt-in

general:
use_uvloop: true
Expand Down
1 change: 1 addition & 0 deletions configs/config_web_default_guardrails.yml
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ middleware:
- *secret_assignment_pattern
- '(?i)dump\s+(all\s+)?documents'
- '(?i)exfiltrate\s+(the\s+)?knowledge\s+base'
- '(?i)\b(?:dump|exfiltrate)(?:\s+and\s+(?:dump|exfiltrate))?\s+(?:(?:all|the)\s+)?(?:documents?|knowledge\s+base)\b'
output:
patterns:
- '(?i)api[_-]?key\s*[:=]'
Expand Down
2 changes: 1 addition & 1 deletion deploy/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ AIQ_DEV_ENV=cli
# -----------------------------------------------------------------------------
NVIDIA_API_KEY=

# Frontier models: required only for config_frontier_models.yml (GPT-5.2 orchestration/planning/writing).
# Frontier models: required only for config_frontier_models.yml (shipped GPT-5.6 Sol/Luna role assignment).
# OPENAI_API_KEY=


Expand Down
2 changes: 1 addition & 1 deletion deploy/compose/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
# docker compose --env-file ../.env -f docker-compose.yaml up -d
#
# Frontend image:
# The frontend is built from frontends/ui/Dockerfile.
# The frontend is built from frontends/ui/deploy/Dockerfile.
#
# =============================================================================

Expand Down
2 changes: 1 addition & 1 deletion docs/source/architecture/agents/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ AI-Q uses a multi-agent architecture where an intent classifier routes queries t
| [Intent Classifier](./intent-classifier.md) | Route queries and determine research depth | Instant | — |
| [Clarifier](./clarifier.md) | Optionally gather missing context and the requested output type before deep research | Interactive | — |
| [Shallow Researcher](./shallow-researcher.md) | Fast, bounded research for simple questions | Fast (30-60s) | Surface |
| [Deep Researcher](./deep-researcher.md) | Advisory source routing, structured planning, concurrent evidence gathering, and writer synthesis | Thorough (2-10min) | Deep |
| [Deep Researcher](./deep-researcher.md) | Advisory source routing, structured planning, concurrent evidence gathering, and writer synthesis | Several minutes; can exceed 10 | Deep |

```{toctree}
:titlesonly:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/contributing/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,5 @@ Refer to each benchmark's README for details. The [Customization guide](../custo
## Debugging

- **Verbose logging:** `./scripts/start_cli.sh --verbose` or set `verbose: true` in workflow config.
- **Phoenix tracing:** Start `phoenix serve`, run the agent with Phoenix tracing enabled in config, then open `http://localhost:6006`.
- **Phoenix tracing:** Start `uvx --from arize-phoenix phoenix serve`, run the agent with Phoenix tracing enabled in config, then open `http://localhost:6006`.
- **Common issues:** Import errors -- ensure `uv pip install -e .`; auth -- check env vars; tool not found -- check config; pre-commit cache -- `pre-commit clean`.
2 changes: 1 addition & 1 deletion docs/source/customization/configuration-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -706,7 +706,7 @@ only the additional sections you need.
| `configs/config_web_azure_ai_search.yml` | Web API | Azure AI Search knowledge retrieval and web search |
| `configs/config_web_frag.yml` | Web API / Helm base | Foundational RAG plus Tavily. Requires separately deployed RAG query and ingestion services. Paper search is commented out. |
| `configs/config_web_opensearch.yml` | Web API | Built-in OpenSearch knowledge backend plus Tavily. Supports unauthenticated or basic self-hosted OpenSearch and SigV4 (`es` or `aoss`); infrastructure and credentials are deployment opt-ins. |
| `configs/config_frontier_models.yml` | Web API | LlamaIndex plus explicit per-agent tools, Nemotron researcher roles, and an OpenAI frontier model for orchestration/planning/writing. Requires `OPENAI_API_KEY`; paper search is commented out. |
| `configs/config_frontier_models.yml` | Web API | Shipped LlamaIndex frontier profile: Nemotron 3 Super for intent/shallow, GPT-5.6 Sol for clarification/orchestration/planning/writing, GPT-5.6 Luna for source routing/research, and Gemma 4 for summaries. Requires `NVIDIA_API_KEY`, `OPENAI_API_KEY`, and `TAVILY_API_KEY` for the enabled Tavily tools; the commented paper-search opt-in requires `SERPER_API_KEY` when enabled. Validate the complete workflow against the configured provider endpoints before deployment. |
| `configs/config_web_default_guardrails.yml` | Web API | LlamaIndex with workflow Guardrails attached explicitly, shallow-agent Guardrails dynamically attached through `workflow_functions`, and async deep-agent Guardrails applied by the AI-Q runner from the same target configuration. |
| `configs/config_web_frag_mcp_auth.yml` | Web API | Foundational RAG plus a protected per-user OAuth MCP source example. Requires a real protected MCP endpoint and shared token-store configuration; it is not a zero-config default. |
| `configs/config_domain_routing_and_skills.yml` | Direct deep-research workflow | Automatic domain routing, Tavily, DuckDuckGo news, Polymarket, LlamaIndex, enabled Serper paper search, built-in skills, and a Modal sandbox. Requires the corresponding service credentials and Modal setup. |
Expand Down
10 changes: 10 additions & 0 deletions docs/source/customization/guardrails.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ SPDX-License-Identifier: Apache-2.0

AI-Q can use NeMo Guardrails through NeMo Agent Toolkit middleware to evaluate selected workflow and agent-boundary inputs and outputs. Guardrails can pass content through unchanged, block content with a configured refusal, or modify selected fields before execution continues.

```{important}
`configs/config_web_default_guardrails.yml` is a reference integration, not a universal end-to-end security policy.
Guardrail policies and acceptable false-positive/false-negative tradeoffs are deployment-specific. Operators must adapt
the rails to their requirements and test every exposed route and execution boundary before relying on them as a
security control.
```

AI-Q provides middleware types for workflow, shallow-research, and deep-research boundaries. In
`configs/config_web_default_guardrails.yml`, the workflow middleware is explicitly attached under `workflow.middleware`.
The shallow middleware is dynamically attached by targeting `shallow_research_agent` under `workflow_functions`.
Expand Down Expand Up @@ -155,4 +162,7 @@ Guardrails middleware is available at these AI-Q boundaries:

The reference profile actively guards all three boundaries described above.

This scope applies only to the middleware and fields explicitly attached in the active configuration. It does not
replace authentication, authorization, network controls, secret handling, or deployment-specific policy validation.

For the complete YAML schema and general configuration conventions, refer to [Configuration Reference](./configuration-reference.md).
6 changes: 4 additions & 2 deletions docs/source/customization/prompts.md
Original file line number Diff line number Diff line change
Expand Up @@ -312,13 +312,15 @@ For advanced Jinja2 patterns (source hierarchy, default values, whitespace contr

## Testing Templates

Set the `DEBUG_PROMPTS` environment variable to log rendered prompts:
Set the `DEBUG_PROMPTS` environment variable to log prompt-rendering metadata:

```bash
DEBUG_PROMPTS=1 .venv/bin/nat run --config_file configs/my_config.yml --input "test query"
```

This logs the fully rendered system prompt before each LLM call, letting you verify variable substitution and conditional rendering.
This logs the rendered prompt size and a stable correlation reference. It does not log the rendered prompt or
user-provided content. Validate exact variable substitution and conditional rendering with unit tests that use
synthetic, non-sensitive inputs.

## Best Practices

Expand Down
16 changes: 16 additions & 0 deletions docs/source/customization/swapping-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,22 @@ SPDX-License-Identifier: Apache-2.0

LLMs are defined in the `llms` section and referenced by agents and tools. You can swap NIM models, change parameters, or add alternative providers.

## Shipped Profiles and Validation Boundary

AI-Q 2.2 ships these exact model assignments and parameters:

| Configuration | Intent and shallow roles | Clarification and deep-research roles | Optional summary role |
| --- | --- | --- | --- |
| `configs/config_cli_default.yml`, `configs/config_web_default_llamaindex.yml` | `nvidia/nemotron-3-super-120b-a12b` | `nvidia/nemotron-3-ultra-550b-a55b` for clarification, orchestration, source routing, research, planning, and writing | `google/gemma-4-31b-it` in the web profile |
| `configs/config_frontier_models.yml` | `nvidia/nemotron-3-super-120b-a12b` | `gpt-5.6-sol` for clarification, orchestration, planning, and writing; `gpt-5.6-luna` for source routing and research | `google/gemma-4-31b-it` |

The checked-in files define the documented compatibility boundary; they are not a substitute for an end-to-end run
against your provider endpoints and credentials. Changing a model, endpoint, role assignment, prompt, or inference
parameter creates a custom profile outside that boundary. OpenAI-compatible transport alone does not establish
workflow compatibility. Other bring-your-own models can require provider-specific prompt, hyperparameter,
tool-calling, and structured-output tuning, and should be treated as experimental until the complete workflow is
evaluated in that exact configuration.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

**Example: NIM model (default)**

```yaml
Expand Down
25 changes: 12 additions & 13 deletions docs/source/deployment/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,21 +51,16 @@ instrument LangChain directly can present a different tree.

### Setup

1. Install Phoenix:
1. Start Phoenix in an isolated `uvx` environment. This installs `arize-phoenix` outside the AI-Q project environment
on the first run:

```bash
uv pip install arize-phoenix
```

2. Start the Phoenix server:

```bash
python -m phoenix.server.main serve
uvx --from arize-phoenix phoenix serve
```

This launches the Phoenix UI at [http://localhost:6006](http://localhost:6006).

3. Enable Phoenix tracing in your YAML config:
2. Enable Phoenix tracing in your YAML config:

```yaml
general:
Expand Down Expand Up @@ -287,7 +282,11 @@ general:

## Verbose Logging

For quick debugging without any external services, enable the built-in verbose callback logger. This prints detailed agent execution information directly to the console.
For quick debugging without any external services, enable the built-in `VerboseTraceCallback` logger. This callback
records execution metadata directly to the console without printing raw prompts, tool arguments, tool results, or
model responses. This metadata-only guarantee applies only to `VerboseTraceCallback`. Phoenix and other exporters,
enabled source adapters, and external providers can still receive or retain raw prompts, tool arguments, tool results,
and model responses; configure and audit their redaction, retention, and access controls independently.

### Enable via CLI

Expand All @@ -306,6 +305,6 @@ workflow:
### What Gets Logged

- Chain starts and completions (orchestrator routing, agent handoffs)
- LLM invocations with model name and token counts
- Tool calls with arguments and return values
- Reasoning content for frontier models that support it
- LLM invocation metadata, such as model and message counts when available
- Tool names and execution metadata
- Content lengths and redaction markers instead of raw request or response content
Loading
Loading