Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,9 @@ with `LLMProvider.configure(LLMRole.<ROLE>, llm)`
An unset role falls back to the provider default (the `orchestrator_llm` model).
There is **no** generic `llm` field on the deep research agent.

**Clarifier** (`src/aiq_agent/agents/clarifier/register.py`) defines `llm` (its
default) and `planner_llm`. It does **not** use `LLMProvider.configure` for the
role — it passes `planner_llm` straight to the agent constructor, and `planner_llm`
falls back to `llm` when unset.
**Clarifier** (`src/aiq_agent/agents/clarifier/register.py`) defines a single
`llm` field. It does **not** use `LLMProvider.configure` for a role — it passes
`llm` straight to the agent constructor.

```yaml
functions:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,8 @@ variable section there yet — so read the `.j2` file when in doubt.
Each agent owns its Jinja2 templates under
`src/aiq_agent/agents/<agent>/prompts/*.j2`. For example, the deep researcher has
`orchestrator.j2`, `planner.j2`, `researcher.j2`, `source_router.j2`, `writer.j2`,
and `source_registry.j2`; the clarifier has `plan_generation.j2` and
`research_clarification.j2`; `shallow_researcher` and `chat_researcher` have their
own as well.
and `source_registry.j2`; the clarifier has `research_clarification.j2`;
`shallow_researcher` and `chat_researcher` have their own as well.

## How templates load and render

Expand Down
2 changes: 0 additions & 2 deletions configs/config_cli_default.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,11 +99,9 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_super_llm
planner_llm: nemotron_super_llm
# tools: omitted -> inherits all from data_source_registry
# exclude_tools: []
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
2 changes: 0 additions & 2 deletions configs/config_frontier_models.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,12 +127,10 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: openai_gpt_5_2
planner_llm: openai_gpt_5_2
tools:
- web_search_tool
- knowledge_search
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
2 changes: 0 additions & 2 deletions configs/config_openshell.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,7 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_super_llm
planner_llm: nemotron_super_llm
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
2 changes: 0 additions & 2 deletions configs/config_web_default_guardrails.yml
Original file line number Diff line number Diff line change
Expand Up @@ -206,9 +206,7 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_super_llm
planner_llm: nemotron_super_llm
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
2 changes: 0 additions & 2 deletions configs/config_web_default_llamaindex.yml
Original file line number Diff line number Diff line change
Expand Up @@ -172,11 +172,9 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_super_llm
planner_llm: nemotron_super_llm
# tools: omitted -> inherits all from data_source_registry
# exclude_tools: []
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
2 changes: 0 additions & 2 deletions configs/config_web_frag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,11 +141,9 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_super_llm
planner_llm: nemotron_super_llm
# tools: omitted -> inherits all from data_source_registry
# exclude_tools: []
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
2 changes: 0 additions & 2 deletions configs/config_web_frag_mcp_auth.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,11 +190,9 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_super_llm
planner_llm: nemotron_super_llm
# tools: omitted -> inherits all from data_source_registry
# exclude_tools: []
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
2 changes: 0 additions & 2 deletions configs/config_web_opensearch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -148,11 +148,9 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_super_llm
planner_llm: nemotron_super_llm
# tools: omitted -> inherits all from data_source_registry
# exclude_tools: []
max_turns: 3
enable_plan_approval: true
log_response_max_chars: 2000
verbose: true

Expand Down
4 changes: 2 additions & 2 deletions docs/notebooks/0_Getting_Started_with_AIQ.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@
"**Optional settings:** \n",
"\n",
"`enable_clarifier: true` lets the agent ask clarifying questions before deep research; \n",
"`enable_plan_approval: true` (on the clarifier) lets users approve or adjust the research plan. \n",
"When the request is vague, the clarifier may also ask you to narrow the scope or clarify the type of output requested. \n",
"\n",
">**Note:** the above human-in-the-loop interactions may not work in a notebook environment; you can experience these using the AI-Q CLI or the web UI deployed later in this notebook.\n",
"\n",
Expand Down Expand Up @@ -622,7 +622,7 @@
"\n",
"**Select data sources:** In the UI, open the **data sources** panel (e.g. from the right side or connections icon). You will see available sources (e.g. Web Search, and optionally Paper Search, Knowledge Layer, or enterprise sources if configured). Toggle **on** the sources you want the agent to use for the next query. Web Search is usually enabled by default.\n",
"\n",
"**Send a query:** Type your question or research request in the chat input and send it. The agent will use only the **enabled** data sources for that query. For deep research, the UI may show clarification or plan approval steps; respond as prompted. Results and the final report will appear in the chat."
"**Send a query:** Type your question or research request in the chat input and send it. The agent will use only the **enabled** data sources for that query. For deep research, the UI may show clarification steps; respond as prompted. Results and the final report will appear in the chat."
]
},
{
Expand Down
72 changes: 10 additions & 62 deletions docs/source/architecture/agents/clarifier.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ SPDX-License-Identifier: Apache-2.0
# Clarifier Agent

The Clarifier Agent provides human-in-the-loop (HITL) interaction before deep
research begins. It gathers clarifications from the user, generates a
structured research plan, and optionally presents the plan for approval.
research begins. It gathers context and, when the request is vague, optionally
asks the user to narrow the scope or clarify the type of output requested.

**Location:** `src/aiq_agent/agents/clarifier/agent.py`

Expand All @@ -16,9 +16,9 @@ structured research plan, and optionally presents the plan for approval.
Deep research is expensive in both time and compute. The Clarifier reduces
wasted effort by:

1. Asking focused clarification questions to narrow the research scope
2. Generating a structured research plan with title and sections
3. Allowing the user to approve, reject, or provide feedback on the plan
1. Gathering context (including optional tool calls such as web search) about the request
2. Asking focused clarification questions only when the request is genuinely ambiguous
3. Optionally clarifying the **type of output** the user wants (for example, report, table, comparison, prediction, or brief answer) when that is unclear

The clarifier runs on the deep research path and also when a shallow query
escalates to deep. It can be disabled entirely using `enable_clarifier: false`
Expand Down Expand Up @@ -46,28 +46,11 @@ graph TD
I -->|no| L[Auto-complete clarification]

H -->|no| L[Clarification complete]
L --> M{enable_plan_approval?}

M -->|no| N[Return ClarifierResult<br/>with clarifier_log]
M -->|yes| O[Generate research plan<br/>using plan_generation.j2]

O --> P[Present plan to user]
P --> Q{User decision?}

Q -->|approve| R[Return ClarifierResult<br/>plan_approved = true]
Q -->|reject| S[Return ClarifierResult<br/>plan_rejected = true]
Q -->|feedback| T{iterations < max_plan_iterations?}
T -->|yes| U[Regenerate plan<br/>with feedback]
U --> P
T -->|no| V[Auto-approve plan]
V --> R
L --> N[Return ClarifierResult<br/>with clarifier_log]

style A fill:#e1f5fe
style N fill:#e8f5e9
style R fill:#e8f5e9
style S fill:#ffebee
style J fill:#fff3e0
style P fill:#fff3e0
```

## State Model
Expand All @@ -82,11 +65,6 @@ graph TD
| `max_turns` | `int` | `3` | Maximum clarification Q&A turns |
| `clarifier_log` | `str` | `""` | Accumulated clarification dialog log |
| `iteration` | `int` | `0` | Current clarification turn counter |
| `plan_title` | `str` or `None` | `None` | Title of the generated research plan |
| `plan_sections` | `list[str]` | `[]` | Section titles for the research plan |
| `plan_approved` | `bool` | `false` | Whether the user approved the plan |
| `plan_rejected` | `bool` | `false` | Whether the user rejected the plan |
| `plan_feedback_history` | `list[str]` | `[]` | History of user feedback on plan iterations |

Computed property:
- `remaining_questions` = `max_turns - iteration`
Expand All @@ -98,13 +76,6 @@ Returned to the orchestrator after the clarification dialog completes:
| Field | Type | Description |
| ----- | ---- | ----------- |
| `clarifier_log` | `str` | Full clarification dialog log |
| `plan_title` | `str` or `None` | Research plan title (if plan approval enabled) |
| `plan_sections` | `list[str]` | Plan section titles |
| `plan_approved` | `bool` | Whether the plan was approved |
| `plan_rejected` | `bool` | Whether the plan was rejected |

The `get_approved_plan_context()` method formats the approved plan as markdown
for injection into the deep researcher's orchestrator prompt.

### ClarificationResponse

Expand All @@ -122,11 +93,8 @@ Configured through `ClarifierConfig` (NeMo Agent Toolkit type name: `clarifier_a
| Parameter | Type | Default | Description |
| --------- | ---- | ------- | ----------- |
| `llm` | `LLMRef` | required | LLM for generating clarification questions |
| `planner_llm` | `LLMRef` or `None` | `None` | Separate LLM for plan generation; falls back to `llm` |
| `tools` | `list[FunctionRef \| FunctionGroupRef]` | `[]` | Tools for context gathering (for example, web search) |
| `max_turns` | `int` | `3` | Maximum clarification Q&A turns before auto-completing |
| `enable_plan_approval` | `bool` | `false` | Enable plan preview and approval after clarification |
| `max_plan_iterations` | `int` | `10` | Maximum plan feedback iterations before auto-approving |
| `log_response_max_chars` | `int` | `2000` | Maximum characters to log from LLM responses |
| `verbose` | `bool` | `false` | Enable verbose logging with `VerboseTraceCallback` |

Expand All @@ -137,12 +105,9 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_llm
planner_llm: nemotron_llm
tools:
- web_search_tool
max_turns: 3
enable_plan_approval: true
max_plan_iterations: 10
verbose: true
```

Expand All @@ -153,7 +118,6 @@ Located in `src/aiq_agent/agents/clarifier/prompts/`:
| Template | Purpose |
| -------- | ------- |
| `research_clarification.j2` | Generates clarification questions. Includes conditional sections for uploaded documents context. Instructs the LLM to respond with JSON containing `needs_clarification` and `clarification_question`. Template variables: `clarifier_result`, `available_documents`, `tools`, `tool_names` |
| `plan_generation.j2` | Generates a structured research plan with title and sections from the clarified query and dialog log. Template variables: `clarifier_context`, `feedback_history` |

## HITL Interaction Patterns

Expand All @@ -167,25 +131,9 @@ User: "Focus on Germany and Japan."
Agent: "Got it. Are you interested in economic impacts from a GDP perspective,
job creation, or both?"
User: "Both GDP impact and job creation."
Agent: [clarification complete, generates plan]
```

When `enable_plan_approval` is `true`:

```
Agent: "Here is the proposed research plan:
Title: Economic Impacts of Renewable Energy in Germany and Japan
Sections:
- GDP Impact Analysis
- Job Creation Metrics
- Comparative Analysis
Do you approve this plan?"
User: "Add a section on policy frameworks."
Agent: [regenerates plan with feedback]
User: "Approve"
Agent: [returns ClarifierResult with plan_approved=true]
Agent: [clarification complete, proceeds to deep research]
```

User responses are matched against keyword sets:
- **Approval:** approve, approved, yes, ok, proceed, continue, go ahead, looks good, y, accept
- **Rejection:** reject, rejected, no, cancel, stop, abort, n
When the desired output form is unclear, the clarifier may instead ask which
type of output you want (for example, a full report, a comparison table, or a
brief answer) before research begins.
2 changes: 1 addition & 1 deletion docs/source/architecture/agents/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ AI-Q uses a multi-agent architecture where an intent classifier routes queries t
| Agent | Purpose | Speed | Depth |
|-------|---------|-------|-------|
| [Intent Classifier](./intent-classifier.md) | Route queries and determine research depth | Instant | — |
| [Clarifier](./clarifier.md) | HITL: clarify ambiguous queries and approve research plans | Interactive | — |
| [Clarifier](./clarifier.md) | HITL: clarify ambiguous queries and the requested output type | Interactive | — |
| [Shallow Researcher](./shallow-researcher.md) | Fast, bounded research for simple questions | Fast (30-60s) | Surface |
| [Deep Researcher](./deep-researcher.md) | Multi-phase deep research with planning and iteration | Thorough (2-10min) | Deep |

Expand Down
11 changes: 2 additions & 9 deletions docs/source/customization/configuration-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,24 +293,19 @@ functions:
clarifier_agent:
_type: clarifier_agent
llm: nemotron_llm
planner_llm: nemotron_llm
tools:
- web_search_tool
max_turns: 3
enable_plan_approval: true
max_plan_iterations: 10
log_response_max_chars: 2000
verbose: true
```

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `llm` | `str` | **required** | LLM for generating clarification questions. |
| `planner_llm` | `str` | `None` | LLM for plan generation. Falls back to `llm` if not specified. |
| `tools` | `list[str]` | `[]` | Tools available for gathering context during clarification. |
| `exclude_tools` | `list[str]` | `[]` | Tool names to exclude when inheriting from the data source registry. |
| `max_turns` | `int` | `3` | Maximum number of clarification Q&A turns before auto-completing. |
| `enable_plan_approval` | `bool` | `false` | Show research plan to the user for approval after clarification. |
| `max_plan_iterations` | `int` | `10` | Maximum plan feedback iterations before auto-approving. |
| `log_response_max_chars` | `int` | `2000` | Maximum characters to log from LLM responses. |
| `verbose` | `bool` | `false` | Enable verbose logging. |

Expand Down Expand Up @@ -475,11 +470,9 @@ functions:
clarifier_agent: # Asks clarifying questions for deep research
_type: clarifier_agent
llm: research_llm
planner_llm: research_llm
tools:
- web_search_tool
max_turns: 3
enable_plan_approval: true
verbose: true

shallow_research_agent: # Fast single-pass research
Expand Down Expand Up @@ -514,7 +507,7 @@ The repository includes several pre-built configurations:

| File | Mode | Features |
|------|------|----------|
| `configs/config_cli_default.yml` | CLI | Web search, paper search, clarifier with plan approval |
| `configs/config_cli_default.yml` | CLI | Web search, paper search, clarifier |
| `configs/config_web_default_llamaindex.yml` | Web API | LlamaIndex knowledge retrieval, web search, paper search |
| `configs/config_web_frag.yml` | Web API | Foundational RAG knowledge retrieval, web search, paper search |

Expand Down
12 changes: 7 additions & 5 deletions docs/source/customization/hitl.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ SPDX-License-Identifier: Apache-2.0
-->
# Human-in-the-Loop (HITL)

Human-in-the-loop (clarifier and plan approval) runs before deep research. To disable it:
The clarifier runs before deep research. It gathers context and, when the
request is vague, may ask you to narrow the scope or clarify the type of output
you want. To disable it:

## Disable the Clarifier Entirely

No plan generation or approval step:
No clarification step before deep research:

```yaml
workflow:
Expand All @@ -17,14 +19,14 @@ workflow:
# ...
```

## Keep Clarifier but Skip Plan Approval
## Limit Clarification Questions

No user approval step before deep research:
Cap how many clarification turns the clarifier may take:

```yaml
functions:
clarifier_agent:
_type: clarifier_agent
enable_plan_approval: false
max_turns: 1
# ...
```
2 changes: 1 addition & 1 deletion docs/source/customization/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ SPDX-License-Identifier: Apache-2.0
- **[Guardrails](./guardrails.md)** — Configure NeMo Guardrails at workflow and agent boundaries
- **[Knowledge Layer](./knowledge-layer.md)** — Add document retrieval (LlamaIndex or Foundational RAG)
- **[Prompts](./prompts.md)** — Modify agent behavior through Jinja2 prompt templates
- **[Human-in-the-Loop](./hitl.md)** — Configure the clarifier and plan approval workflow
- **[Human-in-the-Loop](./hitl.md)** — Configure the clarifier
3 changes: 0 additions & 3 deletions docs/source/customization/prompts.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ Each agent in the AI-Q blueprint uses [Jinja2](https://jinja.palletsprojects.com
| `src/aiq_agent/agents/deep_researcher/prompts/planner.j2` | Deep Research Planner | Generates evidence-grounded research plans with TOC structure and search queries |
| `src/aiq_agent/agents/deep_researcher/prompts/researcher.j2` | Deep Research Researcher | Gathers and synthesizes information from search tools with inline citations |
| `src/aiq_agent/agents/clarifier/prompts/research_clarification.j2` | Clarifier | Determines whether a research request needs clarification, asks focused follow-up questions |
| `src/aiq_agent/agents/clarifier/prompts/plan_generation.j2` | Clarifier (Plan) | Generates a lightweight research plan after clarification is complete |

## Template Directory Structure

Expand All @@ -35,7 +34,6 @@ src/aiq_agent/agents/
researcher.j2 # Sub-researcher prompt
clarifier/
prompts/
plan_generation.j2 # Research plan generation
research_clarification.j2 # Clarification prompt
chat_researcher/
prompts/
Expand Down Expand Up @@ -177,7 +175,6 @@ Each template has well-defined sections you can target:
- **Deep Research Planner** (`planner.j2`) — TOC structure, query generation guidelines, research cycle instructions, output JSON schema
- **Deep Research Researcher** (`researcher.j2`) — Research protocol, source prioritization, tool call budget, citation format
- **Clarifier** (`research_clarification.j2`) — What counts as "sufficiently specified", question style, multi-turn policy
- **Plan Generation** (`plan_generation.j2`) — Plan structure, section naming rules, output JSON format

### Creating a New Template

Expand Down
Loading
Loading