feat(shallow-research): replace escalation heuristics with validated assessment

docs/source/architecture/agents/shallow-researcher.md

@@ -76,6 +76,54 @@ graph LR
 The recursion limit is set to `(max_llm_turns * 2) + 10` to account for
 the agent-tools round trips plus headroom.
 
+## Post-Shallow Escalation Assessment
+
+When the shallow researcher runs inside the top-level chat workflow, a
+successful final answer can be evaluated for escalation to deep research.
+This assessment is not part of the research loop and does not invoke the
+clarifier, planner, researchers, or writer. It is one additional bounded call
+to the existing shallow-research LLM with:
+
+- no bound tools
+- temperature set to 0
+- a maximum output of 256 tokens
+- reasoning disabled when the model provider exposes a per-call thinking switch
+- at most 3,000 JSON-encoded characters from the query and 12,000 from the final answer
+- a hard 16,000-character limit for the complete serialized assessment payload
+- the number of unique sources retrieved in this invocation and whether the tool budget was exhausted
+
+The assessor returns JSON matching one of three statuses:
+
+| Status | Meaning |
+| ------ | ------- |
+| `sufficient` | The shallow answer satisfies the central request, or any limitation does not justify deep research. |
+| `material_gap` | An explicit or central requirement is unsupported, and a concrete deep-research strategy could address it. |
+| `material_conflict` | Retrieved evidence conflicts on a conclusion-critical point, and a concrete deep-research strategy could reconcile it. |
+
+`material_conflict` also has a deterministic evidence guard: fewer than two
+unique sources cannot establish a source conflict and resolves to `sufficient`.
+
+The assessment is deliberately conservative. Budget exhaustion only explains
+why the shallow loop stopped; it does not by itself justify spending a deep
+research budget. Generic breadth, minor omissions, opportunities to improve
+the answer, infrastructure failures, and information that deep research has
+no credible way to obtain also remain on the shallow path.
+
+Malformed JSON, invalid field combinations, empty responses, timeouts, and
+model failures all fail closed to `sufficient`. Source, authentication, and
+other shallow execution failures skip the assessment entirely. Standalone
+uses of the shallow researcher and top-level workflows with
+`enable_escalation: false` do not make the additional model call.
+The top-level workflow also skips the call when its deep-research tool
+validation says that deep research cannot execute.
+
+The top-level `ChatResearcherAgent`, rather than the shallow researcher,
+controls routing. A valid `material_gap` or `material_conflict` recommendation
+routes to the clarifier before deep research; it never invokes the deep
+researcher directly. Initial broad or complex queries should still be routed
+to deep research by the intent classifier rather than relying on this
+post-shallow fallback.
+
 ## State Model
 
 ### ShallowResearchAgentState
@@ -89,6 +137,9 @@ the agent-tools round trips plus headroom.
 | `available_documents` | `list[AvailableDocument]` or `None` | `None` | User-uploaded documents with summaries |
 | `collection_name` | `str` or `None` | `None` | Knowledge collection name |
 | `tool_iterations` | `int` | `0` | Counter tracking total tool calls made |
+| `retrieved_source_count` | `int` | `0` | Unique sources returned during this shallow invocation; does not include prior conversation turns |
+| `assess_escalation` | `bool` | `false` | Enables the bounded post-answer assessment when invoked by the top-level chat workflow |
+| `escalation_assessment` | `ShallowEscalationAssessment` or `None` | `None` | Validated assessment result; invalid or failed assessments resolve to `sufficient` |
 
 ## Configuration
 

docs/source/architecture/overview.md

@@ -62,14 +62,31 @@ graph LR
    `clarifier`.
 
 2. **`should_escalate`** -- After shallow research completes, the graph
-   evaluates whether the response warrants escalation to deep research. It
-   checks for empty responses and escalation keywords ("unable to find",
-   "need more research", "i don't have enough information") in the last
-   800 characters of the AI response. When escalation triggers, the graph
-   routes to the `clarifier` node (not directly to `deep_research`), so it
-   can gather any missing context or output-shape preference before research.
-   Research planning then occurs inside the deep-research workflow.
-   Escalation is gated by the `enable_escalation` config flag.
+   evaluates whether the response warrants escalation to deep research.
+   When escalation is enabled and shallow research completes successfully,
+   the existing shallow-research LLM receives one bounded, JSON-only
+   assessment call with no tools. The assessor sees bounded portions of the
+   original query and final shallow answer, the unique source count for that
+   shallow invocation, and whether the shallow tool budget was exhausted. The
+   complete serialized assessment input is capped at 16,000 characters. It can
+   classify the answer as `sufficient`, `material_gap`, or `material_conflict`.
+
+   Only an explicit or central unmet requirement, or a source conflict that
+   could change the central conclusion, can trigger escalation. A material
+   outcome must also name a concrete deep-research strategy. Budget
+   exhaustion, generic task breadth, minor omissions, and answers that could
+   merely be improved are not escalation triggers. A source conflict requires
+   at least two unique retrieved sources. Invalid, empty, timed-out, or failed
+   assessments default to `sufficient`; shallow research errors are not
+   assessed. The call is also skipped when deep-research tools are unavailable.
+   When a valid material outcome triggers escalation, the graph
+   routes through the `clarifier` node (not directly to `deep_research`). If
+   clarification is enabled and not skipped, it gathers missing context or
+   output-shape preferences; otherwise, the node continues directly to deep
+   research. Research planning then occurs inside the deep-research workflow.
+   For compatibility, an explicit `ShallowResult.escalate_to_deep=true`
+   follows the same clarifier-node route. Escalation is gated by the
+   `enable_escalation` config flag.
 
 ## ChatResearcherState
 
@@ -84,6 +101,7 @@ The central state model carries data through the entire workflow:
 | `depth_decision` | `DepthDecision` or `None` | Routing decision: `shallow` or `deep` |
 | `final_report` | `str` or `None` | Final report output from deep research |
 | `shallow_result` | `ShallowResult` or `None` | Result from shallow research path |
+| `shallow_assessment` | `ShallowEscalationAssessment` or `None` | Validated post-shallow escalation recommendation |
 | `clarifier_result` | `str` or `None` | Clarification log containing missing context or output-shape preferences |
 | `original_query` | `str` or `None` | Preserved user query for deep research |
 | `available_documents` | `list[AvailableDocument]` or `None` | User-uploaded documents with summaries |
@@ -121,6 +139,12 @@ The central state model carries data through the entire workflow:
   through benchmarks (FreshQA, Deep Research Bench) and can evolve as
   evaluation scores improve.
 
+- **Conservative escalation**: Post-shallow escalation uses a validated,
+  fail-closed assessment instead of matching phrases in the answer. The
+  assessment is intentionally a single tool-free model call, and automatic
+  escalation is reserved for material gaps or conflicts that deep research
+  has a concrete way to address.
+
 - **Citation verification and auditability**: Research paths capture sources
   for deterministic citation checks and URL sanitization. Deep-research
   citation verification is enabled by default and configurable with

docs/source/customization/configuration-reference.md

@@ -456,7 +456,7 @@ workflow:
 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
 | `_type` | `str` | **required** | Workflow type. Use `chat_deepresearcher_agent` for the full pipeline. |
-| `enable_escalation` | `bool` | `true` | Allow the intent classifier to route queries to deep research. When `false`, all research queries use shallow research only. |
+| `enable_escalation` | `bool` | `false` | Enable the bounded post-shallow assessment and allow a material shallow result to route to deep research. This setting does not affect queries that the intent classifier initially routes to deep research. |
 | `enable_clarifier` | `bool` | `true` | Run the clarifier agent before deep research to gather user requirements. |
 | `use_async_deep_research` | `bool` | `false` | Submit deep research as an async background job (requires [Dask](https://www.dask.org/) scheduler). |
 | `max_history` | `int` | `20` | Maximum number of messages to keep in conversation history before trimming. |

docs/source/customization/prompts.md

@@ -13,6 +13,7 @@ Each agent in the AI-Q blueprint uses [Jinja2](https://jinja.palletsprojects.com
 |----------|----------|---------|
 | `src/aiq_agent/agents/chat_researcher/prompts/intent_classification.j2` | Intent Classifier | Classifies queries as meta or research, determines depth (shallow/deep), generates meta responses |
 | `src/aiq_agent/agents/shallow_researcher/prompts/researcher.j2` | Shallow Researcher | Defines the research persona, tool usage strategy, source hierarchy, and citation rules |
+| `src/aiq_agent/agents/shallow_researcher/prompts/escalation_assessment.j2` | Shallow Escalation Assessor | Classifies completed shallow answers as sufficient, materially incomplete, or materially conflicted |
 | `src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2` | Deep Research Orchestrator | Coordinates ordered routing, planning, batched research, and writer delegation; it does not call source tools directly |
 | `src/aiq_agent/agents/deep_researcher/prompts/source_router.j2` | Source Router | Selects an advisory route from the request-allowed source catalog before planning |
 | `src/aiq_agent/agents/deep_researcher/prompts/planner.j2` | Deep Research Planner | Grounds and returns a structured `ResearchPlan` with independent `ResearchQuery` objects |
@@ -29,7 +30,8 @@ Each agent stores its prompts in a `prompts/` subdirectory co-located with the a
 src/aiq_agent/agents/
     shallow_researcher/
         prompts/
-            researcher.j2              # Single system prompt
+            researcher.j2              # Research system prompt
+            escalation_assessment.j2   # Post-shallow routing assessment
     deep_researcher/
         prompts/
             orchestrator.j2            # Orchestrator prompt

docs/source/resources/faq.md

@@ -40,7 +40,14 @@ questions it asks.
 
 **What happens when shallow research escalates to deep?**
 
-If `enable_escalation: true` in the workflow config, the orchestrator evaluates the shallow research result. If it detects insufficient coverage (response too short, "unable to find" keywords), it escalates to the clarifier and then deep research. The clarifier asks only for missing context; planning happens inside the deep-research workflow. Refer to [Architecture Overview](../architecture/overview.md).
+If `enable_escalation: true` in the workflow config, a successful shallow
+answer receives one bounded, tool-free assessment call. Escalation is reserved
+for a material unmet requirement or a conclusion-critical source conflict that
+deep research has a concrete way to address; response length and keywords do
+not trigger it. A material result routes through the clarifier node and then to
+deep research. The clarifier asks only for missing context or output-shape
+preferences when it is enabled and not skipped; planning happens inside the
+deep-research workflow. Refer to [Architecture Overview](../architecture/overview.md).
 
 **How does deep research choose data sources?**
 

pyproject.toml

@@ -22,6 +22,7 @@ package-dir = {"" = "src"}
 
 [tool.setuptools.package-data]
 "aiq_agent.agents.deep_researcher" = ["skills/**/*"]
+"aiq_agent.agents.shallow_researcher" = ["prompts/*.j2"]
 
 [project]
 name = "aiq-agent"