feat(skills): add aiq-configure-workflow skill for creating custom config workflows

.agents/skills/README.md

@@ -24,7 +24,7 @@ AI-Q has two distinct kinds of skill, separated by audience:
 | :-- | :-- | :-- |
 | **Audience** | Developers changing the AI-Q repo | Users calling a running AI-Q server |
 | **Location** | `.agents/skills/` (this directory) | top-level `skills/` |
-| **Examples** | `aiq-add-data-source`, `aiq-add-tool`, `aiq-release-qa`, `aiq-prepare-pr`, `aiq-customize-prompts-models`, `aiq-maintain-ci` | `aiq-deploy`, `aiq-research` |
+| **Examples** | `aiq-add-data-source`, `aiq-add-tool`, `aiq-configure-workflow`, `aiq-release-qa`, `aiq-prepare-pr`, `aiq-customize-prompts-models`, `aiq-maintain-ci` | `aiq-deploy`, `aiq-research` |
 | **Assumes** | A repo checkout and dev toolchain | A reachable AI-Q backend |
 
 Consumer skills under `skills/` are authored to be self-contained and exportable

.agents/skills/aiq-add-data-source/SKILL.md

@@ -89,6 +89,7 @@ format failures for the new source package.
 
 ## Related Skills
 
+- `aiq-configure-workflow`
 - `aiq-add-tool`
 - `aiq-release-qa`
 - `aiq-prepare-pr`

.agents/skills/aiq-add-tool/SKILL.md

@@ -94,6 +94,7 @@ format failures for the new tool package.
 
 ## Related Skills
 
+- `aiq-configure-workflow`
 - `aiq-add-data-source`
 - `aiq-release-qa`
 - `aiq-prepare-pr`

.agents/skills/aiq-configure-workflow/SKILL.md

@@ -0,0 +1,89 @@
+---
+name: aiq-configure-workflow
+description: Use when composing, adapting, or validating an AI-Q workflow YAML under configs/ — selecting a shipped profile, enabling tools and data_source_registry sources, wiring agents and the chat_deepresearcher_agent workflow, configuring general.telemetry (Phoenix, LangSmith, Weave, OTEL) and general.front_end aiq_api settings, and pre-flighting cross-references before deploy or serve. Hand off deploy to aiq-deploy, live research to aiq-research, prompt/model edits to aiq-customize-prompts-models, and new source code to aiq-add-tool or aiq-add-data-source.
+license: Apache-2.0
+compatibility: Claude Code, Codex, Cursor, OpenCode, and Agent Skills-compatible tools.
+metadata:
+  version: "0.1.0"
+  source-repo: "NVIDIA-AI-Blueprints/aiq"
+  tags: "aiq nemo-agent-toolkit config yaml workflow data-source-registry telemetry aiq_api"
+allowed-tools: Read Bash Edit
+---
+
+# Configure AI-Q Workflows
+
+Use this skill when a developer or operator needs a **new `configs/config_*.yml`**
+file.
+
+## Start Here
+
+- Confirm this is **config composition** — not deploy (`aiq-deploy`), live research
+  (`aiq-research`), prompt edits (`aiq-customize-prompts-models`), or new NAT
+  packages (`aiq-add-tool` / `aiq-add-data-source`).
+- Copy the closest shipped `configs/*.yml` profile; merge feature blocks from others.
+- **Every produced config must pass `validate_config.py` before hand-off.**
+
+## Authoritative References
+
+- `docs/source/customization/configuration-reference.md` — all fields and defaults
+- `docs/source/customization/tools-and-sources.md`
+- `docs/source/deployment/observability.md` — tracing setup detail
+- `frontends/aiq_api/README.md`
+- `configs/config_web_default_llamaindex.yml` / `configs/config_cli_default.yml`
+
+Bundle:
+
+- [references/config-profiles.md](references/config-profiles.md) — pick a starting profile.
+- [references/composing-config.md](references/composing-config.md) — holistic config map
+  (`general`, `llms`, `functions`, `workflow`, telemetry, `aiq_api`) and how to tune them.
+- [references/env-vars.md](references/env-vars.md) — environment variables by config feature.
+- [references/config-schema.md](references/config-schema.md) — validator checks only.
+- [assets/config-scaffold.yml](assets/config-scaffold.yml) — fallback scaffold.
+
+## Workflow
+
+1. **Scaffold** — `cp configs/<profile>.yml configs/config_<name>.yml` (or
+   [assets/config-scaffold.yml](assets/config-scaffold.yml) + merge blocks).
+2. **Compose** — [references/composing-config.md](references/composing-config.md):
+   adjust registry, tools, agents, LLMs, telemetry, `aiq_api`, workflow flags.
+   Use `config_web_default_llamaindex.yml` as the live default for web `general:`
+   blocks; `configuration-reference.md` for every option. Use
+   [references/env-vars.md](references/env-vars.md) for feature-specific env vars.
+3. **Validate (required)** —
+
+```bash
+uv run python .agents/skills/aiq-configure-workflow/scripts/validate_config.py configs/config_<name>.yml
+```
+
+Fix every `ERROR:`; re-run until exit code 0. Then hand off to `aiq-deploy` or:
+
+```bash
+dotenv -f deploy/.env run nat serve --config_file configs/config_<name>.yml --port 8000
+```
+
+## Validation
+
+```bash
+uv run python .agents/skills/aiq-configure-workflow/scripts/validate_config.py <config.yml>
+```
+
+See [references/config-schema.md](references/config-schema.md). Expected: exit 0.
+
+## Common Mistakes
+
+- Skipping `validate_config.py` on a new config.
+- Undefined `llms:` alias or registry tool not declared under `functions:`.
+- Missing required workflow agents (`intent_classifier`, `shallow_research_agent`,
+  `deep_research_agent`).
+- `use_async_deep_research: true` without `general.front_end` (`aiq_api`).
+- Inventing feature YAML — copy from a shipped profile.
+
+## Related Skills
+
+- `aiq-deploy`
+- `aiq-research`
+- `aiq-customize-prompts-models`
+- `aiq-add-tool`
+- `aiq-add-data-source`
+- `aiq-release-qa`
+- `aiq-prepare-pr`

.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml

@@ -0,0 +1,52 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Fallback scaffold only — prefer copying the closest configs/*.yml profile
+# (see .agents/skills/aiq-configure-workflow/references/config-profiles.md).
+# Merge feature blocks from other shipped profiles; do not invent fields.
+#
+# Usage:
+#   cp .agents/skills/aiq-configure-workflow/assets/config-scaffold.yml configs/my_workflow.yml
+
+general:
+  telemetry:
+    logging:
+      console:
+        _type: console
+        level: INFO
+  # Web/API: copy general.front_end from config_web_default_llamaindex.yml
+
+llms:
+  main_llm:
+    _type: nim
+    model_name: nvidia/nemotron-3-super-120b-a12b
+    base_url: "https://integrate.api.nvidia.com/v1"
+    temperature: 0.7
+    max_tokens: 65536
+    num_retries: 5
+
+functions:
+  data_sources:
+    _type: data_source_registry
+    sources: []   # add sources — see aiq-add-data-source registry reference
+
+  intent_classifier:
+    _type: intent_classifier
+    llm: main_llm
+
+  shallow_research_agent:
+    _type: shallow_research_agent
+    llm: main_llm
+
+  deep_research_agent:
+    _type: deep_research_agent
+    orchestrator_llm: main_llm
+    planner_llm: main_llm
+    researcher_llm: main_llm
+    writer_llm: main_llm
+
+workflow:
+  _type: chat_deepresearcher_agent
+  enable_escalation: true
+  enable_clarifier: false   # set true and add clarifier_agent under functions: to enable
+  checkpoint_db: ${AIQ_CHECKPOINT_DB:-./checkpoints.db}

.agents/skills/aiq-configure-workflow/references/composing-config.md

@@ -0,0 +1,207 @@
+# Composing a workflow config
+
+Holistic map for `configs/config_*.yml`. **Canonical field tables and defaults:**
+`docs/source/customization/configuration-reference.md`. **Worked examples:**
+`configs/config_web_default_llamaindex.yml` (web) and `configs/config_cli_default.yml`
+(CLI).
+
+## How to edit
+
+1. Copy the nearest shipped profile ([config-profiles.md](config-profiles.md)).
+2. Merge feature blocks from other `configs/*.yml` files (knowledge, guardrails,
+   sandbox, MCP auth) — do not invent fields.
+3. Tune sections below; cross-check names against the canonical reference.
+4. Run `validate_config.py` ([config-schema.md](config-schema.md)).
+
+Sibling skills for depth (do not duplicate here):
+
+- Registry / UI → `.agents/skills/aiq-add-data-source/references/registry-and-ui.md`
+- LLM roles → `.agents/skills/aiq-customize-prompts-models/references/model-selection.md`
+- New NAT packages → `aiq-add-tool` / `aiq-add-data-source`
+- Secrets → `skills/aiq-deploy/references/env-and-secrets.md`
+
+## Top-level sections
+
+```yaml
+general:      # use_uvloop, telemetry, front_end (web only)
+llms:         # named aliases → referenced by agents
+functions:    # data_source_registry, tools, agents (+ optional feature functions)
+workflow:     # chat_deepresearcher_agent orchestrator flags
+```
+
+`${VAR}` / `${VAR:-default}` anywhere; secrets in `deploy/.env`.
+
+---
+
+## `general:`
+
+| Area | CLI | Web (`nat serve`) | Where defaults live |
+|------|-----|-------------------|---------------------|
+| `use_uvloop` | optional | recommended `true` | `config_web_default_llamaindex.yml` |
+| `telemetry` | yes | yes | both shipped profiles |
+| `front_end` | **omit** | **required** (`aiq_api`) | `config_web_default_llamaindex.yml` |
+
+### Telemetry
+
+**Console** (always available):
+
+```yaml
+general:
+  telemetry:
+    logging:
+      console:
+        _type: console
+        level: INFO   # DEBUG | INFO | WARNING | ERROR
+```
+
+**Tracing** — enable under `general.telemetry.tracing` (exporters can coexist).
+Uncomment the matching block in any `config_web_*.yml` or copy from
+`docs/source/deployment/observability.md`:
+
+| Backend | YAML `_type` | Setup notes |
+|---------|--------------|-------------|
+| Phoenix | `phoenix` | `phoenix serve`; set `endpoint`, `project` |
+| LangSmith | `langsmith` or env-only | `LANGCHAIN_TRACING_V2`, `LANGCHAIN_API_KEY`, `LANGCHAIN_PROJECT` |
+| Weave | `weave` | `WANDB_API_KEY`; `project`, optional `redact_pii` |
+| OpenTelemetry | `otelcollector_redaction` | `endpoint`; redaction + batch fields |
+
+`verbose: true` on `workflow:` or agents adds console detail without a tracer.
+
+### `front_end` (`aiq_api`)
+
+Required for UI, REST, WebSocket chat, async jobs, and `aiq-research`. Full
+block with comments: `config_web_default_llamaindex.yml` lines 18–47.
+
+| Field | Default (web profile) | Tune when |
+|-------|----------------------|-----------|
+| `_type` | `aiq_api` | always |
+| `runner_class` | `aiq_api.plugin.AIQAPIWorker` | rarely |
+| `db_url` | `${NAT_JOB_STORE_DB_URL:-sqlite+aiosqlite:///./jobs.db}` | Postgres prod |
+| `expiry_seconds` | `86400` (600–604800) | job retention policy |
+| `cors` | localhost regex | production UI origin |
+
+**Env (not YAML):** `NAT_JOB_STORE_DB_URL`, `NAT_DASK_SCHEDULER_ADDRESS`,
+`REQUIRE_AUTH`, `AIQ_TRACE_*` — `frontends/aiq_api/README.md`.
+
+`knowledge_retrieval` under `functions:` auto-enables `/v1/collections` and
+`/v1/documents`.
+
+---
+
+## `llms:`
+
+Define **aliases** under `llms:`; agents reference them by key (`llm`,
+`orchestrator_llm`, `planner_llm`, etc.). Swap models by editing aliases or
+repointing role fields — no Python changes.
+
+| Parameter | Typical use | Notes |
+|-----------|-------------|-------|
+| `_type` | `nim` or `openai` | Provider plugin |
+| `model_name` | required | e.g. `nvidia/nemotron-3-super-120b-a12b` |
+| `base_url` | NIM / compatible endpoint | Set explicitly for hosted NIM |
+| `api_key` | optional | Falls back to `NVIDIA_API_KEY` for NIM |
+| `temperature`, `top_p`, `max_tokens` | per role | See role table in configuration-reference |
+| `num_retries` | resilience | default `5` |
+| `chat_template_kwargs` | e.g. `enable_thinking: true` | chain-of-thought models |
+
+**Role-specific starting points** (temperature / max_tokens): intent classifier
+(moderate), shallow researcher (low), deep orchestrator/writer (high + thinking),
+summary LLM (short output). Full table: `configuration-reference.md` §
+`llms` → "Common LLM Configurations".
+
+**Deeper guidance:** `docs/source/customization/swapping-models.md` and
+`.agents/skills/aiq-customize-prompts-models/references/model-selection.md`.
+
+---
+
+## `functions:` — tools, registry, agents
+
+Each entry is a named function under `functions:`. The YAML `_type` selects the
+NAT plugin; the **key you choose** (e.g. `web_search_tool`) is what registry
+`tools:` lists and agents reference.
+
+### Option index: retrieval / search tools
+
+Full parameter tables: `configuration-reference.md` § `functions`. Examples in
+shipped configs under `configs/`.
+
+| `_type` | Purpose | Key options to tune | Env / profile |
+|---------|---------|---------------------|---------------|
+| `tavily_web_search` | Web search | `max_results`, `advanced_search`, `max_content_length`, `api_base_url` | `TAVILY_API_KEY`; all web profiles |
+| `exa_web_search` | Web search (Exa) | `max_results`, `search_type` (`auto`/`fast`/`deep`), `full_text`, `highlights` | `EXA_API_KEY` |
+| `paper_search` | Academic papers | `provider` (`serper`/`serpapi`/`searchapi`), `max_results` | `SERPER_API_KEY`, etc.; commented in most profiles |
+| `knowledge_retrieval` | Document RAG | `backend` (`llamaindex`/`foundational_rag`/`opensearch`), `top_k`, `collection_name`, backend-specific URLs/auth | `config_web_default_llamaindex.yml`, `config_web_frag.yml`, `config_web_opensearch.yml` |
+
+Enable/disable for agents: register in `data_source_registry`, then inherit or
+`exclude_tools` (below). Knowledge-layer backends: `docs/source/customization/knowledge-layer.md`.
+
+### `data_source_registry`
+
+Single source of truth for tools → UI toggles → agent inherit. One block with
+`_type: data_source_registry` (often `data_sources`):
+
+```yaml
+  data_sources:
+    _type: data_source_registry
+    sources:
+      - id: web_search
+        name: "Web Search"
+        tools: [web_search_tool, advanced_web_search_tool]
+  web_search_tool:
+    _type: tavily_web_search
+    max_results: 5
+```
+
+- Omit agent `tools:` → inherit all registry tools; use `exclude_tools` to specialize
+  (`config_web_default_llamaindex.yml` shallow vs deep).
+- Enable a tool: declare under `functions:`, add to a source's `tools:`, set env vars.
+
+Optional **feature** function blocks (guardrails middleware, sandbox, skills,
+domain catalog) are not search tools — copy YAML from the profile listed in
+[config-profiles.md](config-profiles.md); field docs in `configuration-reference.md`
+and feature guides under `docs/source/customization/`.
+
+### Option index: agents
+
+| `_type` | Key options to tune | Doc anchor |
+|---------|---------------------|------------|
+| `intent_classifier` | `llm`, `tools`, `llm_timeout`, `verbose` | `configuration-reference.md` § `intent_classifier` |
+| `clarifier_agent` | `llm`, `max_turns`, `exclude_tools`, `verbose` | § `clarifier_agent` |
+| `shallow_research_agent` | `llm`, `max_llm_turns`, `max_tool_iterations`, `exclude_tools` | § `shallow_research_agent` |
+| `deep_research_agent` | role LLMs, `exclude_tools`, `enable_source_router`, `domain_catalog_path`, `enable_citation_verification`, `skills`, `sandbox`, concurrency caps | § `deep_research_agent` |
+
+### Agents (required by workflow)
+
+| Function | When | LLM fields |
+|----------|------|------------|
+| `intent_classifier` | always | `llm` |
+| `shallow_research_agent` | always | `llm` |
+| `deep_research_agent` | always | `orchestrator_llm`, `planner_llm`, `researcher_llm`, `writer_llm`, `source_router_llm` |
+| `clarifier_agent` | `workflow.enable_clarifier: true` | `llm` |
+
+---
+
+## `workflow:`
+
+```yaml
+workflow:
+  _type: chat_deepresearcher_agent
+  enable_escalation: true       # false → shallow only
+  enable_clarifier: true
+  use_async_deep_research: true   # needs general.front_end
+  max_history: 20
+  checkpoint_db: ${AIQ_CHECKPOINT_DB:-./checkpoints.db}
+  verbose: true
+```
+
+Full defaults table: `configuration-reference.md` § `workflow`.
+
+---
+
+## Validate
+
+```bash
+uv run python .agents/skills/aiq-configure-workflow/scripts/validate_config.py configs/config_<name>.yml
+```
+
+What it checks: [config-schema.md](config-schema.md). Fix all errors before deploy.