fix: update available default model successors - #414
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis PR replaces legacy model profiles with role-specific Nemotron Super, Ultra, and Ultra Writer configurations. It updates model defaults and documentation, adds embedding compatibility validation for Chroma and OpenSearch, and expands configuration tests. ChangesModel configuration and workflow wiring
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Comment |
a12e47c to
7a3faa3
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/source/extending/adding-a-tool.md (1)
279-292: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUse Nano for the shallow-research example.
This example assigns Nemotron 3 Ultra to
shallow_research_agent. The updated default role mapping assigns Nemotron Nano 3.5 Preview to shallow research. Changeresearch_llmto Nano, or state that this is an intentional higher-capability override.Proposed fix
research_llm: _type: nim - model_name: nvidia/nemotron-3-ultra-550b-a55b + model_name: nvidia/nemotron-nano-3.5-preview🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/extending/adding-a-tool.md` around lines 279 - 292, Update the research_llm configuration in the shallow_research_agent example to use the Nano model matching the default shallow-research role mapping, or explicitly document that Nemotron 3 Ultra is an intentional higher-capability override.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/aiq-customize-prompts-models/references/model-selection.md:
- Around line 57-61: Update the model-selection example so the writer_llm
assignment references a model defined in the preceding llms configuration.
Either add the missing nemotron_ultra_writer_llm definition before the role
assignments or change writer_llm to use the existing nemotron_ultra_llm entry;
preserve the other role mappings.
In `@configs/config_web_default_guardrails.yml`:
- Around line 81-82: Remove one of the three consecutive blank lines near the
indicated location in config_web_default_guardrails.yml, leaving no more than
two consecutive blank lines so YAMLlint passes.
In `@docs/notebooks/0_Getting_Started_with_AIQ.ipynb`:
- Line 404: Update the llms model-role description in the notebook to match the
current example configuration: identify Nemotron Nano as handling intent
classification and shallow research, and Nemotron Ultra as handling the
deep-research roles. Remove the stale claim that Nemotron Ultra handles intent,
while preserving the existing role-specific settings description.
- Around line 781-787: Update the Together.ai migration guidance near the
Nemotron LLM example to cover all configured definitions:
nemotron_nano_intent_llm, nemotron_nano_agent_llm, nemotron_ultra_llm, and
nemotron_ultra_writer_llm. Ensure the example and surrounding instructions
consistently direct users to update every applicable block so shallow research
and report writing no longer use NIM.
In `@docs/notebooks/2_Deep_Researcher_Customization.ipynb`:
- Line 331: Align the notebook example heading with its configuration: update
the heading to state that Nemotron serves only the researcher, or change the
example configuration so planner_llm also uses nemotron_ultra_llm. Review the
surrounding documentation for any stale references and ensure all descriptions
match the current planner_llm and researcher_llm assignments.
In `@docs/source/customization/swapping-models.md`:
- Around line 27-40: Correct the mismatch in the “NIM with thinking” example by
either renaming the heading to reflect that enable_thinking is false or changing
chat_template_kwargs.enable_thinking to true if the example is intended to
demonstrate thinking.
In `@sources/knowledge_layer/src/llamaindex/adapter.py`:
- Line 73: Update the LlamaIndexIngestor class docstring to document
nvidia/nemotron-3-nano-omni-30b-a3b-reasoning as the default VLM model, matching
DEFAULT_VLM_MODEL and leaving the runtime configuration unchanged.
In `@tests/aiq_agent/test_default_model_profiles.py`:
- Around line 79-86: Update the test loop covering orchestrator, router,
researcher, planner, and writer profiles to separately assert that
function["writer_llm"] uses the dedicated "nemotron_ultra_writer_llm" alias
before validating its resolved ULTRA_MODEL, while keeping the existing model
assertions for the other roles.
- Around line 14-18: The CONFIG_GLOBS tuple in the test does not include a
pattern matching the agent-skill scaffold configuration at
.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml, so
role-mapping regressions in that file will not be caught. Add a new glob pattern
to CONFIG_GLOBS that matches the scaffold configuration file path to include it
in the test parameter set, ensuring the scaffold is validated alongside the
other configurations.
---
Outside diff comments:
In `@docs/source/extending/adding-a-tool.md`:
- Around line 279-292: Update the research_llm configuration in the
shallow_research_agent example to use the Nano model matching the default
shallow-research role mapping, or explicitly document that Nemotron 3 Ultra is
an intentional higher-capability override.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: b76c8167-1a98-404d-9525-efa8234def5e
📒 Files selected for processing (57)
.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-configure-workflow/references/config-profiles.md.agents/skills/aiq-customize-prompts-models/references/model-selection.md.secrets.baselineREADME.mdconfigs/config_cli_default.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_openshell.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_opensearch.ymldeploy/.env.exampledocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/source/architecture/agents/deep-researcher.mddocs/source/customization/configuration-reference.mddocs/source/customization/knowledge-layer.mddocs/source/customization/mcp-tools.mddocs/source/customization/swapping-models.mddocs/source/customization/tools-and-sources.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/examples/cli-with-local-nims.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/examples/minimal-shallow-only.mddocs/source/extending/adding-a-tool.mddocs/source/get-started/installation.mddocs/source/profiling/index.mddocs/source/resources/troubleshooting.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.ymlfrontends/benchmarks/deepsearch_qa/README.mdfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymlfrontends/ui/src/features/chat/lib/intermediate-step-parser.tsmcp/tests/test_config_and_packaging.pymcp/tests/test_preclassification_integration.pysources/google_scholar_paper_search/README.mdsources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.mdsources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/llamaindex/adapter.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/register.pysrc/aiq_agent/tokenomics/README.mdtests/aiq_agent/test_default_model_profiles.pytests/knowledge_layer_tests/test_azure_ai_search.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (17)
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages into skills.
Files:
docs/source/customization/tools-and-sources.mddocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/get-started/installation.mddocs/source/extending/adding-a-tool.mddocs/source/customization/mcp-tools.mddocs/source/customization/swapping-models.mddocs/source/profiling/index.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/minimal-shallow-only.mddocs/source/resources/troubleshooting.mddocs/source/examples/cli-with-local-nims.mddocs/source/customization/knowledge-layer.mddocs/source/customization/configuration-reference.mddocs/source/architecture/agents/deep-researcher.md
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes inside this repository, avoid editing adjacent repositories, and scope changes to the smallest relevant independent package, especially undersources/.
Run the narrowest relevant validation command first and broaden to the full suite only when a change crosses shared boundaries.
Keep pull requests scoped, exclude unrelated files and generated artifacts, never include secrets, and provide validation commands and results.
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
Files:
docs/source/customization/tools-and-sources.mdmcp/tests/test_config_and_packaging.pydocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/get-started/installation.mdfrontends/benchmarks/deepsearch_qa/README.mddocs/source/extending/adding-a-tool.mdsources/google_scholar_paper_search/README.mdfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.ymlmcp/tests/test_preclassification_integration.pysources/knowledge_layer/src/azure_ai_search/adapter.pyfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymldocs/source/customization/mcp-tools.mdsources/knowledge_layer/src/opensearch/adapter.pytests/knowledge_layer_tests/test_azure_ai_search.pydocs/source/customization/swapping-models.mdsources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.mddocs/source/profiling/index.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/deployment/aws-opensearch-serverless.mdfrontends/ui/src/features/chat/lib/intermediate-step-parser.tsconfigs/config_domain_routing_and_skills.ymlsrc/aiq_agent/tokenomics/README.mdREADME.mdfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymldocs/source/examples/full-pipeline-llamaindex.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlconfigs/config_cli_default.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymldocs/source/examples/minimal-shallow-only.mddocs/source/resources/troubleshooting.mdconfigs/config_openshell.ymlconfigs/config_frontier_models.ymldocs/source/examples/cli-with-local-nims.mddocs/source/customization/knowledge-layer.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlconfigs/config_mcp.ymlsources/knowledge_layer/src/register.pysources/knowledge_layer/src/llamaindex/adapter.pydocs/notebooks/1_Deep_Researcher_Web_Search.ipynbconfigs/config_web_azure_ai_search.ymldocs/notebooks/0_Getting_Started_with_AIQ.ipynbtests/aiq_agent/test_default_model_profiles.pyconfigs/config_web_default_guardrails.ymldocs/notebooks/2_Deep_Researcher_Customization.ipynbconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymldocs/source/customization/configuration-reference.mddocs/source/architecture/agents/deep-researcher.mdconfigs/config_web_opensearch.ymlconfigs/config_web_frag_mcp_auth.yml
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/customization/tools-and-sources.mddocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/get-started/installation.mddocs/source/extending/adding-a-tool.mddocs/source/customization/mcp-tools.mddocs/source/customization/swapping-models.mddocs/source/profiling/index.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/deployment/aws-opensearch-serverless.mdREADME.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/minimal-shallow-only.mddocs/source/resources/troubleshooting.mddocs/source/examples/cli-with-local-nims.mddocs/source/customization/knowledge-layer.mddocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/source/customization/configuration-reference.mddocs/source/architecture/agents/deep-researcher.md
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format and lint Python with Ruff using a 120-character line length, Python 3.11 target, rules E/F/W/I/PL/UP, and single-line imports; avoid reformatting unrelated code.
Never print or log secret values, including through tool output or error messages.
**/*.py: Runuv run ruff check .anduv run ruff format --check .for root Python changes.
Runuv run pytestfor root project Python changes.
Files:
mcp/tests/test_config_and_packaging.pymcp/tests/test_preclassification_integration.pysources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/opensearch/adapter.pytests/knowledge_layer_tests/test_azure_ai_search.pysources/knowledge_layer/src/register.pysources/knowledge_layer/src/llamaindex/adapter.pytests/aiq_agent/test_default_model_profiles.py
mcp/**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For MCP changes, run the MCP development dependency setup and
uv run --project mcp --extra dev pytest mcp/tests.
Files:
mcp/tests/test_config_and_packaging.pymcp/tests/test_preclassification_integration.py
{src/aiq_agent/knowledge/**,sources/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/knowledge/**,sources/**}: Review data-source and knowledge-layer changes for optional dependency boundaries, external API error handling,
retry/rate-limit behavior, deterministic tests, and registration consistency. New source packages should include
package metadata, plugin registration when applicable, and source-level tests.
Files:
sources/google_scholar_paper_search/README.mdsources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.mdsources/knowledge_layer/src/register.pysources/knowledge_layer/src/llamaindex/adapter.py
**/*.{yaml,yml}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For deployment changes, run the relevant Helm or Compose validation and describe the environment used.
Files:
frontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymlconfigs/config_domain_routing_and_skills.ymlfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlconfigs/config_cli_default.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlconfigs/config_openshell.ymlconfigs/config_frontier_models.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlconfigs/config_mcp.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/config_web_opensearch.ymlconfigs/config_web_frag_mcp_auth.yml
sources/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Implement new tools and data sources as NeMo Agent Toolkit functions registered with
@register_function; configuration schemas must inherit fromFunctionBaseConfig.
Files:
sources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/register.pysources/knowledge_layer/src/llamaindex/adapter.py
.agents/skills/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Use repository maintainer skills from
.agents/skills/for applicable workflows; keep API-consumer skills inskills/separate because they are not deployed application runtime skills.
Files:
.agents/skills/aiq-customize-prompts-models/references/model-selection.md.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/config-profiles.md
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}
⚙️ CodeRabbit configuration file
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.
Files:
.agents/skills/aiq-customize-prompts-models/references/model-selection.md.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/config-profiles.md
frontends/ui/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
frontends/ui/**/*.{ts,tsx}: Use the existing Next.js, React, TypeScript, Tailwind, and KUI patterns; reuse KUI components instead of introducing new equivalent UI components.
Access the backend through the proxy andBACKEND_URL, and preserve authentication-aware UI states.
Validate UI-affecting changes withnpm run lint,npm run type-check, andnpm run test:ci; include a screenshot for visible changes.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
frontends/ui/**/*.{js,jsx,ts,tsx,css,scss}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For UI changes, run
npm ci,npm run lint,npm run type-check,npm run test:ci, andnpm run buildfromfrontends/ui.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
frontends/ui/**/*
⚙️ CodeRabbit configuration file
frontends/ui/**/*: Review UI changes for strict TypeScript behavior, API contract alignment, auth/session handling, accessible controls,
resilient loading and error states, and report/chat state consistency. Prefer existing UI patterns and require tests
for changed user-visible workflows.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
configs/**/*.y{a,}ml
📄 CodeRabbit inference engine (AGENTS.md)
Use
_typevalues derived from the registered configuration class names in workflow YAML configuration.
Files:
configs/config_domain_routing_and_skills.ymlconfigs/config_cli_default.ymlconfigs/config_openshell.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/config_web_opensearch.ymlconfigs/config_web_frag_mcp_auth.yml
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_domain_routing_and_skills.ymlconfigs/config_cli_default.ymlconfigs/config_openshell.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/config_web_opensearch.ymlconfigs/config_web_frag_mcp_auth.yml
src/aiq_agent/tokenomics/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/tokenomics/**/*: Review tokenomics changes for deterministic accounting, pricing-data assumptions, rounding behavior, and report
compatibility. Flag pricing or aggregation changes without representative tests and clear documentation updates.
Files:
src/aiq_agent/tokenomics/README.md
**/{.env,.env.*}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, resolving API keys at runtime.
Files:
deploy/.env.example
🪛 LanguageTool
docs/source/customization/swapping-models.md
[style] ~27-~27: Consider a different adjective to strengthen your wording.
Context: ...le: NIM with thinking (for example, for deep research)** ```yaml llms: nemotron_u...
(DEEP_PROFOUND)
[style] ~82-~82: Consider a different adjective to strengthen your wording.
Context: ... and a local downloadable Ultra NIM for deep research: ```yaml llms: hosted_nano_...
(DEEP_PROFOUND)
sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md
[uncategorized] ~145-~145: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...| false | Extract tables from PDFs as markdown using pdfplumber | | `AIQ_EXTRACT_IMAGE...
(MARKDOWN_NNP)
.agents/skills/aiq-configure-workflow/references/config-profiles.md
[style] ~20-~20: Consider a different adjective to strengthen your wording.
Context: ...nt/shallow; Ultra for clarification and deep research 4. Optional features — co...
(DEEP_PROFOUND)
docs/source/customization/knowledge-layer.md
[uncategorized] ~287-~287: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...| false | Extract tables from PDFs as markdown | | AIQ_EXTRACT_IMAGES | false | Ex...
(MARKDOWN_NNP)
docs/source/customization/configuration-reference.md
[style] ~116-~116: To form a complete sentence, be sure to include a subject.
Context: ...l|str|None` | API endpoint URL. Should always be set explicitly for NVIDIA NIM...
(MISSING_IT_THERE)
🪛 YAMLlint (1.37.1)
configs/config_web_default_guardrails.yml
[error] 82-82: too many blank lines (3 > 2)
(empty-lines)
🔇 Additional comments (52)
mcp/tests/test_config_and_packaging.py (1)
77-77: LGTM!mcp/tests/test_preclassification_integration.py (1)
30-30: LGTM!tests/aiq_agent/test_default_model_profiles.py (1)
1-13: LGTM!Also applies to: 20-78, 89-101
tests/knowledge_layer_tests/test_azure_ai_search.py (1)
420-420: LGTM!docs/notebooks/0_Getting_Started_with_AIQ.ipynb (1)
73-76: LGTM!Also applies to: 93-93, 302-351, 366-388, 420-420
docs/notebooks/1_Deep_Researcher_Web_Search.ipynb (1)
344-348: LGTM!Also applies to: 434-456, 467-476, 603-665
docs/notebooks/2_Deep_Researcher_Customization.ipynb (1)
178-178: LGTM!Also applies to: 203-225, 241-251, 311-313, 477-477, 502-524, 542-552, 565-565
docs/source/examples/cli-with-local-nims.md (1)
14-14: LGTM!Also applies to: 28-28, 63-84, 107-114, 123-135, 199-199
frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.yml (1)
21-43: LGTM!Also applies to: 58-62
docs/source/examples/hybrid-frontier-model.md (1)
22-45: LGTM!Also applies to: 76-91
docs/source/examples/full-pipeline-llamaindex.md (1)
46-102: LGTM!Also applies to: 143-143, 152-152, 162-162, 172-176
docs/source/examples/minimal-shallow-only.md (1)
28-42: LGTM!frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.yml (1)
21-43: LGTM!Also applies to: 58-62
frontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.yml (1)
21-28: LGTM!frontends/benchmarks/deepsearch_qa/README.md (1)
77-77: LGTM!frontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.yml (1)
2-2: LGTM!Also applies to: 17-39, 56-60
frontends/benchmarks/freshqa/configs/config_full_workflow.yml (1)
2-2: LGTM!Also applies to: 12-60, 76-80, 89-93
frontends/benchmarks/freshqa/configs/config_shallow_research_only.yml (1)
2-2: LGTM!Also applies to: 16-27, 42-42
docs/source/examples/full-pipeline-web.md (1)
63-115: LGTM!Also applies to: 160-160, 173-173, 187-187, 201-205
deploy/.env.example (1)
188-192: LGTM!sources/google_scholar_paper_search/README.md (1)
119-119: LGTM!sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md (1)
142-149: LGTM!Also applies to: 220-220, 482-482
sources/knowledge_layer/src/llamaindex/adapter.py (1)
27-27: LGTM!Also applies to: 450-450, 531-531
sources/knowledge_layer/src/register.py (1)
258-258: 🗄️ Data Integrity & IntegrationDefine one migration policy for the new embedding default.
If persisted indexes are reused, vectors from the previous model are not comparable with vectors from
nvidia/nemotron-3-embed-1b. The unchanged 2048 dimension and environment override do not prevent mixed-model reads.
sources/knowledge_layer/src/register.py#L258-L258: propagate model-version metadata or require re-embedding for configured backends.sources/knowledge_layer/src/azure_ai_search/adapter.py#L92-L92: validate the stored model before Azure index reuse.sources/knowledge_layer/src/llamaindex/adapter.py#L485-L488: validate the Chroma collection before ingestion and retrieval.sources/knowledge_layer/src/llamaindex/adapter.py#L1606-L1606: reject queries against collections created with a different embedding model.sources/knowledge_layer/src/opensearch/adapter.py#L76-L76: validate the OpenSearch index model before reuse.frontends/ui/src/features/chat/lib/intermediate-step-parser.ts (1)
93-93: LGTM!Also applies to: 133-133, 145-145
.secrets.baseline (1)
136-136: 🔒 Security & PrivacyValidate the generated baseline against the current checkout. Regenerate or validate
.secrets.baselinefrom the exact tree and confirm that theREADME.mdfinding belongs at line 395. A stale line mapping can leave secret-detection results out of sync..agents/skills/aiq-configure-workflow/assets/config-scaffold.yml (1)
20-52: LGTM!Also applies to: 61-72
configs/config_cli_default.yml (1)
19-67: LGTM!Also applies to: 122-129, 138-138, 152-152
configs/config_domain_routing_and_skills.yml (1)
50-80: LGTM!Also applies to: 219-219
configs/config_frontier_models.yml (1)
51-75: LGTM!Also applies to: 98-101, 135-135, 154-154
configs/config_mcp.yml (1)
29-77: LGTM!Also applies to: 102-114, 127-127
configs/config_openshell.yml (1)
31-86: LGTM!Also applies to: 127-139, 196-196
configs/config_web_azure_ai_search.yml (1)
50-106: LGTM!Also applies to: 194-202, 212-212, 227-227
configs/config_web_default_guardrails.yml (1)
31-79: LGTM!Also applies to: 83-88, 224-249
configs/config_web_default_llamaindex.yml (1)
50-106: LGTM!Also applies to: 195-202, 211-211, 226-226
configs/config_web_frag.yml (1)
52-100: LGTM!Also applies to: 165-171, 180-180, 194-194
configs/config_web_frag_mcp_auth.yml (1)
59-107: LGTM!Also applies to: 214-220, 229-229, 243-243
configs/config_web_opensearch.yml (1)
51-99: LGTM!Also applies to: 157-157, 172-178, 187-187, 201-201
.agents/skills/aiq-configure-workflow/references/composing-config.md (1)
100-100: LGTM!.agents/skills/aiq-configure-workflow/references/config-profiles.md (1)
20-20: LGTM!README.md (1)
111-120: LGTM!Also applies to: 153-157, 290-300
docs/source/resources/troubleshooting.md (1)
38-43: LGTM!Also applies to: 45-76
docs/source/get-started/installation.md (1)
28-32: LGTM!docs/source/profiling/index.md (1)
148-154: LGTM!src/aiq_agent/tokenomics/README.md (1)
40-47: LGTM!Also applies to: 62-62
docs/source/architecture/agents/deep-researcher.md (1)
239-243: LGTM!Also applies to: 264-264
docs/source/customization/configuration-reference.md (1)
95-122: LGTM!Also applies to: 131-135, 335-335, 368-368, 382-382, 457-461, 591-639, 660-687
docs/source/customization/knowledge-layer.md (1)
115-115: LGTM!Also applies to: 129-129, 253-253, 283-291, 310-310
docs/source/customization/mcp-tools.md (1)
117-117: LGTM!docs/source/customization/tools-and-sources.md (1)
140-140: 📐 Maintainability & Code QualityVerify the remaining LLM references in this example.
This example now uses
nemotron_nano_intent_llm, but its clarifier, shallow-research, and deep-research entries still usenemotron_llmandnemotron_llm_deep. Update those entries to the role-specific Nano and Ultra profile names if this block is intended to document the shipping default profile.As per coding guidelines, “Update canonical documentation under
docs/source/when behavior, configuration, or workflows change.”#!/bin/bash set -euo pipefail rg -n -C 4 \ 'nemotron_(nano_intent_llm|nano_agent_llm|ultra_llm|ultra_writer_llm|llm_deep|llm)' \ docs/source/customization/tools-and-sources.md \ configs/config_cli_default.ymldocs/source/deployment/aws-opensearch-serverless.md (1)
334-338: LGTM!docs/source/evaluation/benchmarks/deepsearch-qa.md (1)
81-81: LGTM!
7a3faa3 to
b895411
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/notebooks/2_Deep_Researcher_Customization.ipynb`:
- Line 565: Update the models mapping in the deep-research customization
notebook to document all configured deep-research roles, adding
source_router_llm mapped to nemotron_ultra_llm and writer_llm mapped to
nemotron_ultra_writer_llm while preserving the existing orchestrator, planner,
and researcher mappings.
In `@docs/source/customization/configuration-reference.md`:
- Around line 131-132: Align the Nano shallow-research documentation with
shipped defaults: in docs/source/customization/configuration-reference.md lines
131-132, update the shallow researcher row to top_p 0.7 with thinking enabled;
in lines 604-615, update nano_agent_llm to top_p 0.7 and enable_thinking true.
Review these sections for stale examples and keep all other settings unchanged.
In `@docs/source/examples/cli-with-local-nims.md`:
- Line 28: Update the local NIM setup instructions around the nemotron image and
docker run command to authenticate with nvcr.io using NGC_API_KEY before
starting the container, pass NGC_API_KEY into the container, and list it in the
Required Environment Variables section.
In `@docs/source/extending/adding-a-tool.md`:
- Line 282: Align both examples with the role-specific model mapping: in
docs/source/extending/adding-a-tool.md lines 282-282, configure
shallow_research_agent with the Nano agent profile; in
sources/google_scholar_paper_search/README.md lines 119-119, define the Ultra
and Ultra Writer profiles and bind the deep-research roles to them instead of
passing the Nano my_llm.
In `@sources/knowledge_layer/src/register.py`:
- Around line 257-259: Prevent persisted-store reuse across embedding-model
changes by adding model-identity validation or versioning the collection/index
namespace around the embed_model configuration and the Chroma, Azure AI Search,
LlamaIndex, and OpenSearch adapter flows. Update
sources/knowledge_layer/src/register.py:257-259,
sources/knowledge_layer/src/azure_ai_search/adapter.py:92-93,
sources/knowledge_layer/src/llamaindex/adapter.py:485-488 and 1606-1607, and
sources/knowledge_layer/src/opensearch/adapter.py:76 so ingestion and retrieval
reject mismatched models or use a model-specific namespace. Document the
required reindex migration in
sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md:142-143 and 220.
In `@src/aiq_agent/agents/shallow_researcher/prompts/researcher.j2`:
- Around line 3-9: Keep the mandatory research instruction in researcher.j2, and
add a runtime guard in the shallow researcher agent that rejects or retries an
initial final AIMessage when tools are available until a successful tool call
provides evidence. Update
tests/aiq_agent/agents/shallow_researcher/test_agent.py lines 290-300 to test
rejecting or retrying no-tool synthesis and accepting synthesis after tool
evidence; remove brittle literal prompt-substring assertions. The prompt site
requires no further direct change beyond retaining the instruction.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 5fffe060-c170-404d-b347-b58fdc6bf4dc
📒 Files selected for processing (59)
.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-configure-workflow/references/config-profiles.md.agents/skills/aiq-customize-prompts-models/references/model-selection.md.secrets.baselineREADME.mdconfigs/config_cli_default.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_openshell.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_opensearch.ymldeploy/.env.exampledocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/source/architecture/agents/deep-researcher.mddocs/source/customization/configuration-reference.mddocs/source/customization/knowledge-layer.mddocs/source/customization/mcp-tools.mddocs/source/customization/swapping-models.mddocs/source/customization/tools-and-sources.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/examples/cli-with-local-nims.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/examples/minimal-shallow-only.mddocs/source/extending/adding-a-tool.mddocs/source/get-started/installation.mddocs/source/profiling/index.mddocs/source/resources/troubleshooting.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.ymlfrontends/benchmarks/deepsearch_qa/README.mdfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymlfrontends/ui/src/features/chat/lib/intermediate-step-parser.tsmcp/tests/test_config_and_packaging.pymcp/tests/test_preclassification_integration.pysources/google_scholar_paper_search/README.mdsources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.mdsources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/llamaindex/adapter.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/register.pysrc/aiq_agent/agents/shallow_researcher/prompts/researcher.j2src/aiq_agent/tokenomics/README.mdtests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/test_default_model_profiles.pytests/knowledge_layer_tests/test_azure_ai_search.py
📜 Review details
⏰ Context from checks skipped due to timeout. (5)
- GitHub Check: UI Unit Tests
- GitHub Check: Run Harbor skill eval
- GitHub Check: Lint and Hooks
- GitHub Check: Pytest and Coverage
- GitHub Check: Script Validation
🧰 Additional context used
📓 Path-based instructions (18)
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages into skills.
Files:
docs/source/customization/tools-and-sources.mddocs/source/profiling/index.mddocs/source/extending/adding-a-tool.mddocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/customization/mcp-tools.mddocs/source/customization/knowledge-layer.mddocs/source/get-started/installation.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/examples/hybrid-frontier-model.mddocs/source/customization/swapping-models.mddocs/source/examples/full-pipeline-web.mddocs/source/resources/troubleshooting.mddocs/source/examples/minimal-shallow-only.mddocs/source/architecture/agents/deep-researcher.mddocs/source/examples/cli-with-local-nims.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/customization/configuration-reference.md
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes inside this repository, avoid editing adjacent repositories, and scope changes to the smallest relevant independent package, especially undersources/.
Run the narrowest relevant validation command first and broaden to the full suite only when a change crosses shared boundaries.
Keep pull requests scoped, exclude unrelated files and generated artifacts, never include secrets, and provide validation commands and results.
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
Files:
docs/source/customization/tools-and-sources.mdtests/aiq_agent/agents/shallow_researcher/test_agent.pysources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/register.pyfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlsrc/aiq_agent/agents/shallow_researcher/prompts/researcher.j2tests/knowledge_layer_tests/test_azure_ai_search.pyconfigs/config_cli_default.ymldocs/source/profiling/index.mddocs/source/extending/adding-a-tool.mdfrontends/ui/src/features/chat/lib/intermediate-step-parser.tsdocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/customization/mcp-tools.mdfrontends/benchmarks/deepsearch_qa/README.mdconfigs/config_domain_routing_and_skills.ymldocs/source/customization/knowledge-layer.mddocs/source/get-started/installation.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/examples/hybrid-frontier-model.mdconfigs/config_web_default_guardrails.ymldocs/source/customization/swapping-models.mddocs/source/examples/full-pipeline-web.mddocs/source/resources/troubleshooting.mddocs/source/examples/minimal-shallow-only.mdmcp/tests/test_config_and_packaging.pysources/google_scholar_paper_search/README.mdfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymldocs/source/architecture/agents/deep-researcher.mdsrc/aiq_agent/tokenomics/README.mdsources/knowledge_layer/src/opensearch/adapter.pyREADME.mdfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.ymldocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/notebooks/1_Deep_Researcher_Web_Search.ipynbtests/aiq_agent/test_default_model_profiles.pydocs/source/examples/cli-with-local-nims.mddocs/source/examples/full-pipeline-llamaindex.mdconfigs/config_web_frag.ymldocs/source/customization/configuration-reference.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlconfigs/config_web_azure_ai_search.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlsources/knowledge_layer/src/llamaindex/adapter.pyconfigs/config_web_default_llamaindex.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_web_frag_mcp_auth.ymlmcp/tests/test_preclassification_integration.pydocs/notebooks/0_Getting_Started_with_AIQ.ipynbconfigs/config_openshell.ymlconfigs/config_web_opensearch.ymlsources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/customization/tools-and-sources.mddocs/source/profiling/index.mddocs/source/extending/adding-a-tool.mddocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/customization/mcp-tools.mddocs/source/customization/knowledge-layer.mddocs/source/get-started/installation.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/examples/hybrid-frontier-model.mddocs/source/customization/swapping-models.mddocs/source/examples/full-pipeline-web.mddocs/source/resources/troubleshooting.mddocs/source/examples/minimal-shallow-only.mddocs/source/architecture/agents/deep-researcher.mdREADME.mddocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/source/examples/cli-with-local-nims.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/customization/configuration-reference.mddocs/notebooks/0_Getting_Started_with_AIQ.ipynb
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format and lint Python with Ruff using a 120-character line length, Python 3.11 target, rules E/F/W/I/PL/UP, and single-line imports; avoid reformatting unrelated code.
Never print or log secret values, including through tool output or error messages.
**/*.py: Runuv run ruff check .anduv run ruff format --check .for root Python changes.
Runuv run pytestfor root project Python changes.
Files:
tests/aiq_agent/agents/shallow_researcher/test_agent.pysources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/register.pytests/knowledge_layer_tests/test_azure_ai_search.pymcp/tests/test_config_and_packaging.pysources/knowledge_layer/src/opensearch/adapter.pytests/aiq_agent/test_default_model_profiles.pysources/knowledge_layer/src/llamaindex/adapter.pymcp/tests/test_preclassification_integration.py
sources/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Implement new tools and data sources as NeMo Agent Toolkit functions registered with
@register_function; configuration schemas must inherit fromFunctionBaseConfig.
Files:
sources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/register.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/llamaindex/adapter.py
{src/aiq_agent/knowledge/**,sources/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/knowledge/**,sources/**}: Review data-source and knowledge-layer changes for optional dependency boundaries, external API error handling,
retry/rate-limit behavior, deterministic tests, and registration consistency. New source packages should include
package metadata, plugin registration when applicable, and source-level tests.
Files:
sources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/register.pysources/google_scholar_paper_search/README.mdsources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/llamaindex/adapter.pysources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md
**/*.{yaml,yml}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For deployment changes, run the relevant Helm or Compose validation and describe the environment used.
Files:
frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlconfigs/config_cli_default.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_web_default_guardrails.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymlfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.ymlconfigs/config_web_frag.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlconfigs/config_web_azure_ai_search.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_openshell.ymlconfigs/config_web_opensearch.yml
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/shallow_researcher/prompts/researcher.j2
configs/**/*.y{a,}ml
📄 CodeRabbit inference engine (AGENTS.md)
Use
_typevalues derived from the registered configuration class names in workflow YAML configuration.
Files:
configs/config_cli_default.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_frag.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_openshell.ymlconfigs/config_web_opensearch.yml
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_cli_default.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_frag.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_openshell.ymlconfigs/config_web_opensearch.yml
frontends/ui/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
frontends/ui/**/*.{ts,tsx}: Use the existing Next.js, React, TypeScript, Tailwind, and KUI patterns; reuse KUI components instead of introducing new equivalent UI components.
Access the backend through the proxy andBACKEND_URL, and preserve authentication-aware UI states.
Validate UI-affecting changes withnpm run lint,npm run type-check, andnpm run test:ci; include a screenshot for visible changes.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
frontends/ui/**/*.{js,jsx,ts,tsx,css,scss}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For UI changes, run
npm ci,npm run lint,npm run type-check,npm run test:ci, andnpm run buildfromfrontends/ui.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
frontends/ui/**/*
⚙️ CodeRabbit configuration file
frontends/ui/**/*: Review UI changes for strict TypeScript behavior, API contract alignment, auth/session handling, accessible controls,
resilient loading and error states, and report/chat state consistency. Prefer existing UI patterns and require tests
for changed user-visible workflows.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
.agents/skills/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Use repository maintainer skills from
.agents/skills/for applicable workflows; keep API-consumer skills inskills/separate because they are not deployed application runtime skills.
Files:
.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-customize-prompts-models/references/model-selection.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/config-profiles.md
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}
⚙️ CodeRabbit configuration file
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.
Files:
.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-customize-prompts-models/references/model-selection.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/config-profiles.md
mcp/**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For MCP changes, run the MCP development dependency setup and
uv run --project mcp --extra dev pytest mcp/tests.
Files:
mcp/tests/test_config_and_packaging.pymcp/tests/test_preclassification_integration.py
src/aiq_agent/tokenomics/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/tokenomics/**/*: Review tokenomics changes for deterministic accounting, pricing-data assumptions, rounding behavior, and report
compatibility. Flag pricing or aggregation changes without representative tests and clear documentation updates.
Files:
src/aiq_agent/tokenomics/README.md
**/{.env,.env.*}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, resolving API keys at runtime.
Files:
deploy/.env.example
🧠 Learnings (1)
📚 Learning: 2026-07-06T23:55:42.908Z
Learnt from: cdgamarose-nv
Repo: NVIDIA-AI-Blueprints/aiq PR: 311
File: src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2:74-81
Timestamp: 2026-07-06T23:55:42.908Z
Learning: In agent test files (e.g., tests/aiq_agent/agents/*/test_agent.py), avoid brittle assertions that match exact substrings from prompt template files (such as *.j2 prompt wording). Prompt wording can change frequently, so instead assert structural/behavioral properties (e.g., that the prompt builder is called, that required sections/fields are present via stable markers, that the model output/agent behavior conforms to an expected schema, or that key actions are taken) rather than matching literal prompt text.
Applied to files:
tests/aiq_agent/agents/shallow_researcher/test_agent.py
🪛 LanguageTool
docs/source/customization/knowledge-layer.md
[uncategorized] ~287-~287: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...| false | Extract tables from PDFs as markdown | | AIQ_EXTRACT_IMAGES | false | Ex...
(MARKDOWN_NNP)
docs/source/customization/swapping-models.md
[style] ~27-~27: Consider a different adjective to strengthen your wording.
Context: ...le: NIM with thinking (for example, for deep research)** ```yaml llms: nemotron_u...
(DEEP_PROFOUND)
[style] ~82-~82: Consider a different adjective to strengthen your wording.
Context: ... and a local downloadable Ultra NIM for deep research: ```yaml llms: hosted_nano_...
(DEEP_PROFOUND)
docs/source/customization/configuration-reference.md
[style] ~116-~116: To form a complete sentence, be sure to include a subject.
Context: ...l|str|None` | API endpoint URL. Should always be set explicitly for NVIDIA NIM...
(MISSING_IT_THERE)
.agents/skills/aiq-configure-workflow/references/config-profiles.md
[style] ~20-~20: Consider a different adjective to strengthen your wording.
Context: ...nt/shallow; Ultra for clarification and deep research 4. Optional features — co...
(DEEP_PROFOUND)
sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md
[uncategorized] ~145-~145: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...| false | Extract tables from PDFs as markdown using pdfplumber | | `AIQ_EXTRACT_IMAGE...
(MARKDOWN_NNP)
🪛 YAMLlint (1.37.1)
configs/config_web_default_guardrails.yml
[error] 82-82: too many blank lines (3 > 2)
(empty-lines)
🔇 Additional comments (61)
tests/aiq_agent/test_default_model_profiles.py (2)
14-18: Existing finding remains.
CONFIG_GLOBSstill excludes.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml. A role-mapping regression in the shipped scaffold will not fail this test. Add that path to the tested configuration set.As per coding guidelines, “Add or update tests for behavior changes.”
Source: Coding guidelines
89-96: Existing finding remains.The loop only validates the writer alias model name.
writer_llm: nemotron_ultra_llmwould still pass. Assertfunction["writer_llm"] == "nemotron_ultra_writer_llm"before resolving its model.As per coding guidelines, “Add or update tests for behavior changes.”
Source: Coding guidelines
.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml (1)
20-66: LGTM!Also applies to: 75-86
configs/config_cli_default.yml (1)
19-67: LGTM!Also applies to: 122-152
configs/config_domain_routing_and_skills.yml (1)
50-80: LGTM!Also applies to: 219-219
configs/config_frontier_models.yml (1)
51-73: LGTM!Also applies to: 98-101, 135-135, 154-154
configs/config_mcp.yml (1)
29-77: LGTM!Also applies to: 102-127
mcp/tests/test_config_and_packaging.py (1)
83-89: LGTM!mcp/tests/test_preclassification_integration.py (1)
30-30: LGTM!docs/source/examples/hybrid-frontier-model.md (1)
22-48: LGTM!Also applies to: 79-94
docs/source/examples/full-pipeline-web.md (1)
63-115: LGTM!Also applies to: 160-205
configs/config_web_default_guardrails.yml (2)
81-82: Remove one blank line.Lines 81-82 still create three consecutive blank lines. YAMLlint reports this as an
empty-linesviolation.
31-80: LGTM!Also applies to: 83-88, 224-249
.agents/skills/aiq-customize-prompts-models/references/model-selection.md (1)
18-23: Definenemotron_ultra_writer_llmin the example.The
writer_llmreference does not resolve from the shownllms:block. A copied example fails configuration resolution.Also applies to: 57-61
docs/source/customization/swapping-models.md (2)
27-40: Correct the thinking-example label.The heading says “NIM with thinking.” The example sets
enable_thinking: false. Rename the heading or enable thinking.
13-25: LGTM!Also applies to: 43-43, 58-128
configs/config_openshell.yml (1)
31-86: LGTM!Also applies to: 127-139, 196-196
configs/config_web_azure_ai_search.yml (1)
50-106: LGTM!Also applies to: 194-202, 212-212, 227-227
.agents/skills/aiq-configure-workflow/references/composing-config.md (1)
100-100: LGTM!.agents/skills/aiq-configure-workflow/references/config-profiles.md (1)
20-20: LGTM!README.md (1)
111-120: LGTM!Also applies to: 153-157, 290-300
docs/source/architecture/agents/deep-researcher.md (1)
239-243: LGTM!Also applies to: 264-264
docs/source/customization/configuration-reference.md (1)
95-122: LGTM!Also applies to: 335-335, 368-368, 382-382, 457-461
docs/source/customization/knowledge-layer.md (1)
115-115: LGTM!Also applies to: 129-129, 253-253, 283-291, 310-310
docs/source/customization/mcp-tools.md (1)
117-117: LGTM!configs/config_web_default_llamaindex.yml (1)
50-106: LGTM!Also applies to: 195-202, 211-211, 226-226
configs/config_web_frag.yml (1)
52-100: LGTM!Also applies to: 165-171, 180-180, 194-194
frontends/benchmarks/freshqa/configs/config_full_workflow.yml (1)
2-2: LGTM!Also applies to: 12-60, 76-80, 89-93
frontends/benchmarks/freshqa/configs/config_shallow_research_only.yml (1)
2-2: LGTM!Also applies to: 16-25, 42-42
docs/source/customization/tools-and-sources.md (1)
140-140: LGTM!docs/source/deployment/aws-opensearch-serverless.md (1)
334-338: LGTM!docs/source/evaluation/benchmarks/deepsearch-qa.md (1)
81-81: LGTM!docs/source/get-started/installation.md (1)
28-32: LGTM!docs/source/profiling/index.md (1)
148-154: LGTM!src/aiq_agent/tokenomics/README.md (1)
40-47: LGTM!Also applies to: 62-62
docs/notebooks/0_Getting_Started_with_AIQ.ipynb (3)
404-404: Correct the model-role description.Line 404 assigns intent to Nemotron Ultra, but Line 366 assigns
intent_classifiertonemotron_nano_intent_llm. State that Nano handles intent and shallow research. State the Ultra deep-research roles separately.As per path instructions, review documentation for stale examples and current behavior.
Source: Path instructions
781-787: Complete the provider migration instructions.Line 781 names only
nemotron_nano_intent_llmandnemotron_ultra_llm. Lines 315-350 also definenemotron_nano_agent_llmandnemotron_ultra_writer_llm. Update all applicable blocks or state that the example changes only selected roles.As per path instructions, review documentation for stale examples and current behavior.
Source: Path instructions
73-76: LGTM!Also applies to: 93-93, 302-388, 420-420
docs/notebooks/2_Deep_Researcher_Customization.ipynb (2)
331-331: Align the heading with the planner assignment.Line 305 says Nemotron serves planner and researcher roles. Lines 329-331 assign
planner_llmtofrontier_llmand onlyresearcher_llmtonemotron_ultra_llm. Update the heading or the configuration.As per path instructions, review documentation for stale examples and current behavior.
Source: Path instructions
178-178: LGTM!Also applies to: 203-251, 311-313, 477-477, 502-552
configs/config_web_frag_mcp_auth.yml (1)
59-107: LGTM!Also applies to: 214-220, 229-229, 243-243
configs/config_web_opensearch.yml (1)
51-99: LGTM!Also applies to: 157-157, 172-178, 187-187, 201-201
docs/notebooks/1_Deep_Researcher_Web_Search.ipynb (1)
344-348: LGTM!Also applies to: 434-456, 467-476, 601-665
docs/source/examples/cli-with-local-nims.md (1)
14-14: LGTM!Also applies to: 63-84, 107-114, 123-135, 199-199
frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.yml (1)
21-43: LGTM!Also applies to: 58-62
frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.yml (1)
21-43: LGTM!Also applies to: 58-62
frontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.yml (1)
21-28: LGTM!frontends/benchmarks/deepsearch_qa/README.md (1)
77-77: LGTM!frontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.yml (1)
2-2: LGTM!Also applies to: 17-39, 56-60
sources/knowledge_layer/src/llamaindex/adapter.py (2)
73-73: Update the stale VLM default in the class docstring.
DEFAULT_VLM_MODELnow usesnvidia/nemotron-3-nano-omni-30b-a3b-reasoning, butLlamaIndexIngestorstill documentsnvidia/llama-3.2-90b-vision-instructat Line 459. Update the docstring to match the runtime default.
27-27: LGTM!Also applies to: 450-450, 531-531
deploy/.env.example (1)
188-192: LGTM!sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md (1)
149-149: LGTM!Also applies to: 482-482
sources/knowledge_layer/src/register.py (1)
258-258: 📐 Maintainability & Code QualityRun the required root Python validation before merge.
As per coding guidelines, root Python changes require
uv run ruff check .,uv run ruff format --check ., anduv run pytest. Run them from the repository root and report the results.Source: Coding guidelines
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts (2)
93-93: LGTM!Also applies to: 133-133, 145-145
93-93: 📐 Maintainability & Code QualityRun the required UI validation before merge.
As per path instructions, changes under
frontends/uirequirenpm ci,npm run lint,npm run type-check,npm run test:ci, andnpm run buildfromfrontends/ui. This diff changes only JSDoc examples, so no screenshot applies.Also applies to: 133-133, 145-145
Source: Path instructions
.secrets.baseline (1)
136-136: LGTM!Also applies to: 358-358
tests/knowledge_layer_tests/test_azure_ai_search.py (1)
420-420: LGTM!docs/source/resources/troubleshooting.md (1)
38-38: LGTM!Also applies to: 45-76
docs/source/examples/minimal-shallow-only.md (1)
28-42: LGTM!docs/source/examples/full-pipeline-llamaindex.md (1)
46-102: LGTM!Also applies to: 143-143, 152-152, 162-162, 172-176
b895411 to
657cbb3
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
AjayThorve
changed the title
There was a problem hiding this comment.
Actionable comments posted: 11
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/aiq-customize-prompts-models/references/model-selection.md:
- Around line 21-23: Update the nemotron_ultra_writer_llm profile by adding an
explicit max_tokens setting for the larger writer output budget, while keeping
model_name solely responsible for selecting the model.
In `@configs/config_cli_default.yml`:
- Around line 19-43: Replace the Ultra-backed nemotron_ultra_intent_llm and
nemotron_ultra_agent_llm profiles with Nano 3.5 Preview profiles for intent
classification and shallow research, while preserving Ultra for clarification
and deep research. Apply this in configs/config_cli_default.yml:19-43,
.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml:49-53,
configs/config_frontier_models.yml:53-77, configs/config_mcp.yml:29-53,
configs/config_openshell.yml:31-55,
configs/config_web_azure_ai_search.yml:50-74,
configs/config_web_default_guardrails.yml:31-55,
configs/config_web_default_llamaindex.yml:50-74,
configs/config_web_frag.yml:52-77, and
configs/config_web_frag_mcp_auth.yml:59-83; then rebind each file’s intent and
agent settings at the referenced locations to the Nano profiles.
In `@configs/config_web_opensearch.yml`:
- Around line 51-75: Replace the Ultra intent/agent profiles with approved
Nemotron Nano 3.5 Preview aliases and bind intent_classifier and
shallow_research_agent to Nano while retaining Ultra for clarification and
deep-research roles. Apply the corresponding alias and role-mapping updates in
configs/config_web_opensearch.yml (51-75, 172-187),
frontends/benchmarks/freshqa/configs/config_full_workflow.yml (12-36, 76-80),
and docs/notebooks/0_Getting_Started_with_AIQ.ipynb (302-326, 366-372). Update
the default role-split documentation and regression guidance in
.agents/skills/aiq-configure-workflow/references/config-profiles.md (20),
README.md (111, 152, 289-299), and the notebook’s migration guidance (404,
781-787) to consistently describe Nano for intent/shallow research and Ultra for
remaining default roles.
In `@docs/notebooks/0_Getting_Started_with_AIQ.ipynb`:
- Line 404: Update the llms description to list only roles actually configured
in the shown YAML: remove “clarification” and any reference to an unconfigured
clarifier_agent, unless you intentionally add and enable that configuration.
In `@docs/notebooks/1_Deep_Researcher_Web_Search.ipynb`:
- Around line 342-349: Update the direct ChatNVIDIA initialization to disable
thinking by passing chat_template_kwargs with enable_thinking set to false
through model_kwargs, while preserving the existing model and generation
settings.
In `@docs/notebooks/2_Deep_Researcher_Customization.ipynb`:
- Around line 241-251: Update the notebook’s introductory model overview and
role table to list all five configured LLM roles: orchestrator, source router,
planner, researcher, and writer. Ensure the planner description references
frontier_llm, while source_router_llm and writer_llm are represented
consistently with the YAML configuration.
In `@docs/source/customization/configuration-reference.md`:
- Line 335: Validate generated embedding vector lengths before OpenSearch bulk
indexing and search requests, rejecting mismatches with a clear error and adding
regression tests for ingestion and query cases. The configuration-reference.md
sites at lines 335 and 368 and aws-opensearch-serverless.md lines 334-338
require no direct documentation change; use their documented 2048-dimension
defaults when implementing validation.
In `@docs/source/customization/knowledge-layer.md`:
- Line 253: Update the canonical knowledge-layer documentation around the
AIQ_EMBED_MODEL configuration to document embedding-model migration: explain
that existing Chroma and OpenSearch stores must be deleted and re-ingested, and
include the OpenSearch embedding-dimension requirement. Match the procedure and
details documented in KNOWLEDGE-LAYER-SETUP.md.
In `@frontends/benchmarks/freshqa/configs/config_shallow_research_only.yml`:
- Around line 16-27: Replace the Nemotron Ultra profile with Nemotron Nano 3.5
Preview for intent classification and shallow research. Update the model
definition and role binding in
frontends/benchmarks/freshqa/configs/config_shallow_research_only.yml (lines
16-27 and 42), then revise the corresponding model references, default-profile
note, and examples in docs/source/customization/configuration-reference.md
(lines 131-132, 591-615, and 660-675),
docs/source/customization/swapping-models.md (lines 78-80), and
docs/source/customization/tools-and-sources.md (line 140) so none select or
document Nemotron Ultra for these roles.
In `@sources/knowledge_layer/src/opensearch/adapter.py`:
- Line 521: Update the index validation flow containing
_validate_index_embedding to also verify the physical index owner after
concurrent creation: read _meta["collection_name"], compare it with
collection_name, and reject mismatches before ingestion. Add a regression test
covering two logical collections racing to create the same index, ensuring the
losing worker rejects the winner’s collection.
In `@tests/aiq_agent/test_default_model_profiles.py`:
- Around line 72-75: Update tests/aiq_agent/test_default_model_profiles.py lines
72-75 so both intent_classifier and shallow_research_agent expectations use the
canonical Nemotron Nano 3.5 Preview model and alias instead of ULTRA_MODEL,
while preserving Ultra expectations for clarification and default deep-research
roles. Update mcp/tests/test_preclassification_integration.py line 30 so
_INTENT_LLM_NAME uses the alias resolving to the Nano 3.5 Preview intent model.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 65b73d6c-44b0-4084-8b14-dccc8b31e19e
📒 Files selected for processing (59)
.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-configure-workflow/references/config-profiles.md.agents/skills/aiq-customize-prompts-models/references/model-selection.md.secrets.baselineREADME.mdconfigs/config_cli_default.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_openshell.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_opensearch.ymldeploy/.env.exampledocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/source/architecture/agents/deep-researcher.mddocs/source/customization/configuration-reference.mddocs/source/customization/knowledge-layer.mddocs/source/customization/mcp-tools.mddocs/source/customization/swapping-models.mddocs/source/customization/tools-and-sources.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/examples/cli-with-local-nims.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/examples/minimal-shallow-only.mddocs/source/extending/adding-a-tool.mddocs/source/get-started/installation.mddocs/source/profiling/index.mddocs/source/resources/troubleshooting.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.ymlfrontends/benchmarks/deepsearch_qa/README.mdfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymlfrontends/ui/src/features/chat/lib/intermediate-step-parser.tsmcp/tests/test_config_and_packaging.pymcp/tests/test_preclassification_integration.pysources/google_scholar_paper_search/README.mdsources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.mdsources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/llamaindex/adapter.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/register.pysrc/aiq_agent/tokenomics/README.mdtests/aiq_agent/test_default_model_profiles.pytests/knowledge_layer_tests/test_azure_ai_search.pytests/knowledge_layer_tests/test_llamaindex_embedding_compatibility.pytests/knowledge_layer_tests/test_opensearch_adapter.py
📜 Review details
⏰ Context from checks skipped due to timeout. (5)
- GitHub Check: UI Unit Tests
- GitHub Check: Run Harbor skill eval
- GitHub Check: Script Validation
- GitHub Check: Lint and Hooks
- GitHub Check: Pytest and Coverage
🧰 Additional context used
📓 Path-based instructions (17)
.agents/skills/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Use repository maintainer skills from
.agents/skills/for applicable workflows; keep API-consumer skills inskills/separate because they are not deployed application runtime skills.
Files:
.agents/skills/aiq-configure-workflow/references/config-profiles.md.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-customize-prompts-models/references/model-selection.md
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}
⚙️ CodeRabbit configuration file
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.
Files:
.agents/skills/aiq-configure-workflow/references/config-profiles.md.agents/skills/aiq-configure-workflow/references/composing-config.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-customize-prompts-models/references/model-selection.md
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format and lint Python with Ruff using a 120-character line length, Python 3.11 target, rules E/F/W/I/PL/UP, and single-line imports; avoid reformatting unrelated code.
Never print or log secret values, including through tool output or error messages.
**/*.py: Runuv run ruff check .anduv run ruff format --check .for root Python changes.
Runuv run pytestfor root project Python changes.
Files:
mcp/tests/test_config_and_packaging.pysources/knowledge_layer/src/azure_ai_search/adapter.pytests/knowledge_layer_tests/test_azure_ai_search.pysources/knowledge_layer/src/register.pymcp/tests/test_preclassification_integration.pytests/knowledge_layer_tests/test_llamaindex_embedding_compatibility.pysources/knowledge_layer/src/opensearch/adapter.pytests/aiq_agent/test_default_model_profiles.pytests/knowledge_layer_tests/test_opensearch_adapter.pysources/knowledge_layer/src/llamaindex/adapter.py
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes inside this repository, avoid editing adjacent repositories, and scope changes to the smallest relevant independent package, especially undersources/.
Run the narrowest relevant validation command first and broaden to the full suite only when a change crosses shared boundaries.
Keep pull requests scoped, exclude unrelated files and generated artifacts, never include secrets, and provide validation commands and results.
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
Files:
mcp/tests/test_config_and_packaging.pydocs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/customization/mcp-tools.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/customization/tools-and-sources.mddocs/source/extending/adding-a-tool.mddocs/source/get-started/installation.mdsources/knowledge_layer/src/azure_ai_search/adapter.pyfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlconfigs/config_domain_routing_and_skills.ymldocs/source/examples/full-pipeline-web.mdtests/knowledge_layer_tests/test_azure_ai_search.pysrc/aiq_agent/tokenomics/README.mdfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlsources/knowledge_layer/src/register.pydocs/source/profiling/index.mdfrontends/benchmarks/deepsearch_qa/README.mddocs/notebooks/1_Deep_Researcher_Web_Search.ipynbmcp/tests/test_preclassification_integration.pydocs/source/customization/knowledge-layer.mdsources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.mdsources/google_scholar_paper_search/README.mdtests/knowledge_layer_tests/test_llamaindex_embedding_compatibility.pyconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymlfrontends/ui/src/features/chat/lib/intermediate-step-parser.tsdocs/source/architecture/agents/deep-researcher.mdconfigs/config_web_frag.ymldocs/source/examples/minimal-shallow-only.mddocs/source/examples/cli-with-local-nims.mddocs/source/resources/troubleshooting.mddocs/notebooks/0_Getting_Started_with_AIQ.ipynbfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymldocs/notebooks/2_Deep_Researcher_Customization.ipynbconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_default_llamaindex.ymldocs/source/customization/swapping-models.mdsources/knowledge_layer/src/opensearch/adapter.pydocs/source/customization/configuration-reference.mddocs/source/examples/full-pipeline-llamaindex.mdconfigs/config_web_azure_ai_search.ymldocs/source/examples/hybrid-frontier-model.mdtests/aiq_agent/test_default_model_profiles.pyfrontends/benchmarks/freshqa/configs/config_full_workflow.ymltests/knowledge_layer_tests/test_opensearch_adapter.pyconfigs/config_web_opensearch.ymlsources/knowledge_layer/src/llamaindex/adapter.pyconfigs/config_frontier_models.ymlconfigs/config_openshell.ymlREADME.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlconfigs/config_mcp.ymlfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.yml
mcp/**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For MCP changes, run the MCP development dependency setup and
uv run --project mcp --extra dev pytest mcp/tests.
Files:
mcp/tests/test_config_and_packaging.pymcp/tests/test_preclassification_integration.py
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages into skills.
Files:
docs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/customization/mcp-tools.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/customization/tools-and-sources.mddocs/source/extending/adding-a-tool.mddocs/source/get-started/installation.mddocs/source/examples/full-pipeline-web.mddocs/source/profiling/index.mddocs/source/customization/knowledge-layer.mddocs/source/architecture/agents/deep-researcher.mddocs/source/examples/minimal-shallow-only.mddocs/source/examples/cli-with-local-nims.mddocs/source/resources/troubleshooting.mddocs/source/customization/swapping-models.mddocs/source/customization/configuration-reference.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/hybrid-frontier-model.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/evaluation/benchmarks/deepsearch-qa.mddocs/source/customization/mcp-tools.mddocs/source/deployment/aws-opensearch-serverless.mddocs/source/customization/tools-and-sources.mddocs/source/extending/adding-a-tool.mddocs/source/get-started/installation.mddocs/source/examples/full-pipeline-web.mddocs/source/profiling/index.mddocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/source/customization/knowledge-layer.mddocs/source/architecture/agents/deep-researcher.mddocs/source/examples/minimal-shallow-only.mddocs/source/examples/cli-with-local-nims.mddocs/source/resources/troubleshooting.mddocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/source/customization/swapping-models.mddocs/source/customization/configuration-reference.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/hybrid-frontier-model.mdREADME.md
sources/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Implement new tools and data sources as NeMo Agent Toolkit functions registered with
@register_function; configuration schemas must inherit fromFunctionBaseConfig.
Files:
sources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/register.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/llamaindex/adapter.py
{src/aiq_agent/knowledge/**,sources/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/knowledge/**,sources/**}: Review data-source and knowledge-layer changes for optional dependency boundaries, external API error handling,
retry/rate-limit behavior, deterministic tests, and registration consistency. New source packages should include
package metadata, plugin registration when applicable, and source-level tests.
Files:
sources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/register.pysources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.mdsources/google_scholar_paper_search/README.mdsources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/llamaindex/adapter.py
**/*.{yaml,yml}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For deployment changes, run the relevant Helm or Compose validation and describe the environment used.
Files:
frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlconfigs/config_domain_routing_and_skills.ymlfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymlconfigs/config_web_frag.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_azure_ai_search.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlconfigs/config_web_opensearch.ymlconfigs/config_frontier_models.ymlconfigs/config_openshell.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlconfigs/config_mcp.ymlfrontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.yml
configs/**/*.y{a,}ml
📄 CodeRabbit inference engine (AGENTS.md)
Use
_typevalues derived from the registered configuration class names in workflow YAML configuration.
Files:
configs/config_domain_routing_and_skills.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymlconfigs/config_web_frag.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_opensearch.ymlconfigs/config_frontier_models.ymlconfigs/config_openshell.ymlconfigs/config_mcp.yml
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_domain_routing_and_skills.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymlconfigs/config_web_frag.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_opensearch.ymlconfigs/config_frontier_models.ymlconfigs/config_openshell.ymlconfigs/config_mcp.yml
src/aiq_agent/tokenomics/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/tokenomics/**/*: Review tokenomics changes for deterministic accounting, pricing-data assumptions, rounding behavior, and report
compatibility. Flag pricing or aggregation changes without representative tests and clear documentation updates.
Files:
src/aiq_agent/tokenomics/README.md
frontends/ui/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
frontends/ui/**/*.{ts,tsx}: Use the existing Next.js, React, TypeScript, Tailwind, and KUI patterns; reuse KUI components instead of introducing new equivalent UI components.
Access the backend through the proxy andBACKEND_URL, and preserve authentication-aware UI states.
Validate UI-affecting changes withnpm run lint,npm run type-check, andnpm run test:ci; include a screenshot for visible changes.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
frontends/ui/**/*.{js,jsx,ts,tsx,css,scss}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For UI changes, run
npm ci,npm run lint,npm run type-check,npm run test:ci, andnpm run buildfromfrontends/ui.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
frontends/ui/**/*
⚙️ CodeRabbit configuration file
frontends/ui/**/*: Review UI changes for strict TypeScript behavior, API contract alignment, auth/session handling, accessible controls,
resilient loading and error states, and report/chat state consistency. Prefer existing UI patterns and require tests
for changed user-visible workflows.
Files:
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts
**/{.env,.env.*}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, resolving API keys at runtime.
Files:
deploy/.env.example
🧠 Learnings (1)
📚 Learning: 2026-08-04T22:10:29.858Z
Learnt from: AjayThorve
Repo: NVIDIA-AI-Blueprints/aiq PR: 414
File: sources/knowledge_layer/src/register.py:257-259
Timestamp: 2026-08-04T22:10:29.858Z
Learning: Within the knowledge layer, embedding-model changes must preserve vector-store compatibility. Chroma must persist and validate an embedding-model marker; OpenSearch must validate both embedding-model identity and embedding dimensions before ingestion and retrieval; incompatible or unmarked stores must fail closed and provide re-ingestion guidance. Azure AI Search must use model-scoped index names and validate the associated model marker.
Applied to files:
sources/knowledge_layer/src/azure_ai_search/adapter.pysources/knowledge_layer/src/register.pysources/knowledge_layer/src/opensearch/adapter.pysources/knowledge_layer/src/llamaindex/adapter.py
🪛 LanguageTool
docs/source/customization/knowledge-layer.md
[uncategorized] ~287-~287: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...| false | Extract tables from PDFs as markdown | | AIQ_EXTRACT_IMAGES | false | Ex...
(MARKDOWN_NNP)
sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md
[uncategorized] ~145-~145: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...| false | Extract tables from PDFs as markdown using pdfplumber | | `AIQ_EXTRACT_IMAGE...
(MARKDOWN_NNP)
docs/source/customization/swapping-models.md
[style] ~27-~27: Consider a different adjective to strengthen your wording.
Context: ...le: NIM with thinking (for example, for deep research)** ```yaml llms: nemotron_u...
(DEEP_PROFOUND)
[style] ~82-~82: Consider a different adjective to strengthen your wording.
Context: ... and a local downloadable Ultra NIM for deep research: ```yaml llms: hosted_shall...
(DEEP_PROFOUND)
docs/source/customization/configuration-reference.md
[style] ~116-~116: To form a complete sentence, be sure to include a subject.
Context: ...l|str|None` | API endpoint URL. Should always be set explicitly for NVIDIA NIM...
(MISSING_IT_THERE)
docs/source/examples/hybrid-frontier-model.md
[style] ~9-~9: Consider a different adjective to strengthen your wording.
Context: ...llow research, and a frontier model for deep research (higher quality reports). ## ...
(DEEP_PROFOUND)
README.md
[style] ~294-~294: Consider a different adjective to strengthen your wording.
Context: ...amaIndex profile using GPT Sol/Luna for deep research and Ultra for intent/shallow. ...
(DEEP_PROFOUND)
🔇 Additional comments (54)
tests/aiq_agent/test_default_model_profiles.py (2)
1-71: LGTM!Also applies to: 90-102
76-87: 📐 Maintainability & Code QualityKeep the existing frontier topology coverage.
test_frontier_profile_uses_validated_gpt_role_splitasserts all five deep-research role aliases forconfig_frontier_models.yml.> Likely an incorrect or invalid review comment.mcp/tests/test_config_and_packaging.py (1)
85-85: LGTM!.secrets.baseline (1)
136-136: LGTM!Also applies to: 309-309, 358-358
deploy/.env.example (1)
188-192: LGTM!sources/knowledge_layer/KNOWLEDGE-LAYER-SETUP.md (1)
142-149: LGTM!Also applies to: 220-220, 272-289, 500-500
sources/knowledge_layer/src/azure_ai_search/adapter.py (1)
92-92: LGTM!sources/knowledge_layer/src/llamaindex/adapter.py (1)
27-27: LGTM!Also applies to: 71-93, 466-466, 475-475, 501-504, 547-547, 794-800, 1329-1334, 1628-1628, 1716-1716
sources/knowledge_layer/src/register.py (1)
258-258: LGTM!docs/source/customization/knowledge-layer.md (1)
115-115: LGTM!Also applies to: 129-129, 310-310
tests/knowledge_layer_tests/test_azure_ai_search.py (1)
420-420: LGTM!tests/knowledge_layer_tests/test_llamaindex_embedding_compatibility.py (1)
13-24: LGTM!tests/knowledge_layer_tests/test_opensearch_adapter.py (1)
481-557: LGTM!docs/source/customization/mcp-tools.md (1)
117-117: 🎯 Functional CorrectnessKeep
nemotron_ultra_agent_llmfor shallow research.The default profiles and documentation assign Nemotron 3 Ultra to this role. Nano references are for the optional VLM, not a shallow-research LLM alias.
> Likely an incorrect or invalid review comment..agents/skills/aiq-configure-workflow/assets/config-scaffold.yml (1)
20-40: LGTM!Also applies to: 55-61
configs/config_cli_default.yml (1)
45-67: LGTM!Also applies to: 127-129, 152-152
configs/config_domain_routing_and_skills.yml (1)
50-80: LGTM!Also applies to: 219-219
configs/config_frontier_models.yml (1)
106-109: LGTM!configs/config_mcp.yml (1)
55-77: LGTM!Also applies to: 104-110, 127-127
configs/config_openshell.yml (1)
57-86: LGTM!Also applies to: 130-135, 196-196
configs/config_web_azure_ai_search.yml (1)
76-106: LGTM!Also applies to: 199-202, 227-227
configs/config_web_default_guardrails.yml (1)
57-87: LGTM!Also applies to: 226-231, 244-248
configs/config_web_default_llamaindex.yml (1)
76-106: LGTM!Also applies to: 200-207, 226-226
configs/config_web_frag.yml (1)
78-100: LGTM!Also applies to: 169-176, 194-194
configs/config_web_frag_mcp_auth.yml (1)
85-107: LGTM!Also applies to: 218-225, 243-243
configs/config_web_opensearch.yml (1)
77-99: LGTM!Also applies to: 157-157, 176-178, 201-201
frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.yml (1)
21-43: LGTM!Also applies to: 58-62
frontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.yml (1)
21-43: LGTM!Also applies to: 58-62
frontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.yml (1)
2-2: LGTM!Also applies to: 17-39, 56-60
frontends/benchmarks/freshqa/configs/config_full_workflow.yml (1)
2-2: LGTM!Also applies to: 38-60, 89-93
.agents/skills/aiq-configure-workflow/references/composing-config.md (1)
100-100: LGTM!.agents/skills/aiq-customize-prompts-models/references/model-selection.md (1)
57-61: LGTM!README.md (1)
112-120: LGTM!Also applies to: 153-156
docs/notebooks/0_Getting_Started_with_AIQ.ipynb (1)
73-76: LGTM!Also applies to: 93-93, 328-350, 377-388, 420-420
docs/notebooks/1_Deep_Researcher_Web_Search.ipynb (1)
434-476: LGTM!Also applies to: 603-665
frontends/benchmarks/freshqa/configs/config_shallow_research_only.yml (1)
2-2: LGTM!docs/notebooks/2_Deep_Researcher_Customization.ipynb (1)
178-178: LGTM!Also applies to: 203-225, 305-313, 331-331, 477-477, 502-524, 542-552, 565-565
docs/source/architecture/agents/deep-researcher.md (1)
239-243: LGTM!Also applies to: 264-264
docs/source/customization/configuration-reference.md (2)
382-382: 🗄️ Data Integrity & IntegrationKeep the intent-classifier LLM reference resolvable.
The standalone example uses
nemotron_ultra_intent_llmat Line [382]. The complete profile definesultra_intent_llmat Line [591] and wires the function to that key at Line [660]. If users combine the examples, the standalone reference does not resolve. Use one canonical key or define the alias, then validate the rendered YAML.Possible alignment
- llm: nemotron_ultra_intent_llm + llm: ultra_intent_llmAlso applies to: 591-591, 660-660
Source: Path instructions
95-122: LGTM!Also applies to: 133-135, 457-461, 617-639, 676-687
docs/source/customization/swapping-models.md (1)
13-43: LGTM!Also applies to: 58-76, 82-112, 125-125
frontends/ui/src/features/chat/lib/intermediate-step-parser.ts (1)
93-93: LGTM!Also applies to: 133-133, 145-145
frontends/benchmarks/deepsearch_qa/README.md (1)
77-77: 🎯 Functional CorrectnessNo change needed. The YAML uses
nvidia/nemotron-3-ultra-550b-a55bwith the NVIDIA API Catalog endpoint and${NVIDIA_API_KEY}.> Likely an incorrect or invalid review comment.frontends/benchmarks/deepresearch_bench/configs/config_tokenomics_pricing.yml (1)
21-25: 🗄️ Data Integrity & IntegrationNo change required.
The parser intentionally uses the input rate when cached pricing is omitted. Existing tests cover cached accounting and exact, substring, and default lookup paths.
docs/source/get-started/installation.md (1)
28-32: 🗄️ Data Integrity & IntegrationVerify the canonical intent and shallow-research model mapping.
The changed documentation uses Nemotron Ultra for intent classification or shallow research. The PR objective states that these roles use Nemotron Nano 3.5 Preview. The supplied stack context describes Ultra role profiles, so confirm the source of truth against the default configurations and regression tests.
docs/source/get-started/installation.md#L28-L32: Update the default hardware table if Nano is canonical.docs/source/resources/troubleshooting.md#L45-L76: Update the default availability and self-hosting guidance if Nano is canonical.docs/source/examples/hybrid-frontier-model.md#L21-L45: Use the canonical intent and shallow-research model profiles.docs/source/examples/hybrid-frontier-model.md#L76-L91: Bind the intent and shallow-research agents to the canonical profiles.docs/source/examples/minimal-shallow-only.md#L28-L42: Use the canonical shallow-research model.docs/source/extending/adding-a-tool.md#L282-L282: Keep the copied shallow-research example on the canonical model tier.docs/source/evaluation/benchmarks/deepsearch-qa.md (1)
81-81: LGTM!docs/source/examples/cli-with-local-nims.md (1)
14-30: LGTM!Also applies to: 65-86, 109-116, 125-137, 155-156, 201-201
docs/source/examples/full-pipeline-llamaindex.md (1)
46-94: LGTM!Also applies to: 96-102, 143-143, 152-152, 162-162, 172-176
docs/source/examples/full-pipeline-web.md (1)
63-114: LGTM!Also applies to: 159-159, 172-172, 186-186, 200-204
docs/source/examples/hybrid-frontier-model.md (1)
8-9: LGTM!docs/source/profiling/index.md (1)
148-151: LGTM!docs/source/resources/troubleshooting.md (1)
38-40: LGTM!sources/google_scholar_paper_search/README.md (2)
117-129: 🎯 Functional CorrectnessVerify the hosted NIM API-key wiring.
These profiles target
https://integrate.api.nvidia.com/v1, but neither profile setsapi_key. Other hosted NIM examples setapi_key: ${NVIDIA_API_KEY}explicitly.Confirm that NAT’s
_type: nimprovider loadsNVIDIA_API_KEYimplicitly. If it does not, add the explicit API-key field to both profiles so a copied configuration can authenticate.
144-148: LGTM!
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/customization/knowledge-layer.md`:
- Around line 270-284: Update the Chroma migration guidance under “Changing the
embedding model” to recommend deleting only the affected collection through the
Knowledge API or UI. If retaining the AIQ_CHROMA_DIR deletion option, explicitly
warn that it removes all named collections and may delete unrelated data.
In `@sources/knowledge_layer/src/opensearch/adapter.py`:
- Line 1591: Update the retrieval flow around _validate_embedding_vectors to
preserve the metadata returned by _validate_index_embedding and call
_validate_index_owner before validating embeddings or performing search. Ensure
an owner mismatch stops retrieval immediately, and add a regression test
verifying neither embedding nor search executes when ownership validation fails.
In `@tests/knowledge_layer_tests/test_opensearch_adapter.py`:
- Around line 575-587: Update
test_retrieval_rejects_mismatched_query_dimension_before_search to track calls
to fake_client.search and assert it was not called after retrieval returns the
dimension error. Configure the fake client’s search behavior or call recording
so any request is detectable, preserving the fail-before-request contract.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: d46a531c-53c5-4bef-b1a1-3b04a0526b18
📒 Files selected for processing (7)
.agents/skills/aiq-customize-prompts-models/references/model-selection.mddocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/source/customization/knowledge-layer.mdsources/knowledge_layer/src/opensearch/adapter.pytests/knowledge_layer_tests/test_opensearch_adapter.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (8)
.agents/skills/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Use repository maintainer skills from
.agents/skills/for applicable workflows; keep API-consumer skills inskills/separate because they are not deployed application runtime skills.
Files:
.agents/skills/aiq-customize-prompts-models/references/model-selection.md
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}
⚙️ CodeRabbit configuration file
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.
Files:
.agents/skills/aiq-customize-prompts-models/references/model-selection.md
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages into skills.
Files:
docs/source/customization/knowledge-layer.md
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes inside this repository, avoid editing adjacent repositories, and scope changes to the smallest relevant independent package, especially undersources/.
Run the narrowest relevant validation command first and broaden to the full suite only when a change crosses shared boundaries.
Keep pull requests scoped, exclude unrelated files and generated artifacts, never include secrets, and provide validation commands and results.
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
Files:
docs/source/customization/knowledge-layer.mddocs/notebooks/1_Deep_Researcher_Web_Search.ipynbtests/knowledge_layer_tests/test_opensearch_adapter.pydocs/notebooks/0_Getting_Started_with_AIQ.ipynbsources/knowledge_layer/src/opensearch/adapter.pydocs/notebooks/2_Deep_Researcher_Customization.ipynb
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/customization/knowledge-layer.mddocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynb
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format and lint Python with Ruff using a 120-character line length, Python 3.11 target, rules E/F/W/I/PL/UP, and single-line imports; avoid reformatting unrelated code.
Never print or log secret values, including through tool output or error messages.
**/*.py: Runuv run ruff check .anduv run ruff format --check .for root Python changes.
Runuv run pytestfor root project Python changes.
Files:
tests/knowledge_layer_tests/test_opensearch_adapter.pysources/knowledge_layer/src/opensearch/adapter.py
sources/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
Implement new tools and data sources as NeMo Agent Toolkit functions registered with
@register_function; configuration schemas must inherit fromFunctionBaseConfig.
Files:
sources/knowledge_layer/src/opensearch/adapter.py
{src/aiq_agent/knowledge/**,sources/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/knowledge/**,sources/**}: Review data-source and knowledge-layer changes for optional dependency boundaries, external API error handling,
retry/rate-limit behavior, deterministic tests, and registration consistency. New source packages should include
package metadata, plugin registration when applicable, and source-level tests.
Files:
sources/knowledge_layer/src/opensearch/adapter.py
🧠 Learnings (1)
📚 Learning: 2026-08-04T22:10:29.858Z
Learnt from: AjayThorve
Repo: NVIDIA-AI-Blueprints/aiq PR: 414
File: sources/knowledge_layer/src/register.py:257-259
Timestamp: 2026-08-04T22:10:29.858Z
Learning: Within the knowledge layer, embedding-model changes must preserve vector-store compatibility. Chroma must persist and validate an embedding-model marker; OpenSearch must validate both embedding-model identity and embedding dimensions before ingestion and retrieval; incompatible or unmarked stores must fail closed and provide re-ingestion guidance. Azure AI Search must use model-scoped index names and validate the associated model marker.
Applied to files:
sources/knowledge_layer/src/opensearch/adapter.py
🔇 Additional comments (7)
docs/notebooks/2_Deep_Researcher_Customization.ipynb (1)
14-14: LGTM!Also applies to: 178-178, 203-225, 241-251, 286-298, 307-315, 333-333, 479-479, 504-526, 544-554, 567-567
docs/notebooks/0_Getting_Started_with_AIQ.ipynb (1)
73-76: LGTM!Also applies to: 93-93, 302-351, 366-388, 404-406, 420-420, 781-787
docs/notebooks/1_Deep_Researcher_Web_Search.ipynb (1)
344-349: LGTM!Also applies to: 435-457, 468-477, 602-666
docs/source/customization/knowledge-layer.md (1)
115-115: LGTM!Also applies to: 129-129, 253-253, 298-306, 325-325
.agents/skills/aiq-customize-prompts-models/references/model-selection.md (1)
18-25: LGTM!Also applies to: 59-63
sources/knowledge_layer/src/opensearch/adapter.py (1)
76-76: LGTM!Also applies to: 500-537, 1414-1417
tests/knowledge_layer_tests/test_opensearch_adapter.py (1)
339-375: LGTM!Also applies to: 1311-1344
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
5007644 to
e8a81c1
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
docs/source/examples/hybrid-frontier-model.md (1)
36-44: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winCorrect the
nemotron_ultra_llmcomment.
nemotron_ultra_llmis a NIM profile. The configuration uses it forclarifier_agenton Line 80.frontier_llmremains the deep-research model on Line 97.Proposed fix
- # Frontier model for deep research (higher quality) + # NIM profile for clarification nemotron_ultra_llm:As per path instructions, review documentation for “stale examples.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/examples/hybrid-frontier-model.md` around lines 36 - 44, Correct the comment above the nemotron_ultra_llm NIM profile to describe its use by clarifier_agent, while keeping frontier_llm identified as the deep-research model. Review the surrounding documentation for any stale example wording and update only references affected by this correction.Source: Path instructions
docs/source/resources/troubleshooting.md (2)
53-60: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winCover the Super endpoint in the self-hosting guidance.
Line 53 says either hosted endpoint can be saturated. The only self-hosting link on Lines 59-60 covers Ultra. Default intent and shallow-research traffic still uses Super. Add a Super self-hosting path, or narrow the statement to Ultra and deep-research roles.
As per path instructions, review documentation for command accuracy and stale examples.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/resources/troubleshooting.md` around lines 53 - 60, Update the “Recommended Mitigation: Self-Host the Affected Model” section to cover both hosted endpoints: add a verified self-hosting link for Nemotron 3 Super alongside the existing Ultra link, and ensure any model names, URLs, and commands match current NVIDIA documentation. Keep the guidance consistent with the default Super intent/shallow-research roles and Ultra clarification/deep-research roles.Source: Path instructions
70-70: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winHandle keyless local NIMs in validation.
api_key: ""is treated as unset, so the validator still requiresNVIDIA_API_KEY; omitting the field has the same result. Update the validator or correct this example and the local-NIM guidance.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/resources/troubleshooting.md` at line 70, Update the troubleshooting documentation example and its local-NIM guidance around the api_key configuration to accurately reflect validator behavior: an empty or omitted api_key still requires NVIDIA_API_KEY. Either document the required environment variable for keyless local NIMs or revise the validator so this configuration is accepted, keeping the example and validation behavior consistent.Source: Path instructions
docs/notebooks/0_Getting_Started_with_AIQ.ipynb (1)
416-416: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUpdate the frontier-model description. The configuration uses
gpt-5.6-solfor orchestration, planning, and writing, andgpt-5.6-lunafor source routing and research. Replace “for orchestration and planning” with these roles and updateGPT-5.2to the configured GPT-5.6 models.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/notebooks/0_Getting_Started_with_AIQ.ipynb` at line 416, Update the frontier model description in the notebook to state that gpt-5.6-sol handles orchestration, planning, and writing, while gpt-5.6-luna handles source routing and research; replace the GPT-5.2 example with the configured GPT-5.6 model names.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/examples/full-pipeline-llamaindex.md`:
- Line 68: Update the comment above nemotron_ultra_llm to describe its
clarification and deep-research role, and avoid labeling it as the
document-summary model; keep the summary_llm documentation as the
summary-profile reference.
In `@tests/aiq_agent/test_default_model_profiles.py`:
- Around line 96-105: Update SCANNED_SUFFIXES in the repository-reference test
to include the .baseline suffix, ensuring files such as .secrets.baseline are
scanned while preserving the existing filtering and violation reporting
behavior.
---
Outside diff comments:
In `@docs/notebooks/0_Getting_Started_with_AIQ.ipynb`:
- Line 416: Update the frontier model description in the notebook to state that
gpt-5.6-sol handles orchestration, planning, and writing, while gpt-5.6-luna
handles source routing and research; replace the GPT-5.2 example with the
configured GPT-5.6 model names.
In `@docs/source/examples/hybrid-frontier-model.md`:
- Around line 36-44: Correct the comment above the nemotron_ultra_llm NIM
profile to describe its use by clarifier_agent, while keeping frontier_llm
identified as the deep-research model. Review the surrounding documentation for
any stale example wording and update only references affected by this
correction.
In `@docs/source/resources/troubleshooting.md`:
- Around line 53-60: Update the “Recommended Mitigation: Self-Host the Affected
Model” section to cover both hosted endpoints: add a verified self-hosting link
for Nemotron 3 Super alongside the existing Ultra link, and ensure any model
names, URLs, and commands match current NVIDIA documentation. Keep the guidance
consistent with the default Super intent/shallow-research roles and Ultra
clarification/deep-research roles.
- Line 70: Update the troubleshooting documentation example and its local-NIM
guidance around the api_key configuration to accurately reflect validator
behavior: an empty or omitted api_key still requires NVIDIA_API_KEY. Either
document the required environment variable for keyless local NIMs or revise the
validator so this configuration is accepted, keeping the example and validation
behavior consistent.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: a466913a-550c-43b5-9eca-ca966fd90c82
📒 Files selected for processing (26)
.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml.agents/skills/aiq-configure-workflow/references/config-profiles.md.secrets.baselineREADME.mdconfigs/config_cli_default.ymlconfigs/config_frontier_models.ymlconfigs/config_mcp.ymlconfigs/config_openshell.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_opensearch.ymldocs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/source/customization/configuration-reference.mddocs/source/customization/swapping-models.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/examples/minimal-shallow-only.mddocs/source/get-started/installation.mddocs/source/resources/troubleshooting.mdfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymltests/aiq_agent/test_default_model_profiles.py
📜 Review details
⏰ Context from checks skipped due to timeout. (4)
- GitHub Check: Run Harbor skill eval
- GitHub Check: Lint and Hooks
- GitHub Check: Script Validation
- GitHub Check: Pytest and Coverage
🧰 Additional context used
📓 Path-based instructions (9)
configs/**/*.y{a,}ml
📄 CodeRabbit inference engine (AGENTS.md)
Use
_typevalues derived from the registered configuration class names in workflow YAML configuration.
Files:
configs/config_frontier_models.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymlconfigs/config_mcp.ymlconfigs/config_openshell.ymlconfigs/config_web_opensearch.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_frag.yml
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes inside this repository, avoid editing adjacent repositories, and scope changes to the smallest relevant independent package, especially undersources/.
Run the narrowest relevant validation command first and broaden to the full suite only when a change crosses shared boundaries.
Keep pull requests scoped, exclude unrelated files and generated artifacts, never include secrets, and provide validation commands and results.
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
Files:
configs/config_frontier_models.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymldocs/source/customization/swapping-models.mddocs/source/get-started/installation.mdconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymldocs/source/examples/minimal-shallow-only.mddocs/source/examples/full-pipeline-llamaindex.mdconfigs/config_mcp.ymldocs/source/resources/troubleshooting.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mdfrontends/benchmarks/freshqa/configs/config_full_workflow.ymldocs/source/customization/configuration-reference.mddocs/notebooks/0_Getting_Started_with_AIQ.ipynbconfigs/config_openshell.ymlconfigs/config_web_opensearch.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_llamaindex.ymltests/aiq_agent/test_default_model_profiles.pyconfigs/config_web_frag_mcp_auth.ymlREADME.mdconfigs/config_web_frag.yml
**/*.{yaml,yml}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For deployment changes, run the relevant Helm or Compose validation and describe the environment used.
Files:
configs/config_frontier_models.ymlfrontends/benchmarks/freshqa/configs/config_shallow_research_only.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymlconfigs/config_mcp.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlconfigs/config_openshell.ymlconfigs/config_web_opensearch.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_frag.yml
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_frontier_models.ymlconfigs/config_web_default_guardrails.ymlconfigs/config_cli_default.ymlconfigs/config_mcp.ymlconfigs/config_openshell.ymlconfigs/config_web_opensearch.ymlconfigs/config_web_azure_ai_search.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag_mcp_auth.ymlconfigs/config_web_frag.yml
.agents/skills/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Use repository maintainer skills from
.agents/skills/for applicable workflows; keep API-consumer skills inskills/separate because they are not deployed application runtime skills.
Files:
.agents/skills/aiq-configure-workflow/references/config-profiles.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}
⚙️ CodeRabbit configuration file
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.
Files:
.agents/skills/aiq-configure-workflow/references/config-profiles.md.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages into skills.
Files:
docs/source/customization/swapping-models.mddocs/source/get-started/installation.mddocs/source/examples/minimal-shallow-only.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/resources/troubleshooting.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/customization/configuration-reference.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/customization/swapping-models.mddocs/source/get-started/installation.mddocs/source/examples/minimal-shallow-only.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/resources/troubleshooting.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mddocs/source/customization/configuration-reference.mddocs/notebooks/0_Getting_Started_with_AIQ.ipynbREADME.md
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format and lint Python with Ruff using a 120-character line length, Python 3.11 target, rules E/F/W/I/PL/UP, and single-line imports; avoid reformatting unrelated code.
Never print or log secret values, including through tool output or error messages.
**/*.py: Runuv run ruff check .anduv run ruff format --check .for root Python changes.
Runuv run pytestfor root project Python changes.
Files:
tests/aiq_agent/test_default_model_profiles.py
🪛 LanguageTool
.agents/skills/aiq-configure-workflow/references/config-profiles.md
[style] ~20-~20: Consider a different adjective to strengthen your wording.
Context: ...iles for intent/shallow, with Ultra for deep research and a larger writer budget 4....
(DEEP_PROFOUND)
README.md
[style] ~295-~295: Consider a different adjective to strengthen your wording.
Context: ...amaIndex profile using GPT Sol/Luna for deep research and Super for intent/shallow. ...
(DEEP_PROFOUND)
🔇 Additional comments (21)
tests/aiq_agent/test_default_model_profiles.py (2)
12-12: LGTM!Also applies to: 21-30
79-90: 🎯 Functional CorrectnessNo additional frontier assertions are needed here.
test_frontier_profile_uses_validated_gpt_role_splitalready checks the exact GPT Sol/Luna role mapping.> Likely an incorrect or invalid review comment..agents/skills/aiq-configure-workflow/references/config-profiles.md (1)
20-20: LGTM!.agents/skills/aiq-configure-workflow/assets/config-scaffold.yml (1)
20-40: 🩺 Stability & AvailabilityKeep the profiles without an explicit
api_key.NIMModelConfig.api_keydefaults toNone;ChatNVIDIAreadsNVIDIA_API_KEYwhen no key is passed. The existingnimvalidation already requiresNVIDIA_API_KEYfor these hosted profiles.> Likely an incorrect or invalid review comment.configs/config_web_azure_ai_search.yml (1)
50-102: LGTM!Also applies to: 190-223
configs/config_web_default_guardrails.yml (1)
31-84: LGTM!Also applies to: 219-244
configs/config_web_default_llamaindex.yml (1)
50-102: LGTM!Also applies to: 191-222
configs/config_web_frag.yml (1)
52-96: LGTM!Also applies to: 161-190
configs/config_web_frag_mcp_auth.yml (1)
59-103: LGTM!Also applies to: 210-239
configs/config_web_opensearch.yml (1)
51-95: LGTM!Also applies to: 153-153, 168-197
frontends/benchmarks/freshqa/configs/config_full_workflow.yml (1)
12-56: LGTM!Also applies to: 72-89
.secrets.baseline (1)
136-136: LGTM!Also applies to: 309-309, 358-358
README.md (1)
111-120: LGTM!Also applies to: 153-157, 290-300
docs/notebooks/0_Getting_Started_with_AIQ.ipynb (1)
73-76: LGTM!Also applies to: 93-93, 324-347, 372-384, 400-402, 777-783
docs/source/customization/configuration-reference.md (1)
95-122: LGTM!Also applies to: 131-135, 335-335, 368-368, 382-382, 457-461, 591-591, 602-635, 656-683
docs/source/customization/swapping-models.md (1)
27-39: LGTM!Also applies to: 54-72, 74-107, 120-120
docs/source/examples/full-pipeline-llamaindex.md (1)
46-66: LGTM!Also applies to: 69-100, 140-173
docs/source/examples/full-pipeline-web.md (1)
63-65: LGTM!Also applies to: 89-115, 172-172, 186-186, 200-204
docs/source/examples/minimal-shallow-only.md (1)
28-41: LGTM!docs/source/get-started/installation.md (1)
28-32: LGTM!docs/source/resources/troubleshooting.md (1)
38-38: LGTM!Also applies to: 45-52, 61-69, 71-76
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
|
/merge |
) #### Overview Update AIQ 2.2 for the August model-deprecation wave while preserving the existing intent and shallow-research behavior until Nano 3.5 is publicly available: - retain Nemotron 3 Super 120B A12B, with its existing aliases and generation profiles, for intent classification and shallow research; - use Nemotron 3 Ultra 550B A55B for clarification and every default deep-research role; - preserve the GPT Sol/Luna deep-role topology introduced by NVIDIA-AI-Blueprints#413 in `config_frontier_models.yml`, while retaining Super for its intent/shallow fast path; - replace the summary, embedding, and VLM defaults with Gemma 4 31B IT, Nemotron 3 Embed 1B, and Nemotron 3 Nano Omni respectively; and - keep public configs on canonical Build model identifiers and NAT's standard `https://integrate.api.nvidia.com/v1` endpoint. The Super exception is intentionally narrow and temporary. Nano 3.5 is excluded because its GA date is August 11. The intent/shallow switch, benchmarked shallow profile, and tool-first runtime guard are isolated in stacked draft NVIDIA-AI-Blueprints#419 so this migration can merge without depending on a not-yet-public endpoint. The migration covers shipping configs, benchmark profiles, knowledge-layer defaults, agent-skill assets, README/docs, launchable examples, and executable notebook config cells. Repository-wide regression tests enforce the Super intent/shallow and Ultra deep-role boundary while rejecting every other replaced model identifier and private inference endpoint reference. This PR also makes persisted embedding stores fail closed when their model metadata is absent, incompatible, or owned by a different logical collection. Chroma and OpenSearch users receive explicit re-ingestion errors instead of silently querying vectors generated by a different embedding model; Azure AI Search already isolates and validates model/dimension compatibility. The branch is rebased on NVIDIA-AI-Blueprints#412. Its updated Brev getting-started notebook is preserved, with only the targeted model references and deep-role configuration changed here. #### DCO sign-off for the squash commit Signed-off-by: Ajay Thorve <athorve@nvidia.com> #### Validation - [x] `uv run pytest` — 2052 passed, 13 skipped - [x] `uv run pre-commit run --all-files` — all hooks passed, including Ruff, YAML, secrets, agent-skill validation, notebook output clearing, and Markdown link checking - [x] Focused knowledge-layer and default-profile tests — 64 passed - [x] Shipping and FreshQA intent/shallow profiles match `release/2.2` exactly - [x] All three changed notebooks parse as valid JSON after output clearing - [x] `git diff --check` - [x] All commits are DCO-signed and have verified GPG signatures - [x] No Nano 3.5 identifier, private inference endpoint, secret, or credential is included #### Where should reviewers start? Start with `tests/aiq_agent/test_default_model_profiles.py` for the enforced model invariants, the shipping configs for the role mapping, and `sources/knowledge_layer/src/{llamaindex,opensearch}/adapter.py` for the embedding compatibility and ownership guards. #### Related Issues - Builds on the frontier-model compatibility work merged in NVIDIA-AI-Blueprints#413. - Preserves the Brev notebook update merged in NVIDIA-AI-Blueprints#412. - Relates to the August model endpoint deprecation migration. ## Summary by CodeRabbit * **New Features** * Added role-specific Nemotron Ultra and Ultra Writer configurations for clarification, deep research, orchestration, and writing. * Retained the existing Nemotron Super profiles for intent classification and shallow research. * Updated default models for summaries, embeddings, and vision processing. * Added validation to detect incompatible, legacy, or incorrectly owned embedding indexes and collections before ingestion or search. * **Documentation** * Refreshed setup guides, examples, notebooks, hardware guidance, and troubleshooting for the updated model lineup. * Added collection-safe migration guidance for embedding indexes and collections. * **Tests** * Expanded coverage for model assignments, embedding compatibility, ownership checks, and fail-before-request behavior. Authors: - Ajay Thorve (https://github.com/AjayThorve) Approvers: - Soumili Nandi (https://github.com/soumilinandi) URL: NVIDIA-AI-Blueprints#414
2 participants
Overview
Update AIQ 2.2 for the August model-deprecation wave while preserving the existing intent and shallow-research behavior until Nano 3.5 is publicly available:
config_frontier_models.yml, while retaining Super for its intent/shallow fast path;https://integrate.api.nvidia.com/v1endpoint.The Super exception is intentionally narrow and temporary. Nano 3.5 is excluded because its GA date is August 11. The intent/shallow switch, benchmarked shallow profile, and tool-first runtime guard are isolated in stacked draft #419 so this migration can merge without depending on a not-yet-public endpoint.
The migration covers shipping configs, benchmark profiles, knowledge-layer defaults, agent-skill assets, README/docs, launchable examples, and executable notebook config cells. Repository-wide regression tests enforce the Super intent/shallow and Ultra deep-role boundary while rejecting every other replaced model identifier and private inference endpoint reference.
This PR also makes persisted embedding stores fail closed when their model metadata is absent, incompatible, or owned by a different logical collection. Chroma and OpenSearch users receive explicit re-ingestion errors instead of silently querying vectors generated by a different embedding model; Azure AI Search already isolates and validates model/dimension compatibility.
The branch is rebased on #412. Its updated Brev getting-started notebook is preserved, with only the targeted model references and deep-role configuration changed here.
DCO sign-off for the squash commit
Signed-off-by: Ajay Thorve athorve@nvidia.com
Validation
uv run pytest— 2052 passed, 13 skippeduv run pre-commit run --all-files— all hooks passed, including Ruff, YAML, secrets, agent-skill validation, notebook output clearing, and Markdown link checkingrelease/2.2exactlygit diff --checkWhere should reviewers start?
Start with
tests/aiq_agent/test_default_model_profiles.pyfor the enforced model invariants, the shipping configs for the role mapping, andsources/knowledge_layer/src/{llamaindex,opensearch}/adapter.pyfor the embedding compatibility and ownership guards.Related Issues
Summary by CodeRabbit
New Features
Documentation
Tests