enhancement: make deep researcher customizable, flexible and less token expensive - #267
Merged
AjayThorve merged 14 commits intoJun 16, 2026
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
cdgamarose-nv
commented
Jun 10, 2026
AjayThorve
self-requested a review
June 11, 2026 21:55
Member
|
@coderabbitai can you review |
|
✅ Action performedFull review finished. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughDeep research configs and docs now route through explicit source-router, planner, researcher, and writer roles, with new domain catalog and source-tool batch settings. The PR also adds DuckDuckGo news and Polymarket source packages, refactors deep-research runtime/orchestration, and updates citation handling and tests. ChangesDeep research platform changes
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes ✨ Finishing Touches🧪 Generate unit tests (beta)
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 22
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
configs/deep_research_domain_catalog.yml (1)
24-64:⚠️ Potential issue | 🟠 MajorFix missing
academic_searchsource ID in the deep research domain catalog
configs/deep_research_domain_catalog.ymlreferencesacademic_search, but the workflow that loads this catalog (configs/config_domain_routing_and_skills.ymlviadomain_catalog_path) definesweb_search,news_search,prediction_market,knowledge_layer, andpaper_search—it does not defineacademic_search. Update the catalog to use an existing source ID (likelypaper_search) or add anacademic_searchentry to thedata_source_registry.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@configs/deep_research_domain_catalog.yml` around lines 24 - 64, The scholarly_technical domain references a non-existent source ID "academic_search"; either replace "academic_search" with the existing "paper_search" in the preferred_source_ids list for domain_id scholarly_technical in configs/deep_research_domain_catalog.yml, or alternatively add a matching "academic_search" entry to the data_source_registry used by configs/config_domain_routing_and_skills.yml; update only the preferred_source_ids for the scholarly_technical domain (or add the new data source definition) so the catalog and registry use the same source ID.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/customization/configuration-reference.md`:
- Around line 498-499: The example config uses undefined keys source_router_llm:
router_llm and writer_llm: writer_llm; update the example so those values
reference actual LLM entries defined in the llms section (e.g., replace
router_llm and writer_llm with existing keys like deep_llm or research_llm), or
add corresponding LLM definitions named router_llm and writer_llm to the llms
block so the references resolve; ensure the referenced LLM identifiers match the
keys used in the llms section (e.g., llms: { deep_llm: {...}, research_llm:
{...} }).
In `@sources/duckduckgo_news_search/src/register.py`:
- Around line 139-141: The except block that currently does "except Exception as
exc" and returns f"Error: News search failed - {exc}" can leak API/internal
details; change it to return a generic message like "Error: News search failed"
when attempt == tool_config.max_retries - 1 and avoid interpolating exc into the
user-facing string, and instead log the full exception to an internal logger at
debug/exception level (do not expose in return). Update the handler around the
symbols attempt, tool_config.max_retries and exc accordingly.
- Around line 63-78: The output in _format_news_result embeds unescaped
user-controlled fields (url, title, body, source, date) into XML/HTML-like
strings; update _format_news_result to HTML-escape these values before inserting
them (escape url for attribute context and title/body/source/date for element
text) using a standard routine (e.g., html.escape) so characters like <, >, &,
and " are encoded; ensure the href attribute value in the <Document href="...">
is properly escaped/quoted and replace uses of url, title, body, source, and
date with their escaped counterparts when building the final string.
In `@sources/duckduckgo_news_search/tests/test_register.py`:
- Around line 68-126: Add tests that cover timeout/retry and HTML-escaping by
creating two new async tests in the same TestDuckDuckGoNewsSearchLive class: (1)
test_timeout_triggers_retry: use _FakeDDGS to raise an Exception("Timeout"),
call _install_fake_ddgs, create DuckDuckGoNewsSearchToolConfig(timeout=0.01,
max_retries=2) and invoke duckduckgo_news_search(...).single_fn("test"), then
assert the output contains an error message like "Error: News search failed" and
that fake.calls length equals the expected retry count; (2)
test_special_characters_in_results_are_escaped: create _FakeDDGS with a result
whose title/body/url include <, >, &, and script tags, install it via
_install_fake_ddgs, call duckduckgo_news_search with default config and assert
the returned output does not include raw "<script>" and that the special
characters are present only in escaped form (e.g., "<script>" or escaped
&, <); reference duckduckgo_news_search, DuckDuckGoNewsSearchToolConfig,
_FakeDDGS, and _install_fake_ddgs to locate code.
In `@sources/polymarket_prediction_market/src/register.py`:
- Around line 256-258: The XML-like Document builder is inserting raw title and
description text (variables like title and body and lines composed into
metadata_lines) which can break the structure if they contain <, >, &, or
quotes; update the code to escape XML/HTML metacharacters (e.g., use
html.escape) before interpolating into the return string in the
Document-returning function, and likewise escape any market line titles in the
_format_market_line function; also apply the same escaping to the other similar
construction mentioned around lines 289-291 so all inserted user-facing fields
are escaped consistently.
- Line 435: The exponential backoff sleep await asyncio.sleep(2**attempt) can
grow unbounded; cap it by introducing a maximum backoff (e.g., MAX_BACKOFF = 30)
or a new parameter (max_backoff) and replace the sleep with await
asyncio.sleep(min(2**attempt, MAX_BACKOFF)) (or min(2**attempt, max_backoff))
inside the retry loop where attempt and max_retries are used (the same block in
register.py that performs retries), ensuring the cap is documented and defaulted
to a reasonable value.
- Around line 406-412: The timeout layering is confusing: asyncio.wait_for(...,
timeout=tool_config.timeout) bounds the overall concurrent attempt of
_search_events and _search_markets to a single "attempt" duration, while
httpx.AsyncClient(timeout=tool_config.timeout) acts as a per-request safety net;
add a short clarifying comment immediately above the AsyncClient/context and the
asyncio.wait_for call stating that wait_for enforces the max time for the
combined attempt and the httpx client timeout applies to each individual HTTP
request, so maintainers understand these are complementary, not additive.
In `@sources/polymarket_prediction_market/tests/test_register.py`:
- Around line 41-121: Add two unit tests in
sources/polymarket_prediction_market/tests/test_register.py: one that calls
polymarket_search (using PolymarketSearchToolConfig and a MagicMock builder)
with mocked _fetch_json returning event/market titles and descriptions
containing XML metacharacters (<, >, &) and assert the resulting output contains
the escaped equivalents (e.g., <, >, &) for both event and market
documents; and a second test that monkeypatches _fetch_json to raise a transient
exception (e.g., ConnectionError) on the first N-1 calls and succeed on the Nth
call, configure PolymarketSearchToolConfig with max_retries>0, run
polymarket_search.single_fn and assert the successful final output and that
_fetch_json was called the expected number of times to prove the retry logic is
used.
In `@src/aiq_agent/agents/deep_researcher/agent.py`:
- Around line 214-215: Replace the generic ValueError raises in this agent
workflow with specific custom exceptions: define WorkflowOutputError and
CitationValidationError (e.g., near the top of
src/aiq_agent/agents/deep_researcher/agent.py or an exceptions module) and use
WorkflowOutputError where you currently raise ValueError for missing
final_message in the writer step (the check referencing final_message) and use
CitationValidationError for the citation validation failure (the ValueError
around citation checks referenced on lines 233-234); import and raise these new
exception types instead of ValueError so callers can distinguish workflow
failures from programming errors and reuse the existing EmptySourceRegistryError
pattern already used at line ~243.
- Around line 59-94: The config option max_loops is dead:
DeepResearchAgentConfig (in register.py) still defines max_loops but
DeepResearcherAgent.__init__ no longer accepts it and run() does a single
agent.ainvoke call, so the config is misleading; remove max_loops from
DeepResearchAgentConfig (or rename it to a supported field) and any references
to it, or if you intend to support retries, add a max_loops parameter back into
DeepResearcherAgent.__init__ and thread it into run() to control a retry loop
around agent.ainvoke; update register.py to either drop max_loops entirely or
forward it to DeepResearcherAgent (and update any docs/comments) so the config
and agent signature are consistent.
In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`:
- Around line 289-294: The recursive collector _collect_builtin_skill_files()
currently rglob("*") and reads every file under BUILTIN_SKILLS_DIR which can
later cause large/binary files to be included and decoded by
_builtin_skill_state_files(); change the collection to only include explicit
supported patterns (e.g., SKILL.md or a whitelist like ["**/SKILL.md",
"**/*.md"]) instead of rglob("*"), keep the existing hidden/__pycache__
exclusion, and update or add a unit test
(test_prepare_state_preloads_builtin_skill_files) to exercise the real
_collect_builtin_skill_files() behavior with a temp skills tree containing both
supported text files and binary/other files to assert only the supported
patterns are returned and no decode failures occur.
In `@src/aiq_agent/agents/deep_researcher/factory.py`:
- Around line 67-72: The think tool currently logs the full thought string at
INFO via logger.info("Thinking: %s", thought), which can leak PII; update think
(function think) to avoid emitting raw user/thought text—either remove the
logger.info call or replace it with a call that first sanitizes/redacts
sensitive content (e.g., mask_sensitive_data(thought)) and log only
non-sensitive metadata (e.g., length, truncated/hashed summary) at a lower
verbosity (DEBUG) so sensitive content is never logged at INFO; ensure the new
helper (e.g., mask_sensitive_data or summarize_for_logging) is used inside think
before any logging.
- Line 385: The recursion_limit=10000 is too large and not tied to the declared
loop budget; change the agent construction so it respects
DeepResearchAgentConfig.max_loops and lowers the recursion cap: when returning
agent.with_config(...) include both a reduced recursion_limit (e.g., something
like max(50, config.max_loops * 10)) and the explicit max_loops value (e.g.,
"max_loops": config.max_loops), or alternatively enforce a step/turn budget
during deepagents/graph construction so each super-step checks against
DeepResearchAgentConfig.max_loops; add a short comment by the agent.with_config
call explaining the relationship between recursion_limit and max_loops.
- Around line 58-64: Update the FILESYSTEM_TOOL_NAMES set to include "grep" and
"execute" so ToolNameSanitizationMiddleware will recognize and sanitize
filesystem tool aliases (the set used to build valid_tool_names which is
combined with tool_set). Modify the constant FILESYSTEM_TOOL_NAMES in factory.py
to add these names (ensuring it aligns with FilesystemMiddleware's include_tools
usage) and add assertions in
tests/aiq_agent/agents/deep_researcher/test_factory.py to assert that "grep" and
"execute" are present in the generated valid_tool_names used by
ToolNameSanitizationMiddleware.
In `@src/aiq_agent/agents/deep_researcher/register.py`:
- Around line 58-61: Remove the deprecated evidence_judge_llm Field from the
model in register.py: delete the evidence_judge_llm declaration and its LLMRef
import (if now unused), update any references/usages (e.g., code that sets or
reads agent.evidence_judge_llm) to stop relying on it, and adjust related
schema/docs/tests to remove mention of this field so configuration surface area
is reduced; ensure no runtime references remain and run tests to confirm nothing
breaks.
- Line 71: The max_loops Field in register.py is unused; either remove it or
wire it into DeepResearcherAgent: if removing, delete the max_loops: int =
Field(default=2) declaration from
src/aiq_agent/agents/deep_researcher/register.py; if wiring, add a max_loops
parameter to DeepResearcherAgent.__init__(self, ..., max_loops: int = 2) and
store it on the agent, then update the code that instantiates
DeepResearcherAgent in register.py to pass the max_loops value from the Field;
ensure the symbol names max_loops and DeepResearcherAgent are consistently used
and update any type hints or tests accordingly.
In `@src/aiq_agent/agents/deep_researcher/tools/research.py`:
- Around line 172-180: The function build_research_batch_tool currently accepts
source_tool_names but never uses it; either remove the parameter from
build_research_batch_tool signature and any callers, or explicitly mark it as
intentionally unused (e.g., add a discard assignment `_ = source_tool_names` or
a short comment) so linters stop flagging it; update the inner
run_research_batch references if you remove the parameter and ensure any callers
of build_research_batch_tool (or tests) are updated accordingly.
In `@src/aiq_agent/agents/deep_researcher/tools/source_routing.py`:
- Around line 232-236: The list comprehension building unmapped_tools repeats
getattr(runtime_tool, "name", "") twice; extract the tool name into a temporary
variable (e.g., tool_name) inside the comprehension or convert to a for-loop so
you call getattr once per runtime_tool, then use tool_name in both the
truthiness check and the get_source_id_for_tool(tool_name) call; update
references to runtime_tool and get_source_id_for_tool accordingly
(unmapped_tools, runtime_tool, get_source_id_for_tool).
In `@src/aiq_agent/agents/deep_researcher/tools/source_tool_batching.py`:
- Around line 35-63: The limit method in SourceToolConcurrencyLimiter can block
forever because await semaphore.acquire() has no timeout; add an optional
timeout parameter to the class (e.g., max_concurrent_timeout: Optional[float])
or to limit(), call asyncio.wait_for(semaphore.acquire(), timeout=...) instead
of awaiting directly, and handle asyncio.TimeoutError by raising a clear
exception or cancelling the operation; ensure you only call semaphore.release()
if the acquire actually succeeded (track an acquired flag) and update
_get_semaphore/SourceToolConcurrencyLimiter signatures to include the new
timeout behavior.
In `@src/aiq_agent/common/data_sources.py`:
- Around line 75-85: The filtering logic does strict, case-sensitive membership
checks (selected = set(data_sources); source_id in selected) which breaks when
caller casing differs from registry ids; normalize casing consistently or
document the requirement. Fix by normalizing the user-provided data_sources into
a canonical form (e.g., lowercasing) when building selected and ensure the value
returned from get_source_id_for_tool is normalized the same way (or update
get_source_id_for_tool to return a normalized id), then use those normalized
selected and source_id variables in the membership check; alternatively add a
clear docstring/comment for the function explaining that data_sources must match
registry id casing exactly. Ensure references to selected, source_id, and
get_source_id_for_tool are updated accordingly.
In `@tests/aiq_agent/auth/__init__.py`:
- Around line 15-27: Remove the duplicate Apache License header present in the
file (the repeated block in lines 15-27) so only the single license header at
the top remains; edit the module-level comment to delete the redundant license
text and ensure there are no other duplicated license blocks elsewhere in this
file (keep the initial header starting at the top and remove the subsequent
copy).
In `@tests/aiq_agent/common/test_citation_verification.py`:
- Around line 549-558: Change the loose assertions that accept extra citations
to exact counts: replace any occurrences in this test
(test_missing_references_section_with_inline_citations_appends_registry_sources)
and the other tests in the same file that currently assert
len(result.valid_citations) >= 2 with assert len(result.valid_citations) == 2 so
the tests expect exactly the two registry-registered citations; locate the
assertions referencing result.valid_citations and update the comparison operator
from >= to ==.
---
Outside diff comments:
In `@configs/deep_research_domain_catalog.yml`:
- Around line 24-64: The scholarly_technical domain references a non-existent
source ID "academic_search"; either replace "academic_search" with the existing
"paper_search" in the preferred_source_ids list for domain_id
scholarly_technical in configs/deep_research_domain_catalog.yml, or
alternatively add a matching "academic_search" entry to the data_source_registry
used by configs/config_domain_routing_and_skills.yml; update only the
preferred_source_ids for the scholarly_technical domain (or add the new data
source definition) so the catalog and registry use the same source ID.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: fda65de3-ab5c-4f89-89c6-29a40f43c332
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (95)
configs/config_cli_default.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_frontier_models.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_web_frag.ymlconfigs/deep_research_domain_catalog.ymldocs/source/architecture/agents/deep-researcher.mddocs/source/customization/configuration-reference.mddocs/source/examples/skills-sandbox/index.mdfrontends/aiq_api/src/aiq_api/auth/request_trace.pyfrontends/aiq_api/src/aiq_api/jobs/_auth_context.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pyfrontends/aiq_api/tests/conftest.pyfrontends/aiq_api/tests/test_auth_errors.pyfrontends/aiq_api/tests/test_job_access.pyfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymlfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymlpyproject.tomlskills/aiq-research/scripts/aiq.pysources/duckduckgo_news_search/README.mdsources/duckduckgo_news_search/pyproject.tomlsources/duckduckgo_news_search/src/__init__.pysources/duckduckgo_news_search/src/register.pysources/duckduckgo_news_search/tests/__init__.pysources/duckduckgo_news_search/tests/test_register.pysources/exa_web_search/pyproject.tomlsources/exa_web_search/src/__init__.pysources/exa_web_search/src/register.pysources/exa_web_search/tests/__init__.pysources/exa_web_search/tests/test_register.pysources/polymarket_prediction_market/README.mdsources/polymarket_prediction_market/pyproject.tomlsources/polymarket_prediction_market/src/__init__.pysources/polymarket_prediction_market/src/register.pysources/polymarket_prediction_market/tests/__init__.pysources/polymarket_prediction_market/tests/test_register.pysrc/aiq_agent/agents/deep_researcher/README.mdsrc/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/models/__init__.pysrc/aiq_agent/agents/deep_researcher/models/state.pysrc/aiq_agent/agents/deep_researcher/models/subagent_contracts.pysrc/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2src/aiq_agent/agents/deep_researcher/prompts/planner.j2src/aiq_agent/agents/deep_researcher/prompts/researcher.j2src/aiq_agent/agents/deep_researcher/prompts/source_registry.j2src/aiq_agent/agents/deep_researcher/prompts/source_router.j2src/aiq_agent/agents/deep_researcher/prompts/writer.j2src/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/skills/research-sandbox/data-table-analysis/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/research-sandbox/forecast-analysis/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/research-sandbox/lightweight-calculation/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/synthesis/long-form-report-writer/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/synthesis/prediction-report-writer/SKILL.mdsrc/aiq_agent/agents/deep_researcher/tools/__init__.pysrc/aiq_agent/agents/deep_researcher/tools/research.pysrc/aiq_agent/agents/deep_researcher/tools/source_registry.pysrc/aiq_agent/agents/deep_researcher/tools/source_routing.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.pysrc/aiq_agent/common/citation_verification.pysrc/aiq_agent/common/data_sources.pysrc/aiq_agent/common/llm_provider.pysrc/aiq_agent/tokenomics/__init__.pysrc/aiq_agent/tokenomics/nat_adapter.pysrc/aiq_agent/tokenomics/pricing.pysrc/aiq_agent/tokenomics/profile.pysrc/aiq_agent/tokenomics/report/__init__.pysrc/aiq_agent/tokenomics/report/__main__.pysrc/aiq_agent/tokenomics/report/_report_base.pysrc/aiq_agent/tokenomics/report/_report_builders.pysrc/aiq_agent/tokenomics/report/_report_stats.pysrc/aiq_agent/tokenomics/report/_report_template_comparison.pysrc/aiq_agent/tokenomics/report/_report_template_single.pytests/aiq_agent/agents/deep_researcher/models/test_subagent_contracts.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/aiq_agent/agents/deep_researcher/test_source_routing.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pytests/aiq_agent/auth/__init__.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/common/test_data_sources.pytests/aiq_agent/common/test_llm_provider.pytests/aiq_agent/jobs/test_runner.pytests/deploy/test_helm_deployment_k8s.pytests/tokenomics/test_nat_adapter.pytests/tokenomics/test_pricing.pytests/tokenomics/test_profile.pytests/tokenomics/test_report_builders.pytests/tokenomics/test_report_stats.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (14)
{src/aiq_agent/knowledge/**,sources/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/knowledge/**,sources/**}: Review data-source and knowledge-layer changes for optional dependency boundaries, external API error handling,
retry/rate-limit behavior, deterministic tests, and registration consistency. New source packages should include
package metadata, plugin registration when applicable, and source-level tests.
Files:
sources/exa_web_search/pyproject.tomlsources/duckduckgo_news_search/tests/__init__.pysources/exa_web_search/src/__init__.pysources/duckduckgo_news_search/README.mdsources/duckduckgo_news_search/src/__init__.pysources/duckduckgo_news_search/pyproject.tomlsources/exa_web_search/tests/test_register.pysources/exa_web_search/tests/__init__.pysources/polymarket_prediction_market/tests/__init__.pysources/polymarket_prediction_market/README.mdsources/exa_web_search/src/register.pysources/polymarket_prediction_market/pyproject.tomlsources/polymarket_prediction_market/src/__init__.pysources/polymarket_prediction_market/tests/test_register.pysources/polymarket_prediction_market/src/register.pysources/duckduckgo_news_search/tests/test_register.pysources/duckduckgo_news_search/src/register.py
**
⚙️ CodeRabbit configuration file
**: # Contributing GuidelinesWe're posting these examples on GitHub to support the NVIDIA LLM community and facilitate feedback.
We invite contributions!Use the following guidelines to contribute to this project.
Pull Requests
Developer workflow for code contributions is as follows:
- Developers must first fork the upstream this repository.
- Git clone the forked repository and push changes to the personal fork.
- Once the code changes are staged on the fork and ready for review, a Pull Request (PR) can be requested to merge the changes from a branch of the fork into a selected branch of upstream.
- Since there is no CI/CD process in place yet, the PR will be accepted and the corresponding issue closed only after adequate testing has been completed, manually, by the developer and/or repository owners reviewing the code.
Signing Your Work
We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
Any contribution which contains commits that are not Signed-Off will not be accepted.
To sign off on a commit, use the--signoff(or-s) option when committing your changes:
$ git commit -s -m "Add cool feature."
This will append the following to your commit message:Signed-off-by: Your Name your@email.com
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or(b)...
Files:
sources/exa_web_search/pyproject.tomlsources/duckduckgo_news_search/tests/__init__.pysources/exa_web_search/src/__init__.pysources/duckduckgo_news_search/README.mdfrontends/aiq_api/tests/conftest.pysources/duckduckgo_news_search/src/__init__.pysources/duckduckgo_news_search/pyproject.tomlfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench.ymlsrc/aiq_agent/agents/deep_researcher/tools/__init__.pysrc/aiq_agent/agents/deep_researcher/prompts/writer.j2src/aiq_agent/agents/deep_researcher/skills/research-sandbox/lightweight-calculation/SKILL.mdskills/aiq-research/scripts/aiq.pyfrontends/benchmarks/deepsearch_qa/configs/config_deepsearch_qa.ymlfrontends/aiq_api/src/aiq_api/jobs/_auth_context.pysrc/aiq_agent/tokenomics/report/_report_builders.pysrc/aiq_agent/tokenomics/report/_report_template_single.pysources/exa_web_search/tests/test_register.pysources/exa_web_search/tests/__init__.pyconfigs/config_frontier_models.ymlfrontends/benchmarks/freshqa/configs/config_full_workflow.ymltests/tokenomics/test_report_stats.pysources/polymarket_prediction_market/tests/__init__.pysrc/aiq_agent/tokenomics/report/__main__.pysources/polymarket_prediction_market/README.mdfrontends/benchmarks/deepresearch_bench/configs/config_deep_research_bench_profiling.ymltests/aiq_agent/auth/__init__.pysrc/aiq_agent/tokenomics/report/_report_stats.pyfrontends/aiq_api/tests/test_job_access.pysources/exa_web_search/src/register.pydocs/source/architecture/agents/deep-researcher.mdtests/aiq_agent/common/test_llm_provider.pytests/tokenomics/test_report_builders.pyfrontends/aiq_api/src/aiq_api/auth/request_trace.pysrc/aiq_agent/tokenomics/__init__.pytests/aiq_agent/common/test_data_sources.pysrc/aiq_agent/tokenomics/profile.pysrc/aiq_agent/agents/deep_researcher/skills/research-sandbox/forecast-analysis/SKILL.mdsrc/aiq_agent/tokenomics/nat_adapter.pyconfigs/deep_research_domain_catalog.ymlsources/polymarket_prediction_market/pyproject.tomlsrc/aiq_agent/agents/deep_researcher/tools/source_registry.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pydocs/source/examples/skills-sandbox/index.mdsrc/aiq_agent/agents/deep_researcher/models/state.pypyproject.tomltests/deploy/test_helm_deployment_k8s.pytests/aiq_agent/common/test_citation_verification.pysrc/aiq_agent/common/data_sources.pysrc/aiq_agent/agents/deep_researcher/README.mdsources/polymarket_prediction_market/src/__init__.pysrc/aiq_agent/agents/deep_researcher/prompts/source_registry.j2src/aiq_agent/tokenomics/report/_report_base.pysrc/aiq_agent/agents/deep_researcher/skills/research-sandbox/data-table-analysis/SKILL.mdconfigs/config_web_frag.ymltests/tokenomics/test_profile.pytests/tokenomics/test_pricing.pysources/polymarket_prediction_market/tests/test_register.pysrc/aiq_agent/tokenomics/pricing.pysrc/aiq_agent/tokenomics/report/_report_template_comparison.pysources/polymarket_prediction_market/src/register.pyconfigs/config_web_default_llamaindex.ymlsrc/aiq_agent/agents/deep_researcher/skills/synthesis/long-form-report-writer/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/synthesis/prediction-report-writer/SKILL.mdtests/tokenomics/test_nat_adapter.pydocs/source/customization/configuration-reference.mdtests/aiq_agent/agents/deep_researcher/test_source_routing.pysrc/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2src/aiq_agent/agents/deep_researcher/prompts/planner.j2configs/config_domain_routing_and_skills.ymlsrc/aiq_agent/agents/deep_researcher/models/subagent_contracts.pysrc/aiq_agent/agents/deep_researcher/models/__init__.pytests/aiq_agent/agents/deep_researcher/models/test_subagent_contracts.pysrc/aiq_agent/tokenomics/report/__init__.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.pysources/duckduckgo_news_search/tests/test_register.pysrc/aiq_agent/common/citation_verification.pysrc/aiq_agent/agents/deep_researcher/prompts/researcher.j2src/aiq_agent/agents/deep_researcher/prompts/source_router.j2frontends/aiq_api/tests/test_auth_errors.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pyconfigs/config_cli_default.ymlsrc/aiq_agent/common/llm_provider.pysrc/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/tools/source_routing.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/tools/research.pysources/duckduckgo_news_search/src/register.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/aiq_agent/agents/deep_researcher/test_agent.py
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
Files:
sources/duckduckgo_news_search/tests/__init__.pysources/exa_web_search/src/__init__.pyfrontends/aiq_api/tests/conftest.pysources/duckduckgo_news_search/src/__init__.pysrc/aiq_agent/agents/deep_researcher/tools/__init__.pyskills/aiq-research/scripts/aiq.pyfrontends/aiq_api/src/aiq_api/jobs/_auth_context.pysrc/aiq_agent/tokenomics/report/_report_builders.pysrc/aiq_agent/tokenomics/report/_report_template_single.pysources/exa_web_search/tests/test_register.pysources/exa_web_search/tests/__init__.pytests/tokenomics/test_report_stats.pysources/polymarket_prediction_market/tests/__init__.pysrc/aiq_agent/tokenomics/report/__main__.pytests/aiq_agent/auth/__init__.pysrc/aiq_agent/tokenomics/report/_report_stats.pyfrontends/aiq_api/tests/test_job_access.pysources/exa_web_search/src/register.pytests/aiq_agent/common/test_llm_provider.pytests/tokenomics/test_report_builders.pyfrontends/aiq_api/src/aiq_api/auth/request_trace.pysrc/aiq_agent/tokenomics/__init__.pytests/aiq_agent/common/test_data_sources.pysrc/aiq_agent/tokenomics/profile.pysrc/aiq_agent/tokenomics/nat_adapter.pysrc/aiq_agent/agents/deep_researcher/tools/source_registry.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pysrc/aiq_agent/agents/deep_researcher/models/state.pytests/deploy/test_helm_deployment_k8s.pytests/aiq_agent/common/test_citation_verification.pysrc/aiq_agent/common/data_sources.pysources/polymarket_prediction_market/src/__init__.pysrc/aiq_agent/tokenomics/report/_report_base.pytests/tokenomics/test_profile.pytests/tokenomics/test_pricing.pysources/polymarket_prediction_market/tests/test_register.pysrc/aiq_agent/tokenomics/pricing.pysrc/aiq_agent/tokenomics/report/_report_template_comparison.pysources/polymarket_prediction_market/src/register.pytests/tokenomics/test_nat_adapter.pytests/aiq_agent/agents/deep_researcher/test_source_routing.pysrc/aiq_agent/agents/deep_researcher/models/subagent_contracts.pysrc/aiq_agent/agents/deep_researcher/models/__init__.pytests/aiq_agent/agents/deep_researcher/models/test_subagent_contracts.pysrc/aiq_agent/tokenomics/report/__init__.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.pysources/duckduckgo_news_search/tests/test_register.pysrc/aiq_agent/common/citation_verification.pyfrontends/aiq_api/tests/test_auth_errors.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pysrc/aiq_agent/common/llm_provider.pysrc/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/tools/source_routing.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/tools/research.pysources/duckduckgo_news_search/src/register.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/aiq_agent/agents/deep_researcher/test_agent.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
frontends/aiq_api/tests/conftest.pysources/exa_web_search/tests/test_register.pytests/tokenomics/test_report_stats.pyfrontends/aiq_api/tests/test_job_access.pytests/aiq_agent/common/test_llm_provider.pytests/tokenomics/test_report_builders.pytests/aiq_agent/common/test_data_sources.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/deploy/test_helm_deployment_k8s.pytests/aiq_agent/common/test_citation_verification.pytests/tokenomics/test_profile.pytests/tokenomics/test_pricing.pysources/polymarket_prediction_market/tests/test_register.pytests/tokenomics/test_nat_adapter.pytests/aiq_agent/agents/deep_researcher/test_source_routing.pytests/aiq_agent/agents/deep_researcher/models/test_subagent_contracts.pysources/duckduckgo_news_search/tests/test_register.pyfrontends/aiq_api/tests/test_auth_errors.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/aiq_agent/agents/deep_researcher/test_agent.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/tools/__init__.pysrc/aiq_agent/agents/deep_researcher/prompts/writer.j2src/aiq_agent/agents/deep_researcher/skills/research-sandbox/lightweight-calculation/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/research-sandbox/forecast-analysis/SKILL.mdsrc/aiq_agent/agents/deep_researcher/tools/source_registry.pysrc/aiq_agent/agents/deep_researcher/models/state.pysrc/aiq_agent/agents/deep_researcher/README.mdsrc/aiq_agent/agents/deep_researcher/prompts/source_registry.j2src/aiq_agent/agents/deep_researcher/skills/research-sandbox/data-table-analysis/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/synthesis/long-form-report-writer/SKILL.mdsrc/aiq_agent/agents/deep_researcher/skills/synthesis/prediction-report-writer/SKILL.mdsrc/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2src/aiq_agent/agents/deep_researcher/prompts/planner.j2src/aiq_agent/agents/deep_researcher/models/subagent_contracts.pysrc/aiq_agent/agents/deep_researcher/models/__init__.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.pysrc/aiq_agent/agents/deep_researcher/prompts/researcher.j2src/aiq_agent/agents/deep_researcher/prompts/source_router.j2src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/tools/source_routing.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/tools/research.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
skills/aiq-research/scripts/aiq.py
📄 CodeRabbit inference engine (skills/aiq-research/SKILL.md)
skills/aiq-research/scripts/aiq.py: Use Python 3.11+ with the helper script atscripts/aiq.pyto call a locally running NVIDIA AI-Q Blueprint server
Resolve the target AI-Q backend URL by checkingAIQ_SERVER_URLenvironment variable first, defaulting tohttp://localhost:8000if not set
Runhealthcommand before sending research requests to verify the backend is reachable
Before sending any user query to a non-local AI-Q backend URL, explicitly confirm in conversation that the URL is trusted
Do not transmit API keys, bearer tokens, cookies, or basic-auth credentials throughAIQ_SERVER_URLor query text; store backend credentials in the AI-Q deployment environment instead
Poll asynchronous deep research jobs usingresearch_poll <JOB_ID>when AI-Q returns a job ID in the response
Present returned research reports with citations and source URLs intact; do not truncate or remove source attribution
Stop on failed jobs and show the returned error; do not retry automatically without user guidance
Verify semantic version compatibility: skill major version must match Blueprint major version; Blueprint minor version must be equal or greater than skill minor version
Files:
skills/aiq-research/scripts/aiq.py
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}
⚙️ CodeRabbit configuration file
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.
Files:
skills/aiq-research/scripts/aiq.py
skills/aiq-research/**
⚙️ CodeRabbit configuration file
skills/aiq-research/**: ---
name: aiq-research
description: |
Use when asked to run deep research or AI-Q research through a reachable NVIDIA AI-Q Blueprint backend.
license: Apache-2.0
permissions:
env:
- AIQ_SERVER_URL
network:
- http://localhost:8000
compatibility: |
Designed for Claude Code, OpenCode, Codex, and Agent Skills-compatible tools. Requires Python 3.11+ and network
access to a running local AI-Q Blueprint server athttp://localhost:8000by default. Non-local backends must be
explicitly trusted by the user and granted by the host tool outside this public skill.
metadata:
version: "2.1.0"
author: "NVIDIA AI-Q Blueprint Team aiq-blueprint@nvidia.com"
github-url: "https://github.com/NVIDIA-AI-Blueprints/aiq"
tags:
- nvidia
- aiq
- blueprint
- deep-research
- research-agents
- agent-skills
languages:
- python
- bash
domain: "research-agents"
allowed-tools: Read BashAIQ Research Skill
Purpose
Use this skill to call a locally running NVIDIA AI-Q Blueprint server through the helper script at
scripts/aiq.py.Use this skill for research-shaped requests, including:
- "deep research on ..."
- "AIQ research ..."
- "research ..."
- "use AI-Q to answer ..."
- "ask AI-Q about ..."
Do not use this skill for install, deploy, start, stop, UI, CLI, Docker, Helm, or troubleshooting requests. Those
belong toaiq-deploy.Prerequisites
Users need:
- Python 3.11+ available as
python3.- A reachable local or self-hosted AI-Q Blueprint backend.
AIQ_SERVER_URLset when the backend is not running athttp://localhost:8000; non-local values must be trusted by
the user before any query is sent.- A backend configured with authentication disabled for this public helper, or a separate authenticated AI-Q skill for
authenticated environments.- Network access from the local machine to the AI-Q backend URL.
- Credentials configured in the backend environment, not in this skill. Thi...
Files:
skills/aiq-research/scripts/aiq.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/_auth_context.pyfrontends/aiq_api/src/aiq_api/auth/request_trace.pyfrontends/aiq_api/src/aiq_api/jobs/runner.py
src/aiq_agent/tokenomics/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/tokenomics/**/*: Review tokenomics changes for deterministic accounting, pricing-data assumptions, rounding behavior, and report
compatibility. Flag pricing or aggregation changes without representative tests and clear documentation updates.
Files:
src/aiq_agent/tokenomics/report/_report_builders.pysrc/aiq_agent/tokenomics/report/_report_template_single.pysrc/aiq_agent/tokenomics/report/__main__.pysrc/aiq_agent/tokenomics/report/_report_stats.pysrc/aiq_agent/tokenomics/__init__.pysrc/aiq_agent/tokenomics/profile.pysrc/aiq_agent/tokenomics/nat_adapter.pysrc/aiq_agent/tokenomics/report/_report_base.pysrc/aiq_agent/tokenomics/pricing.pysrc/aiq_agent/tokenomics/report/_report_template_comparison.pysrc/aiq_agent/tokenomics/report/__init__.py
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_frontier_models.ymlconfigs/deep_research_domain_catalog.ymlconfigs/config_web_frag.ymlconfigs/config_web_default_llamaindex.ymlconfigs/config_domain_routing_and_skills.ymlconfigs/config_cli_default.yml
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/architecture/agents/deep-researcher.mddocs/source/examples/skills-sandbox/index.mddocs/source/customization/configuration-reference.md
{src/aiq_agent/auth/**,frontends/aiq_api/src/aiq_api/auth/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/auth/**,frontends/aiq_api/src/aiq_api/auth/**}: Review authentication changes for issuer/audience validation, token parsing, error hygiene, logging safety,
and compatibility with local and deployed modes. Do not accept changes that expose tokens, weaken validation,
or blur trusted server-side identity with client-supplied fields.
Files:
frontends/aiq_api/src/aiq_api/auth/request_trace.py
{.github/**,ci/**,.pre-commit-config.yaml,pyproject.toml,uv.lock}
⚙️ CodeRabbit configuration file
{.github/**,ci/**,.pre-commit-config.yaml,pyproject.toml,uv.lock}: Review automation and packaging changes for least-privilege permissions, pinned versions where appropriate,
copy-pr-bot pull-request/ branch behavior, reproducible uv/npm setup, secret handling, and consistency with
the documented validation matrix.
Files:
pyproject.toml
🪛 ast-grep (0.43.0)
tests/aiq_agent/agents/deep_researcher/test_source_routing.py
[info] 213-213: use jsonify instead of json.dumps for JSON output
Context: json.dumps(payload)
Note: Security best practice.
(use-jsonify)
src/aiq_agent/agents/deep_researcher/tools/source_routing.py
[info] 268-276: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
source_catalog_payload(
tools,
allowed_source_ids=allowed_source_ids,
domain_catalog_path=domain_catalog_path,
),
indent=2,
ensure_ascii=False,
)
Note: Security best practice.
(use-jsonify)
src/aiq_agent/agents/deep_researcher/tools/research.py
[info] 42-42: use jsonify instead of json.dumps for JSON output
Context: json.dumps(query.model_dump(mode="json"), indent=2, ensure_ascii=False)
Note: Security best practice.
(use-jsonify)
[info] 103-103: use jsonify instead of json.dumps for JSON output
Context: json.dumps(query.model_dump(mode="json"), sort_keys=True, ensure_ascii=False)
Note: Security best practice.
(use-jsonify)
[info] 114-114: use jsonify instead of json.dumps for JSON output
Context: json.dumps(note.model_dump(mode="json", exclude_none=True), indent=2, ensure_ascii=False)
Note: Security best practice.
(use-jsonify)
[info] 212-216: use jsonify instead of json.dumps for JSON output
Context: json.dumps(
[note.model_dump(mode="json", exclude_none=True) for note in notes],
indent=2,
ensure_ascii=False,
)
Note: Security best practice.
(use-jsonify)
src/aiq_agent/agents/deep_researcher/custom_middleware.py
[warning] 245-245: Do not make http calls without encryption
Context: "http://"
Note: [CWE-319].
(requests-http)
🪛 LanguageTool
src/aiq_agent/agents/deep_researcher/skills/research-sandbox/forecast-analysis/SKILL.md
[style] ~73-~73: This phrase is redundant. Consider writing “point” or “time”.
Context: ...nce of market-implied expectations at a point in time and should be labeled as such.
(MOMENT_IN_TIME)
src/aiq_agent/agents/deep_researcher/skills/synthesis/long-form-report-writer/SKILL.md
[style] ~13-~13: Consider a different adjective to strengthen your wording.
Context: ...learly asks for a comprehensive report, deep analysis, publication-ready writeup, or...
(DEEP_PROFOUND)
[style] ~72-~72: ‘exact same’ might be wordy. Consider a shorter alternative.
Context: ...re not valid final citations unless the exact same URL or citation key appears in `get_ver...
(EN_WORDINESS_PREMIUM_EXACT_SAME)
[style] ~99-~99: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...actual claim has an inline citation. 4. Confirm the Sources section includes every cite...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
src/aiq_agent/agents/deep_researcher/skills/synthesis/prediction-report-writer/SKILL.md
[style] ~27-~27: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...l evidence supporting the prediction. - Extract directional evidence opposing the predi...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~30-~30: Consider using “the surrounding report”.
Context: ...al forecast before drafting, then write the report around it. ## Forecast Discipline - Commit to a...
(NOUN_AROUND_IT)
[style] ~39-~39: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...lect the matching option or bucket. - For ranked or categorical outcomes: rank ca...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~53-~53: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...oint estimate and plausible range. - For multiple-choice or threshold questions,...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~118-~118: ‘exact same’ might be wordy. Consider a shorter alternative.
Context: ...re not valid final citations unless the exact same URL or citation key appears in `get_ver...
(EN_WORDINESS_PREMIUM_EXACT_SAME)
[style] ~148-~148: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ainties are concrete and actionable. 4. Confirm every material factual claim has an inl...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~149-~149: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...actual claim has an inline citation. 5. Confirm the Sources section includes every cite...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~150-~150: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ive source labels alone are invalid. 6. Confirm no internal files, agents, prompts, or ...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~151-~151: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...rompts, or tool names are mentioned. 7. Confirm internal evidence_judgment scores and...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
Signed-off-by: Chantal D Gama Rose <cdgamarose@nvidia.com>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
sources/polymarket_prediction_market/src/register.py (1)
439-446: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick winSanitize exception messages in error returns.
Lines 443 and 446 expose raw exception details to the user via
f"Error: Polymarket search failed - {exc}"andf"Error: Polymarket search failed - {last_error}". This is inconsistent with the sanitized error handling in the DuckDuckGo source (which logs the exception and returns a generic message). The exception text could leak API details, internal paths, or HTTP error bodies.🔒 Proposed fix
except Exception as exc: # noqa: BLE001 - source APIs can raise transport-specific exceptions last_error = exc if attempt == tool_config.max_retries - 1: - logger.warning("Polymarket search failed for query %r: %s", query, exc) - return f"Error: Polymarket search failed - {exc}" + logger.exception( + "Polymarket search failed after %s attempts for query %r", + tool_config.max_retries, + query, + ) + return "Error: Polymarket search failed" await asyncio.sleep(min(2**attempt, MAX_RETRY_BACKOFF_SECONDS)) - return f"Error: Polymarket search failed - {last_error}" + return "Error: Polymarket search failed after all retries"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sources/polymarket_prediction_market/src/register.py` around lines 439 - 446, The current error returns embed raw exception text via exc and last_error; update the Polymarket search error handling (the except block that references exc, last_error, logger, tool_config.max_retries and MAX_RETRY_BACKOFF_SECONDS) to avoid returning exception details to callers: keep logging the full exception with logger.warning (include exc info) but change returned strings on both the final retry and the final return to a generic message like "Error: Polymarket search failed" (no exception interpolation); ensure last_error is still captured for logging but not exposed in returned messages.tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
1-24: 📐 Maintainability & Code Quality | 🔴 CriticalRuff gates pass, but pytest gate is blocked by dependency import errors.
ruff check+ruff format --checksucceeded on the requested files.pytestfails before running tests:tests/aiq_agent/conftest.pycan’t import due tosrc/aiq_agent/common/__init__.py→natimportingpyzotero, which errors withImportError: cannot import name 'InvalidItemFields' from pyzotero.zotero_errors.Install the full repo-pinned dependencies (including the correct
pyzoterotransitive version fornvidia-nat*/nat) and rerun the same pytest subset.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/aiq_agent/agents/deep_researcher/test_agent.py` around lines 1 - 24, Pytest is failing due to an import error from the installed pyzotero version (ImportError: cannot import name 'InvalidItemFields') when importing aiq_agent.common (nat) as used by tests/aiq_agent/conftest.py; fix by installing the repository-pinned dependencies (including the correct transitive pyzotero version required by the nvidia-nat*/nat packages) and then re-run the failing pytest subset (e.g., pip install -r requirements.txt or use the repo lockfile / poetry/pipenv equivalent to install pinned deps), ensuring that pyzotero resolves to the version exposing InvalidItemFields before running pytest for tests/aiq_agent/... .Source: Coding guidelines
♻️ Duplicate comments (1)
src/aiq_agent/agents/deep_researcher/factory.py (1)
58-65: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winInclude
executein the filesystem tool allowlist for sandbox runs.Line 58-65 now includes
grep, butexecuteis still missing. IfFilesystemMiddlewareexposesexecute(sandbox backend), sanitizer-valid-name filtering can still reject normalized/suffixedexecutecalls and cause avoidable tool-call failures. Please add"execute"and extend the sanitizer assertions intests/aiq_agent/agents/deep_researcher/test_factory.py.Proposed minimal patch
FILESYSTEM_TOOL_NAMES = { "edit_file", + "execute", "grep", "glob", "ls", "read_file", "write_file", }#!/bin/bash set -euo pipefail python - <<'PY' import inspect try: from deepagents.middleware.filesystem import FilesystemMiddleware src = inspect.getsource(FilesystemMiddleware) print("contains 'grep':", "grep" in src) print("contains 'execute':", "execute" in src) except Exception as exc: print("IMPORT_OR_INTROSPECTION_FAILED:", repr(exc)) PY rg -n "FILESYSTEM_TOOL_NAMES|valid_tool_names|grep|execute" \ src/aiq_agent/agents/deep_researcher/factory.py \ tests/aiq_agent/agents/deep_researcher/test_factory.py🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/factory.py` around lines 58 - 65, The FILESYSTEM_TOOL_NAMES set is missing "execute", causing sanitizer rejections when FilesystemMiddleware exposes execute; add the string "execute" to the FILESYSTEM_TOOL_NAMES set in factory.py (referencing the FILESYSTEM_TOOL_NAMES symbol) and update the sanitizer assertions in tests/aiq_agent/agents/deep_researcher/test_factory.py to expect normalized/suffixed "execute" variants the same way existing checks handle "grep" (adjust the valid_tool_names/allowlist assertions in the test that reference FilesystemMiddleware behavior).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@sources/polymarket_prediction_market/src/register.py`:
- Around line 439-446: The current error returns embed raw exception text via
exc and last_error; update the Polymarket search error handling (the except
block that references exc, last_error, logger, tool_config.max_retries and
MAX_RETRY_BACKOFF_SECONDS) to avoid returning exception details to callers: keep
logging the full exception with logger.warning (include exc info) but change
returned strings on both the final retry and the final return to a generic
message like "Error: Polymarket search failed" (no exception interpolation);
ensure last_error is still captured for logging but not exposed in returned
messages.
In `@tests/aiq_agent/agents/deep_researcher/test_agent.py`:
- Around line 1-24: Pytest is failing due to an import error from the installed
pyzotero version (ImportError: cannot import name 'InvalidItemFields') when
importing aiq_agent.common (nat) as used by tests/aiq_agent/conftest.py; fix by
installing the repository-pinned dependencies (including the correct transitive
pyzotero version required by the nvidia-nat*/nat packages) and then re-run the
failing pytest subset (e.g., pip install -r requirements.txt or use the repo
lockfile / poetry/pipenv equivalent to install pinned deps), ensuring that
pyzotero resolves to the version exposing InvalidItemFields before running
pytest for tests/aiq_agent/... .
---
Duplicate comments:
In `@src/aiq_agent/agents/deep_researcher/factory.py`:
- Around line 58-65: The FILESYSTEM_TOOL_NAMES set is missing "execute", causing
sanitizer rejections when FilesystemMiddleware exposes execute; add the string
"execute" to the FILESYSTEM_TOOL_NAMES set in factory.py (referencing the
FILESYSTEM_TOOL_NAMES symbol) and update the sanitizer assertions in
tests/aiq_agent/agents/deep_researcher/test_factory.py to expect
normalized/suffixed "execute" variants the same way existing checks handle
"grep" (adjust the valid_tool_names/allowlist assertions in the test that
reference FilesystemMiddleware behavior).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: dda884ed-d2c5-4d22-b389-9cd1881fab4c
📒 Files selected for processing (30)
docs/notebooks/0_Getting_Started_with_AIQ.ipynbdocs/notebooks/1_Deep_Researcher_Web_Search.ipynbdocs/notebooks/2_Deep_Researcher_Customization.ipynbdocs/source/architecture/agents/deep-researcher.mddocs/source/customization/configuration-reference.mddocs/source/examples/cli-with-local-nims.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/full-pipeline-web.mddocs/source/examples/hybrid-frontier-model.mdsources/duckduckgo_news_search/src/register.pysources/duckduckgo_news_search/tests/test_register.pysources/polymarket_prediction_market/src/register.pysources/polymarket_prediction_market/tests/test_register.pysrc/aiq_agent/agents/deep_researcher/README.mdsrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/tools/research.pysrc/aiq_agent/agents/deep_researcher/tools/source_routing.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.pysrc/aiq_agent/common/citation_verification.pysrc/aiq_agent/common/data_sources.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/aiq_agent/agents/deep_researcher/test_source_routing.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/common/test_data_sources.pytests/aiq_agent/jobs/test_runner.py
💤 Files with no reviewable changes (12)
- docs/source/examples/hybrid-frontier-model.md
- docs/source/architecture/agents/deep-researcher.md
- docs/source/examples/cli-with-local-nims.md
- docs/source/examples/full-pipeline-llamaindex.md
- src/aiq_agent/agents/deep_researcher/README.md
- docs/notebooks/0_Getting_Started_with_AIQ.ipynb
- src/aiq_agent/agents/deep_researcher/register.py
- docs/source/examples/full-pipeline-web.md
- src/aiq_agent/agents/deep_researcher/tools/research.py
- docs/notebooks/1_Deep_Researcher_Web_Search.ipynb
- docs/notebooks/2_Deep_Researcher_Customization.ipynb
- tests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/customization/configuration-reference.md
**
⚙️ CodeRabbit configuration file
**: # Contributing GuidelinesWe're posting these examples on GitHub to support the NVIDIA LLM community and facilitate feedback.
We invite contributions!Use the following guidelines to contribute to this project.
Pull Requests
Developer workflow for code contributions is as follows:
- Developers must first fork the upstream this repository.
- Git clone the forked repository and push changes to the personal fork.
- Once the code changes are staged on the fork and ready for review, a Pull Request (PR) can be requested to merge the changes from a branch of the fork into a selected branch of upstream.
- Since there is no CI/CD process in place yet, the PR will be accepted and the corresponding issue closed only after adequate testing has been completed, manually, by the developer and/or repository owners reviewing the code.
Signing Your Work
We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
Any contribution which contains commits that are not Signed-Off will not be accepted.
To sign off on a commit, use the--signoff(or-s) option when committing your changes:
$ git commit -s -m "Add cool feature."
This will append the following to your commit message:Signed-off-by: Your Name your@email.com
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or(b)...
Files:
docs/source/customization/configuration-reference.mdsources/duckduckgo_news_search/tests/test_register.pysrc/aiq_agent/agents/deep_researcher/tools/source_routing.pytests/aiq_agent/agents/deep_researcher/test_source_routing.pytests/aiq_agent/common/test_data_sources.pysources/polymarket_prediction_market/tests/test_register.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pysrc/aiq_agent/common/data_sources.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysources/duckduckgo_news_search/src/register.pytests/aiq_agent/common/test_citation_verification.pysources/polymarket_prediction_market/src/register.pysrc/aiq_agent/agents/deep_researcher/factory.pytests/aiq_agent/agents/deep_researcher/test_factory.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.pysrc/aiq_agent/common/citation_verification.pytests/aiq_agent/agents/deep_researcher/test_agent.py
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
Files:
sources/duckduckgo_news_search/tests/test_register.pysrc/aiq_agent/agents/deep_researcher/tools/source_routing.pytests/aiq_agent/agents/deep_researcher/test_source_routing.pytests/aiq_agent/common/test_data_sources.pysources/polymarket_prediction_market/tests/test_register.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pysrc/aiq_agent/common/data_sources.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysources/duckduckgo_news_search/src/register.pytests/aiq_agent/common/test_citation_verification.pysources/polymarket_prediction_market/src/register.pysrc/aiq_agent/agents/deep_researcher/factory.pytests/aiq_agent/agents/deep_researcher/test_factory.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.pysrc/aiq_agent/common/citation_verification.pytests/aiq_agent/agents/deep_researcher/test_agent.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
sources/duckduckgo_news_search/tests/test_register.pytests/aiq_agent/agents/deep_researcher/test_source_routing.pytests/aiq_agent/common/test_data_sources.pysources/polymarket_prediction_market/tests/test_register.pytests/aiq_agent/agents/deep_researcher/test_source_tool_batching.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/aiq_agent/agents/deep_researcher/test_agent.py
{src/aiq_agent/knowledge/**,sources/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/knowledge/**,sources/**}: Review data-source and knowledge-layer changes for optional dependency boundaries, external API error handling,
retry/rate-limit behavior, deterministic tests, and registration consistency. New source packages should include
package metadata, plugin registration when applicable, and source-level tests.
Files:
sources/duckduckgo_news_search/tests/test_register.pysources/polymarket_prediction_market/tests/test_register.pysources/duckduckgo_news_search/src/register.pysources/polymarket_prediction_market/src/register.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/tools/source_routing.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/tools/source_tool_batching.py
🔇 Additional comments (16)
docs/source/customization/configuration-reference.md (1)
496-498: LGTM!sources/duckduckgo_news_search/tests/test_register.py (1)
23-24: LGTM!Also applies to: 69-86, 148-188
src/aiq_agent/agents/deep_researcher/tools/source_routing.py (1)
56-72: LGTM!Also applies to: 81-89, 210-220
src/aiq_agent/common/data_sources.py (1)
63-64: LGTM!Also applies to: 76-76, 84-84
tests/aiq_agent/agents/deep_researcher/test_source_routing.py (1)
180-194: LGTM!Also applies to: 233-245, 249-257
tests/aiq_agent/common/test_data_sources.py (1)
249-250: LGTM!sources/duckduckgo_news_search/src/register.py (1)
1-156: LGTM!sources/polymarket_prediction_market/src/register.py (1)
1-38: LGTM!Also applies to: 42-121, 127-438, 448-452
sources/polymarket_prediction_market/tests/test_register.py (1)
1-257: LGTM!src/aiq_agent/agents/deep_researcher/tools/source_tool_batching.py (1)
1-216: LGTM!src/aiq_agent/common/citation_verification.py (1)
1-1126: LGTM!tests/aiq_agent/agents/deep_researcher/test_source_tool_batching.py (1)
1-318: LGTM!src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
24-24: LGTM!Also applies to: 43-43, 50-50, 176-203, 248-252, 313-323
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
210-212: LGTM!tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
214-247: LGTM!Also applies to: 252-285
tests/aiq_agent/common/test_citation_verification.py (1)
557-557: LGTM!Also applies to: 569-569, 597-597
5 tasks
AjayThorve
reviewed
Jun 14, 2026
AjayThorve
left a comment
AjayThorve
left a comment
Member
There was a problem hiding this comment.
Reviewed this PR against the head commit and ran the changed test suites locally (422 passed, 1 failed). The refactor is solid overall — clean decomposition into source-router/planner/researcher-batch/writer, good unit coverage, and max_loops removed cleanly. Flagging the critical/high issues I'm most confident in as inline comments.
One more, not inline (the stale lines aren't in this diff): the config_skills.yml → config_domain_routing_and_skills.yml rename left runnable --config_file configs/config_skills.yml commands in docs/source/examples/skills-sandbox/index.md and a reference in docs/source/examples/index.md that now 404.
AjayThorve
reviewed
Jun 15, 2026
AjayThorve
reviewed
Jun 15, 2026
AjayThorve
reviewed
Jun 15, 2026
AjayThorve
reviewed
Jun 16, 2026
Signed-off-by: Chantal D Gama Rose <cdgamarose@nvidia.com>
Collaborator
Author
|
/ok to test d4dda47 |
Member
|
/merge |
AjayThorve
merged commit Jun 16, 2026
8638256
into
NVIDIA-AI-Blueprints:develop
9 of 10 checks passed
|
This is a big improvement structurally, especially separating source routing, researcher batches, and writer-only synthesis. One thing I’d watch closely is the runtime envelope now that deep research has more moving parts and the loop limit was removed. Source routing + batched researchers + nested skills can create a lot of model/tool calls unless there is a central pre-call guard. A pattern like this would make the pipeline safer: const decision = canRunResearchStep({
jobId,
stage, // router | planner | researcher | writer
agentId,
model,
sourceId,
batchSize,
concurrency,
estimatedTokens,
remainingBudget,
});Before each LLM/tool batch, I’d check:
The structured contracts and citation verification are strong additions. I’d just make sure the new router/researcher/writer split also has a deterministic budget/concurrency boundary before calls execute, not only observability after the job has already expanded. |
5 tasks
5 tasks
zhongxuanwang-nv
added a commit
to zhongxuanwang-nv/aiq
that referenced
this pull request
Jun 24, 2026
…intainer skills DEVSKILLS-8 (under epic AIQ-3362) tracks later maintainer skills until their surfaces stabilize or become active release work. Two of those gates have now opened, so these skills move from the backlog into the set: - aiq-customize-prompts-models: prompt/model customization became active release work in 2.2 (per-agent LLM role fields orchestrator_llm/source_router_llm/ researcher_llm/planner_llm/writer_llm via NVIDIA-AI-Blueprints#267, plus the documented Jinja2 prompt templates and swapping-models guide). Routes editing src/aiq_agent/agents/*/prompts/*.j2 and assigning LLMs per role in the llms section and agent config. - aiq-maintain-ci: CI/governance became active release work (4 workflows, contributor governance setup, the skill-eval regression gate, copy-pr-bot mirroring, and an expanded pre-commit hook set). Routes changes to .github/workflows, .pre-commit-config.yaml, CODEOWNERS, .coderabbit.yaml, and the .github/skill-eval harness. Each skill is a SKILL.md plus two references and a .claude/skills compatibility symlink, following the existing maintainer-skill conventions. The other two DEVSKILLS-8 skills stay deferred: aiq-auth-data-source-integration (protected- source UX/API deferred to 2.2/2.3 per PR NVIDIA-AI-Blueprints#212) and aiq-ui-change (2.2 UI auth controls not yet settled). Doc example listings (agent-skills.md table, README) that enumerate maintainer skills are reworked in the separate PR NVIDIA-AI-Blueprints#281; kept disjoint here. Validation: scripts/validate_skills.py (8 skills OK), pytest tests/test_agent_skills.py (4 passed), and pre-commit (detect-secrets, validate-skills, markdown-link-check) on the new files all pass. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com>
5 tasks
cdgamarose-nv
added a commit
that referenced
this pull request
Jun 25, 2026
) * feat(skills): add aiq-customize-prompts-models and aiq-maintain-ci maintainer skills DEVSKILLS-8 (under epic AIQ-3362) tracks later maintainer skills until their surfaces stabilize or become active release work. Two of those gates have now opened, so these skills move from the backlog into the set: - aiq-customize-prompts-models: prompt/model customization became active release work in 2.2 (per-agent LLM role fields orchestrator_llm/source_router_llm/ researcher_llm/planner_llm/writer_llm via #267, plus the documented Jinja2 prompt templates and swapping-models guide). Routes editing src/aiq_agent/agents/*/prompts/*.j2 and assigning LLMs per role in the llms section and agent config. - aiq-maintain-ci: CI/governance became active release work (4 workflows, contributor governance setup, the skill-eval regression gate, copy-pr-bot mirroring, and an expanded pre-commit hook set). Routes changes to .github/workflows, .pre-commit-config.yaml, CODEOWNERS, .coderabbit.yaml, and the .github/skill-eval harness. Each skill is a SKILL.md plus two references and a .claude/skills compatibility symlink, following the existing maintainer-skill conventions. The other two DEVSKILLS-8 skills stay deferred: aiq-auth-data-source-integration (protected- source UX/API deferred to 2.2/2.3 per PR #212) and aiq-ui-change (2.2 UI auth controls not yet settled). Doc example listings (agent-skills.md table, README) that enumerate maintainer skills are reworked in the separate PR #281; kept disjoint here. Validation: scripts/validate_skills.py (8 skills OK), pytest tests/test_agent_skills.py (4 passed), and pre-commit (detect-secrets, validate-skills, markdown-link-check) on the new files all pass. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com> * docs(skills): fix accuracy issues in the two new maintainer skills Address PR #282 review feedback; every change verified against the repo. aiq-maintain-ci: - pytest and helm-lint are pre-commit `stages: [push]`, so `pre-commit run --all-files` does NOT run them (the prior "can be heavier than expected" note was backwards). Document `--hook-stage push` and that CI runs them as the dedicated test/helm-lint jobs; note the pre-commit job's SKIP= set. - Correct ui.yml job ids (install/lint/type-check/unit-test/build); name the skills-eval stages (detect-changes -> generate-datasets, which is creds-free -> harbor-eval); drop the tangential aiq-add-tool from Related Skills. aiq-customize-prompts-models: - The LLMProvider.configure(LLMRole.X) role binding is the deep-researcher pattern (add a field->role table: ORCHESTRATOR/ROUTER/RESEARCHER/PLANNER/ REPORT_WRITER); the clarifier passes planner_llm to its constructor instead. Unset deep-research roles fall back to orchestrator_llm (no generic `llm` field). Add source_router_llm to the example. - Adding a NEW template needs a one-line load_prompt wiring in the agent (prompts.md Step 3); soften "without changing agent code". Note prompts.md does not document every template's variables (source_router/writer/ source_registry) -- the .j2 files are authoritative. both: - Validation smoke must pass `--config_file <your config>` (a bare start_cli.sh runs the fixed default config); scope pytest to the agent's test dir. Validation: validate_skills.py (8 OK), pytest tests/test_agent_skills.py (4 passed), pre-commit (detect-secrets, validate-skills, markdown-link-check) on the changed files all pass. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com> * docs(skills): list the two new skills in the maintainer skill tables Now that #281 has merged, enumerate aiq-customize-prompts-models and aiq-maintain-ci alongside the other four maintainer skills in the .agents/skills/README.md and docs/source/integration/agent-skills.md examples tables, and add their .claude/skills symlinks to the documented symlink block. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com> * docs(skills): warn against hardcoding queries/domains in prompts Address review feedback on aiq-customize-prompts-models: add guidance that prompt templates must stay task-agnostic and not hard-code specific queries, domains, or source/tool names, since source/domain selection is data-driven via the data_source_registry and source_router.j2. Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com> --------- Signed-off-by: Zhongxuan Wang <daniewang@nvidia.com> Co-authored-by: Chantal D Gama Rose <cdgamarose@nvidia.com>
AjayThorve
added a commit
that referenced
this pull request
Jul 1, 2026
#### Overview
Adds **report-aware follow-up** to the deep-research blueprint. After a
report job completes, a user can ask about it, make cosmetic edits, or
start delta research that reuses the prior report — without forcing
every follow-up through a dedicated "report mode." A chat **router**
picks a semantic route from the message plus an optional
`active_report_job_id`; the active report is treated as *context*, never
as a command.
**Modes**
- **report ask** — inline, bounded LLM answer from the parent report
only. No tools, no live research, no child artifact.
- **report cosmetic edit** — an internal `report_rewriter` async child
job for mechanical/aesthetic edits that do not need new evidence. The
parent report stays immutable.
- **report delta research** — the existing deep researcher, seeded with
parent-report context, for fresh evidence, deeper analysis, or a new
analytical perspective on the same report topic.
- **standalone research** — normal shallow/deep research when the
request is unrelated to the active report or asks for a separate report.
**Intent routing examples**
- `what are the risks in this report?` → `report_ask`
- `make this shorter` → `report_cosmetic_edit`
- `format the key takeaways as bullets` → `report_cosmetic_edit`
- `rewrite this report from a player-performance POV` →
`report_delta_research`
- `redo this with newer evidence on 2026 logistics` →
`report_delta_research`
- `write a separate report on player performance trends across 2014,
2018, and 2022` → `standalone_research`
**Key pieces**
- One new internal agent — `report_rewriter` — registered with
`public=False` (hidden from `GET /agents`; direct `/submit` of
internal-only agents is rejected, and the gate is also enforced at the
submission boundary).
- A durable parent-report **context resolver** that reconstructs the
report + sources from job output/events, **authorizes the caller before
any read**, and seeds `/shared/*` files into child runs.
- New endpoint `POST /v1/jobs/async/job/{job_id}/report/edit` (per-job
ownership auth); `GET .../report` extended with `parent_job_id` /
`interaction_action` / `result_kind`.
- Chat (`POST /chat` / WebSocket) accepts `active_report_job_id`; the UI
forwards the latest completed report job id and renders the report-edit
child job through the existing report-streaming path.
- DeepAgents runtime normalizes seeded `/shared/*` parent-context files
so delta research can read `/shared/original_report.md` and
`/shared/source_summary.md` reliably.
> Rebased onto `develop` (which now includes #267); this branch is a
standalone PR. The most recent commits harden the feature for
production: the chat path works under the default `REQUIRE_AUTH=false`,
a caller-supplied `job_id` can no longer delete another job's state,
submission failures roll back cleanly, the UI consumes the report-edit
response, and report-delta research can reuse seeded parent context
without creating `/shared/shared` or malformed file records.
#### Validation
All commands run from the repo root unless noted.
**Lint / format / unit tests (backend)**
```bash
uv run ruff check .
uv run ruff format --check .
uv run pytest # full suite
# or the suites this PR touches:
uv run pytest frontends/aiq_api/tests/ \
tests/aiq_agent/agents/chat_researcher/ \
tests/aiq_agent/agents/report_rewriter/ \
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py \
tests/aiq_agent/jobs/test_runner.py
```
**UI**
```bash
cd frontends/ui
npm run lint && npm run type-check && npm run test:ci
# focused: npx vitest run src/features/chat/hooks/use-websocket-chat.spec.ts \
# src/adapters/api/websocket-client.spec.ts
```
**End-to-end (manual)** — report follow-up needs async jobs, so bring up
the full stack (PostgreSQL job store + embedded Dask scheduler/worker +
web), e.g. via `deploy/compose` or
`scripts/start_server_in_debug_mode.sh` with `deploy/.env`, then
exercise the surfaces below.
**What to test**
1. **Internal-agent gating** — `GET /v1/jobs/async/agents` does *not*
list `report_rewriter`; `POST /v1/jobs/async/submit` with
`agent_type=report_rewriter` returns `400 Agent type is internal-only:
report_rewriter`.
2. **HTTP report edit** — `POST /v1/jobs/async/job/{id}/report/edit` on
a completed report → a `report_rewriter` child job; `GET .../report` on
the child returns a revised report plus `parent_job_id`,
`interaction_action="edit"`, `result_kind="report"`. The parent report
is unchanged.
3. **Chat routing with `active_report_job_id`** — verify each semantic
route:
- **report ask**: `What are the top three takeaways from this report?`;
`Where does the report say the evidence is weak or incomplete?`
- **report cosmetic edit**: `Make this report shorter while preserving
the sources.`; `Format the key takeaways as bullets.`; `Remove the
one-table comparison section and keep the rest unchanged.`
- **report delta research**: `Rewrite this report from a
player-performance POV.`; `Redo this report with newer evidence on 2026
logistics and host-city operations.`; `Add a section on fan travel
emissions for 2026.`
- **standalone research**: `Write a separate report on player
performance trends across the 2014, 2018, and 2022 World Cups.`;
`Research the economics of Olympic host cities since 2000.`
4. **Parent-context seeding for delta research** — delta research should
read `/shared/original_report.md` and `/shared/source_summary.md`
successfully, not see `/shared/shared/`, and not fail with `string
indices must be integers` or `'str' object has no attribute 'get'` from
filesystem tools.
5. **Authorization** — works for anonymous callers under
`REQUIRE_AUTH=false`; under `REQUIRE_AUTH=true`, a non-owner is rejected
with `404` before any report content is read.
6. **Robustness / edge cases** — a colliding caller-supplied `job_id`
returns `409` and does **not** delete the existing job; whitespace-only
`input` returns `422`; report ask/edit degrade to a chat message (not an
opaque workflow error) if context resolution fails.
**Evidence** — backend suites (`frontends/aiq_api/tests` +
`tests/aiq_agent/...`) and UI specs pass; `ruff` and `tsc` clean.
Verified live against a Postgres + Dask stack with `deploy/.env`:
internal agent hidden + `/submit` rejected (400); HTTP and chat
report-edit produce a revised child report with correct lineage; chat
report-ask answers from the report only; `job_id` collision → 409 with
the victim preserved; blank input → 422; ownership-mismatch → 404 under
`REQUIRE_AUTH=true`; report-delta routing kicks off deep research with
seeded parent context.
- [x] I ran the relevant local checks or explained why they are not
applicable.
- [x] I added or updated tests for behavior changes.
- [x] I updated documentation for user-facing or contributor-facing
changes.
- [x] I confirmed this PR does not include secrets, credentials, or
internal-only data.
- [x] I certify this contribution under the Developer Certificate of
Origin (DCO) and signed my commits with `git commit -s` or an equivalent
sign-off.
#### Where should reviewers start?
Read in this order — security-sensitive paths first:
1. **`frontends/aiq_api/src/aiq_api/jobs/report_context.py`** — durable
report/source reconstruction and `resolve_authorized_report_context()`,
which **authorizes the caller before any read** and seeds `/shared/*`
for child runs. The core security boundary.
2. **`frontends/aiq_api/src/aiq_api/routes/jobs.py`** — the
`report/edit` endpoint, the `public` agent filter on `/agents` +
`/submit`, the new `JobReportResponse` fields, and the request
validators (blank-input → 422, `job_id` collision → 409).
3. **`frontends/aiq_api/src/aiq_api/jobs/submit.py`** —
`submit_agent_job()` ownership recording, the
`JobIdConflictError`/`InternalAgentError` gates, and the rollback that
only deletes state this submission created.
4. **`src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py`**
+ **`agent.py`** + **`register.py`** — the semantic route classifier
(`report_ask` / `report_cosmetic_edit` / `report_delta_research` /
`standalone_research`), the bounded tool-free report-ask path, and how
each entry point resolves the principal (same contract as the HTTP
routes).
5. **`src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`** —
route-aware `/shared/*` file seeding for delta research.
6. **`src/aiq_agent/agents/report_rewriter/`** — the single new internal
agent (a bounded, tool-less single-LLM rewrite).
7. **`frontends/ui/src/features/chat/hooks/use-websocket-chat.ts`** +
`adapters/api/websocket-client.ts` — forwards `active_report_job_id` and
routes the report-edit child job through the existing report-streaming
path.
Tests mirror these: `test_report_context.py`, `test_report_edit.py`,
`test_submit_collision.py`, `test_submit_internal_agent.py`,
`test_job_access.py`, `test_agent_registry_visibility.py`,
`test_intent_classifier.py`, `test_deepagents_runtime.py` (backend), and
`use-websocket-chat.spec.ts` (UI).
#### Related Issues
- Relates to #267 (deep-researcher structured output / customization —
merged into `develop`).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Report editing and follow-up workflows to revise completed reports
* Report Q&A capability to ask questions about existing reports
* Conversation-scoped job tracking for multi-turn interactions
* Agent visibility controls restricting certain agents from public
submission
* **Improvements**
* Enhanced input validation and error responses for job submissions
* Expanded REST API documentation for report operations
* Better job ID conflict detection and error handling
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5 tasks
This was referenced Jul 8, 2026
This was referenced Jul 28, 2026
3 tasks
This was referenced Aug 4, 2026
This was referenced Aug 11, 2026
This was referenced Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds structured-output deep research with advisory source routing, batched researcher workers, writer-only final synthesis, and new news / prediction-market sources.
Changes
SourceRoutingPlan,ResearchPlan,ResearchNotes, answer strategy, research queries, and evidence judgments.run_research_batch, source-tool batching/throttling, compact verified-source lists from ResearchNotes, and stricter/shared/output.mdfinal-answer extraction.config_domain_routing_and_skills.yml, newdeep_research_domain_catalog.yml,source_router_llm,writer_llm, routing enablement, andconcurrency limits.
reports.
config handling.
Summary by CodeRabbit