fix: resolve VDR 2.2 release-readiness issues - #429
Conversation
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe PR hardens sensitive-content logging, strengthens citation verification and artifact propagation, adds explicit conversation-ID handling, improves guardrail response compatibility, and updates shipped model, deployment, backup, observability, and CLI documentation. ChangesSecurity and workflow hardening
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/customization/swapping-models.md`:
- Around line 15-23: Update docs/source/customization/configuration-reference.md
at line 709 to state that configs/config_frontier_models.yml requires
TAVILY_API_KEY for its enabled Tavily tools, while keeping SERPER_API_KEY
associated only with the commented-out paper-search opt-in. No direct change is
required in docs/source/customization/swapping-models.md at line 15-23; it is
the related profile reference.
In `@docs/source/deployment/observability.md`:
- Around line 285-288: Revise the observability guidance around
VerboseTraceCallback to scope the metadata-only guarantee exclusively to that
console callback, clarifying that Phoenix and other exporters may still receive
raw prompts, tool arguments, tool results, or model responses. Explicitly
require separate retention and access controls for enabled source adapters,
external providers, and exporters.
In `@docs/source/deployment/production.md`:
- Around line 49-59: Update the backup command instructions around the
PostgreSQL dump examples to create archives in a restricted directory outside
the repository, applying umask 077 or equivalent before writing them. Add
concise guidance to encrypt the dumps and enforce access controls before copying
them to backup storage.
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Line 809: Update the job execution flow around
context_state.user_id.set(owner_user_id) to retain the returned reset token,
then reset the user context in the existing finally block alongside
conversation_id. Ensure the reset always runs after the job, including failures
and jobs without an owner_user_id, so reused workers cannot retain prior owner
identity.
In `@frontends/ui/README.md`:
- Line 196: Update the Docker build command in the README to work when executed
from frontends/ui/ by referencing the repository-root Dockerfile and using the
repository root as the build context, while preserving the existing image tag.
In `@src/aiq_agent/agents/chat_researcher/agent.py`:
- Line 183: Update the error logging in the deep research tools validation flow
to pass error_msg directly to logger.error instead of wrapping it with
log_content_metadata. Keep the existing validation failure context and
user-facing AIMessage behavior unchanged.
In `@src/aiq_agent/agents/deep_researcher/custom_middleware.py`:
- Around line 958-963: Replace the raw tool_name argument in the retry failure
log with a safe metadata representation, such as the existing
log_content_metadata helper, while preserving the attempt, retry limit, error
type, and error detail fields. Add a retry-path test using a secret marker as
the model-provided tool name and assert that the marker is absent from emitted
logs.
In `@src/aiq_agent/agents/deep_researcher/register.py`:
- Around line 302-307: The catch-all error handlers omit traceback information
while retaining redacted exception details. In
src/aiq_agent/agents/deep_researcher/register.py lines 302-307, update the
generic except Exception logger.error call to include exc_info=True; make the
same change in src/aiq_agent/agents/chat_researcher/agent.py lines 393-397.
Preserve the existing redacted message and metadata arguments.
In `@src/aiq_agent/agents/report_rewriter/agent.py`:
- Around line 197-200: Compute _effective_parent_sources(original_report,
parent_context) once in run and pass the resulting parent source allowlist into
rewrite_report, reusing it for _verified_cited_urls instead of recalculating it
inside rewrite_report.
- Around line 202-204: Update the callback loop in the report rewriter agent to
break immediately after invoking the first callback that exposes
emit_final_report, matching the behavior of ShallowResearcherAgent.run and
DeepResearcherAgent.run and preventing duplicate final-report and citation
events.
In `@src/aiq_agent/agents/shallow_researcher/agent.py`:
- Around line 408-421: Update the final verification flow in the agent method
containing verify_citations so content is replaced with
final_verification.verified_report before deriving cited URLs, emitting via
emit_final_report, and writing the final message. Preserve the existing callback
behavior while ensuring the published report and cited URLs come from the same
final verification result.
In `@tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py`:
- Around line 1754-1759: Update the test around
SourceRoutingPersistenceMiddleware.after_agent() to capture the raised
RuntimeError with pytest.raises and assert its message does not contain
error_detail, while retaining the existing caplog assertions that verify the
backend detail is logged.
In `@tests/aiq_agent/jobs/test_runner.py`:
- Around line 494-495: Update the test around mock_job_store.submit_job to
document that job_args[14] is run_agent_job’s parent_conversation_id, and bind
the submitted positional arguments against run_agent_job’s signature before
asserting the expected value so argument reordering cannot silently alter the
mapping.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 27266ab4-bd2d-49ee-98e9-76b8418f2010
📒 Files selected for processing (59)
.secrets.baselineREADME.mdconfigs/config_frontier_models.ymlconfigs/config_web_default_guardrails.ymldeploy/.env.exampledeploy/compose/docker-compose.yamldocs/source/architecture/agents/index.mddocs/source/contributing/testing.mddocs/source/customization/configuration-reference.mddocs/source/customization/guardrails.mddocs/source/customization/prompts.mddocs/source/customization/swapping-models.mddocs/source/deployment/observability.mddocs/source/deployment/production.mddocs/source/evaluation/benchmarks/deep-research-bench.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/examples/hybrid-frontier-model.mddocs/source/examples/index.mddocs/source/get-started/quick-start.mddocs/source/index.mddocs/source/integration/agent-skills.mddocs/source/resources/faq.mdfrontends/aiq_api/src/aiq_api/jobs/callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/event_store.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pyfrontends/aiq_api/src/aiq_api/jobs/submit.pyfrontends/aiq_api/src/aiq_api/routes/jobs.pyfrontends/aiq_api/tests/test_job_submit_data_sources.pyfrontends/benchmarks/deepresearch_bench/README.mdfrontends/ui/README.mdskills/aiq-research/scripts/aiq.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pysrc/aiq_agent/agents/chat_researcher/register.pysrc/aiq_agent/agents/clarifier/agent.pysrc/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/common/callbacks.pysrc/aiq_agent/common/citation_verification.pysrc/aiq_agent/common/logging_utils.pysrc/aiq_agent/guardrails/interface/middleware.pysrc/aiq_agent/guardrails/workflow/middleware.pytests/aiq_agent/agents/chat_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/agents/report_rewriter/test_agent.pytests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/common/test_callbacks.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/common/test_logging_utils.pytests/aiq_agent/guardrails/test_guardrails_interface.pytests/aiq_agent/guardrails/test_guardrails_registration.pytests/aiq_agent/guardrails/test_workflow_guardrails.pytests/aiq_agent/jobs/test_runner.pytests/scripts/test_aiq_research_helper.py
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Run Harbor skill eval
- GitHub Check: Pytest and Coverage
🧰 Additional context used
📓 Path-based instructions (17)
**/*
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat eachsources/*package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.
Files:
docs/source/architecture/agents/index.mdfrontends/benchmarks/deepresearch_bench/README.mddocs/source/examples/index.mdconfigs/config_frontier_models.ymldocs/source/examples/full-pipeline-llamaindex.mddocs/source/customization/configuration-reference.mddocs/source/integration/agent-skills.mddocs/source/customization/prompts.mdsrc/aiq_agent/common/logging_utils.pydocs/source/contributing/testing.mdfrontends/ui/README.mddocs/source/evaluation/benchmarks/deep-research-bench.mdfrontends/aiq_api/tests/test_job_submit_data_sources.pydocs/source/examples/hybrid-frontier-model.mddeploy/compose/docker-compose.yamldocs/source/index.mdsrc/aiq_agent/agents/deep_researcher/register.pyskills/aiq-research/scripts/aiq.pydocs/source/customization/guardrails.mdsrc/aiq_agent/agents/clarifier/agent.pydocs/source/customization/swapping-models.mddocs/source/resources/faq.mdconfigs/config_web_default_guardrails.ymlsrc/aiq_agent/agents/report_rewriter/agent.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/agents/report_rewriter/test_agent.pydocs/source/get-started/quick-start.mdtests/aiq_agent/common/test_logging_utils.pytests/aiq_agent/agents/chat_researcher/test_agent.pysrc/aiq_agent/guardrails/interface/middleware.pyREADME.mddocs/source/deployment/production.mdsrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/agents/chat_researcher/register.pydocs/source/deployment/observability.mdfrontends/aiq_api/src/aiq_api/routes/jobs.pytests/aiq_agent/agents/shallow_researcher/test_agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/common/test_callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/event_store.pyfrontends/aiq_api/src/aiq_api/jobs/callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/submit.pytests/aiq_agent/guardrails/test_workflow_guardrails.pytests/aiq_agent/guardrails/test_guardrails_interface.pysrc/aiq_agent/guardrails/workflow/middleware.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/scripts/test_aiq_research_helper.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/common/callbacks.pysrc/aiq_agent/common/citation_verification.pytests/aiq_agent/guardrails/test_guardrails_registration.py
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, and resolve API keys at runtime.
Files:
docs/source/architecture/agents/index.mdfrontends/benchmarks/deepresearch_bench/README.mddocs/source/examples/index.mdconfigs/config_frontier_models.ymldocs/source/examples/full-pipeline-llamaindex.mddocs/source/customization/configuration-reference.mddocs/source/integration/agent-skills.mddocs/source/customization/prompts.mdsrc/aiq_agent/common/logging_utils.pydocs/source/contributing/testing.mdfrontends/ui/README.mddocs/source/evaluation/benchmarks/deep-research-bench.mdfrontends/aiq_api/tests/test_job_submit_data_sources.pydocs/source/examples/hybrid-frontier-model.mddeploy/compose/docker-compose.yamldocs/source/index.mdsrc/aiq_agent/agents/deep_researcher/register.pyskills/aiq-research/scripts/aiq.pydocs/source/customization/guardrails.mdsrc/aiq_agent/agents/clarifier/agent.pydocs/source/customization/swapping-models.mddocs/source/resources/faq.mdconfigs/config_web_default_guardrails.ymlsrc/aiq_agent/agents/report_rewriter/agent.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/agents/report_rewriter/test_agent.pydocs/source/get-started/quick-start.mdtests/aiq_agent/common/test_logging_utils.pytests/aiq_agent/agents/chat_researcher/test_agent.pysrc/aiq_agent/guardrails/interface/middleware.pyREADME.mddocs/source/deployment/production.mdsrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/agents/chat_researcher/register.pydocs/source/deployment/observability.mdfrontends/aiq_api/src/aiq_api/routes/jobs.pytests/aiq_agent/agents/shallow_researcher/test_agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/common/test_callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/event_store.pyfrontends/aiq_api/src/aiq_api/jobs/callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/submit.pytests/aiq_agent/guardrails/test_workflow_guardrails.pytests/aiq_agent/guardrails/test_guardrails_interface.pysrc/aiq_agent/guardrails/workflow/middleware.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/scripts/test_aiq_research_helper.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/common/callbacks.pysrc/aiq_agent/common/citation_verification.pytests/aiq_agent/guardrails/test_guardrails_registration.py
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.
Files:
docs/source/architecture/agents/index.mddocs/source/examples/index.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/customization/configuration-reference.mddocs/source/integration/agent-skills.mddocs/source/customization/prompts.mddocs/source/contributing/testing.mddocs/source/evaluation/benchmarks/deep-research-bench.mddocs/source/examples/hybrid-frontier-model.mddocs/source/index.mddocs/source/customization/guardrails.mddocs/source/customization/swapping-models.mddocs/source/resources/faq.mddocs/source/get-started/quick-start.mddocs/source/deployment/production.mddocs/source/deployment/observability.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/architecture/agents/index.mddocs/source/examples/index.mddocs/source/examples/full-pipeline-llamaindex.mddocs/source/customization/configuration-reference.mddocs/source/integration/agent-skills.mddocs/source/customization/prompts.mddocs/source/contributing/testing.mddocs/source/evaluation/benchmarks/deep-research-bench.mddocs/source/examples/hybrid-frontier-model.mddocs/source/index.mddocs/source/customization/guardrails.mddocs/source/customization/swapping-models.mddocs/source/resources/faq.mddocs/source/get-started/quick-start.mdREADME.mddocs/source/deployment/production.mddocs/source/deployment/observability.md
**/*.{yaml,yml}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
For deployment changes, run the relevant Helm or Compose validation and describe the environment used.
Files:
configs/config_frontier_models.ymldeploy/compose/docker-compose.yamlconfigs/config_web_default_guardrails.yml
configs/**/*.{yml,yaml}
📄 CodeRabbit inference engine (AGENTS.md)
Use
_typenames that come from the registered configuration class.
Files:
configs/config_frontier_models.ymlconfigs/config_web_default_guardrails.yml
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_frontier_models.ymldeploy/compose/docker-compose.yamlconfigs/config_web_default_guardrails.yml
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*.py: Runuv run ruff check .anduv run ruff format --check .for root Python changes.
Runuv run pytestfor root project Python changes.
Files:
src/aiq_agent/common/logging_utils.pyfrontends/aiq_api/tests/test_job_submit_data_sources.pysrc/aiq_agent/agents/deep_researcher/register.pyskills/aiq-research/scripts/aiq.pysrc/aiq_agent/agents/clarifier/agent.pysrc/aiq_agent/agents/report_rewriter/agent.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/agents/report_rewriter/test_agent.pytests/aiq_agent/common/test_logging_utils.pytests/aiq_agent/agents/chat_researcher/test_agent.pysrc/aiq_agent/guardrails/interface/middleware.pysrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/agents/chat_researcher/register.pyfrontends/aiq_api/src/aiq_api/routes/jobs.pytests/aiq_agent/agents/shallow_researcher/test_agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/common/test_callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/event_store.pyfrontends/aiq_api/src/aiq_api/jobs/callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/submit.pytests/aiq_agent/guardrails/test_workflow_guardrails.pytests/aiq_agent/guardrails/test_guardrails_interface.pysrc/aiq_agent/guardrails/workflow/middleware.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/scripts/test_aiq_research_helper.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/common/callbacks.pysrc/aiq_agent/common/citation_verification.pytests/aiq_agent/guardrails/test_guardrails_registration.py
**/*.{py,pyi}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{py,pyi}: Format and lint Python code with Ruff using line length 120, Python 3.11 targeting, rules E, F, W, I, PL, and UP, with single-line imports; do not reformat unrelated code.
Missing-secret paths must degrade gracefully by stubbing or skipping rather than crashing or leaking secrets.
Files:
src/aiq_agent/common/logging_utils.pyfrontends/aiq_api/tests/test_job_submit_data_sources.pysrc/aiq_agent/agents/deep_researcher/register.pyskills/aiq-research/scripts/aiq.pysrc/aiq_agent/agents/clarifier/agent.pysrc/aiq_agent/agents/report_rewriter/agent.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/agents/report_rewriter/test_agent.pytests/aiq_agent/common/test_logging_utils.pytests/aiq_agent/agents/chat_researcher/test_agent.pysrc/aiq_agent/guardrails/interface/middleware.pysrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/agents/chat_researcher/register.pyfrontends/aiq_api/src/aiq_api/routes/jobs.pytests/aiq_agent/agents/shallow_researcher/test_agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/common/test_callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/event_store.pyfrontends/aiq_api/src/aiq_api/jobs/callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/submit.pytests/aiq_agent/guardrails/test_workflow_guardrails.pytests/aiq_agent/guardrails/test_guardrails_interface.pysrc/aiq_agent/guardrails/workflow/middleware.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/scripts/test_aiq_research_helper.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/common/callbacks.pysrc/aiq_agent/common/citation_verification.pytests/aiq_agent/guardrails/test_guardrails_registration.py
**/*.{py,pyi,js,jsx,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Never print or log secret values, including in tool output or error messages.
Files:
src/aiq_agent/common/logging_utils.pyfrontends/aiq_api/tests/test_job_submit_data_sources.pysrc/aiq_agent/agents/deep_researcher/register.pyskills/aiq-research/scripts/aiq.pysrc/aiq_agent/agents/clarifier/agent.pysrc/aiq_agent/agents/report_rewriter/agent.pytests/aiq_agent/common/test_citation_verification.pytests/aiq_agent/agents/report_rewriter/test_agent.pytests/aiq_agent/common/test_logging_utils.pytests/aiq_agent/agents/chat_researcher/test_agent.pysrc/aiq_agent/guardrails/interface/middleware.pysrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/agents/chat_researcher/register.pyfrontends/aiq_api/src/aiq_api/routes/jobs.pytests/aiq_agent/agents/shallow_researcher/test_agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/common/test_callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/event_store.pyfrontends/aiq_api/src/aiq_api/jobs/callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/submit.pytests/aiq_agent/guardrails/test_workflow_guardrails.pytests/aiq_agent/guardrails/test_guardrails_interface.pysrc/aiq_agent/guardrails/workflow/middleware.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/scripts/test_aiq_research_helper.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/common/callbacks.pysrc/aiq_agent/common/citation_verification.pytests/aiq_agent/guardrails/test_guardrails_registration.py
src/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/**/*.py: Respect authenticated data sources by honoringrequires_auth, passing through per-user tokens, and using backend token validators; apply owner guardrails before loading protected report or artifact context into an agent.
Do not weaken or bypassAuthMiddleware, authentication validators, or authentication gating without prior design discussion.
Files:
src/aiq_agent/common/logging_utils.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/clarifier/agent.pysrc/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/guardrails/interface/middleware.pysrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/agents/chat_researcher/register.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pysrc/aiq_agent/guardrails/workflow/middleware.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/common/callbacks.pysrc/aiq_agent/common/citation_verification.py
docs/source/contributing/**/*
📄 CodeRabbit inference engine (docs/source/contributing/code-style.md)
Run
pre-commit run --all-files; its checks include Ruff fixes and formatting, root and MCP lock checks, secret detection, notebook output clearing, and Markdown link checking.
Files:
docs/source/contributing/testing.md
frontends/ui/**/*
⚙️ CodeRabbit configuration file
frontends/ui/**/*: Review UI changes for strict TypeScript behavior, API contract alignment, auth/session handling, accessible controls,
resilient loading and error states, and report/chat state consistency. Prefer existing UI patterns and require tests
for changed user-visible workflows.
Files:
frontends/ui/README.md
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/clarifier/agent.pysrc/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/register.pysrc/aiq_agent/agents/chat_researcher/register.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/chat_researcher/nodes/intent_classifier.pysrc/aiq_agent/agents/chat_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/agent.py
skills/aiq-research/scripts/aiq.py
📄 CodeRabbit inference engine (skills/aiq-research/SKILL.md)
skills/aiq-research/scripts/aiq.py: Use Python 3.11+ and Python standard-library HTTP modules; the helper script must not require third-party Python packages.
When polling requires an execution method with escalated permissions, request explicit user approval first and explain why; tell the user when deep research is running in the background.
Files:
skills/aiq-research/scripts/aiq.py
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}
⚙️ CodeRabbit configuration file
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.
Files:
skills/aiq-research/scripts/aiq.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/routes/jobs.pyfrontends/aiq_api/src/aiq_api/jobs/event_store.pyfrontends/aiq_api/src/aiq_api/jobs/callbacks.pyfrontends/aiq_api/src/aiq_api/jobs/submit.pyfrontends/aiq_api/src/aiq_api/jobs/runner.py
🧠 Learnings (2)
📚 Learning: 2026-06-14T17:49:00.640Z
Learnt from: torkian
Repo: NVIDIA-AI-Blueprints/aiq PR: 273
File: frontends/aiq_api/tests/test_sse_reconnect_cursor.py:384-401
Timestamp: 2026-06-14T17:49:00.640Z
Learning: When using `unittest.mock.patch` for code that imports dependencies inside functions/generators (e.g., inside `aiq_api.routes.jobs`), don’t patch via an attribute that doesn’t exist on the consuming module. If the generator does `from ..jobs.event_store import EventStore` inside the generator body, then `aiq_api.routes.jobs` will not have an `EventStore` attribute; patch the source class/method in its defining module instead (e.g., `aiq_api.jobs.event_store.EventStore.get_events_async`). Patching `aiq_api.routes.jobs.EventStore...` would raise `AttributeError` because that symbol is not present at module scope.
Applied to files:
frontends/aiq_api/tests/test_job_submit_data_sources.py
📚 Learning: 2026-07-06T23:55:42.908Z
Learnt from: cdgamarose-nv
Repo: NVIDIA-AI-Blueprints/aiq PR: 311
File: src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2:74-81
Timestamp: 2026-07-06T23:55:42.908Z
Learning: In agent test files (e.g., tests/aiq_agent/agents/*/test_agent.py), avoid brittle assertions that match exact substrings from prompt template files (such as *.j2 prompt wording). Prompt wording can change frequently, so instead assert structural/behavioral properties (e.g., that the prompt builder is called, that required sections/fields are present via stable markers, that the model output/agent behavior conforms to an expected schema, or that key actions are taken) rather than matching literal prompt text.
Applied to files:
tests/aiq_agent/agents/report_rewriter/test_agent.pytests/aiq_agent/agents/chat_researcher/test_agent.pytests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_agent.py
🪛 ast-grep (0.45.1)
tests/scripts/test_aiq_research_helper.py
[error] 51-56: Command coming from incoming request
Context: subprocess.run(
[sys.executable, str(_SCRIPT), help_flag],
check=False,
capture_output=True,
text=True,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
tests/aiq_agent/guardrails/test_guardrails_registration.py
[warning] 71-71: Regex pattern passed to re is built from a non-literal (variable, call, concatenation, or f-string) value. If that value is attacker-controlled it can introduce a malicious pattern with catastrophic backtracking (ReDoS). Use a hardcoded literal pattern, or validate/escape untrusted input with re.escape() and bound the regex complexity before compiling.
Context: re.search(pattern, prompt)
Note: [CWE-1333] Inefficient Regular Expression Complexity.
(redos-non-literal-regex-python)
[warning] 72-72: Regex pattern passed to re is built from a non-literal (variable, call, concatenation, or f-string) value. If that value is attacker-controlled it can introduce a malicious pattern with catastrophic backtracking (ReDoS). Use a hardcoded literal pattern, or validate/escape untrusted input with re.escape() and bound the regex complexity before compiling.
Context: re.search(pattern, prompt)
Note: [CWE-1333] Inefficient Regular Expression Complexity.
(redos-non-literal-regex-python)
🪛 LanguageTool
docs/source/examples/hybrid-frontier-model.md
[style] ~9-~9: Consider a different adjective to strengthen your wording.
Context: ...with GPT Sol/Luna for clarification and deep research. ## Shipped Configuration | ...
(DEEP_PROFOUND)
docs/source/index.md
[style] ~8-~8: Consider a different adjective to strengthen your wording.
Context: ...int An NVIDIA blueprint for AI-powered deep research, built on the NeMo Agent Toolk...
(DEEP_PROFOUND)
README.md
[style] ~297-~297: Consider a different adjective to strengthen your wording.
Context: ...amaIndex profile using GPT Sol/Luna for deep research and Super for intent/shallow. ...
(DEEP_PROFOUND)
🔇 Additional comments (61)
src/aiq_agent/common/logging_utils.py (1)
4-4: LGTM!Also applies to: 13-24
src/aiq_agent/common/callbacks.py (1)
23-25: LGTM!Also applies to: 80-111, 145-145, 170-170, 183-183, 194-207, 226-251
src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py (1)
33-33: LGTM!Also applies to: 225-248, 278-282
src/aiq_agent/agents/chat_researcher/register.py (1)
34-34: LGTM!Also applies to: 327-331, 646-647
src/aiq_agent/agents/clarifier/agent.py (1)
66-66: LGTM!Also applies to: 262-262, 668-668
frontends/aiq_api/src/aiq_api/jobs/event_store.py (1)
31-32: LGTM!Also applies to: 206-206, 233-233, 448-454, 513-519
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
40-40: LGTM!Also applies to: 503-514, 533-538, 664-665, 782-790, 849-849, 1101-1109, 1144-1146
tests/aiq_agent/agents/chat_researcher/test_agent.py (1)
33-33: LGTM!Also applies to: 221-248
tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py (1)
61-61: LGTM!Also applies to: 1366-1380, 1922-1938
tests/aiq_agent/common/test_callbacks.py (1)
28-28: LGTM!Also applies to: 202-218, 277-277, 596-635
tests/aiq_agent/common/test_citation_verification.py (1)
18-19: LGTM!Also applies to: 660-684
tests/aiq_agent/common/test_logging_utils.py (1)
6-6: LGTM!Also applies to: 22-31
frontends/aiq_api/src/aiq_api/jobs/submit.py (1)
173-173: LGTM!Also applies to: 196-198, 329-336, 462-462
frontends/aiq_api/src/aiq_api/routes/jobs.py (2)
43-43: LGTM!Also applies to: 728-728, 812-812
1681-1687: LGTM!src/aiq_agent/guardrails/interface/middleware.py (1)
30-47: LGTM!Also applies to: 58-65
src/aiq_agent/guardrails/workflow/middleware.py (1)
90-91: LGTM!Also applies to: 116-131, 232-243
configs/config_web_default_guardrails.yml (1)
178-178: LGTM!frontends/aiq_api/tests/test_job_submit_data_sources.py (1)
191-204: LGTM!tests/aiq_agent/guardrails/test_guardrails_interface.py (1)
18-18: LGTM!Also applies to: 27-29, 104-132
tests/aiq_agent/guardrails/test_guardrails_registration.py (1)
18-18: LGTM!Also applies to: 54-73
tests/aiq_agent/guardrails/test_workflow_guardrails.py (1)
39-57: LGTM!Also applies to: 113-119, 354-372, 568-613, 641-641, 675-675
docs/source/integration/agent-skills.md (1)
200-214: LGTM!skills/aiq-research/scripts/aiq.py (1)
748-750: LGTM!tests/scripts/test_aiq_research_helper.py (1)
4-8: LGTM!Also applies to: 20-20, 47-61
src/aiq_agent/common/citation_verification.py (3)
109-115: LGTM!Also applies to: 196-228, 352-357, 640-642, 699-699, 820-820, 913-913, 980-984
1103-1104: LGTM!Also applies to: 1164-1186, 1199-1207, 1241-1243
1329-1329: LGTM!Also applies to: 1389-1396, 1437-1439, 1461-1461, 1497-1508
src/aiq_agent/agents/deep_researcher/agent.py (2)
33-39: LGTM!Also applies to: 302-302, 359-373
399-415: LGTM!Also applies to: 428-439
frontends/aiq_api/src/aiq_api/jobs/callbacks.py (2)
131-144: LGTM!Also applies to: 465-477
394-414: 🗄️ Data Integrity & IntegrationNo change needed. Final-report processing clears
sources_citedand replaces it with the final report’scited_urls, so earliercitation_useevents do not remain in aggregated job metadata.> Likely an incorrect or invalid review comment.src/aiq_agent/agents/deep_researcher/custom_middleware.py (1)
45-45: LGTM!Also applies to: 726-740, 1087-1089, 1254-1259, 1368-1373
tests/aiq_agent/agents/report_rewriter/test_agent.py (1)
8-8: LGTM!Also applies to: 37-38, 57-57, 95-96, 126-129
src/aiq_agent/agents/shallow_researcher/register.py (1)
31-31: LGTM!Also applies to: 168-173, 187-192
src/aiq_agent/agents/report_rewriter/agent.py (1)
86-96: LGTM!src/aiq_agent/agents/chat_researcher/agent.py (1)
49-49: LGTM!Also applies to: 220-223, 250-254, 265-269, 322-326, 382-390, 611-611
src/aiq_agent/agents/deep_researcher/register.py (1)
37-37: LGTM!src/aiq_agent/agents/shallow_researcher/agent.py (1)
48-48: LGTM!Also applies to: 220-223, 260-264, 322-324
tests/aiq_agent/agents/shallow_researcher/test_agent.py (1)
569-603: LGTM!Also applies to: 634-638, 649-649
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
20-20: LGTM!Also applies to: 49-53, 1586-1613, 1806-1810, 2286-2355, 2383-2433, 2435-2513
tests/aiq_agent/jobs/test_runner.py (1)
466-493: LGTM!Also applies to: 653-775, 894-894, 935-936, 971-990, 1005-1007, 1585-1627, 1738-1748, 1915-1954, 2067-2076, 2097-2150, 2169-2252
.secrets.baseline (1)
136-136: LGTM!Also applies to: 358-358
README.md (1)
55-55: LGTM!Also applies to: 119-123, 297-297, 450-450, 502-502
configs/config_frontier_models.yml (1)
1-8: LGTM!deploy/.env.example (1)
19-19: LGTM!docs/source/examples/hybrid-frontier-model.md (2)
8-30: LGTM!Also applies to: 34-47, 57-59
49-53: 🗄️ Data Integrity & IntegrationVerify that
BACKEND_CONFIGreaches Compose interpolation.The document tells users to set
BACKEND_CONFIGindeploy/.env. Indeploy/compose/docker-compose.yamlLines 31-51,CONFIG_FILE=${BACKEND_CONFIG:-...}is interpolated by Compose, whileenv_file: ../.envinjects variables into the container. Unless the standard startup command passesdeploy/.envwith--env-fileor exportsBACKEND_CONFIG, Compose uses the default LlamaIndex configuration instead ofconfig_frontier_models.yml. Make that requirement explicit, or verify that the linked procedure already supplies it.docs/source/examples/index.md (1)
17-17: LGTM!docs/source/get-started/quick-start.md (1)
94-95: LGTM!docs/source/index.md (1)
8-12: LGTM!docs/source/resources/faq.md (1)
18-22: LGTM!Also applies to: 33-33
frontends/benchmarks/deepresearch_bench/README.md (1)
65-65: LGTM!deploy/compose/docker-compose.yaml (1)
26-26: LGTM!docs/source/architecture/agents/index.md (1)
15-15: LGTM!docs/source/contributing/testing.md (1)
27-27: LGTM!docs/source/customization/guardrails.md (1)
10-16: LGTM!Also applies to: 165-167
docs/source/customization/prompts.md (1)
315-323: LGTM!docs/source/deployment/observability.md (1)
54-63: LGTM!docs/source/evaluation/benchmarks/deep-research-bench.md (1)
68-68: LGTM!docs/source/examples/full-pipeline-llamaindex.md (1)
239-247: 🗄️ Data Integrity & IntegrationThe documented
conversation-idcontract is correct. The route propagates the header through asynchronous submission, and retrieval uses it as the collection withcollection_nameas the fallback.
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/source/deployment/production.md (1)
73-107: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winMake the restore check fail closed and assert the expected schema.
The block has no shell fail-fast handling. If
pg_restorefails, cleanup can determine the final exit status. The\dtcommands only list relations and do not assert the required tables in either restored database. Addset -euo pipefail, anEXITcleanup trap that preserves the original status,ON_ERROR_STOP=1for inspection, and explicit assertions for tables such asjob_info,job_events, and the checkpoint tables.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/deployment/production.md` around lines 73 - 107, Update the restore verification block to use fail-fast shell settings with set -euo pipefail and an EXIT cleanup trap that removes both disposable databases while preserving the original command status. Add ON_ERROR_STOP=1 to the inspection commands and replace \dt-only listing with explicit checks that required tables, including job_info, job_events, and the checkpoint tables, exist in each restored database.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 53-64: Update the backup script around the pg_dump commands to
enable set -euo pipefail, write each dump to a unique temporary file, and use
unique final archive names that avoid replacing prior backups. Ensure both dumps
complete successfully before moving either temporary file into its final
destination, while preserving the existing AIQ_BACKUP_DIR validation and
restrictive permissions.
---
Outside diff comments:
In `@docs/source/deployment/production.md`:
- Around line 73-107: Update the restore verification block to use fail-fast
shell settings with set -euo pipefail and an EXIT cleanup trap that removes both
disposable databases while preserving the original command status. Add
ON_ERROR_STOP=1 to the inspection commands and replace \dt-only listing with
explicit checks that required tables, including job_info, job_events, and the
checkpoint tables, exist in each restored database.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: b47b1723-4dc3-40fc-a72c-24b809efb033
📒 Files selected for processing (11)
docs/source/customization/configuration-reference.mddocs/source/deployment/observability.mddocs/source/deployment/production.mdfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/agents/report_rewriter/test_agent.pytests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
- GitHub Check: Run Harbor skill eval
- GitHub Check: Pytest and Coverage
- GitHub Check: Lint and Hooks
🧰 Additional context used
📓 Path-based instructions (10)
**/*
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat eachsources/*package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.
Files:
docs/source/customization/configuration-reference.mddocs/source/deployment/observability.mdsrc/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pydocs/source/deployment/production.mdtests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pytests/aiq_agent/agents/report_rewriter/test_agent.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/jobs/test_runner.py
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, and resolve API keys at runtime.
Files:
docs/source/customization/configuration-reference.mddocs/source/deployment/observability.mdsrc/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pydocs/source/deployment/production.mdtests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pytests/aiq_agent/agents/report_rewriter/test_agent.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/jobs/test_runner.py
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.
Files:
docs/source/customization/configuration-reference.mddocs/source/deployment/observability.mddocs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/customization/configuration-reference.mddocs/source/deployment/observability.mddocs/source/deployment/production.md
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*.py: Runuv run ruff check .anduv run ruff format --check .for root Python changes.
Runuv run pytestfor root project Python changes.
Files:
src/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pytests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pytests/aiq_agent/agents/report_rewriter/test_agent.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/jobs/test_runner.py
**/*.{py,pyi}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{py,pyi}: Format and lint Python code with Ruff using line length 120, Python 3.11 targeting, rules E, F, W, I, PL, and UP, with single-line imports; do not reformat unrelated code.
Missing-secret paths must degrade gracefully by stubbing or skipping rather than crashing or leaking secrets.
Files:
src/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pytests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pytests/aiq_agent/agents/report_rewriter/test_agent.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/jobs/test_runner.py
**/*.{py,pyi,js,jsx,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Never print or log secret values, including in tool output or error messages.
Files:
src/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pytests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pytests/aiq_agent/agents/report_rewriter/test_agent.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/jobs/test_runner.py
src/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/**/*.py: Respect authenticated data sources by honoringrequires_auth, passing through per-user tokens, and using backend token validators; apply owner guardrails before loading protected report or artifact context into an agent.
Do not weaken or bypassAuthMiddleware, authentication validators, or authentication gating without prior design discussion.
Files:
src/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/report_rewriter/agent.pysrc/aiq_agent/agents/shallow_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
🧠 Learnings (2)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T05:43:32.564Z
Learning: Refer to each benchmark's README for benchmark-specific testing details.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T05:43:32.564Z
Learning: Use the customization guide when adding evaluation harnesses.
📚 Learning: 2026-07-06T23:55:42.908Z
Learnt from: cdgamarose-nv
Repo: NVIDIA-AI-Blueprints/aiq PR: 311
File: src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2:74-81
Timestamp: 2026-07-06T23:55:42.908Z
Learning: In agent test files (e.g., tests/aiq_agent/agents/*/test_agent.py), avoid brittle assertions that match exact substrings from prompt template files (such as *.j2 prompt wording). Prompt wording can change frequently, so instead assert structural/behavioral properties (e.g., that the prompt builder is called, that required sections/fields are present via stable markers, that the model output/agent behavior conforms to an expected schema, or that key actions are taken) rather than matching literal prompt text.
Applied to files:
tests/aiq_agent/agents/shallow_researcher/test_agent.pytests/aiq_agent/agents/report_rewriter/test_agent.py
🔇 Additional comments (12)
src/aiq_agent/agents/report_rewriter/agent.py (1)
205-205: LGTM!src/aiq_agent/agents/shallow_researcher/agent.py (1)
220-224: LGTM!Also applies to: 260-264, 321-325, 408-422
tests/aiq_agent/agents/shallow_researcher/test_agent.py (1)
755-790: LGTM!tests/aiq_agent/jobs/test_runner.py (1)
66-66: LGTM!Also applies to: 496-499, 667-788
tests/aiq_agent/agents/report_rewriter/test_agent.py (1)
38-39: LGTM!Also applies to: 59-59
docs/source/deployment/production.md (1)
54-56: Reject unsafeAIQ_BACKUP_DIRvalues instead of only documenting the requirement.The parameter expansion only checks that the variable is set.
AIQ_BACKUP_DIR=./backupsor a path inside the repository still passes and can write sensitive dumps into the working tree. Resolve the path and fail unless it is absolute and outside the repository.As per path instructions, review this deployment command for accuracy; as per coding guidelines, do not leave generated local artifacts in the repository.
Sources: Coding guidelines, Path instructions
docs/source/customization/configuration-reference.md (1)
709-709: LGTM!docs/source/deployment/observability.md (1)
54-63: LGTM!Also applies to: 285-289, 308-310
src/aiq_agent/agents/deep_researcher/custom_middleware.py (1)
45-45: LGTM!Also applies to: 726-740, 958-963, 1087-1089, 1254-1259, 1368-1373
frontends/aiq_api/src/aiq_api/jobs/runner.py (2)
1142-1146: 🔒 Security & PrivacyMake context restoration unconditional.
The reset calls occur after
_flush_event_store(), thread joining, and_teardown_sandbox(). If any cleanup operation raises, thefinallyblock exits before the context resets. A reused worker can retain the previous job's user identity, conversation ID, or authentication token.Wrap the cleanup body in an inner
tryand move all context resets into itsfinally. Add a test that forces cleanup failure and verifies that the outer context remains unchanged. The suppliedtests/aiq_agent/jobs/test_runner.pytest at Lines 669-788 covers normal agent failure, not cleanup failure.As per path instructions: “Treat API, auth, and job-runner changes as externally visible contracts” and check “cross-user data isolation.”
Source: Path instructions
40-40: LGTM!Also applies to: 51-51, 425-425, 504-510, 533-533, 664-666, 783-794, 808-809, 829-829, 847-847, 1099-1107
tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py (1)
53-53: LGTM!Also applies to: 62-62, 1062-1085, 1391-1405, 1767-1785, 1948-1964
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/source/deployment/production.md (1)
90-117: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winMake the restore check fail closed.
Add
set -euo pipefailand-v ON_ERROR_STOP=1to everypsqlcommand. Register anEXITtrap before creating the disposable databases to remove both databases after failures or interruption. Otherwise, a failedpg_restorecan be followed by successful cleanup, causing the restore check to report success.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/deployment/production.md` around lines 90 - 117, Update the restore-check shell block to begin with set -euo pipefail, add -v ON_ERROR_STOP=1 to every psql invocation including table checks and cleanup, and register an EXIT trap before creating either disposable database that drops both restore-check databases. Preserve the existing restore and verification commands while ensuring failures or interruptions trigger cleanup and a nonzero exit.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 69-74: Define the PostgreSQL container name once as the
AIQ_POSTGRES_CONTAINER environment variable and replace every hard-coded
aiq-postgres reference in the backup and restore commands, including the
commands near the later referenced sections. Remove the need for operators to
edit individual commands manually while preserving the existing dump and restore
behavior.
- Around line 65-67: Initialize jobs_tmp and checkpoints_tmp before the mktemp
calls, then install the cleanup trap before creating either temporary archive.
Make the trap null-safe so it can safely run when one or both variables are
still unset, while preserving cleanup of successfully created files.
- Around line 72-74: Update the production backup procedure around the pg_dump
command to use the same checkpoint database configured and initialized by the
Helm deployment, consistently using aiq_checkpoints for the archive. Do not
leave the documented backup targeting aiq_jobs unless deployment-specific backup
and restore commands are added.
---
Outside diff comments:
In `@docs/source/deployment/production.md`:
- Around line 90-117: Update the restore-check shell block to begin with set
-euo pipefail, add -v ON_ERROR_STOP=1 to every psql invocation including table
checks and cleanup, and register an EXIT trap before creating either disposable
database that drops both restore-check databases. Preserve the existing restore
and verification commands while ensuring failures or interruptions trigger
cleanup and a nonzero exit.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 775697ea-163b-4b3b-9423-40ee92d1354e
📒 Files selected for processing (1)
docs/source/deployment/production.md
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
- GitHub Check: Run Harbor skill eval
- GitHub Check: Lint and Hooks
- GitHub Check: Pytest and Coverage
🧰 Additional context used
📓 Path-based instructions (4)
**/*
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat eachsources/*package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.
Files:
docs/source/deployment/production.md
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, and resolve API keys at runtime.
Files:
docs/source/deployment/production.md
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.
Files:
docs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/deployment/production.md
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T05:53:12.966Z
Learning: Run the project test suite with `uv run pytest`; use coverage with `uv run pytest --cov=src/aiq_agent --cov-report=html`, and run MCP tests using the MCP project environment and lockfile.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T05:53:12.966Z
Learning: Refer to each benchmark's README for benchmark-specific testing details and consult the Customization guide when adding evaluation harnesses.
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 83-91: Update the backup procedure around the jobs_tmp and
checkpoints_tmp pg_dump commands to define a consistent cross-database recovery
window: pause workers and related writes before both dumps and resume them
afterward, or document and implement a coordinated snapshot procedure with its
recovery guarantee. Ensure the documented procedure covers both aiq_jobs and
aiq_checkpoints rather than treating the dumps as independent snapshots.
- Around line 70-82: Update the backup cleanup flow around cleanup and the two
archive moves to track whether both final moves completed successfully. On any
partial failure, remove both published archive files as well as temporary files;
only preserve the final archives after both moves succeed, or alternatively
publish a completion manifest after that point.
- Around line 111-129: Update the restore verification block to enable set -euo
pipefail, define a null-safe cleanup function that drops both disposable
databases while tolerating cleanup failures, and install it with an EXIT trap
before either database is created. Remove the existing normal-completion DROP
commands and clear the trap only after both restore checks complete and both
databases have been dropped.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: ae942662-d5df-462a-9439-915e111976d7
📒 Files selected for processing (1)
docs/source/deployment/production.md
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
- GitHub Check: Run Harbor skill eval
- GitHub Check: Pytest and Coverage
- GitHub Check: Lint and Hooks
🧰 Additional context used
📓 Path-based instructions (4)
**/*
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat eachsources/*package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.
Files:
docs/source/deployment/production.md
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, and resolve API keys at runtime.
Files:
docs/source/deployment/production.md
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.
Files:
docs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/deployment/production.md
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T06:05:41.029Z
Learning: Use `./scripts/start_cli.sh --verbose` or set `verbose: true` in workflow configuration when debugging with verbose logging.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T06:05:41.029Z
Learning: Use Phoenix tracing for debugging by starting the Phoenix server, enabling tracing in agent configuration, and opening `http://localhost:6006`.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T06:05:41.029Z
Learning: For common issues, reinstall the package in editable mode for import errors, check environment variables for authentication failures, check configuration for missing tools, and run `pre-commit clean` for pre-commit cache issues.
🔇 Additional comments (2)
docs/source/deployment/production.md (2)
43-55: LGTM!
98-107: LGTM!
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 50-52: Update the backup procedure surrounding the two PostgreSQL
snapshots to document recovery when set -e stops on a pg_dump or
archive-publication failure. Ensure the failure path confirms cleanup has
completed and resumes the paused API, workers, and other writers before any
retry, or add a pause/resume wrapper with an EXIT trap that guarantees this
behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 2e69162d-16c3-4d0e-8fee-b1ccdd345add
📒 Files selected for processing (1)
docs/source/deployment/production.md
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
- GitHub Check: Run Harbor skill eval
- GitHub Check: Pytest and Coverage
- GitHub Check: Lint and Hooks
🧰 Additional context used
📓 Path-based instructions (4)
**/*
📄 CodeRabbit inference engine (CONTRIBUTING.md)
**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.
**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat eachsources/*package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.
Files:
docs/source/deployment/production.md
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}
📄 CodeRabbit inference engine (AGENTS.md)
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and
SecretStr, and resolve API keys at runtime.
Files:
docs/source/deployment/production.md
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update canonical documentation under
docs/source/when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.
Files:
docs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/deployment/production.md
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T06:13:39.273Z
Learning: Run the documented test commands with `uv`, including coverage, targeted tests, and the separate MCP project test environment.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T06:13:39.273Z
Learning: Refer to each benchmark's README for benchmark-specific testing details and consult the customization guide when adding evaluation harnesses.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq
Timestamp: 2026-08-11T06:13:39.273Z
Learning: Use verbose CLI logging, Phoenix tracing, and the documented troubleshooting steps when debugging.
🔇 Additional comments (2)
docs/source/deployment/production.md (2)
74-74: LGTM!Also applies to: 85-87, 103-103
119-121: LGTM!Also applies to: 123-131
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
tanleach
left a comment
There was a problem hiding this comment.
Good updates to docs and logging.
|
/ok to test e16dc33 |
|
/merge |
Overview
Fix the deterministic AI-Q 2.2 VDR findings that are addressable without adding release-scope features:
conversation-idfrom the async HTTP submit route before knowledge tools are constructed;This intentionally does not add a progress API, automatic batch splitting, PDF/WebP support, UI math changes, new model/provider integrations, a semantic guardrail platform, connectors, benchmarks, or case studies.
DCO sign-off for the squash commit
Signed-off-by: Ajay Thorve athorve@nvidia.com
Signed-off-by: Ajay Thorve AjayThorve@users.noreply.github.com
Validation
Automated validation:
uv run pytest -q— 2,135 passed, 13 skipped.uv run ruff check .— passed.uv run ruff format --check .— 383 files formatted.PATH="$PWD/.venv/bin:$PATH" make -C docs html— passed with three pre-existing missing-xref warnings.uv run python scripts/validate_skills.py skills— two skills valid.git diff --check— passed.deploy/.envsecret values were compared against the diff — zero matches.Configuration and operational checks using the real, ignored root
deploy/.env:nat validatepassed forconfig_web_default_llamaindex.yml,config_web_default_guardrails.yml, andconfig_frontier_models.yml.uvx --from arize-phoenix phoenix serve --helppassed.docker build -f deploy/Dockerfile -t aiq-vdr-ui:local .completed all 23 steps fromfrontends/ui.aiq.py -handaiq.py --helpboth exited zero; livehealthandagentsconsumer-skill checks passed.Live E2E checks used
./scripts/start_server_in_debug_mode.sh, which sourced the real rootdeploy/.env:conversation-id: vdr-e2e-aug10; the no-header control used configuredtest_collectionand returned the expected no-source failure./generateand/chatfor exact and configured reworded prompts in 6–8 ms.The root environment does not contain an OpenAI credential. The exact GPT Sol/Luna role split was instead validated live through NVIDIA Inference Hub using the shipped
_type: openaicompatibility path and a temporary, uncommitted endpoint override. The full CUDA-X job completed successfully in 9m07s; 28 sources were found, 12 were cited, and all 12 visible report URLs exactly matched the authoritative job-state URLs. The override was restored after validation, and the public profile remains provider-neutral.git commit -sor an equivalent sign-off.Where should reviewers start?
src/aiq_agent/common/citation_verification.pyandsrc/aiq_agent/agents/deep_researcher/agent.pyfor final citation integrity.frontends/aiq_api/src/aiq_api/jobs/{submit,runner,callbacks,event_store}.pyandroutes/jobs.pyfor async routing, state parity, and log redaction.src/aiq_agent/guardrails/,configs/config_web_default_guardrails.yml, and their focused tests for bounded guardrail parity.docs/,README.md, andskills/aiq-research/scripts/aiq.pyfor VDR documentation and consumer-skill corrections.Related Issues
Summary by CodeRabbit
New Features
-hand--helpCLI behavior.Security
Documentation