Skip to content

fix: resolve VDR 2.2 release-readiness issues - #429

Merged
rapids-bot[bot] merged 7 commits into
release/2.2from
ajay/vdr-2.2-release-fixes
Aug 11, 2026
Merged

fix: resolve VDR 2.2 release-readiness issues#429
rapids-bot[bot] merged 7 commits into
release/2.2from
ajay/vdr-2.2-release-fixes

Conversation

@AjayThorve

@AjayThorve AjayThorve commented Aug 11, 2026

Copy link
Copy Markdown
Member

Overview

Fix the deterministic AI-Q 2.2 VDR findings that are addressable without adding release-scope features:

  • preserve verified citations through final report/artifact processing and make final report URLs authoritative in async job state;
  • propagate conversation-id from the async HTTP submit route before knowledge tools are constructed;
  • apply configured guardrails consistently to direct, shallow-worker, and deep-worker execution, while preserving typed route responses and bounded termination;
  • redact prompt, tool, error, and event-store content from built-in logs;
  • correct the Phoenix, PostgreSQL backup/restore, frontend Docker, LlamaIndex routing, CLI help, and runtime documentation;
  • document the exact shipped model profiles and require deployment-specific validation for other BYO provider/model combinations rather than claiming universal support;
  • clarify the guardrails example and governed/self-hostable backend positioning.

This intentionally does not add a progress API, automatic batch splitting, PDF/WebP support, UI math changes, new model/provider integrations, a semantic guardrail platform, connectors, benchmarks, or case studies.

DCO sign-off for the squash commit

Signed-off-by: Ajay Thorve athorve@nvidia.com
Signed-off-by: Ajay Thorve AjayThorve@users.noreply.github.com

Validation

Automated validation:

  • uv run pytest -q — 2,135 passed, 13 skipped.
  • uv run ruff check . — passed.
  • uv run ruff format --check . — 383 files formatted.
  • PATH="$PWD/.venv/bin:$PATH" make -C docs html — passed with three pre-existing missing-xref warnings.
  • uv run python scripts/validate_skills.py skills — two skills valid.
  • git diff --check — passed.
  • Exact root deploy/.env secret values were compared against the diff — zero matches.

Configuration and operational checks using the real, ignored root deploy/.env:

  • nat validate passed for config_web_default_llamaindex.yml, config_web_default_guardrails.yml, and config_frontier_models.yml.
  • uvx --from arize-phoenix phoenix serve --help passed.
  • PostgreSQL backup and restore verification passed for both shipped databases.
  • docker build -f deploy/Dockerfile -t aiq-vdr-ui:local . completed all 23 steps from frontends/ui.
  • aiq.py -h and aiq.py --help both exited zero; live health and agents consumer-skill checks passed.

Live E2E checks used ./scripts/start_server_in_debug_mode.sh, which sourced the real root deploy/.env:

  • LlamaIndex async routing retrieved the uploaded canary only with conversation-id: vdr-e2e-aug10; the no-header control used configured test_collection and returned the expected no-source failure.
  • Guardrails returned the same typed HTTP 200 refusal envelope from /generate and /chat for exact and configured reworded prompts in 6–8 ms.
  • Guarded shallow and deep async jobs both completed successfully with refusal reports and zero tool events.
  • A fake credential sentinel appeared zero times in server logs.
  • Final CUDA-X deep research completed successfully in 11m16s; 21 sources were found, 15 were cited, and the 15 visible report URLs exactly matched the 15 authoritative job-state URLs (zero missing or extra). The final citation event carried all 15 URLs, and the raw prompt appeared zero times in server logs.

The root environment does not contain an OpenAI credential. The exact GPT Sol/Luna role split was instead validated live through NVIDIA Inference Hub using the shipped _type: openai compatibility path and a temporary, uncommitted endpoint override. The full CUDA-X job completed successfully in 9m07s; 28 sources were found, 12 were cited, and all 12 visible report URLs exactly matched the authoritative job-state URLs. The override was restored after validation, and the public profile remains provider-neutral.

  • I ran the relevant local checks or explained why they are not applicable.
  • I added or updated tests for behavior changes.
  • I updated documentation for user-facing or contributor-facing changes.
  • I confirmed this PR does not include secrets, credentials, or internal-only data.
  • I certify this contribution under the Developer Certificate of Origin (DCO) and signed my commits with git commit -s or an equivalent sign-off.
  • I replaced the DCO sign-off placeholder with my GitHub commit identity and kept the required angle brackets around the email address.

Where should reviewers start?

  1. src/aiq_agent/common/citation_verification.py and src/aiq_agent/agents/deep_researcher/agent.py for final citation integrity.
  2. frontends/aiq_api/src/aiq_api/jobs/{submit,runner,callbacks,event_store}.py and routes/jobs.py for async routing, state parity, and log redaction.
  3. src/aiq_agent/guardrails/, configs/config_web_default_guardrails.yml, and their focused tests for bounded guardrail parity.
  4. docs/, README.md, and skills/aiq-research/scripts/aiq.py for VDR documentation and consumer-skill corrections.

Related Issues

  • AI-Q 2.2 VDR release review; no public issue.

Summary by CodeRabbit

  • New Features

    • Added optional conversation ID support for job submissions and retrieval collection selection.
    • Improved citation verification so final reports retain only validated, visible source URLs.
    • Added broader guardrail input support and schema-compatible refusal responses.
    • Added successful -h and --help CLI behavior.
    • Updated the shipped frontier model profile and deployment validation guidance.
  • Security

    • Sensitive content is now represented by safe metadata in logs.
    • Added protection against document or knowledge-base exfiltration requests.
  • Documentation

    • Updated deployment, observability, backup, tracing, model, and research timing guidance.

Signed-off-by: Ajay Thorve <athorve@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The PR hardens sensitive-content logging, strengthens citation verification and artifact propagation, adds explicit conversation-ID handling, improves guardrail response compatibility, and updates shipped model, deployment, backup, observability, and CLI documentation.

Changes

Security and workflow hardening

Layer / File(s) Summary
Citation integrity and report metadata
src/aiq_agent/common/citation_verification.py, src/aiq_agent/agents/*, frontends/aiq_api/src/aiq_api/jobs/*
Reports revalidate citations after sanitization. Final-report events and job artifacts now contain only verified visible URLs. Citation loss raises CitationIntegrityError.
Metadata-safe logging
src/aiq_agent/common/logging_utils.py, src/aiq_agent/common/callbacks.py, src/aiq_agent/agents/*, frontends/aiq_api/src/aiq_api/jobs/*
Content-bearing logs now use lengths, stable references, exception types, and aggregate counts. SQL diagnostics hide bound parameters.
Job context and guardrails
frontends/aiq_api/src/aiq_api/jobs/*, frontends/aiq_api/src/aiq_api/routes/jobs.py, src/aiq_agent/guardrails/*
Submission accepts conversation-id. Workers bind and restore the context. Middleware applies to configured worker functions. Guardrails support additional input fields and typed refusal responses.
Configuration and deployment documentation
configs/*, deploy/*, docs/source/*, README.md, skills/aiq-research/scripts/aiq.py
Documentation now describes shipped model profiles, deployment validation, redacted logging, container backups, uvx Phoenix startup, updated research durations, Docker paths, and successful CLI help handling.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.09% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title uses valid Conventional Commits syntax, describes the release-readiness fixes, and meets the length and punctuation requirements.
Description check ✅ Passed The description includes the required overview, DCO sign-off, detailed validation, checklist, reviewer guidance, and related-issues sections.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ajay/vdr-2.2-release-fixes

Comment @coderabbitai help to get the list of available commands.

@AjayThorve
AjayThorve marked this pull request as ready for review August 11, 2026 04:58
@AjayThorve
AjayThorve requested a review from a team August 11, 2026 04:58

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 13

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/customization/swapping-models.md`:
- Around line 15-23: Update docs/source/customization/configuration-reference.md
at line 709 to state that configs/config_frontier_models.yml requires
TAVILY_API_KEY for its enabled Tavily tools, while keeping SERPER_API_KEY
associated only with the commented-out paper-search opt-in. No direct change is
required in docs/source/customization/swapping-models.md at line 15-23; it is
the related profile reference.

In `@docs/source/deployment/observability.md`:
- Around line 285-288: Revise the observability guidance around
VerboseTraceCallback to scope the metadata-only guarantee exclusively to that
console callback, clarifying that Phoenix and other exporters may still receive
raw prompts, tool arguments, tool results, or model responses. Explicitly
require separate retention and access controls for enabled source adapters,
external providers, and exporters.

In `@docs/source/deployment/production.md`:
- Around line 49-59: Update the backup command instructions around the
PostgreSQL dump examples to create archives in a restricted directory outside
the repository, applying umask 077 or equivalent before writing them. Add
concise guidance to encrypt the dumps and enforce access controls before copying
them to backup storage.

In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Line 809: Update the job execution flow around
context_state.user_id.set(owner_user_id) to retain the returned reset token,
then reset the user context in the existing finally block alongside
conversation_id. Ensure the reset always runs after the job, including failures
and jobs without an owner_user_id, so reused workers cannot retain prior owner
identity.

In `@frontends/ui/README.md`:
- Line 196: Update the Docker build command in the README to work when executed
from frontends/ui/ by referencing the repository-root Dockerfile and using the
repository root as the build context, while preserving the existing image tag.

In `@src/aiq_agent/agents/chat_researcher/agent.py`:
- Line 183: Update the error logging in the deep research tools validation flow
to pass error_msg directly to logger.error instead of wrapping it with
log_content_metadata. Keep the existing validation failure context and
user-facing AIMessage behavior unchanged.

In `@src/aiq_agent/agents/deep_researcher/custom_middleware.py`:
- Around line 958-963: Replace the raw tool_name argument in the retry failure
log with a safe metadata representation, such as the existing
log_content_metadata helper, while preserving the attempt, retry limit, error
type, and error detail fields. Add a retry-path test using a secret marker as
the model-provided tool name and assert that the marker is absent from emitted
logs.

In `@src/aiq_agent/agents/deep_researcher/register.py`:
- Around line 302-307: The catch-all error handlers omit traceback information
while retaining redacted exception details. In
src/aiq_agent/agents/deep_researcher/register.py lines 302-307, update the
generic except Exception logger.error call to include exc_info=True; make the
same change in src/aiq_agent/agents/chat_researcher/agent.py lines 393-397.
Preserve the existing redacted message and metadata arguments.

In `@src/aiq_agent/agents/report_rewriter/agent.py`:
- Around line 197-200: Compute _effective_parent_sources(original_report,
parent_context) once in run and pass the resulting parent source allowlist into
rewrite_report, reusing it for _verified_cited_urls instead of recalculating it
inside rewrite_report.
- Around line 202-204: Update the callback loop in the report rewriter agent to
break immediately after invoking the first callback that exposes
emit_final_report, matching the behavior of ShallowResearcherAgent.run and
DeepResearcherAgent.run and preventing duplicate final-report and citation
events.

In `@src/aiq_agent/agents/shallow_researcher/agent.py`:
- Around line 408-421: Update the final verification flow in the agent method
containing verify_citations so content is replaced with
final_verification.verified_report before deriving cited URLs, emitting via
emit_final_report, and writing the final message. Preserve the existing callback
behavior while ensuring the published report and cited URLs come from the same
final verification result.

In `@tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py`:
- Around line 1754-1759: Update the test around
SourceRoutingPersistenceMiddleware.after_agent() to capture the raised
RuntimeError with pytest.raises and assert its message does not contain
error_detail, while retaining the existing caplog assertions that verify the
backend detail is logged.

In `@tests/aiq_agent/jobs/test_runner.py`:
- Around line 494-495: Update the test around mock_job_store.submit_job to
document that job_args[14] is run_agent_job’s parent_conversation_id, and bind
the submitted positional arguments against run_agent_job’s signature before
asserting the expected value so argument reordering cannot silently alter the
mapping.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 27266ab4-bd2d-49ee-98e9-76b8418f2010

📥 Commits

Reviewing files that changed from the base of the PR and between 160d05e and 55ecb71.

📒 Files selected for processing (59)
  • .secrets.baseline
  • README.md
  • configs/config_frontier_models.yml
  • configs/config_web_default_guardrails.yml
  • deploy/.env.example
  • deploy/compose/docker-compose.yaml
  • docs/source/architecture/agents/index.md
  • docs/source/contributing/testing.md
  • docs/source/customization/configuration-reference.md
  • docs/source/customization/guardrails.md
  • docs/source/customization/prompts.md
  • docs/source/customization/swapping-models.md
  • docs/source/deployment/observability.md
  • docs/source/deployment/production.md
  • docs/source/evaluation/benchmarks/deep-research-bench.md
  • docs/source/examples/full-pipeline-llamaindex.md
  • docs/source/examples/hybrid-frontier-model.md
  • docs/source/examples/index.md
  • docs/source/get-started/quick-start.md
  • docs/source/index.md
  • docs/source/integration/agent-skills.md
  • docs/source/resources/faq.md
  • frontends/aiq_api/src/aiq_api/jobs/callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/event_store.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • frontends/aiq_api/src/aiq_api/jobs/submit.py
  • frontends/aiq_api/src/aiq_api/routes/jobs.py
  • frontends/aiq_api/tests/test_job_submit_data_sources.py
  • frontends/benchmarks/deepresearch_bench/README.md
  • frontends/ui/README.md
  • skills/aiq-research/scripts/aiq.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • src/aiq_agent/agents/clarifier/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/deep_researcher/register.py
  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/common/callbacks.py
  • src/aiq_agent/common/citation_verification.py
  • src/aiq_agent/common/logging_utils.py
  • src/aiq_agent/guardrails/interface/middleware.py
  • src/aiq_agent/guardrails/workflow/middleware.py
  • tests/aiq_agent/agents/chat_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/common/test_callbacks.py
  • tests/aiq_agent/common/test_citation_verification.py
  • tests/aiq_agent/common/test_logging_utils.py
  • tests/aiq_agent/guardrails/test_guardrails_interface.py
  • tests/aiq_agent/guardrails/test_guardrails_registration.py
  • tests/aiq_agent/guardrails/test_workflow_guardrails.py
  • tests/aiq_agent/jobs/test_runner.py
  • tests/scripts/test_aiq_research_helper.py
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
  • GitHub Check: Run Harbor skill eval
  • GitHub Check: Pytest and Coverage
🧰 Additional context used
📓 Path-based instructions (17)
**/*

📄 CodeRabbit inference engine (CONTRIBUTING.md)

**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.

**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat each sources/* package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.

Files:

  • docs/source/architecture/agents/index.md
  • frontends/benchmarks/deepresearch_bench/README.md
  • docs/source/examples/index.md
  • configs/config_frontier_models.yml
  • docs/source/examples/full-pipeline-llamaindex.md
  • docs/source/customization/configuration-reference.md
  • docs/source/integration/agent-skills.md
  • docs/source/customization/prompts.md
  • src/aiq_agent/common/logging_utils.py
  • docs/source/contributing/testing.md
  • frontends/ui/README.md
  • docs/source/evaluation/benchmarks/deep-research-bench.md
  • frontends/aiq_api/tests/test_job_submit_data_sources.py
  • docs/source/examples/hybrid-frontier-model.md
  • deploy/compose/docker-compose.yaml
  • docs/source/index.md
  • src/aiq_agent/agents/deep_researcher/register.py
  • skills/aiq-research/scripts/aiq.py
  • docs/source/customization/guardrails.md
  • src/aiq_agent/agents/clarifier/agent.py
  • docs/source/customization/swapping-models.md
  • docs/source/resources/faq.md
  • configs/config_web_default_guardrails.yml
  • src/aiq_agent/agents/report_rewriter/agent.py
  • tests/aiq_agent/common/test_citation_verification.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • docs/source/get-started/quick-start.md
  • tests/aiq_agent/common/test_logging_utils.py
  • tests/aiq_agent/agents/chat_researcher/test_agent.py
  • src/aiq_agent/guardrails/interface/middleware.py
  • README.md
  • docs/source/deployment/production.md
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • docs/source/deployment/observability.md
  • frontends/aiq_api/src/aiq_api/routes/jobs.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • tests/aiq_agent/common/test_callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/event_store.py
  • frontends/aiq_api/src/aiq_api/jobs/callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/submit.py
  • tests/aiq_agent/guardrails/test_workflow_guardrails.py
  • tests/aiq_agent/guardrails/test_guardrails_interface.py
  • src/aiq_agent/guardrails/workflow/middleware.py
  • tests/aiq_agent/agents/deep_researcher/test_agent.py
  • tests/scripts/test_aiq_research_helper.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
  • tests/aiq_agent/jobs/test_runner.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • src/aiq_agent/common/callbacks.py
  • src/aiq_agent/common/citation_verification.py
  • tests/aiq_agent/guardrails/test_guardrails_registration.py
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}

📄 CodeRabbit inference engine (AGENTS.md)

Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr, and resolve API keys at runtime.

Files:

  • docs/source/architecture/agents/index.md
  • frontends/benchmarks/deepresearch_bench/README.md
  • docs/source/examples/index.md
  • configs/config_frontier_models.yml
  • docs/source/examples/full-pipeline-llamaindex.md
  • docs/source/customization/configuration-reference.md
  • docs/source/integration/agent-skills.md
  • docs/source/customization/prompts.md
  • src/aiq_agent/common/logging_utils.py
  • docs/source/contributing/testing.md
  • frontends/ui/README.md
  • docs/source/evaluation/benchmarks/deep-research-bench.md
  • frontends/aiq_api/tests/test_job_submit_data_sources.py
  • docs/source/examples/hybrid-frontier-model.md
  • deploy/compose/docker-compose.yaml
  • docs/source/index.md
  • src/aiq_agent/agents/deep_researcher/register.py
  • skills/aiq-research/scripts/aiq.py
  • docs/source/customization/guardrails.md
  • src/aiq_agent/agents/clarifier/agent.py
  • docs/source/customization/swapping-models.md
  • docs/source/resources/faq.md
  • configs/config_web_default_guardrails.yml
  • src/aiq_agent/agents/report_rewriter/agent.py
  • tests/aiq_agent/common/test_citation_verification.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • docs/source/get-started/quick-start.md
  • tests/aiq_agent/common/test_logging_utils.py
  • tests/aiq_agent/agents/chat_researcher/test_agent.py
  • src/aiq_agent/guardrails/interface/middleware.py
  • README.md
  • docs/source/deployment/production.md
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • docs/source/deployment/observability.md
  • frontends/aiq_api/src/aiq_api/routes/jobs.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • tests/aiq_agent/common/test_callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/event_store.py
  • frontends/aiq_api/src/aiq_api/jobs/callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/submit.py
  • tests/aiq_agent/guardrails/test_workflow_guardrails.py
  • tests/aiq_agent/guardrails/test_guardrails_interface.py
  • src/aiq_agent/guardrails/workflow/middleware.py
  • tests/aiq_agent/agents/deep_researcher/test_agent.py
  • tests/scripts/test_aiq_research_helper.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
  • tests/aiq_agent/jobs/test_runner.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • src/aiq_agent/common/callbacks.py
  • src/aiq_agent/common/citation_verification.py
  • tests/aiq_agent/guardrails/test_guardrails_registration.py
docs/source/**/*

📄 CodeRabbit inference engine (AGENTS.md)

Update canonical documentation under docs/source/ when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.

Files:

  • docs/source/architecture/agents/index.md
  • docs/source/examples/index.md
  • docs/source/examples/full-pipeline-llamaindex.md
  • docs/source/customization/configuration-reference.md
  • docs/source/integration/agent-skills.md
  • docs/source/customization/prompts.md
  • docs/source/contributing/testing.md
  • docs/source/evaluation/benchmarks/deep-research-bench.md
  • docs/source/examples/hybrid-frontier-model.md
  • docs/source/index.md
  • docs/source/customization/guardrails.md
  • docs/source/customization/swapping-models.md
  • docs/source/resources/faq.md
  • docs/source/get-started/quick-start.md
  • docs/source/deployment/production.md
  • docs/source/deployment/observability.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.

Files:

  • docs/source/architecture/agents/index.md
  • docs/source/examples/index.md
  • docs/source/examples/full-pipeline-llamaindex.md
  • docs/source/customization/configuration-reference.md
  • docs/source/integration/agent-skills.md
  • docs/source/customization/prompts.md
  • docs/source/contributing/testing.md
  • docs/source/evaluation/benchmarks/deep-research-bench.md
  • docs/source/examples/hybrid-frontier-model.md
  • docs/source/index.md
  • docs/source/customization/guardrails.md
  • docs/source/customization/swapping-models.md
  • docs/source/resources/faq.md
  • docs/source/get-started/quick-start.md
  • README.md
  • docs/source/deployment/production.md
  • docs/source/deployment/observability.md
**/*.{yaml,yml}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

For deployment changes, run the relevant Helm or Compose validation and describe the environment used.

Files:

  • configs/config_frontier_models.yml
  • deploy/compose/docker-compose.yaml
  • configs/config_web_default_guardrails.yml
configs/**/*.{yml,yaml}

📄 CodeRabbit inference engine (AGENTS.md)

Use _type names that come from the registered configuration class.

Files:

  • configs/config_frontier_models.yml
  • configs/config_web_default_guardrails.yml
{deploy/**,configs/**}

⚙️ CodeRabbit configuration file

{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.

Files:

  • configs/config_frontier_models.yml
  • deploy/compose/docker-compose.yaml
  • configs/config_web_default_guardrails.yml
**/*.py

📄 CodeRabbit inference engine (CONTRIBUTING.md)

**/*.py: Run uv run ruff check . and uv run ruff format --check . for root Python changes.
Run uv run pytest for root project Python changes.

Files:

  • src/aiq_agent/common/logging_utils.py
  • frontends/aiq_api/tests/test_job_submit_data_sources.py
  • src/aiq_agent/agents/deep_researcher/register.py
  • skills/aiq-research/scripts/aiq.py
  • src/aiq_agent/agents/clarifier/agent.py
  • src/aiq_agent/agents/report_rewriter/agent.py
  • tests/aiq_agent/common/test_citation_verification.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • tests/aiq_agent/common/test_logging_utils.py
  • tests/aiq_agent/agents/chat_researcher/test_agent.py
  • src/aiq_agent/guardrails/interface/middleware.py
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • frontends/aiq_api/src/aiq_api/routes/jobs.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • tests/aiq_agent/common/test_callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/event_store.py
  • frontends/aiq_api/src/aiq_api/jobs/callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/submit.py
  • tests/aiq_agent/guardrails/test_workflow_guardrails.py
  • tests/aiq_agent/guardrails/test_guardrails_interface.py
  • src/aiq_agent/guardrails/workflow/middleware.py
  • tests/aiq_agent/agents/deep_researcher/test_agent.py
  • tests/scripts/test_aiq_research_helper.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
  • tests/aiq_agent/jobs/test_runner.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • src/aiq_agent/common/callbacks.py
  • src/aiq_agent/common/citation_verification.py
  • tests/aiq_agent/guardrails/test_guardrails_registration.py
**/*.{py,pyi}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{py,pyi}: Format and lint Python code with Ruff using line length 120, Python 3.11 targeting, rules E, F, W, I, PL, and UP, with single-line imports; do not reformat unrelated code.
Missing-secret paths must degrade gracefully by stubbing or skipping rather than crashing or leaking secrets.

Files:

  • src/aiq_agent/common/logging_utils.py
  • frontends/aiq_api/tests/test_job_submit_data_sources.py
  • src/aiq_agent/agents/deep_researcher/register.py
  • skills/aiq-research/scripts/aiq.py
  • src/aiq_agent/agents/clarifier/agent.py
  • src/aiq_agent/agents/report_rewriter/agent.py
  • tests/aiq_agent/common/test_citation_verification.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • tests/aiq_agent/common/test_logging_utils.py
  • tests/aiq_agent/agents/chat_researcher/test_agent.py
  • src/aiq_agent/guardrails/interface/middleware.py
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • frontends/aiq_api/src/aiq_api/routes/jobs.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • tests/aiq_agent/common/test_callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/event_store.py
  • frontends/aiq_api/src/aiq_api/jobs/callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/submit.py
  • tests/aiq_agent/guardrails/test_workflow_guardrails.py
  • tests/aiq_agent/guardrails/test_guardrails_interface.py
  • src/aiq_agent/guardrails/workflow/middleware.py
  • tests/aiq_agent/agents/deep_researcher/test_agent.py
  • tests/scripts/test_aiq_research_helper.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
  • tests/aiq_agent/jobs/test_runner.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • src/aiq_agent/common/callbacks.py
  • src/aiq_agent/common/citation_verification.py
  • tests/aiq_agent/guardrails/test_guardrails_registration.py
**/*.{py,pyi,js,jsx,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Never print or log secret values, including in tool output or error messages.

Files:

  • src/aiq_agent/common/logging_utils.py
  • frontends/aiq_api/tests/test_job_submit_data_sources.py
  • src/aiq_agent/agents/deep_researcher/register.py
  • skills/aiq-research/scripts/aiq.py
  • src/aiq_agent/agents/clarifier/agent.py
  • src/aiq_agent/agents/report_rewriter/agent.py
  • tests/aiq_agent/common/test_citation_verification.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • tests/aiq_agent/common/test_logging_utils.py
  • tests/aiq_agent/agents/chat_researcher/test_agent.py
  • src/aiq_agent/guardrails/interface/middleware.py
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • frontends/aiq_api/src/aiq_api/routes/jobs.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • tests/aiq_agent/common/test_callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/event_store.py
  • frontends/aiq_api/src/aiq_api/jobs/callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/submit.py
  • tests/aiq_agent/guardrails/test_workflow_guardrails.py
  • tests/aiq_agent/guardrails/test_guardrails_interface.py
  • src/aiq_agent/guardrails/workflow/middleware.py
  • tests/aiq_agent/agents/deep_researcher/test_agent.py
  • tests/scripts/test_aiq_research_helper.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
  • tests/aiq_agent/jobs/test_runner.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • src/aiq_agent/common/callbacks.py
  • src/aiq_agent/common/citation_verification.py
  • tests/aiq_agent/guardrails/test_guardrails_registration.py
src/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

src/**/*.py: Respect authenticated data sources by honoring requires_auth, passing through per-user tokens, and using backend token validators; apply owner guardrails before loading protected report or artifact context into an agent.
Do not weaken or bypass AuthMiddleware, authentication validators, or authentication gating without prior design discussion.

Files:

  • src/aiq_agent/common/logging_utils.py
  • src/aiq_agent/agents/deep_researcher/register.py
  • src/aiq_agent/agents/clarifier/agent.py
  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/guardrails/interface/middleware.py
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • src/aiq_agent/guardrails/workflow/middleware.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
  • src/aiq_agent/common/callbacks.py
  • src/aiq_agent/common/citation_verification.py
docs/source/contributing/**/*

📄 CodeRabbit inference engine (docs/source/contributing/code-style.md)

Run pre-commit run --all-files; its checks include Ruff fixes and formatting, root and MCP lock checks, secret detection, notebook output clearing, and Markdown link checking.

Files:

  • docs/source/contributing/testing.md
frontends/ui/**/*

⚙️ CodeRabbit configuration file

frontends/ui/**/*: Review UI changes for strict TypeScript behavior, API contract alignment, auth/session handling, accessible controls,
resilient loading and error states, and report/chat state consistency. Prefer existing UI patterns and require tests
for changed user-visible workflows.

Files:

  • frontends/ui/README.md
src/aiq_agent/agents/**/*

⚙️ CodeRabbit configuration file

src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.

Files:

  • src/aiq_agent/agents/deep_researcher/register.py
  • src/aiq_agent/agents/clarifier/agent.py
  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/register.py
  • src/aiq_agent/agents/chat_researcher/register.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py
  • src/aiq_agent/agents/chat_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/agent.py
skills/aiq-research/scripts/aiq.py

📄 CodeRabbit inference engine (skills/aiq-research/SKILL.md)

skills/aiq-research/scripts/aiq.py: Use Python 3.11+ and Python standard-library HTTP modules; the helper script must not require third-party Python packages.
When polling requires an execution method with escalated permissions, request explicit user approval first and explain why; tell the user when deep research is running in the background.

Files:

  • skills/aiq-research/scripts/aiq.py
{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}

⚙️ CodeRabbit configuration file

{skills/**,.agents/skills/**,.claude/skills/**,.github/skill-eval/**}: Review Agent Skill and skill-eval changes for valid skill metadata, deterministic eval specs, safe handling of
credentials, and clear generated-output boundaries. Do not flag SKILL.md files for missing SPDX headers when the
entrypoint intentionally starts with YAML frontmatter.

Files:

  • skills/aiq-research/scripts/aiq.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}

⚙️ CodeRabbit configuration file

{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.

Files:

  • frontends/aiq_api/src/aiq_api/routes/jobs.py
  • frontends/aiq_api/src/aiq_api/jobs/event_store.py
  • frontends/aiq_api/src/aiq_api/jobs/callbacks.py
  • frontends/aiq_api/src/aiq_api/jobs/submit.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
🧠 Learnings (2)
📚 Learning: 2026-06-14T17:49:00.640Z
Learnt from: torkian
Repo: NVIDIA-AI-Blueprints/aiq PR: 273
File: frontends/aiq_api/tests/test_sse_reconnect_cursor.py:384-401
Timestamp: 2026-06-14T17:49:00.640Z
Learning: When using `unittest.mock.patch` for code that imports dependencies inside functions/generators (e.g., inside `aiq_api.routes.jobs`), don’t patch via an attribute that doesn’t exist on the consuming module. If the generator does `from ..jobs.event_store import EventStore` inside the generator body, then `aiq_api.routes.jobs` will not have an `EventStore` attribute; patch the source class/method in its defining module instead (e.g., `aiq_api.jobs.event_store.EventStore.get_events_async`). Patching `aiq_api.routes.jobs.EventStore...` would raise `AttributeError` because that symbol is not present at module scope.

Applied to files:

  • frontends/aiq_api/tests/test_job_submit_data_sources.py
📚 Learning: 2026-07-06T23:55:42.908Z
Learnt from: cdgamarose-nv
Repo: NVIDIA-AI-Blueprints/aiq PR: 311
File: src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2:74-81
Timestamp: 2026-07-06T23:55:42.908Z
Learning: In agent test files (e.g., tests/aiq_agent/agents/*/test_agent.py), avoid brittle assertions that match exact substrings from prompt template files (such as *.j2 prompt wording). Prompt wording can change frequently, so instead assert structural/behavioral properties (e.g., that the prompt builder is called, that required sections/fields are present via stable markers, that the model output/agent behavior conforms to an expected schema, or that key actions are taken) rather than matching literal prompt text.

Applied to files:

  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • tests/aiq_agent/agents/chat_researcher/test_agent.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_agent.py
🪛 ast-grep (0.45.1)
tests/scripts/test_aiq_research_helper.py

[error] 51-56: Command coming from incoming request
Context: subprocess.run(
[sys.executable, str(_SCRIPT), help_flag],
check=False,
capture_output=True,
text=True,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

(subprocess-from-request)

tests/aiq_agent/guardrails/test_guardrails_registration.py

[warning] 71-71: Regex pattern passed to re is built from a non-literal (variable, call, concatenation, or f-string) value. If that value is attacker-controlled it can introduce a malicious pattern with catastrophic backtracking (ReDoS). Use a hardcoded literal pattern, or validate/escape untrusted input with re.escape() and bound the regex complexity before compiling.
Context: re.search(pattern, prompt)
Note: [CWE-1333] Inefficient Regular Expression Complexity.

(redos-non-literal-regex-python)


[warning] 72-72: Regex pattern passed to re is built from a non-literal (variable, call, concatenation, or f-string) value. If that value is attacker-controlled it can introduce a malicious pattern with catastrophic backtracking (ReDoS). Use a hardcoded literal pattern, or validate/escape untrusted input with re.escape() and bound the regex complexity before compiling.
Context: re.search(pattern, prompt)
Note: [CWE-1333] Inefficient Regular Expression Complexity.

(redos-non-literal-regex-python)

🪛 LanguageTool
docs/source/examples/hybrid-frontier-model.md

[style] ~9-~9: Consider a different adjective to strengthen your wording.
Context: ...with GPT Sol/Luna for clarification and deep research. ## Shipped Configuration | ...

(DEEP_PROFOUND)

docs/source/index.md

[style] ~8-~8: Consider a different adjective to strengthen your wording.
Context: ...int An NVIDIA blueprint for AI-powered deep research, built on the NeMo Agent Toolk...

(DEEP_PROFOUND)

README.md

[style] ~297-~297: Consider a different adjective to strengthen your wording.
Context: ...amaIndex profile using GPT Sol/Luna for deep research and Super for intent/shallow. ...

(DEEP_PROFOUND)

🔇 Additional comments (61)
src/aiq_agent/common/logging_utils.py (1)

4-4: LGTM!

Also applies to: 13-24

src/aiq_agent/common/callbacks.py (1)

23-25: LGTM!

Also applies to: 80-111, 145-145, 170-170, 183-183, 194-207, 226-251

src/aiq_agent/agents/chat_researcher/nodes/intent_classifier.py (1)

33-33: LGTM!

Also applies to: 225-248, 278-282

src/aiq_agent/agents/chat_researcher/register.py (1)

34-34: LGTM!

Also applies to: 327-331, 646-647

src/aiq_agent/agents/clarifier/agent.py (1)

66-66: LGTM!

Also applies to: 262-262, 668-668

frontends/aiq_api/src/aiq_api/jobs/event_store.py (1)

31-32: LGTM!

Also applies to: 206-206, 233-233, 448-454, 513-519

frontends/aiq_api/src/aiq_api/jobs/runner.py (1)

40-40: LGTM!

Also applies to: 503-514, 533-538, 664-665, 782-790, 849-849, 1101-1109, 1144-1146

tests/aiq_agent/agents/chat_researcher/test_agent.py (1)

33-33: LGTM!

Also applies to: 221-248

tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py (1)

61-61: LGTM!

Also applies to: 1366-1380, 1922-1938

tests/aiq_agent/common/test_callbacks.py (1)

28-28: LGTM!

Also applies to: 202-218, 277-277, 596-635

tests/aiq_agent/common/test_citation_verification.py (1)

18-19: LGTM!

Also applies to: 660-684

tests/aiq_agent/common/test_logging_utils.py (1)

6-6: LGTM!

Also applies to: 22-31

frontends/aiq_api/src/aiq_api/jobs/submit.py (1)

173-173: LGTM!

Also applies to: 196-198, 329-336, 462-462

frontends/aiq_api/src/aiq_api/routes/jobs.py (2)

43-43: LGTM!

Also applies to: 728-728, 812-812


1681-1687: LGTM!

src/aiq_agent/guardrails/interface/middleware.py (1)

30-47: LGTM!

Also applies to: 58-65

src/aiq_agent/guardrails/workflow/middleware.py (1)

90-91: LGTM!

Also applies to: 116-131, 232-243

configs/config_web_default_guardrails.yml (1)

178-178: LGTM!

frontends/aiq_api/tests/test_job_submit_data_sources.py (1)

191-204: LGTM!

tests/aiq_agent/guardrails/test_guardrails_interface.py (1)

18-18: LGTM!

Also applies to: 27-29, 104-132

tests/aiq_agent/guardrails/test_guardrails_registration.py (1)

18-18: LGTM!

Also applies to: 54-73

tests/aiq_agent/guardrails/test_workflow_guardrails.py (1)

39-57: LGTM!

Also applies to: 113-119, 354-372, 568-613, 641-641, 675-675

docs/source/integration/agent-skills.md (1)

200-214: LGTM!

skills/aiq-research/scripts/aiq.py (1)

748-750: LGTM!

tests/scripts/test_aiq_research_helper.py (1)

4-8: LGTM!

Also applies to: 20-20, 47-61

src/aiq_agent/common/citation_verification.py (3)

109-115: LGTM!

Also applies to: 196-228, 352-357, 640-642, 699-699, 820-820, 913-913, 980-984


1103-1104: LGTM!

Also applies to: 1164-1186, 1199-1207, 1241-1243


1329-1329: LGTM!

Also applies to: 1389-1396, 1437-1439, 1461-1461, 1497-1508

src/aiq_agent/agents/deep_researcher/agent.py (2)

33-39: LGTM!

Also applies to: 302-302, 359-373


399-415: LGTM!

Also applies to: 428-439

frontends/aiq_api/src/aiq_api/jobs/callbacks.py (2)

131-144: LGTM!

Also applies to: 465-477


394-414: 🗄️ Data Integrity & Integration

No change needed. Final-report processing clears sources_cited and replaces it with the final report’s cited_urls, so earlier citation_use events do not remain in aggregated job metadata.

			> Likely an incorrect or invalid review comment.
src/aiq_agent/agents/deep_researcher/custom_middleware.py (1)

45-45: LGTM!

Also applies to: 726-740, 1087-1089, 1254-1259, 1368-1373

tests/aiq_agent/agents/report_rewriter/test_agent.py (1)

8-8: LGTM!

Also applies to: 37-38, 57-57, 95-96, 126-129

src/aiq_agent/agents/shallow_researcher/register.py (1)

31-31: LGTM!

Also applies to: 168-173, 187-192

src/aiq_agent/agents/report_rewriter/agent.py (1)

86-96: LGTM!

src/aiq_agent/agents/chat_researcher/agent.py (1)

49-49: LGTM!

Also applies to: 220-223, 250-254, 265-269, 322-326, 382-390, 611-611

src/aiq_agent/agents/deep_researcher/register.py (1)

37-37: LGTM!

src/aiq_agent/agents/shallow_researcher/agent.py (1)

48-48: LGTM!

Also applies to: 220-223, 260-264, 322-324

tests/aiq_agent/agents/shallow_researcher/test_agent.py (1)

569-603: LGTM!

Also applies to: 634-638, 649-649

tests/aiq_agent/agents/deep_researcher/test_agent.py (1)

20-20: LGTM!

Also applies to: 49-53, 1586-1613, 1806-1810, 2286-2355, 2383-2433, 2435-2513

tests/aiq_agent/jobs/test_runner.py (1)

466-493: LGTM!

Also applies to: 653-775, 894-894, 935-936, 971-990, 1005-1007, 1585-1627, 1738-1748, 1915-1954, 2067-2076, 2097-2150, 2169-2252

.secrets.baseline (1)

136-136: LGTM!

Also applies to: 358-358

README.md (1)

55-55: LGTM!

Also applies to: 119-123, 297-297, 450-450, 502-502

configs/config_frontier_models.yml (1)

1-8: LGTM!

deploy/.env.example (1)

19-19: LGTM!

docs/source/examples/hybrid-frontier-model.md (2)

8-30: LGTM!

Also applies to: 34-47, 57-59


49-53: 🗄️ Data Integrity & Integration

Verify that BACKEND_CONFIG reaches Compose interpolation.

The document tells users to set BACKEND_CONFIG in deploy/.env. In deploy/compose/docker-compose.yaml Lines 31-51, CONFIG_FILE=${BACKEND_CONFIG:-...} is interpolated by Compose, while env_file: ../.env injects variables into the container. Unless the standard startup command passes deploy/.env with --env-file or exports BACKEND_CONFIG, Compose uses the default LlamaIndex configuration instead of config_frontier_models.yml. Make that requirement explicit, or verify that the linked procedure already supplies it.

docs/source/examples/index.md (1)

17-17: LGTM!

docs/source/get-started/quick-start.md (1)

94-95: LGTM!

docs/source/index.md (1)

8-12: LGTM!

docs/source/resources/faq.md (1)

18-22: LGTM!

Also applies to: 33-33

frontends/benchmarks/deepresearch_bench/README.md (1)

65-65: LGTM!

deploy/compose/docker-compose.yaml (1)

26-26: LGTM!

docs/source/architecture/agents/index.md (1)

15-15: LGTM!

docs/source/contributing/testing.md (1)

27-27: LGTM!

docs/source/customization/guardrails.md (1)

10-16: LGTM!

Also applies to: 165-167

docs/source/customization/prompts.md (1)

315-323: LGTM!

docs/source/deployment/observability.md (1)

54-63: LGTM!

docs/source/evaluation/benchmarks/deep-research-bench.md (1)

68-68: LGTM!

docs/source/examples/full-pipeline-llamaindex.md (1)

239-247: 🗄️ Data Integrity & Integration

The documented conversation-id contract is correct. The route propagates the header through asynchronous submission, and retrieval uses it as the collection with collection_name as the fallback.

Comment thread docs/source/customization/swapping-models.md
Comment thread docs/source/deployment/observability.md Outdated
Comment thread docs/source/deployment/production.md Outdated
Comment thread frontends/aiq_api/src/aiq_api/jobs/runner.py Outdated
Comment thread frontends/ui/README.md
Comment thread src/aiq_agent/agents/report_rewriter/agent.py
Comment thread src/aiq_agent/agents/report_rewriter/agent.py
Comment thread src/aiq_agent/agents/shallow_researcher/agent.py
Comment thread tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
Comment thread tests/aiq_agent/jobs/test_runner.py Outdated
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
@AjayThorve AjayThorve changed the title fix VDR 2.2 release readiness issues fix: resolve VDR 2.2 release-readiness issues Aug 11, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/source/deployment/production.md (1)

73-107: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Make the restore check fail closed and assert the expected schema.

The block has no shell fail-fast handling. If pg_restore fails, cleanup can determine the final exit status. The \dt commands only list relations and do not assert the required tables in either restored database. Add set -euo pipefail, an EXIT cleanup trap that preserves the original status, ON_ERROR_STOP=1 for inspection, and explicit assertions for tables such as job_info, job_events, and the checkpoint tables.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/source/deployment/production.md` around lines 73 - 107, Update the
restore verification block to use fail-fast shell settings with set -euo
pipefail and an EXIT cleanup trap that removes both disposable databases while
preserving the original command status. Add ON_ERROR_STOP=1 to the inspection
commands and replace \dt-only listing with explicit checks that required tables,
including job_info, job_events, and the checkpoint tables, exist in each
restored database.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 53-64: Update the backup script around the pg_dump commands to
enable set -euo pipefail, write each dump to a unique temporary file, and use
unique final archive names that avoid replacing prior backups. Ensure both dumps
complete successfully before moving either temporary file into its final
destination, while preserving the existing AIQ_BACKUP_DIR validation and
restrictive permissions.

---

Outside diff comments:
In `@docs/source/deployment/production.md`:
- Around line 73-107: Update the restore verification block to use fail-fast
shell settings with set -euo pipefail and an EXIT cleanup trap that removes both
disposable databases while preserving the original command status. Add
ON_ERROR_STOP=1 to the inspection commands and replace \dt-only listing with
explicit checks that required tables, including job_info, job_events, and the
checkpoint tables, exist in each restored database.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: b47b1723-4dc3-40fc-a72c-24b809efb033

📥 Commits

Reviewing files that changed from the base of the PR and between 55ecb71 and ded5243.

📒 Files selected for processing (11)
  • docs/source/customization/configuration-reference.md
  • docs/source/deployment/observability.md
  • docs/source/deployment/production.md
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/jobs/test_runner.py
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: Run Harbor skill eval
  • GitHub Check: Pytest and Coverage
  • GitHub Check: Lint and Hooks
🧰 Additional context used
📓 Path-based instructions (10)
**/*

📄 CodeRabbit inference engine (CONTRIBUTING.md)

**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.

**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat each sources/* package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.

Files:

  • docs/source/customization/configuration-reference.md
  • docs/source/deployment/observability.md
  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • docs/source/deployment/production.md
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • tests/aiq_agent/jobs/test_runner.py
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}

📄 CodeRabbit inference engine (AGENTS.md)

Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr, and resolve API keys at runtime.

Files:

  • docs/source/customization/configuration-reference.md
  • docs/source/deployment/observability.md
  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • docs/source/deployment/production.md
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • tests/aiq_agent/jobs/test_runner.py
docs/source/**/*

📄 CodeRabbit inference engine (AGENTS.md)

Update canonical documentation under docs/source/ when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.

Files:

  • docs/source/customization/configuration-reference.md
  • docs/source/deployment/observability.md
  • docs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.

Files:

  • docs/source/customization/configuration-reference.md
  • docs/source/deployment/observability.md
  • docs/source/deployment/production.md
**/*.py

📄 CodeRabbit inference engine (CONTRIBUTING.md)

**/*.py: Run uv run ruff check . and uv run ruff format --check . for root Python changes.
Run uv run pytest for root project Python changes.

Files:

  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • tests/aiq_agent/jobs/test_runner.py
**/*.{py,pyi}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{py,pyi}: Format and lint Python code with Ruff using line length 120, Python 3.11 targeting, rules E, F, W, I, PL, and UP, with single-line imports; do not reformat unrelated code.
Missing-secret paths must degrade gracefully by stubbing or skipping rather than crashing or leaking secrets.

Files:

  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • tests/aiq_agent/jobs/test_runner.py
**/*.{py,pyi,js,jsx,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Never print or log secret values, including in tool output or error messages.

Files:

  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
  • frontends/aiq_api/src/aiq_api/jobs/runner.py
  • tests/aiq_agent/jobs/test_runner.py
src/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

src/**/*.py: Respect authenticated data sources by honoring requires_auth, passing through per-user tokens, and using backend token validators; apply owner guardrails before loading protected report or artifact context into an agent.
Do not weaken or bypass AuthMiddleware, authentication validators, or authentication gating without prior design discussion.

Files:

  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
src/aiq_agent/agents/**/*

⚙️ CodeRabbit configuration file

src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.

Files:

  • src/aiq_agent/agents/report_rewriter/agent.py
  • src/aiq_agent/agents/shallow_researcher/agent.py
  • src/aiq_agent/agents/deep_researcher/custom_middleware.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}

⚙️ CodeRabbit configuration file

{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.

Files:

  • frontends/aiq_api/src/aiq_api/jobs/runner.py
🧠 Learnings (2)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T05:43:32.564Z
Learning: Refer to each benchmark's README for benchmark-specific testing details.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T05:43:32.564Z
Learning: Use the customization guide when adding evaluation harnesses.
📚 Learning: 2026-07-06T23:55:42.908Z
Learnt from: cdgamarose-nv
Repo: NVIDIA-AI-Blueprints/aiq PR: 311
File: src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2:74-81
Timestamp: 2026-07-06T23:55:42.908Z
Learning: In agent test files (e.g., tests/aiq_agent/agents/*/test_agent.py), avoid brittle assertions that match exact substrings from prompt template files (such as *.j2 prompt wording). Prompt wording can change frequently, so instead assert structural/behavioral properties (e.g., that the prompt builder is called, that required sections/fields are present via stable markers, that the model output/agent behavior conforms to an expected schema, or that key actions are taken) rather than matching literal prompt text.

Applied to files:

  • tests/aiq_agent/agents/shallow_researcher/test_agent.py
  • tests/aiq_agent/agents/report_rewriter/test_agent.py
🔇 Additional comments (12)
src/aiq_agent/agents/report_rewriter/agent.py (1)

205-205: LGTM!

src/aiq_agent/agents/shallow_researcher/agent.py (1)

220-224: LGTM!

Also applies to: 260-264, 321-325, 408-422

tests/aiq_agent/agents/shallow_researcher/test_agent.py (1)

755-790: LGTM!

tests/aiq_agent/jobs/test_runner.py (1)

66-66: LGTM!

Also applies to: 496-499, 667-788

tests/aiq_agent/agents/report_rewriter/test_agent.py (1)

38-39: LGTM!

Also applies to: 59-59

docs/source/deployment/production.md (1)

54-56: Reject unsafe AIQ_BACKUP_DIR values instead of only documenting the requirement.

The parameter expansion only checks that the variable is set. AIQ_BACKUP_DIR=./backups or a path inside the repository still passes and can write sensitive dumps into the working tree. Resolve the path and fail unless it is absolute and outside the repository.

As per path instructions, review this deployment command for accuracy; as per coding guidelines, do not leave generated local artifacts in the repository.

Sources: Coding guidelines, Path instructions

docs/source/customization/configuration-reference.md (1)

709-709: LGTM!

docs/source/deployment/observability.md (1)

54-63: LGTM!

Also applies to: 285-289, 308-310

src/aiq_agent/agents/deep_researcher/custom_middleware.py (1)

45-45: LGTM!

Also applies to: 726-740, 958-963, 1087-1089, 1254-1259, 1368-1373

frontends/aiq_api/src/aiq_api/jobs/runner.py (2)

1142-1146: 🔒 Security & Privacy

Make context restoration unconditional.

The reset calls occur after _flush_event_store(), thread joining, and _teardown_sandbox(). If any cleanup operation raises, the finally block exits before the context resets. A reused worker can retain the previous job's user identity, conversation ID, or authentication token.

Wrap the cleanup body in an inner try and move all context resets into its finally. Add a test that forces cleanup failure and verifies that the outer context remains unchanged. The supplied tests/aiq_agent/jobs/test_runner.py test at Lines 669-788 covers normal agent failure, not cleanup failure.

As per path instructions: “Treat API, auth, and job-runner changes as externally visible contracts” and check “cross-user data isolation.”

Source: Path instructions


40-40: LGTM!

Also applies to: 51-51, 425-425, 504-510, 533-533, 664-666, 783-794, 808-809, 829-829, 847-847, 1099-1107

tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py (1)

53-53: LGTM!

Also applies to: 62-62, 1062-1085, 1391-1405, 1767-1785, 1948-1964

Comment thread docs/source/deployment/production.md Outdated
Signed-off-by: Ajay Thorve <athorve@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/source/deployment/production.md (1)

90-117: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Make the restore check fail closed.

Add set -euo pipefail and -v ON_ERROR_STOP=1 to every psql command. Register an EXIT trap before creating the disposable databases to remove both databases after failures or interruption. Otherwise, a failed pg_restore can be followed by successful cleanup, causing the restore check to report success.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/source/deployment/production.md` around lines 90 - 117, Update the
restore-check shell block to begin with set -euo pipefail, add -v
ON_ERROR_STOP=1 to every psql invocation including table checks and cleanup, and
register an EXIT trap before creating either disposable database that drops both
restore-check databases. Preserve the existing restore and verification commands
while ensuring failures or interruptions trigger cleanup and a nonzero exit.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 69-74: Define the PostgreSQL container name once as the
AIQ_POSTGRES_CONTAINER environment variable and replace every hard-coded
aiq-postgres reference in the backup and restore commands, including the
commands near the later referenced sections. Remove the need for operators to
edit individual commands manually while preserving the existing dump and restore
behavior.
- Around line 65-67: Initialize jobs_tmp and checkpoints_tmp before the mktemp
calls, then install the cleanup trap before creating either temporary archive.
Make the trap null-safe so it can safely run when one or both variables are
still unset, while preserving cleanup of successfully created files.
- Around line 72-74: Update the production backup procedure around the pg_dump
command to use the same checkpoint database configured and initialized by the
Helm deployment, consistently using aiq_checkpoints for the archive. Do not
leave the documented backup targeting aiq_jobs unless deployment-specific backup
and restore commands are added.

---

Outside diff comments:
In `@docs/source/deployment/production.md`:
- Around line 90-117: Update the restore-check shell block to begin with set
-euo pipefail, add -v ON_ERROR_STOP=1 to every psql invocation including table
checks and cleanup, and register an EXIT trap before creating either disposable
database that drops both restore-check databases. Preserve the existing restore
and verification commands while ensuring failures or interruptions trigger
cleanup and a nonzero exit.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 775697ea-163b-4b3b-9423-40ee92d1354e

📥 Commits

Reviewing files that changed from the base of the PR and between ded5243 and de941cb.

📒 Files selected for processing (1)
  • docs/source/deployment/production.md
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: Run Harbor skill eval
  • GitHub Check: Lint and Hooks
  • GitHub Check: Pytest and Coverage
🧰 Additional context used
📓 Path-based instructions (4)
**/*

📄 CodeRabbit inference engine (CONTRIBUTING.md)

**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.

**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat each sources/* package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.

Files:

  • docs/source/deployment/production.md
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}

📄 CodeRabbit inference engine (AGENTS.md)

Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr, and resolve API keys at runtime.

Files:

  • docs/source/deployment/production.md
docs/source/**/*

📄 CodeRabbit inference engine (AGENTS.md)

Update canonical documentation under docs/source/ when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.

Files:

  • docs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.

Files:

  • docs/source/deployment/production.md
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T05:53:12.966Z
Learning: Run the project test suite with `uv run pytest`; use coverage with `uv run pytest --cov=src/aiq_agent --cov-report=html`, and run MCP tests using the MCP project environment and lockfile.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T05:53:12.966Z
Learning: Refer to each benchmark's README for benchmark-specific testing details and consult the Customization guide when adding evaluation harnesses.

Comment thread docs/source/deployment/production.md Outdated
Comment thread docs/source/deployment/production.md Outdated
Comment thread docs/source/deployment/production.md Outdated
Signed-off-by: Ajay Thorve <athorve@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 83-91: Update the backup procedure around the jobs_tmp and
checkpoints_tmp pg_dump commands to define a consistent cross-database recovery
window: pause workers and related writes before both dumps and resume them
afterward, or document and implement a coordinated snapshot procedure with its
recovery guarantee. Ensure the documented procedure covers both aiq_jobs and
aiq_checkpoints rather than treating the dumps as independent snapshots.
- Around line 70-82: Update the backup cleanup flow around cleanup and the two
archive moves to track whether both final moves completed successfully. On any
partial failure, remove both published archive files as well as temporary files;
only preserve the final archives after both moves succeed, or alternatively
publish a completion manifest after that point.
- Around line 111-129: Update the restore verification block to enable set -euo
pipefail, define a null-safe cleanup function that drops both disposable
databases while tolerating cleanup failures, and install it with an EXIT trap
before either database is created. Remove the existing normal-completion DROP
commands and clear the trap only after both restore checks complete and both
databases have been dropped.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: ae942662-d5df-462a-9439-915e111976d7

📥 Commits

Reviewing files that changed from the base of the PR and between de941cb and 0b0acc9.

📒 Files selected for processing (1)
  • docs/source/deployment/production.md
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: Run Harbor skill eval
  • GitHub Check: Pytest and Coverage
  • GitHub Check: Lint and Hooks
🧰 Additional context used
📓 Path-based instructions (4)
**/*

📄 CodeRabbit inference engine (CONTRIBUTING.md)

**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.

**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat each sources/* package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.

Files:

  • docs/source/deployment/production.md
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}

📄 CodeRabbit inference engine (AGENTS.md)

Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr, and resolve API keys at runtime.

Files:

  • docs/source/deployment/production.md
docs/source/**/*

📄 CodeRabbit inference engine (AGENTS.md)

Update canonical documentation under docs/source/ when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.

Files:

  • docs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.

Files:

  • docs/source/deployment/production.md
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T06:05:41.029Z
Learning: Use `./scripts/start_cli.sh --verbose` or set `verbose: true` in workflow configuration when debugging with verbose logging.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T06:05:41.029Z
Learning: Use Phoenix tracing for debugging by starting the Phoenix server, enabling tracing in agent configuration, and opening `http://localhost:6006`.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T06:05:41.029Z
Learning: For common issues, reinstall the package in editable mode for import errors, check environment variables for authentication failures, check configuration for missing tools, and run `pre-commit clean` for pre-commit cache issues.
🔇 Additional comments (2)
docs/source/deployment/production.md (2)

43-55: LGTM!


98-107: LGTM!

Comment thread docs/source/deployment/production.md
Comment thread docs/source/deployment/production.md
Comment thread docs/source/deployment/production.md
Signed-off-by: Ajay Thorve <athorve@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/deployment/production.md`:
- Around line 50-52: Update the backup procedure surrounding the two PostgreSQL
snapshots to document recovery when set -e stops on a pg_dump or
archive-publication failure. Ensure the failure path confirms cleanup has
completed and resumes the paused API, workers, and other writers before any
retry, or add a pause/resume wrapper with an EXIT trap that guarantees this
behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 2e69162d-16c3-4d0e-8fee-b1ccdd345add

📥 Commits

Reviewing files that changed from the base of the PR and between 0b0acc9 and 53a9f0e.

📒 Files selected for processing (1)
  • docs/source/deployment/production.md
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: Run Harbor skill eval
  • GitHub Check: Pytest and Coverage
  • GitHub Check: Lint and Hooks
🧰 Additional context used
📓 Path-based instructions (4)
**/*

📄 CodeRabbit inference engine (CONTRIBUTING.md)

**/*: Do not include secrets, credentials, private hostnames, internal-only logs, customer data, or generated local artifacts.
Add or update tests for behavior changes.

**/*: For substantial behavior, authentication, UI, or architecture changes, open a design discussion before coding rather than landing a large unreviewed change.
Keep changes scoped to this repository and avoid editing adjacent repositories; treat each sources/* package independently and prefer the smallest package-scoped change.
Keep pull requests scoped, avoid unrelated files and generated artifacts, provide validation evidence, and ensure every commit has DCO sign-off.

Files:

  • docs/source/deployment/production.md
**/*.{py,pyi,js,jsx,ts,tsx,yml,yaml,json,env,md}

📄 CodeRabbit inference engine (AGENTS.md)

Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr, and resolve API keys at runtime.

Files:

  • docs/source/deployment/production.md
docs/source/**/*

📄 CodeRabbit inference engine (AGENTS.md)

Update canonical documentation under docs/source/ when behavior, configuration, or workflows change; do not duplicate full documentation pages in skills.

Files:

  • docs/source/deployment/production.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.

Files:

  • docs/source/deployment/production.md
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T06:13:39.273Z
Learning: Run the documented test commands with `uv`, including coverage, targeted tests, and the separate MCP project test environment.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T06:13:39.273Z
Learning: Refer to each benchmark's README for benchmark-specific testing details and consult the customization guide when adding evaluation harnesses.
Learnt from: CR
Repo: NVIDIA-AI-Blueprints/aiq

Timestamp: 2026-08-11T06:13:39.273Z
Learning: Use verbose CLI logging, Phoenix tracing, and the documented troubleshooting steps when debugging.
🔇 Additional comments (2)
docs/source/deployment/production.md (2)

74-74: LGTM!

Also applies to: 85-87, 103-103


119-121: LGTM!

Also applies to: 123-131

Comment thread docs/source/deployment/production.md
Signed-off-by: Ajay Thorve <athorve@nvidia.com>
Signed-off-by: Ajay Thorve <athorve@nvidia.com>

@tanleach tanleach left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good updates to docs and logging.

@tanleach

Copy link
Copy Markdown
Collaborator

/ok to test e16dc33

@tanleach

Copy link
Copy Markdown
Collaborator

/merge

@rapids-bot
rapids-bot Bot merged commit 81a93cc into release/2.2 Aug 11, 2026
11 checks passed
@AjayThorve AjayThorve added this to the v2.2 milestone Aug 12, 2026
@coderabbitai coderabbitai Bot mentioned this pull request Aug 14, 2026
6 tasks
@AjayThorve
AjayThorve deleted the ajay/vdr-2.2-release-fixes branch September 2, 2026 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants