feat: isolate and attest OpenShell jobs - #298
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughOpenShell execution now creates policy-bound sandboxes per deep-research job, verifies effective policy and certified versions, and performs fail-closed cleanup. Agent execution adds cancellation finalization, path and output guards, artifact redaction, updated tooling, documentation, and extensive validation. ChangesPer-job OpenShell runtime
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
204-216: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
_cleanup_failedis sticky across mid-life retries, causing false-negative terminal cleanup reports.
_safe_close(and its cousin inopenshell.py's_exit_context) setsself._cleanup_failed = Trueon any close exception, but_safe_closeis also invoked from_reset_session()when tearing down a stale session before recreating it on a recoverable error — a mid-job event unrelated to terminal cleanup. Since_cleanup_failedis never reset, a transient failure during that retry path permanently poisonscleanup_succeeded, sofinalize()will report "failed" for a job whose actual terminalclose()succeeded without error. This undermines the truthful-cleanup-reporting goal called out in the PR description.Consider tracking retry-teardown failures separately from terminal-cleanup failures, or resetting
_cleanup_failedat the start of_reset_session()since a fresh session is about to replace the stale one.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py` around lines 204 - 216, The cleanup failure flag in `_safe_close` is being left sticky across session retries, so a stale-session close error can incorrectly make `cleanup_succeeded` fail later. Update `BaseSandbox._safe_close` and the retry path in `_reset_session()` to distinguish mid-life teardown from final terminal cleanup, or reset `_cleanup_failed` when starting a fresh session replacement. Keep the terminal reporting used by `cleanup_succeeded`/`finalize()` tied only to the actual final close, and apply the same handling to the corresponding `_exit_context` logic in `openshell.py` if it shares the same flag.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 667-673: In runner.py, the sandbox cleanup path around finalize()
only logs when an exception is raised, so a falsy cleanup result is missed.
Update the finalize handling in the job runner to inspect the return value from
sandbox_runtime.finalize(...) and emit the same warning path when it returns
False or another falsy cleanup_succeeded value, while still keeping exception
handling non-fatal. Use the existing finalize, job_id, and logger.warning call
site to keep the behavior aligned with the current cleanup flow.
In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`:
- Around line 278-298: The finalize() idempotency guard in DeepResearcherRuntime
only protects the _finalized flag, so concurrent calls can skip the actual
cleanup and return a stale success from cleanup_succeeded. Update finalize() to
keep the same lock across the full cleanup path, including terminate()/close(),
the cleanup result read, and the cleanup event emission, so only one caller
performs cleanup and others wait for the real outcome. Use the existing
finalize(), _finalize_lock, _finalized, _emit_cleanup, terminate(), and close()
symbols to make the change.
In `@src/aiq_agent/agents/deep_researcher/register.py`:
- Around line 298-306: Add a unit test in test_agent.py that covers _run
cancellation behavior in DeepResearcher register.py: simulate
asyncio.CancelledError during _run, assert the exception is re-raised, and
verify active_agent.finalize is called with interrupted=True only when
owns_active_agent is true. Use the _run coroutine and the owns_active_agent /
active_agent.finalize path to confirm no finalization happens for non-owned
agents and that the finally block triggers the correct interrupted flag for
owned agents.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 368-423: `_os_context` is being accessed concurrently between
`_create_session()` and out-of-band cleanup paths like `close()`, `terminate()`,
and `_terminate_session()`, which can race during cancellation. Protect all
reads/writes/clears of `self._os_context` with the existing `self._state_lock`,
matching how `self._session` is guarded in the base class, and ensure
`_exit_context()` and the `_create_session()` setup/teardown path in
`OpenShellSandboxProvider` use the same lock consistently.
- Around line 406-424: The attestation success event is emitted too early in
OpenShellSandbox creation, before session construction is guaranteed to succeed.
In openshell.py, adjust the _create_session flow so _attest(self, os_sandbox)
and its “succeeded” emission happen only after OpenShellSandbox(...) is
successfully constructed, or add a failure compensation path if adapter
construction throws. Keep the logging and cleanup around os_sandbox.__enter__
and _exit_context() unchanged, but ensure the attestation state reflects the
final outcome of _create_session rather than the pre-construction step.
In `@tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py`:
- Around line 151-441: Add a regression test around OpenShellSandboxProvider
that exercises _reset_session() after a stale-session close failure, then
performs a later successful terminal close and verifies cleanup_succeeded
remains True. Use the existing _reset_session and close behavior in the
provider/test setup to force the first cleanup failure, then assert the second
close does not stay poisoned by _cleanup_failed.
---
Outside diff comments:
In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py`:
- Around line 204-216: The cleanup failure flag in `_safe_close` is being left
sticky across session retries, so a stale-session close error can incorrectly
make `cleanup_succeeded` fail later. Update `BaseSandbox._safe_close` and the
retry path in `_reset_session()` to distinguish mid-life teardown from final
terminal cleanup, or reset `_cleanup_failed` when starting a fresh session
replacement. Keep the terminal reporting used by
`cleanup_succeeded`/`finalize()` tied only to the actual final close, and apply
the same handling to the corresponding `_exit_context` logic in `openshell.py`
if it shares the same flag.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 63446849-608f-4056-8b4b-58895c90df18
📒 Files selected for processing (16)
configs/config_openshell.ymlconfigs/openshell/aiq-research-policy.yamldocs/source/architecture/agents/sandbox.mdfrontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/README.mdscripts/setup_openshell.shsrc/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.mdsrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (10)
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
scripts/README.mdtests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/agent.pyconfigs/config_openshell.ymlfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyconfigs/openshell/aiq-research-policy.yamldocs/source/architecture/agents/sandbox.mdsrc/aiq_agent/agents/deep_researcher/sandbox/config.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pyscripts/setup_openshell.shsrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.md
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/agent.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.md
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_openshell.ymlconfigs/openshell/aiq-research-policy.yaml
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update the docs under docs/source/ when behavior, configuration, or workflows change
Files:
docs/source/architecture/agents/sandbox.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/architecture/agents/sandbox.md
**/*config*.py
📄 CodeRabbit inference engine (AGENTS.md)
Config schemas must inherit from FunctionBaseConfig and YAML _type names must come from the registered config class
Files:
src/aiq_agent/agents/deep_researcher/sandbox/config.py
🔇 Additional comments (20)
configs/config_openshell.yml (1)
1-2: LGTM!Also applies to: 142-165
configs/openshell/aiq-research-policy.yaml (1)
24-29: LGTM!scripts/setup_openshell.sh (2)
17-17: LGTM!Also applies to: 51-51, 61-61, 80-80, 93-95, 108-108, 1010-1010, 1062-1062, 1075-1075
155-162: LGTM!Validation of
LANDLOCK_COMPATIBILITYruns inresolve_policy()beforeemit_policy_header()writes it into the now-unquoted heredoc, so only the two whitelisted literals ever reach the generated policy file — no injection risk from the quoting change.Also applies to: 878-884, 898-898, 922-925
docs/source/architecture/agents/sandbox.md (1)
9-11: LGTM!Description of per-job physical sandboxes, attestation, fail-closed network/Landlock behavior, and
sandbox.attestation/sandbox.cleanupevents is consistent with the config and setup-script changes in this cohort.Also applies to: 24-30, 40-46, 58-58
scripts/README.md (1)
54-65: LGTM!Matches the setup script's new default (
CREATE_SANDBOX=false),--landlock-compatibility/--create-shared-debug-sandboxflags, and Landlock defaults.Also applies to: 190-190
src/aiq_agent/agents/deep_researcher/sandbox/README.md (2)
40-50: LGTM!Fail-closed conditions, attestation/policy-revision requirements, debug shared-sandbox opt-in, and troubleshooting guidance are consistent with the per-job sandbox design described across the config, policy, and setup-script changes in this cohort.
Also applies to: 65-65, 160-165, 211-217, 238-254, 277-279, 287-302
122-152: 📐 Maintainability & Code QualityThe sandbox example matches the current schema.
SandboxConfigexposesnetwork: NetworkPolicywithmode/allow, andproviders.openshell.imageis a real field, so this example is consistent with the code.> Likely an incorrect or invalid review comment.src/aiq_agent/agents/deep_researcher/sandbox/config.py (1)
93-152: LGTM!src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
22-22: LGTM!Also applies to: 36-36, 90-127, 136-162, 191-193, 207-208, 495-504, 517-517
src/aiq_agent/agents/deep_researcher/agent.py (1)
155-158: LGTM!src/aiq_agent/agents/deep_researcher/register.py (1)
18-18: LGTM!Also applies to: 240-245, 272-272
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
649-650: LGTM!src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (3)
130-196: LGTM!
199-262: LGTM!Also applies to: 291-292, 359-359
231-243: 🎯 Functional Correctness
openis an explicit unrestricted mode, so_validate_policy_networkonly needs to enforceblockedandallowlist.> Likely an incorrect or invalid review comment.src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-157
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
16-40: LGTM!Also applies to: 68-148, 564-578
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
35-35: LGTM!Also applies to: 306-377
tests/aiq_agent/jobs/test_runner.py (1)
1911-1920: LGTM!
|
/ok to test b6b058e |
|
Final CodeRabbit remediation is in |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
390-390: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winDo not delete attached debug sandboxes by default.
This shared-name branch attaches to an existing sandbox, but
oscfg.delete_on_exitnow defaults toTrue, so normal cleanup can delete a sandbox the job did not create. Keep deletion disabled for shared/debug attachment unless there is a separate explicit opt-in.Suggested fix
- sandbox_kwargs.update(sandbox=shared_name, delete_on_exit=oscfg.delete_on_exit) + sandbox_kwargs.update(sandbox=shared_name, delete_on_exit=False)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` at line 390, The shared-name attachment path in openshell.py is passing through delete_on_exit from oscfg, which can accidentally delete an existing debug sandbox that this job did not create. Update the sandbox_kwargs.update(...) call in the shared-name branch to disable deletion by default for attached sandboxes, and only allow cleanup when there is an explicit opt-in separate from the shared/debug attach flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Line 390: The shared-name attachment path in openshell.py is passing through
delete_on_exit from oscfg, which can accidentally delete an existing debug
sandbox that this job did not create. Update the sandbox_kwargs.update(...) call
in the shared-name branch to disable deletion by default for attached sandboxes,
and only allow cleanup when there is an explicit opt-in separate from the
shared/debug attach flow.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 2ad4ae39-8901-4264-be74-600780d237d0
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
🔇 Additional comments (9)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
658-683: LGTM!src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
150-162: LGTM!Also applies to: 194-209, 279-322
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-46: LGTM!Also applies to: 271-295, 316-334, 407-430, 432-501, 528-600
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-157, 210-216
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
26-27: LGTM!Also applies to: 37-37, 82-83, 123-130, 400-438, 490-497, 500-529, 532-562, 565-620, 623-674, 814-841
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-21: LGTM!Also applies to: 308-330, 332-334, 340-359, 361-380, 381-429
tests/aiq_agent/jobs/test_runner.py (1)
1911-1919: LGTM!Also applies to: 1921-1930
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-336: LGTM!scripts/smoke_openshell_isolation.py (1)
29-52: LGTM!Also applies to: 55-72, 75-96, 98-142, 145-154, 156-188
e5b69ab to
b05c93b
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
649-650: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUnguarded
event_store.flush()in terminalfinallyblock can mask the job result.Every other cleanup call in this
finallypath (_teardown_sandbox) is deliberately exception-safe, per its own docstring: cleanup must never replace the job result. This newevent_store.flush()call has no try/except — if_flush()raises, the exception propagates out of thefinallyblock and overrides whatever result/exception the job actually produced. It's also invoked synchronously on the event loop rather than viaasyncio.to_threadlike the sandbox teardown right above it, so if_flush()does any blocking I/O it will stall the worker.🛠️ Proposed fix: make flush best-effort and non-blocking
- if event_store is not None and hasattr(event_store, "flush"): - event_store.flush() + if event_store is not None and hasattr(event_store, "flush"): + try: + await asyncio.to_thread(event_store.flush) + except Exception: # noqa: BLE001 - flush must never replace the job result + logger.warning("Event store flush failed for job %s", job_id, exc_info=True)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontends/aiq_api/src/aiq_api/jobs/runner.py` around lines 649 - 650, The new event_store.flush() call in the runner’s terminal finally path should be made best-effort like _teardown_sandbox so it cannot override the job outcome. Update the cleanup in runner.py around the event_store handling to wrap flush in exception-safe handling, and if the flush implementation may block, invoke it off the event loop (consistent with the sandbox teardown pattern) instead of calling it synchronously. Use the existing runner cleanup block and event_store.flush as the locating symbols.src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
403-405: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winRespect
delete_on_exitfor job-owned sandboxes.The per-job path always passes
delete_on_exit=True, so a configureddelete_on_exit=Falseis ignored. Keep the shared-attachment override atFalse, but useoscfg.delete_on_exitfor owned sandboxes.Proposed fix
sandbox_kwargs.update( spec=_build_sandbox_spec(policy=policy, image=oscfg.image, job_id=self.job_id), - delete_on_exit=True, + delete_on_exit=oscfg.delete_on_exit, )As per path instructions, “Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts, HITL behavior, and failure handling.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` around lines 403 - 405, The sandbox setup in openshell.py is hardcoding delete_on_exit=True for the job-owned path, which overrides the configured behavior. Update the sandbox_kwargs.update call in the owned-sandbox creation flow to use oscfg.delete_on_exit, while keeping the shared-attachment override path explicitly set to False. Make sure the change is applied in the same branch that builds the sandbox spec via _build_sandbox_spec and assigns self.job_id.Source: Path instructions
src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
96-106: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winEnforce the shared-sandbox debug opt-in in this config model.
Line 154 treats
existing_sandbox_name/sandbox_nameas a shared attachment, but the validator never rejectsallow_shared_sandbox=False. Add an explicit check here so the NAT/runtime config cannot accept a non-isolated sandbox unless the debug opt-in is present.Proposed fix
if self.provider == "openshell": shared_name = self.existing_sandbox_name or self.sandbox_name + if shared_name and not self.allow_shared_sandbox: + raise ValueError("existing_sandbox_name/sandbox_name requires allow_shared_sandbox=true") if not shared_name and not self.attest: raise ValueError("Per-job OpenShell creation requires attest=true")As per path instructions, “Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts, HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation, async cancellation, checkpointing, or data-source selection without focused tests and docs.”
Also applies to: 149-157
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py` around lines 96 - 106, The shared-sandbox debug opt-in is not enforced in this config model, so a non-isolated attachment can be accepted even when allow_shared_sandbox is false. Add an explicit validation check in the same config class that defines existing_sandbox_name, sandbox_name, and allow_shared_sandbox to reject any use of the shared sandbox fields unless the debug opt-in is enabled. Make sure the validator covers both the primary and deprecated alias fields consistently.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 649-650: The new event_store.flush() call in the runner’s terminal
finally path should be made best-effort like _teardown_sandbox so it cannot
override the job outcome. Update the cleanup in runner.py around the event_store
handling to wrap flush in exception-safe handling, and if the flush
implementation may block, invoke it off the event loop (consistent with the
sandbox teardown pattern) instead of calling it synchronously. Use the existing
runner cleanup block and event_store.flush as the locating symbols.
In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`:
- Around line 96-106: The shared-sandbox debug opt-in is not enforced in this
config model, so a non-isolated attachment can be accepted even when
allow_shared_sandbox is false. Add an explicit validation check in the same
config class that defines existing_sandbox_name, sandbox_name, and
allow_shared_sandbox to reject any use of the shared sandbox fields unless the
debug opt-in is enabled. Make sure the validator covers both the primary and
deprecated alias fields consistently.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 403-405: The sandbox setup in openshell.py is hardcoding
delete_on_exit=True for the job-owned path, which overrides the configured
behavior. Update the sandbox_kwargs.update call in the owned-sandbox creation
flow to use oscfg.delete_on_exit, while keeping the shared-attachment override
path explicitly set to False. Make sure the change is applied in the same branch
that builds the sandbox spec via _build_sandbox_spec and assigns self.job_id.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 1a4f355e-9777-4ffb-8f9e-c7419c6c4804
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
🔇 Additional comments (9)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
658-683: LGTM! Thefinalizefast-path now logs a warning on a falsycleanup_succeededresult, matching the previously requested fix and theTestTerminalTeardowntest suite (test_runtime_finalizer_false_result_is_logged).src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
36-36: LGTM!Also applies to: 92-95, 108-143, 160-162, 191-209, 279-322
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-36: LGTM!Also applies to: 46-46, 132-260, 272-273, 295-295, 316-316, 334-334, 390-392, 408-503, 530-602
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-157, 204-216
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
16-43: LGTM!Also applies to: 71-161, 163-270, 271-705, 828-870
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-21: LGTM!Also applies to: 37-37, 307-331, 332-335, 337-429
tests/aiq_agent/jobs/test_runner.py (1)
1911-1919: LGTM!Also applies to: 1921-1931
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-337: LGTM!scripts/smoke_openshell_isolation.py (1)
1-52: LGTM!Also applies to: 55-73, 75-96, 98-142, 145-155, 156-184, 185-188, 192-197
b05c93b to
24264c4
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
666-672: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDocstring doesn't mention the new
finalize()-first path.The docstring only describes the terminate()/close() routing, but the primary path (lines 675-682) now delegates to
finalize()when present, falling back to terminate/close only for runtimes without it. Worth a one-line update for future maintainers.📝 Suggested docstring update
"""Release sandbox resources on a terminal path (best-effort, never raises). + Prefers ``finalize(interrupted=...)`` when the runtime exposes it (logs a warning on a + falsy/failed result). Falls back to the legacy routing below for runtimes without a + ``finalize`` method: Interrupted jobs (cancel/timeout) call ``terminate()`` so a still-running ``execute`` is forcibly preempted; normal paths call ``close()`` gracefully. Both are idempotent. This runs off the event loop (``asyncio.to_thread``) so the SDK session close cannot block the worker. """🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontends/aiq_api/src/aiq_api/jobs/runner.py` around lines 666 - 672, Update the _teardown_sandbox docstring in runner.py to mention the new finalize()-first behavior before the existing terminate()/close() fallback description. Keep the note brief but explicit that sandbox_runtime.finalize() is used when available, with terminate() for interrupted jobs and close() for the remaining fallback path. Use the _teardown_sandbox symbol so maintainers can quickly find the routing logic.src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
155-156: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winSanitize base lifecycle exception logging.
Both event emission and session cleanup are best-effort paths; avoid
exc_info=Trueso secret-bearing callback/SDK details are not written to logs.Suggested fix
- except Exception: # noqa: BLE001 - event persistence is non-critical - logger.warning("Sandbox event emission failed for %s", self.sandbox_name, exc_info=True) + except Exception as exc: # noqa: BLE001 - event persistence is non-critical + logger.warning("Sandbox event emission failed for %s (%s)", self.sandbox_name, type(exc).__name__) @@ - except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("Sandbox %s cleanup failed", self.sandbox_name, exc_info=True) + logger.warning("Sandbox %s cleanup failed (%s)", self.sandbox_name, type(exc).__name__)As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”
Also applies to: 209-211
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py` around lines 155 - 156, The best-effort exception logging in the sandbox lifecycle currently includes exc_info=True, which can leak secret-bearing callback or SDK details into logs. Update the exception handlers in the base lifecycle methods (including the sandbox event emission path and the related session cleanup path) to log only a sanitized warning message without exception stack traces or raw exception objects, keeping the existing logger.warning calls and using the same identifiers like self.sandbox_name for context.Source: Coding guidelines
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
598-602: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winSanitize SDK cleanup exception logging.
The cleanup path should report failure without logging raw SDK exception text or traceback details.
Suggested fix
- except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("OpenShell sandbox %s context cleanup failed", self.sandbox_name, exc_info=True) + logger.warning( + "OpenShell sandbox %s context cleanup failed (%s)", + self.sandbox_name, + type(exc).__name__, + )As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` around lines 598 - 602, The OpenShell cleanup path in the ctx.__exit__ exception handler is logging raw SDK exception details via exc_info=True, which can expose sensitive data. Update the cleanup logging in openshell.py to report only that sandbox context cleanup failed, without including the exception text or traceback, while still preserving the _cleanup_failed flag and the existing warning in the cleanup flow.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`:
- Around line 294-295: The cleanup warning in the terminal exception handler is
logging raw exception details via exc_info=True, which can leak sensitive
SDK/event-sink messages. Update the exception handling in the sandbox cleanup
path around the cleanup logic in deepagents_runtime.py (including the related
block near the other cleanup handler mentioned in the comment) to log only the
exception type or a sanitized summary, and remove full traceback/exception
payload logging from logger.warning while keeping the failure signal.
- Around line 154-156: The shared-sandbox alias handling in DeepAgentsRuntime is
too permissive because self.existing_sandbox_name or self.sandbox_name silently
prefers one value when both are set. Update the validation around shared_name so
DeepAgentsRuntime explicitly detects when existing_sandbox_name and deprecated
sandbox_name are both provided with different values and raises a ValueError
instead of choosing one. Keep the existing allow_shared_sandbox guard, and use
the existing_sandbox_name/sandbox_name checks in the same block to enforce a
single unambiguous sandbox name.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 419-424: The attestation log in the OpenShell sandbox flow is
including an environment-specific gateway value that should not be emitted.
Update the logger.info call in the attestation path of the OpenShell provider to
remove oscfg.gateway from the message and its argument list, while keeping the
remaining identifiers like backend.id, sandbox_ref.name, policy_version, and
shared so the log stays useful without exposing deployment identifiers.
---
Outside diff comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 666-672: Update the _teardown_sandbox docstring in runner.py to
mention the new finalize()-first behavior before the existing
terminate()/close() fallback description. Keep the note brief but explicit that
sandbox_runtime.finalize() is used when available, with terminate() for
interrupted jobs and close() for the remaining fallback path. Use the
_teardown_sandbox symbol so maintainers can quickly find the routing logic.
In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py`:
- Around line 155-156: The best-effort exception logging in the sandbox
lifecycle currently includes exc_info=True, which can leak secret-bearing
callback or SDK details into logs. Update the exception handlers in the base
lifecycle methods (including the sandbox event emission path and the related
session cleanup path) to log only a sanitized warning message without exception
stack traces or raw exception objects, keeping the existing logger.warning calls
and using the same identifiers like self.sandbox_name for context.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 598-602: The OpenShell cleanup path in the ctx.__exit__ exception
handler is logging raw SDK exception details via exc_info=True, which can expose
sensitive data. Update the cleanup logging in openshell.py to report only that
sandbox context cleanup failed, without including the exception text or
traceback, while still preserving the _cleanup_failed flag and the existing
warning in the cleanup flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 870175a1-89e7-4cd7-b6c1-709d76de1100
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pyscripts/smoke_openshell_isolation.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pyscripts/smoke_openshell_isolation.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
🔇 Additional comments (10)
frontends/aiq_api/src/aiq_api/jobs/runner.py (2)
675-682: 🩺 Stability & AvailabilityFalsy
finalize()return now logged — past review comment resolved.This correctly closes the gap flagged previously: a non-raising
finalize()failure is no longer silently dropped.
639-664: LGTM!src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
193-196: LGTM!Also applies to: 210-211, 281-293, 297-303
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-46: LGTM!Also applies to: 271-295, 316-334, 390-418, 427-503, 530-597
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-154, 213-216
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
26-27: LGTM!Also applies to: 37-37, 82-83, 123-130, 400-438, 458-498, 525-532, 535-709, 849-876
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-21: LGTM!Also applies to: 336-343, 388-437
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-337: LGTM!tests/aiq_agent/jobs/test_runner.py (1)
1911-1924: LGTM!Also applies to: 1935-1945
scripts/smoke_openshell_isolation.py (1)
75-95: LGTM!Also applies to: 166-171
24264c4 to
8e2287b
Compare
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
597-601: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winSanitize OpenShell context cleanup failures.
exc_info=Truecan log raw SDK exception messages during terminal cleanup. Log only the exception type while preserving_cleanup_failed = True. As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”Suggested fix
- except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("OpenShell sandbox %s context cleanup failed", self.sandbox_name, exc_info=True) + logger.warning( + "OpenShell sandbox %s context cleanup failed (%s)", + self.sandbox_name, + type(exc).__name__, + )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` around lines 597 - 601, The OpenShell cleanup warning in the context exit path is leaking raw exception details via exc_info=True, which can expose sensitive SDK messages. In the ctx.__exit__ exception handler inside the OpenShell sandbox cleanup logic, keep setting _cleanup_failed = True but change the logger.warning call to report only the exception type (using the caught exception object) without full traceback or message details, and keep the context tied to the existing sandbox_name identifier.Source: Coding guidelines
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
149-156: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winDo not log raw event-sink or cleanup exceptions.
Both handlers use
exc_info=True, which can emit secret-bearing exception messages from SDKs or persistence sinks. Keep these best-effort paths non-fatal but log onlytype(exc).__name__. As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”Suggested fix
- except Exception: # noqa: BLE001 - event persistence is non-critical - logger.warning("Sandbox event emission failed for %s", self.sandbox_name, exc_info=True) + except Exception as exc: # noqa: BLE001 - event persistence is non-critical + logger.warning("Sandbox event emission failed for %s (%s)", self.sandbox_name, type(exc).__name__) @@ - except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("Sandbox %s cleanup failed", self.sandbox_name, exc_info=True) + logger.warning("Sandbox %s cleanup failed (%s)", self.sandbox_name, type(exc).__name__)Also applies to: 204-211
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py` around lines 149 - 156, The best-effort error handlers in _emit_event and the cleanup path are logging exception details with exc_info=True, which can leak secret-bearing messages from SDKs or persistence sinks. Update these handlers to keep failures non-fatal but log only the exception type name (for example via the caught Exception object), and remove traceback/raw exception output from the logger.warning calls while preserving sandbox_name context.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 680-681: The sandbox cleanup finalizer in `runner.py` still logs
full tracebacks via `logger.warning(..., exc_info=True)`, which can leak
secret-bearing details from unexpected SDK/event-sink exceptions. Update the
`finalize()` exception handler to log only the exception type/class name (using
the existing `job_id` context) and remove traceback emission so `Sandbox cleanup
failed for job %s` stays sanitized.
In `@tests/aiq_agent/jobs/test_runner.py`:
- Around line 1935-1945: The _teardown_sandbox path in aiq_api.jobs.runner still
logs finalize() exceptions with exc_info=True, which can leak raw exception
messages. Update the exception handling around runtime.finalize() to log only
the exception type or a sanitized summary, and add a regression test alongside
test_runtime_finalizer_false_result_is_logged that raises a RuntimeError with
sensitive text to confirm the warning does not expose the secret.
---
Outside diff comments:
In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py`:
- Around line 149-156: The best-effort error handlers in _emit_event and the
cleanup path are logging exception details with exc_info=True, which can leak
secret-bearing messages from SDKs or persistence sinks. Update these handlers to
keep failures non-fatal but log only the exception type name (for example via
the caught Exception object), and remove traceback/raw exception output from the
logger.warning calls while preserving sandbox_name context.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 597-601: The OpenShell cleanup warning in the context exit path is
leaking raw exception details via exc_info=True, which can expose sensitive SDK
messages. In the ctx.__exit__ exception handler inside the OpenShell sandbox
cleanup logic, keep setting _cleanup_failed = True but change the logger.warning
call to report only the exception type (using the caught exception object)
without full traceback or message details, and keep the context tied to the
existing sandbox_name identifier.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 30bb7492-02b8-46c4-87b0-b11bb1797bbf
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
🔇 Additional comments (9)
src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
154-162: LGTM!Also applies to: 202-217, 287-309, 328-329
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-46: LGTM!Also applies to: 272-295, 316-334, 390-502, 529-596
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 213-216
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
641-663: LGTM!Also applies to: 678-679
scripts/smoke_openshell_isolation.py (1)
75-95: LGTM!Also applies to: 166-189
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
26-27: LGTM!Also applies to: 37-37, 82-83, 123-130, 271-318, 411-449, 469-509, 536-720, 860-887
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-337: LGTM!tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-22: LGTM!Also applies to: 337-352, 398-463
tests/aiq_agent/jobs/test_runner.py (1)
1911-1934: LGTM!
|
/ok to test 9cec3b4 |
AjayThorve
left a comment
AjayThorve
left a comment
There was a problem hiding this comment.
I found five remaining blockers on the current head.
|
Can we split durable OpenShell provisioning from gateway lifecycle before merging? |
|
Lifecycle follow-up: commit |
|
/ok to test 0c25ad5 |
|
/ok to test 7baf293 |
|
/ok to test 170bdbf |
AjayThorve
left a comment
AjayThorve
left a comment
There was a problem hiding this comment.
One test-ownership concern on the current head.
|
Can we add a canonical OpenShell setup and deployment guide under |
|
Addressed in The guide now owns the operator contract: security boundary; Linux/Docker, macOS demo, Podman evaluation, remote gateway, and Windows/WSL support matrix; pinned CLI/SDK/gateway/adapter compatibility; lifecycle/service ownership; exact Linux, macOS, and remote flows; policy/config and Landlock pairing; expected per-job behavior; pytest-owned acceptance; sanitized inspection; and safe cleanup/troubleshooting. One-way references were added from The normal Sphinx build succeeds and the new guide/backlinks render without warnings. The requested warning-as-error build completes but remains nonzero on two pre-existing |
KyleZheng1284
added
the
BLOCKED
ChatGPT Codex Connector
BLOCKED — upstream OpenShell policy state and SDK capabilitiesMarking this PR blocked for the AI-Q 2.2 OpenShell integration. The policy-status failure has now been reproduced without AI-Q by creating a sandbox directly through the OpenShell 0.0.77 CLI on macOS/Docker Desktop:
AI-Q therefore correctly times out instead of exposing the execution adapter without authoritative The upstream lifecycle/API work is tracked in NVIDIA/OpenShell#2159. That issue also tracks the Python SDK gap for request-level labels and selector-based listing; template/container labels alone do not make Unblock criteria
Until those gates are met, this PR must not claim production acceptance or merge by weakening attestation, treating Current scope note: the policy-state contradiction is confirmed on macOS with OpenShell 0.0.77; a minimal Linux reproduction is still required to determine whether the runtime manifestation is driver/platform-specific. |
|
/ok to test ddc1b6d |
|
If i run |
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
there are some deps mismatches since the published langchain-openshell pkg |
|
/ok to test d74c86c |
|
@coderabbitai full review |
✅ Action performedFull review finished. |
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/deployment/openshell.md`:
- Around line 264-270: Remove the trailing line-continuation backslash from the
final “--policy offline” line in the macOS setup command, matching the completed
command blocks elsewhere in the document.
In `@scripts/openshell/check_openshell_readiness.py`:
- Around line 224-267: Track the SDK-returned sandbox name immediately after
client.create in the probe flow, and use it for all cleanup operations when
available. Update the finally block associated with the readiness probe to call
get, delete, wait_deleted, and _verify_absent with the returned name, while
retaining sandbox_name as the fallback if creation fails or no sandbox is
returned.
In `@scripts/openshell/install_gateway.sh`:
- Around line 96-115: Update the formula validation in the installed-formula
guard to reject bare “openshell” unconditionally. Only allow the fully qualified
“nvidia/openshell/openshell” value; remove the official_tap_present check and
ensure any other formula name triggers the existing ambiguity failure.
In `@src/aiq_agent/agents/deep_researcher/custom_middleware.py`:
- Around line 49-54: Make unresolved sandbox placeholder detection
whitespace-tolerant in the middleware’s existing guard logic: replace or
supplement _UNRESOLVED_SANDBOX_PATH_TOKENS with a regex that matches both
sandbox_artifact_dir and sandbox_workdir placeholders regardless of surrounding
whitespace, including compact forms such as "{{sandbox_workdir}}". Apply the
same detection to the related handling around the execute path.
In `@src/aiq_agent/agents/deep_researcher/register.py`:
- Around line 240-243: Ensure DeepResearcherAgent.__init__ finalizes the eagerly
created DeepAgentsRuntime when post-construction setup fails: wrap
_load_prompts() and middleware/tool initialization in try/except, call the
runtime’s finalize() method in the exception path, then re-raise the original
exception so register.py can handle it safely.
In `@src/aiq_agent/agents/deep_researcher/sandbox/config.py`:
- Line 111: Update the `ready_timeout_seconds` field in the sandbox
configuration and the corresponding
`DeepResearchSandboxConfig.ready_timeout_seconds` definition to use `Field(...,
gt=0, allow_inf_nan=False)`, matching the validation applied to other timeout
fields.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 182-188: In the process identity validation loop, separate type
validation from root-value validation: in the run_as_user/run_as_group checks,
first raise a clear error indicating process.{field} must be a non-empty string
when the value is not a string or is blank, then separately reject normalized
values of "0" or "root" with the existing non-root error.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 0ad4a925-dcf7-4f16-b7d6-4efdf6d92d1c
📒 Files selected for processing (53)
configs/config_openshell.ymlconfigs/openshell/README.mdconfigs/openshell/aiq-research-policy.yamldeploy/openshell/Dockerfile.aiq-demodocs/source/architecture/agents/deep-researcher.mddocs/source/architecture/agents/sandbox.mddocs/source/deployment/index.mddocs/source/deployment/openshell.mddocs/source/deployment/production.mddocs/source/index.mddocs/source/integration/rest-api.mddocs/source/resources/troubleshooting.mdfrontends/aiq_api/src/aiq_api/jobs/runner.pypyproject.tomlscripts/README.mdscripts/openshell/check_openshell_readiness.pyscripts/openshell/check_versions.pyscripts/openshell/install_gateway.shscripts/openshell/setup_openshell.shscripts/openshell/smoke_openshell_isolation.pyscripts/openshell/start_openshell_gateway.shscripts/openshell/version_contract.pyscripts/start_e2e.shsrc/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/prompts/writer.j2src/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.mdsrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/manager.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/store.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/logging_utils.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/sandbox/registry.pysrc/aiq_agent/agents/deep_researcher/skills/research/chart-generation/SKILL.mdtests/aiq_agent/agents/deep_researcher/sandbox/test_artifacts.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/sandbox/test_sandbox_runtime.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_chart_artifact_contract.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/aiq_agent/jobs/test_runner.pytests/scripts/test_openshell_gateway_installer.pytests/scripts/test_openshell_lifecycle_scripts.pytests/scripts/test_openshell_readiness_checker.pytests/scripts/test_openshell_smoke_wrapper.pytests/scripts/test_openshell_version_tools.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (11)
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update the docs under docs/source/ when behavior, configuration, or workflows change
Files:
docs/source/architecture/agents/deep-researcher.mddocs/source/deployment/production.mddocs/source/deployment/index.mddocs/source/resources/troubleshooting.mddocs/source/index.mddocs/source/integration/rest-api.mddocs/source/architecture/agents/sandbox.mddocs/source/deployment/openshell.md
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
docs/source/architecture/agents/deep-researcher.mdconfigs/openshell/README.mddocs/source/deployment/production.mddocs/source/deployment/index.mddocs/source/resources/troubleshooting.mdpyproject.tomldeploy/openshell/Dockerfile.aiq-demosrc/aiq_agent/agents/deep_researcher/agent.pydocs/source/index.mdsrc/aiq_agent/agents/deep_researcher/sandbox/logging_utils.pyscripts/openshell/smoke_openshell_isolation.pydocs/source/integration/rest-api.mdtests/aiq_agent/agents/deep_researcher/test_chart_artifact_contract.pysrc/aiq_agent/agents/deep_researcher/sandbox/registry.pysrc/aiq_agent/agents/deep_researcher/prompts/writer.j2tests/aiq_agent/agents/deep_researcher/sandbox/test_artifacts.pysrc/aiq_agent/agents/deep_researcher/skills/research/chart-generation/SKILL.mdtests/aiq_agent/agents/deep_researcher/test_agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/store.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/manager.pysrc/aiq_agent/agents/deep_researcher/factory.pyscripts/openshell/version_contract.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/scripts/test_openshell_smoke_wrapper.pytests/aiq_agent/agents/deep_researcher/sandbox/test_sandbox_runtime.pyconfigs/config_openshell.ymldocs/source/architecture/agents/sandbox.mdconfigs/openshell/aiq-research-policy.yamlscripts/start_e2e.shscripts/openshell/start_openshell_gateway.shtests/scripts/test_openshell_version_tools.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pydocs/source/deployment/openshell.mdtests/aiq_agent/jobs/test_runner.pytests/scripts/test_openshell_gateway_installer.pytests/scripts/test_openshell_lifecycle_scripts.pyscripts/openshell/install_gateway.shscripts/openshell/check_versions.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.pytests/scripts/test_openshell_readiness_checker.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.mdscripts/README.mdscripts/openshell/check_openshell_readiness.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pyscripts/openshell/setup_openshell.sh
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/architecture/agents/deep-researcher.mddocs/source/deployment/production.mddocs/source/deployment/index.mddocs/source/resources/troubleshooting.mddocs/source/index.mddocs/source/integration/rest-api.mddocs/source/architecture/agents/sandbox.mddocs/source/deployment/openshell.md
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/openshell/README.mddeploy/openshell/Dockerfile.aiq-democonfigs/config_openshell.ymlconfigs/openshell/aiq-research-policy.yaml
{.github/**,ci/**,.pre-commit-config.yaml,pyproject.toml,uv.lock}
⚙️ CodeRabbit configuration file
{.github/**,ci/**,.pre-commit-config.yaml,pyproject.toml,uv.lock}: Review automation and packaging changes for least-privilege permissions, pinned versions where appropriate,
copy-pr-bot pull-request/ branch behavior, reproducible uv/npm setup, secret handling, and consistency with
the documented validation matrix.
Files:
pyproject.toml
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/logging_utils.pyscripts/openshell/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/test_chart_artifact_contract.pysrc/aiq_agent/agents/deep_researcher/sandbox/registry.pytests/aiq_agent/agents/deep_researcher/sandbox/test_artifacts.pytests/aiq_agent/agents/deep_researcher/test_agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/store.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/manager.pysrc/aiq_agent/agents/deep_researcher/factory.pyscripts/openshell/version_contract.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/scripts/test_openshell_smoke_wrapper.pytests/aiq_agent/agents/deep_researcher/sandbox/test_sandbox_runtime.pytests/scripts/test_openshell_version_tools.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/jobs/test_runner.pytests/scripts/test_openshell_gateway_installer.pytests/scripts/test_openshell_lifecycle_scripts.pyscripts/openshell/check_versions.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.pytests/scripts/test_openshell_readiness_checker.pyscripts/openshell/check_openshell_readiness.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/logging_utils.pysrc/aiq_agent/agents/deep_researcher/sandbox/registry.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/store.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/manager.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/logging_utils.pysrc/aiq_agent/agents/deep_researcher/sandbox/registry.pysrc/aiq_agent/agents/deep_researcher/prompts/writer.j2src/aiq_agent/agents/deep_researcher/skills/research/chart-generation/SKILL.mdsrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/store.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/artifacts/manager.pysrc/aiq_agent/agents/deep_researcher/factory.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.mdsrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_chart_artifact_contract.pytests/aiq_agent/agents/deep_researcher/sandbox/test_artifacts.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_factory.pytests/scripts/test_openshell_smoke_wrapper.pytests/aiq_agent/agents/deep_researcher/sandbox/test_sandbox_runtime.pytests/scripts/test_openshell_version_tools.pytests/aiq_agent/jobs/test_runner.pytests/scripts/test_openshell_gateway_installer.pytests/scripts/test_openshell_lifecycle_scripts.pytests/aiq_agent/agents/deep_researcher/test_custom_middleware.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.pytests/scripts/test_openshell_readiness_checker.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
**/*config*.py
📄 CodeRabbit inference engine (AGENTS.md)
Config schemas must inherit from FunctionBaseConfig and YAML _type names must come from the registered config class
Files:
src/aiq_agent/agents/deep_researcher/sandbox/config.py
🧠 Learnings (1)
📚 Learning: 2026-07-06T23:55:42.908Z
Learnt from: cdgamarose-nv
Repo: NVIDIA-AI-Blueprints/aiq PR: 311
File: src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2:74-81
Timestamp: 2026-07-06T23:55:42.908Z
Learning: In agent test files (e.g., tests/aiq_agent/agents/*/test_agent.py), avoid brittle assertions that match exact substrings from prompt template files (such as *.j2 prompt wording). Prompt wording can change frequently, so instead assert structural/behavioral properties (e.g., that the prompt builder is called, that required sections/fields are present via stable markers, that the model output/agent behavior conforms to an expected schema, or that key actions are taken) rather than matching literal prompt text.
Applied to files:
tests/aiq_agent/agents/deep_researcher/test_agent.py
🪛 ast-grep (0.44.1)
scripts/openshell/smoke_openshell_isolation.py
[error] 81-86: Command coming from incoming request
Context: subprocess.run( # noqa: S603 - fixed argv, no shell, operator-selected interpreter
_command(),
cwd=_REPO_ROOT,
env=_environment(args),
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
scripts/openshell/version_contract.py
[info] 76-76: use jsonify instead of json.dumps for JSON output
Context: json.dumps(asdict(contract), sort_keys=True)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
tests/scripts/test_openshell_gateway_installer.py
[error] 94-101: Command coming from incoming request
Context: subprocess.run(
[str(_INSTALLER), *args],
cwd=_REPO_ROOT,
env=effective_env,
text=True,
capture_output=True,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
tests/scripts/test_openshell_lifecycle_scripts.py
[info] 100-100: use jsonify instead of json.dumps for JSON output
Context: json.dumps([gateway])
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
[error] 108-122: Command coming from incoming request
Context: subprocess.run(
[
str(_GATEWAY_SCRIPT),
"--reuse-existing",
"--gateway-name",
"enterprise",
"--policy-file",
str(policy),
],
cwd=_REPO_ROOT,
env=env,
text=True,
capture_output=True,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
[error] 229-235: Command coming from incoming request
Context: subprocess.run(
[str(_SETUP_SCRIPT), "--gateway-name", "old"],
cwd=_REPO_ROOT,
text=True,
capture_output=True,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
[error] 250-256: Command coming from incoming request
Context: subprocess.run(
[str(_SETUP_SCRIPT), "--openshell-version", version, "--skip-build", "--policy", "offline"],
cwd=_REPO_ROOT,
text=True,
capture_output=True,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
[error] 281-287: Command coming from incoming request
Context: subprocess.run(
[str(_E2E_SCRIPT), "--help"],
cwd=_REPO_ROOT,
text=True,
capture_output=True,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
scripts/openshell/check_versions.py
[error] 46-46: Use of unsanitized data to create processes
Context: subprocess.run(command, check=False, capture_output=True, text=True, timeout=15)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(os-system-unsanitized-data)
[error] 46-46: Command coming from incoming request
Context: subprocess.run(command, check=False, capture_output=True, text=True, timeout=15)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
[info] 229-229: use jsonify instead of json.dumps for JSON output
Context: json.dumps(asdict(report), sort_keys=True)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.py
[info] 408-408: use jsonify instead of json.dumps for JSON output
Context: json.dumps(events, default=str)
Note: [CWE-116] Improper Encoding or Escaping of Output.
(use-jsonify)
scripts/openshell/check_openshell_readiness.py
[error] 53-59: Command coming from incoming request
Context: subprocess.run(
[str(binary), "--version"],
check=False,
capture_output=True,
text=True,
timeout=10,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
🪛 LanguageTool
docs/source/deployment/openshell.md
[style] ~48-~48: Consider a different adjective to strengthen your wording.
Context: ...penShell executes code generated during deep research. AI-Q orchestration, inference...
(DEEP_PROFOUND)
scripts/README.md
[style] ~88-~88: Consider a different adjective to strengthen your wording.
Context: ...Mode Starts the NAT FastAPI server for deep research with async job support. ```ba...
(DEEP_PROFOUND)
🪛 OpenGrep (1.23.0)
tests/aiq_agent/agents/deep_researcher/sandbox/test_sandbox_runtime.py
[ERROR] 463-463: SQL query built via f-string passed to execute()/executemany(). Use parameterized queries with placeholders instead.
(coderabbit.sql-injection.python-fstring-execute)
🔇 Additional comments (63)
docs/source/architecture/agents/deep-researcher.md (1)
14-14: LGTM!configs/openshell/README.md (1)
1-16: LGTM!src/aiq_agent/agents/deep_researcher/sandbox/logging_utils.py (1)
11-28: LGTM!src/aiq_agent/agents/deep_researcher/sandbox/registry.py (1)
30-30: LGTM!Also applies to: 84-92
src/aiq_agent/agents/deep_researcher/sandbox/artifacts/store.py (1)
40-40: LGTM!Also applies to: 219-241, 382-408
src/aiq_agent/agents/deep_researcher/sandbox/artifacts/manager.py (1)
39-39: LGTM!Also applies to: 219-230, 258-269, 303-314, 382-392
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
22-48: Race, attestation-ordering, secret-logging, and CIDR-bypass fixes all confirmed in this revision.Verified against the past-review history:
_state_lock-guarded context ownership in_enter_context/_finish_context_entry/_exit_context/_active_os_contextcloses the previously flagged race;_emit_attestation(status="succeeded")now fires only afterOpenShellSandbox(...)construction succeeds;oscfg.gatewayis no longer logged;_validate_policy_networkrejects hostless/allowed_ipsendpoints; hash comparison uses OpenShell's own reported hash rather than a custom recipe. No regressions found.Also applies to: 111-116, 351-351, 372-372, 390-390, 428-509, 510-744, 745-790, 802-857, 893-912
docs/source/deployment/production.md (1)
116-124: 📐 Maintainability & Code QualityCross-links resolve to the matching headings in
docs/source/deployment/openshell.md.> Likely an incorrect or invalid review comment.tests/scripts/test_openshell_gateway_installer.py (2)
93-102: Static analysis CWE-78 hint is a false positive.
ast-grepflagssubprocess.run([str(_INSTALLER), *args], ...)as "Command coming from incoming request." This is test-harness code invoking a fixed, repo-relative script path (_INSTALLER) with test-controlled literal arguments andcwd=_REPO_ROOT— there's no untrusted/network input here.Source: Linters/SAST tools
1-249: LGTM!docs/source/deployment/index.md (1)
42-43: LGTM!docs/source/resources/troubleshooting.md (1)
40-40: LGTM!src/aiq_agent/agents/deep_researcher/agent.py (1)
157-160: LGTM!src/aiq_agent/agents/deep_researcher/skills/research/chart-generation/SKILL.md (1)
58-63: LGTM!Also applies to: 95-116, 130-139, 148-165
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
297-342: Cancellation-ownership test is correct and resolves the earlier coverage gap.Traced both parametrized branches against
register.py::_run: forowns_active_agent=False,sandbox_configisNoneanddata_sourcesunset, so the guard condition staysFalse,active_agentremains the template agent, and nofinalizecall happens on cancellation — matchingtemplate_agent.finalize.assert_not_called(). Forowns_active_agent=True, the second constructed agent becomesactive_agent, cancellation setsinterrupted=True, andfinallycallsrequest_agent.finalize(interrupted=True)exactly once. Matches the previously requested coverage (see prior review comment onregister.py:298-306, addressed in commit e5b69ab).frontends/aiq_api/src/aiq_api/jobs/runner.py (2)
932-960: Falsy-return logging and exception sanitization from prior review are correctly addressed.
_teardown_sandboxnow logs a warning whenfinalize()returnsFalseif not finalize(interrupted=interrupted): logger.warning("Sandbox cleanup reported failure for job %s", job_id), and exceptions are logged with type-only detail rather than a full traceback except Exception as exc: logger.warning("Sandbox cleanup failed for job %s (%s)", job_id, type(exc).name), matching both previously-requested fixes.
886-893: Double_teardown_sandbox/flush invocation is safe (idempotent) but worth a quick sanity note.
_teardown_sandboxcan run twice for cancelled jobs (once explicitly in theexcept asyncio.CancelledErrorblock, once again infinally). This is safe becauseDeepAgentsRuntime.finalize()caches its result behind_finalize_lock/_finalizedand returns the cached outcome on repeat calls, and the legacyclose()/terminate()fallback is documented as idempotent. No action needed — just confirming the double-call doesn't double-execute cleanup side effects.pyproject.toml (1)
85-89: LGTM!deploy/openshell/Dockerfile.aiq-demo (1)
8-13: LGTM!src/aiq_agent/agents/deep_researcher/factory.py (1)
47-49: LGTM!Also applies to: 474-474, 503-506
scripts/openshell/version_contract.py (1)
1-87: LGTM!tests/aiq_agent/agents/deep_researcher/test_factory.py (1)
27-28: LGTM!Also applies to: 247-247, 296-327
tests/scripts/test_openshell_lifecycle_scripts.py (1)
1-304: LGTM!src/aiq_agent/agents/deep_researcher/custom_middleware.py (1)
169-292: LGTM!Also applies to: 683-727
docs/source/index.md (1)
104-104: LGTM!scripts/openshell/smoke_openshell_isolation.py (1)
1-93: LGTM!tests/aiq_agent/agents/deep_researcher/test_chart_artifact_contract.py (1)
1-38: LGTM!src/aiq_agent/agents/deep_researcher/prompts/writer.j2 (1)
80-84: LGTM!tests/aiq_agent/agents/deep_researcher/sandbox/test_artifacts.py (1)
22-22: LGTM!Also applies to: 122-136, 426-439
tests/aiq_agent/jobs/test_runner.py (1)
2447-2459: LGTM!Also applies to: 2461-2492, 2520-2530
src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (2)
91-134: LGTM! Previously flagged concerns (conflicting shared-sandbox aliases, idempotency-lock scope, sanitized exception logging, no-provider event emission) are all resolved and covered by new tests.Also applies to: 143-155, 174-177, 206-233, 295-302, 343-398, 574-597
156-172: 🎯 Functional CorrectnessNo action needed on the return annotation
from __future__ import annotationsis already present, so the self-referentialDeepResearchSandboxConfigreturn type is safe here.> Likely an incorrect or invalid review comment.tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
41-70: LGTM!Also applies to: 335-364, 415-534
docs/source/integration/rest-api.md (1)
321-325: LGTM!configs/config_openshell.yml (1)
130-172: LGTM!docs/source/architecture/agents/sandbox.md (1)
9-80: LGTM!configs/openshell/aiq-research-policy.yaml (1)
14-28: LGTM!scripts/openshell/start_openshell_gateway.sh (1)
141-271: LGTM!src/aiq_agent/agents/deep_researcher/sandbox/config.py (1)
138-181: LGTM!tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
105-234: LGTM!Also applies to: 371-908, 1184-1345
scripts/openshell/setup_openshell.sh (1)
25-45: LGTM!Also applies to: 240-336, 621-669, 767-840
scripts/openshell/check_versions.py (2)
45-50: False-positive security hints.The
os-system-unsanitized-data/subprocess-from-requesthints on_run()(line 46) don't apply: this is a standalone local diagnostics CLI, and every call site passes a fixed, hardcoded argument list (brew/openshellpaths resolved viashutil.which/local.venvpath), not attacker- or request-controlled input.
1-238: LGTM! Version-mismatch classification (ambiguous_gateway_installation,packaged_gateway_missing,component_version_mismatch,remote_gateway_version_mismatch,gateway_unavailable) correctly matches the doc's troubleshooting table, and_print_human/JSON output only ever surface allowlisted fields.scripts/openshell/check_openshell_readiness.py (3)
1-1: Path directory concern already resolved in a prior review round ("done" per past comments); no action needed here.
52-66: False-positive subprocess hint.
subprocess-from-requeston_version_from_clidoesn't apply —binaryis an operator-supplied--openshell-binCLI path for a local readiness probe, not data from an inbound network request.
105-171: LGTM! The authoritative-policy attestation loop (source/content/hash/revision agreement,PENDING-vs-LOADEDhandling) matches the doc's stated fail-closed contract.tests/scripts/test_openshell_smoke_wrapper.py (2)
1-101: LGTM!
73-85: 🎯 Functional Correctness
_command()is already deterministic. It returns a fixed pytest argv and does not readsys.argv, so pytest CLI args cannot leak into this path.> Likely an incorrect or invalid review comment.tests/aiq_agent/agents/deep_researcher/sandbox/test_sandbox_runtime.py (1)
254-272: LGTM! Sanitized-logging assertions correctly mirrorlog_sandbox_failure's behavior inbase.py(reason codes present, exception text/credential-shaped strings absent fromcaplog.text).Note: the OpenGrep "SQL query built via f-string" hint on line 463 is a false positive —
provider.execute("echo hi")is the sandbox execute call, not a SQL statement.Also applies to: 274-288, 347-359, 455-466
src/aiq_agent/agents/deep_researcher/sandbox/base.py (2)
159-178: LGTM! Event emission is correctly best-effort: failures are caught, sanitized vialog_sandbox_failure, and never propagate to the caller.
232-264: LGTM!_safe_close/_record_cleanup_failure/cleanup_succeeded/cleanup_failure_reason_codesare consistently guarded by_cleanup_state_lock, and the "every cleanup attempt including retry teardown" semantics match the reset/discard call sites (_session_or_create,_reset_session).docs/source/deployment/openshell.md (1)
1-516: LGTM otherwise — the deployment guide's environment-variable table, lifecycle-ownership matrix, and troubleshooting entries are internally consistent with the scripts/tests reviewed elsewhere in this stack (e.g.AIQ_OPENSHELL_LIVE_ALLOW_BEST_EFFORTmatches the smoke-wrapper test).tests/scripts/test_openshell_version_tools.py (1)
1-202: LGTM! Test coverage forinspect_components/maincorrectly exercises every classification branch (ambiguous_gateway_installation,packaged_gateway_missing,gateway_unavailable,remote_gateway_version_mismatch,component_version_mismatch) and the JSON allowlist contract, matchingcheck_versions.py's implementation.scripts/start_e2e.sh (3)
22-31: LGTM!Also applies to: 44-59, 81-86
122-140: LGTM!Also applies to: 189-201, 253-258
141-148: 🎯 Functional CorrectnessAbsolute
--config_filepaths are already rejected earlier.scripts/start_e2e.shchecks"$PROJECT_ROOT/$CONFIG_FILE"before reachingcheck_openshell_component_versions(), so this function does not silently skip on an absolute path. The same repo-relative assumption is used throughout the script; only normalize the path once if absolute configs should be supported.> Likely an incorrect or invalid review comment.scripts/openshell/install_gateway.sh (1)
1-95: LGTM!Also applies to: 116-209
tests/aiq_agent/agents/deep_researcher/test_custom_middleware.py (1)
24-38: LGTM!Also applies to: 50-56, 215-393, 972-988
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.py (1)
1-461: LGTM!tests/scripts/test_openshell_readiness_checker.py (1)
1-410: LGTM!src/aiq_agent/agents/deep_researcher/sandbox/README.md (1)
40-58: LGTM!Also applies to: 73-73, 138-139, 156-163, 226-250, 310-313, 322-334
scripts/README.md (2)
145-165: LGTM!Also applies to: 181-181
52-84: 📐 Maintainability & Code QualityNo change needed for the OpenShell env override docs.
AIQ_OPENSHELL_REQUIRE_HARD_LANDLOCKis already wired throughconfigs/config_openshell.ymlandsrc/aiq_agent/agents/deep_researcher/sandbox/config.py.
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
#### Overview Refresh the AI-Q documentation against the live 2.2 milestone and current `develop` implementation while keeping branch-facing documentation portable across future release cuts. - keeps the root README version-free: “What’s New” highlights current capabilities without embedding release numbers, RC status, or a branch-cut lifecycle - keeps version-specific 2.2 targeting in the existing changelog, which is the detailed unreleased ledger; no separate release-notes document is introduced - makes the roadmap describe implementation in the checked-out branch rather than implying availability in a published release - updates configuration, deployment, quick-start, profiling, and docs-navigation wording to describe current behavior instead of a “2.2 candidate” - documents the newly merged artifact lifecycle (#314), Helm release-namespace behavior (#309), and async trace hierarchy (#321) - keeps the remaining open capabilities explicit: no per-job isolated/attested OpenShell lifecycle (#298) and no standalone public AI-Q MCP server (#319) - resolves Linette's review feedback across link-referral wording, terminology, and documentation clarity - preserves runtime boundaries for advisory routing, focused configuration profiles, MCP reconnect behavior, narrow forward-only encryption, best-effort artifact capture, and best-effort tokenomics phase attribution Milestone audit as of July 10, 2026: 49 items (47 PRs and 2 issues), including 41 PRs merged to `develop`, one PR merged only to `release/2.1`, three open PRs (#298, #319, and this documentation PR), and two closed-unmerged PRs superseded by merged work. AI-Q `v2.1.0` remains the latest stable release. `v2.2.0-rc1` is a prerelease snapshot from `develop`; there is no final `v2.2.0` tag yet, and `release/2.2` has not been cut. These lifecycle details intentionally remain outside the develop-facing README. #### Validation - `make -C docs SPHINXBUILD=../.venv/bin/sphinx-build SPHINXOPTS='-W --keep-going -n' html` - `make -C docs SPHINXBUILD=../.venv/bin/sphinx-build linkcheck` - `pre-commit run --all-files` - `git diff --check origin/develop...HEAD` - independent release-lifecycle, version-free wording, milestone, review-thread, and semantic whole-diff reviews found no remaining Critical or Important issues - [x] I ran the relevant local checks or explained why they are not applicable. - [x] I added or updated tests for behavior changes. Not applicable: this PR changes documentation only. - [x] I updated documentation for user-facing or contributor-facing changes. - [x] I confirmed this PR does not include secrets, credentials, or internal-only data. - [x] I certify this contribution under the Developer Certificate of Origin (DCO) and signed my commits with `git commit -s` or an equivalent sign-off. #### Where should reviewers start? 1. `README.md` for the version-free “What’s New” highlights and current-branch roadmap semantics. 2. `CHANGELOG.md` for the detailed unreleased 2.2 ledger and release lifecycle. 3. `docs/source/architecture/agents/deep-researcher.md` for the routed planner/researcher/writer contract. 4. `docs/source/architecture/data-flow.md` and `docs/source/integration/rest-api.md` for artifact checkpoint, SSE, replay, and authorization semantics. 5. `docs/source/deployment/kubernetes.md` and `docs/source/deployment/observability.md` for the newly merged namespace and trace-hierarchy behavior. #### Related Issues - Relates to the [AI-Q v2.2 milestone](https://github.com/NVIDIA-AI-Blueprints/aiq/milestone/1). - Documents merged changes from #309, #314, and #321. - Keeps open #298 and #319 explicitly out of the candidate scope. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated “Unreleased” release notes targeting AI-Q v2.2.0, including deep research workflow changes, async job reporting, and sandbox/artifact behavior. * Added/expanded REST API documentation for event-derived job state and durable artifact listing/streaming. * Documented OpenSearch support for knowledge retrieval and multiple paper-search providers, plus refined configuration, guardrails, MCP OAuth behavior, and observability trace hierarchy. * **Chores** * Refreshed secrets baseline metadata timestamps/line references only. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Ajay Thorve <athorve@nvidia.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
…ct-resolution Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> # Conflicts: # docs/source/architecture/agents/deep-researcher.md # docs/source/architecture/agents/sandbox.md
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/integration/rest-api.md`:
- Around line 353-366: Update the “Get Artifact Content” documentation and the
related artifact endpoint wording to describe access as job-scoped, with
authentication conditional on REQUIRE_AUTH. Revise the curl -OJ example to
include the required authentication header for authenticated deployments while
clarifying that no-auth mode permits access with a valid job ID only in
trusted-local environments.
- Around line 297-306: Clarify the public event contract for file_path in the
artifact.update payload documentation: define it explicitly as a sanitized,
job-relative display path rather than a filesystem or sandbox path, and ensure
the example and surrounding wording use that meaning consistently. If the
contract cannot guarantee this semantics, remove file_path from the public event
payload documentation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: ef5f8b55-914a-47b8-a710-59fd5d0805b6
📒 Files selected for processing (3)
docs/source/architecture/agents/sandbox.mddocs/source/index.mddocs/source/integration/rest-api.md
📜 Review details
⏰ Context from checks skipped due to timeout. (1)
- GitHub Check: Run Harbor skill eval
🧰 Additional context used
📓 Path-based instructions (2)
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update the docs under docs/source/ when behavior, configuration, or workflows change
Files:
docs/source/index.mddocs/source/integration/rest-api.mddocs/source/architecture/agents/sandbox.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/index.mddocs/source/integration/rest-api.mddocs/source/architecture/agents/sandbox.md
🔇 Additional comments (3)
docs/source/index.md (1)
105-105: LGTM!docs/source/integration/rest-api.md (1)
17-17: LGTM!Also applies to: 36-38, 238-242, 283-296, 307-352, 367-381, 402-405, 424-425
docs/source/architecture/agents/sandbox.md (1)
9-14: LGTM!Also applies to: 18-20, 25-37, 52-57, 61-73, 83-85
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/integration/rest-api.md`:
- Around line 353-366: Update the “Get Artifact Content” documentation and the
related artifact endpoint wording to describe access as job-scoped, with
authentication conditional on REQUIRE_AUTH. Revise the curl -OJ example to
include the required authentication header for authenticated deployments while
clarifying that no-auth mode permits access with a valid job ID only in
trusted-local environments.
- Around line 297-306: Clarify the public event contract for file_path in the
artifact.update payload documentation: define it explicitly as a sanitized,
job-relative display path rather than a filesystem or sandbox path, and ensure
the example and surrounding wording use that meaning consistently. If the
contract cannot guarantee this semantics, remove file_path from the public event
payload documentation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: ef5f8b55-914a-47b8-a710-59fd5d0805b6
📒 Files selected for processing (3)
docs/source/architecture/agents/sandbox.mddocs/source/index.mddocs/source/integration/rest-api.md
📜 Review details
🔇 Additional comments (3)
docs/source/index.md (1)
105-105: LGTM!docs/source/integration/rest-api.md (1)
17-17: LGTM!Also applies to: 36-38, 238-242, 283-296, 307-352, 367-381, 402-405, 424-425
docs/source/architecture/agents/sandbox.md (1)
9-14: LGTM!Also applies to: 18-20, 25-37, 52-57, 61-73, 83-85
🛑 Comments failed to post (2)
docs/source/integration/rest-api.md (2)
297-306: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash set -euo pipefail printf '\n== Relevant doc locations ==\n' git ls-files 'docs/source/integration/rest-api.md' 'docs/source/architecture/data-flow.md' | sed 's#^`#-` #' printf '\n== Search for file_path and artifact.update references ==\n' rg -n --no-heading 'file_path|artifact\.update|artifact\.warning|content endpoint|sandbox path|display filename|logical path' docs/source -S printf '\n== Read target doc slices ==\n' sed -n '280,320p' docs/source/integration/rest-api.md printf '\n---\n' sed -n '420,440p' docs/source/integration/rest-api.md printf '\n---\n' sed -n '1,220p' docs/source/architecture/data-flow.mdRepository: NVIDIA-AI-Blueprints/aiq
Length of output: 19572
🏁 Script executed:
#!/bin/bash set -euo pipefail echo "Searching docs for file_path definition..." rg -n --no-heading 'file_path' docs/source -S || trueRepository: NVIDIA-AI-Blueprints/aiq
Length of output: 2885
Clarify
file_pathsemantics. Indocs/source/integration/rest-api.md:427-431, the payload example uses a basename, but the contract still doesn’t say whetherfile_pathis a sanitized display path or a filesystem/sandbox path. Define it as job-relative/non-filesystem, or remove it from the public event.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/integration/rest-api.md` around lines 297 - 306, Clarify the public event contract for file_path in the artifact.update payload documentation: define it explicitly as a sanitized, job-relative display path rather than a filesystem or sandbox path, and ensure the example and surrounding wording use that meaning consistently. If the contract cannot guarantee this semantics, remove file_path from the public event payload documentation.Source: Path instructions
353-366: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Keep artifact authentication wording and examples consistent with
REQUIRE_AUTH.The content URL is described as authenticated, but this file also documents
REQUIRE_AUTH=false, where callers with a valid job ID can access artifacts, and thecurl -OJexample provides no credentials. Describe the URL as job-scoped, qualify authentication as conditional, and show the required auth header for authenticated deployments.As per path instructions, review documentation for public vs internal boundary clarity and command accuracy.
Also applies to: 427-431
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/source/integration/rest-api.md` around lines 353 - 366, Update the “Get Artifact Content” documentation and the related artifact endpoint wording to describe access as job-scoped, with authentication conditional on REQUIRE_AUTH. Revise the curl -OJ example to include the required authentication header for authenticated deployments while clarifying that no-auth mode permits access with a valid job ID only in trusted-local environments.Source: Path instructions
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/aiq_agent/agents/deep_researcher/agent.py`:
- Around line 157-164: The constructor cleanup around DeepAgentsRuntime.finalize
must treat a False return as a cleanup failure, not only raised exceptions;
update the handling in agent.py to log a warning for either outcome while
preserving the original construction error. Add test coverage in test_agent.py
for finalize(interrupted=False) returning False and verify the cleanup warning
is emitted.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 26455218-471d-4394-bb59-e498cf874351
📒 Files selected for processing (2)
src/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/agents/deep_researcher/test_agent.py
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
- GitHub Check: Run Harbor skill eval
- GitHub Check: Pytest and Coverage
- GitHub Check: Lint and Hooks
🧰 Additional context used
📓 Path-based instructions (4)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
src/aiq_agent/agents/deep_researcher/agent.pytests/aiq_agent/agents/deep_researcher/test_agent.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/agent.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/agent.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.py
🧠 Learnings (1)
📚 Learning: 2026-07-06T23:55:42.908Z
Learnt from: cdgamarose-nv
Repo: NVIDIA-AI-Blueprints/aiq PR: 311
File: src/aiq_agent/agents/deep_researcher/prompts/orchestrator.j2:74-81
Timestamp: 2026-07-06T23:55:42.908Z
Learning: In agent test files (e.g., tests/aiq_agent/agents/*/test_agent.py), avoid brittle assertions that match exact substrings from prompt template files (such as *.j2 prompt wording). Prompt wording can change frequently, so instead assert structural/behavioral properties (e.g., that the prompt builder is called, that required sections/fields are present via stable markers, that the model output/agent behavior conforms to an expected schema, or that key actions are taken) rather than matching literal prompt text.
Applied to files:
tests/aiq_agent/agents/deep_researcher/test_agent.py
🔇 Additional comments (4)
src/aiq_agent/agents/deep_researcher/agent.py (1)
129-156: LGTM!Also applies to: 167-169
tests/aiq_agent/agents/deep_researcher/test_agent.py (3)
297-342: LGTM!
1039-1066: LGTM!
1067-1087: 🩺 Stability & AvailabilityNo additional test needed here:
DeepAgentsRuntime.finalize()already records failed cleanup status when it returnsFalse, so the constructor cleanup path does not lose that signal.> Likely an incorrect or invalid review comment.
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
AjayThorve
left a comment
AjayThorve
left a comment
There was a problem hiding this comment.
Looks good, thanks for this
#### Overview Create and own one physical OpenShell sandbox per AI-Q deep-research job. This change provides per-job isolation, fail-closed policy attestation, discoverable ownership labels, truthful cleanup, authenticated gateway readiness checks, pytest-owned live acceptance, and a canonical operator guide. The previous upstream blocker is resolved by [NVIDIA/OpenShell#2170](NVIDIA/OpenShell#2170) and released in OpenShell `0.0.80`. AI-Q now requires `openshell>=0.0.80,<0.1`; the setup script defaults to and enforces the `0.0.80` supported floor. Runtime security decisions remain capability- and state-based rather than branching on an OpenShell version. Key behavior: - Create one owned physical sandbox lazily for each job and reuse it for subsequent execution within that job. - Apply `aiq=deep-research` and a normalized `aiq-job-id` to both OpenShell request metadata and template/container metadata. Active jobs can be queried with `openshell sandbox list --selector aiq=deep-research`. - Keep shared-sandbox attachment behind explicit `allow_shared_sandbox=true` debug configuration. Shared attachment is non-production and does not transfer cleanup ownership to the job. - Enforce AI-Q's declared network boundary: - blocked mode rejects every configured endpoint; - allowlist mode requires non-empty normalized hosts contained in `network.allow`; - hostless endpoints and `allowed_ips`/CIDR exceptions are rejected. - Attest the exact submitted policy through authoritative OpenShell policy-status and effective-config RPCs before exposing the execution adapter. - Require `READY`, a `LOADED` revision without a load error, sandbox policy source, matching positive current/active/revision/config versions, structurally identical policies, and matching deterministic hashes. - Treat a matching effective policy that remains `PENDING` as `policy_status_inconsistent`; never weaken attestation because the sandbox is otherwise executable. - Use the canonical read-only `/proc` baseline so OpenShell does not enrich the submitted policy and create an unexpected revision. - Delete and verify deletion of partially created sandboxes after attestation, startup, execution, timeout, failure, or cancellation errors. - Emit structured, sanitized `sandbox.attestation` and `sandbox.cleanup` events without exception messages, policy contents, credentials, SDK response bodies, or tracebacks. - Preserve artifact harvest-before-cleanup ordering from `develop`/PR NVIDIA-AI-Blueprints#314. Generated files remain job-scoped, receive durable artifact references, and can be rendered or downloaded after the physical sandbox is deleted. - Add one writer-local corrective turn when the writer claims `Wrote /shared/output.md` without creating a non-empty output file. A second false completion fails with the stable `writer_output_missing` reason instead of restarting research. - Split provisioning from gateway lifecycle: - `setup_openshell.sh` installs the pinned dependencies, generates policy, and builds the image; - `start_openshell_gateway.sh` validates an authenticated registered or packaged gateway and performs a disposable strict readiness probe; - AI-Q never launches a raw gateway binary or broadly kills externally owned processes. - Make the readiness probe verify CLI/SDK/gateway version agreement, request labels, selector behavior, `READY`, `LOADED`, effective policy source/content/hash/version, command execution, deletion, and confirmed absence. - Move live assertions, resources, and verified teardown into pytest. `scripts/smoke_openshell_isolation.py` is now only a thin compatibility launcher. - Add `docs/source/deployment/openshell.md` as the canonical operator guide for platform support, ownership, policy/config pairing, startup, acceptance, and troubleshooting. #### Reviewer Test Instructions On macOS with Docker Desktop, this is a functional local-demo flow using explicit Landlock `best_effort`: ```bash gh pr checkout 298 ./scripts/setup.sh source .venv/bin/activate /opt/homebrew/bin/bash ./scripts/setup_openshell.sh \ --openshell-version 0.0.80 \ --local-demo \ --policy offline export AIQ_OPENSHELL_GATEWAY_NAME=openshell export AIQ_OPENSHELL_IMAGE=aiq-openshell-demo:latest export AIQ_OPENSHELL_POLICY_FILE="$PWD/configs/openshell/generated/aiq-openshell-policy.yaml" export AIQ_OPENSHELL_EXPECTED_GATEWAY_VERSION=0.0.80 export AIQ_OPENSHELL_REQUIRE_HARD_LANDLOCK=false nat validate --config_file configs/config_openshell.yml ./scripts/start_openshell_gateway.sh \ --gateway-name openshell \ --image-name aiq-openshell-demo:latest \ --policy-file configs/openshell/generated/aiq-openshell-policy.yaml ./scripts/start_e2e.sh \ --config_file configs/config_openshell.yml ``` Open `http://localhost:3000`, create two sessions, and start two concurrent deep-research jobs that request a CSV and PNG chart. While both jobs are running: ```bash .venv/bin/openshell sandbox list \ -g openshell \ --selector aiq=deep-research \ -o json ``` Expected behavior: 1. Each job creates one distinct physical sandbox when sandbox execution is first needed. 2. Later execution calls from the same job reuse that sandbox. 3. Both sandboxes have `aiq=deep-research` and distinct `aiq-job-id` labels. 4. Attestation succeeds before sandbox execution is exposed to the agent. 5. Generated files appear under the correct job and remain downloadable after cleanup. 6. PNG artifacts render through durable artifact references in the final report. 7. A false writer completion receives one writer-local retry. 8. Cancelling one job deletes only that job's sandbox; the other job remains usable. 9. Completing the remaining job deletes its sandbox. 10. The selector returns no matching sandboxes after both jobs terminate. Run the automated macOS live suite with: ```bash .venv/bin/python scripts/smoke_openshell_isolation.py \ --gateway openshell \ --policy configs/openshell/generated/aiq-openshell-policy.yaml \ --image aiq-openshell-demo:latest \ --expected-gateway-version 0.0.80 \ --allow-best-effort-landlock ``` Expected result: all three live isolation/attestation, failure-cleanup/redaction, and shared-policy-mismatch tests pass. A passing macOS run is functional demo evidence only. Production acceptance requires Linux, Docker, and a policy/config pairing that both require Landlock `hard_requirement`, as documented in `docs/source/deployment/openshell.md`. #### Validation Current-head scoped regression suite: ```bash .venv/bin/pytest -q \ tests/aiq_agent/agents/deep_researcher \ tests/aiq_agent/jobs/test_runner.py \ tests/scripts/test_openshell_lifecycle_scripts.py \ tests/scripts/test_openshell_readiness_checker.py \ tests/scripts/test_openshell_smoke_wrapper.py ``` Result: **485 passed, 3 skipped in 6.58s**. The three live tests were collected and skipped before optional OpenShell imports or gateway access because live testing was not enabled. Lint, formatting, and shell syntax: ```bash .venv/bin/ruff check \ src/aiq_agent/agents/deep_researcher \ tests/aiq_agent/agents/deep_researcher \ tests/aiq_agent/jobs/test_runner.py \ tests/scripts \ scripts/check_openshell_readiness.py \ scripts/smoke_openshell_isolation.py .venv/bin/ruff format --check \ src/aiq_agent/agents/deep_researcher \ tests/aiq_agent/agents/deep_researcher \ tests/aiq_agent/jobs/test_runner.py \ tests/scripts \ scripts/check_openshell_readiness.py \ scripts/smoke_openshell_isolation.py bash -n \ scripts/setup_openshell.sh \ scripts/start_openshell_gateway.sh \ scripts/start_e2e.sh ``` Result: **all checks passed; 53 files already formatted; shell syntax passed**. Configuration validation: ```bash AIQ_OPENSHELL_REQUIRE_HARD_LANDLOCK=false \ .venv/bin/nat validate \ --config_file configs/config_openshell.yml ``` Result: **configuration valid**. Documentation: ```bash cd docs PATH="../.venv/bin:$PATH" make html ``` Result: **build succeeded**. Sphinx reported two existing unresolved links in `docs/source/integration/agent-skills.md`; neither warning originates from the OpenShell documentation. Live OpenShell validation recorded on macOS/Docker Desktop after the `0.0.80` rebaseline: - **3 live tests passed**. - Proved concurrent per-job isolation and selector visibility. - Proved source/content/hash/revision attestation. - Proved isolated cancellation and verified terminal deletion. - Proved failure cleanup and credential-canary redaction. - Proved shared-policy mismatch rejection. Linux/Docker acceptance with Landlock `hard_requirement` has **not** been run on this macOS host and remains the production acceptance gate. - [x] I ran the relevant local checks or explained why they are not applicable. - [x] I added or updated tests for behavior changes. - [x] I updated documentation for user-facing or contributor-facing changes. - [x] I confirmed this PR does not include secrets, credentials, or internal-only data. - [x] I certify this contribution under the Developer Certificate of Origin (DCO) and signed my commits with `git commit -s` or an equivalent sign-off. #### Where should reviewers start? 1. `src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` - Per-job creation, labels, network validation, authoritative attestation, and cleanup. 2. `scripts/check_openshell_readiness.py` and `scripts/start_openshell_gateway.sh` - Version/capability validation, selector proof, policy verification, execution, and verified deletion. 3. `tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.py` - Pytest-owned live isolation, cancellation, cleanup/redaction, and shared-policy rejection. 4. `src/aiq_agent/agents/deep_researcher/custom_middleware.py` - Deterministic filesystem arguments, sandbox-path validation, and the bounded writer-output guard. 5. `docs/source/deployment/openshell.md` - Canonical supported-platform, ownership, provisioning, startup, acceptance, and troubleshooting contract. #### Related Issues - Relates to NVIDIA-AI-Blueprints#287 - AIQ-3531 - Partial AIQ-3433 lifecycle coverage - Upstream issue: [NVIDIA/OpenShell#2159](NVIDIA/OpenShell#2159) - Upstream fix released in OpenShell `0.0.80`: [NVIDIA/OpenShell#2170](NVIDIA/OpenShell#2170) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Experimental per-job OpenShell sandboxes with policy binding, attestation, and network allowlist support; debug shared-sandbox attachment is explicitly opt-in. * Added OpenShell lifecycle tooling (setup, gateway startup, readiness/version checks, smoke/live acceptance). * Strengthened chart-generation/artifact workflow with runtime-driven chart execution and output validation. * **Bug Fixes** * Improved deep-research job cancellation/terminal teardown reliability with safer event store flushing and sandbox finalization. * Tightened fail-closed policy/network handling and reduced sensitive error text leakage in logs and events. * **Documentation** * Updated deep-research sandboxing/ownership architecture guidance, OpenShell deployment/operator docs, and REST SSE artifact event details. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Kyle Zheng <kyzheng@nvidia.com> Co-authored-by: Ajay Thorve <AjayThorve@users.noreply.github.com>
3 participants
Overview
Create and own one physical OpenShell sandbox per AI-Q deep-research job. This change provides per-job isolation, fail-closed policy attestation, discoverable ownership labels, truthful cleanup, authenticated gateway readiness checks, pytest-owned live acceptance, and a canonical operator guide.
The previous upstream blocker is resolved by NVIDIA/OpenShell#2170 and released in OpenShell
0.0.80. AI-Q now requiresopenshell>=0.0.80,<0.1; the setup script defaults to and enforces the0.0.80supported floor. Runtime security decisions remain capability- and state-based rather than branching on an OpenShell version.Key behavior:
aiq=deep-researchand a normalizedaiq-job-idto both OpenShell request metadata and template/container metadata. Active jobs can be queried withopenshell sandbox list --selector aiq=deep-research.allow_shared_sandbox=truedebug configuration. Shared attachment is non-production and does not transfer cleanup ownership to the job.network.allow;allowed_ips/CIDR exceptions are rejected.READY, aLOADEDrevision without a load error, sandbox policy source, matching positive current/active/revision/config versions, structurally identical policies, and matching deterministic hashes.PENDINGaspolicy_status_inconsistent; never weaken attestation because the sandbox is otherwise executable./procbaseline so OpenShell does not enrich the submitted policy and create an unexpected revision.sandbox.attestationandsandbox.cleanupevents without exception messages, policy contents, credentials, SDK response bodies, or tracebacks.develop/PR feat: persist and surface sandbox artifacts #314. Generated files remain job-scoped, receive durable artifact references, and can be rendered or downloaded after the physical sandbox is deleted.Wrote /shared/output.mdwithout creating a non-empty output file. A second false completion fails with the stablewriter_output_missingreason instead of restarting research.setup_openshell.shinstalls the pinned dependencies, generates policy, and builds the image;start_openshell_gateway.shvalidates an authenticated registered or packaged gateway and performs a disposable strict readiness probe;READY,LOADED, effective policy source/content/hash/version, command execution, deletion, and confirmed absence.scripts/smoke_openshell_isolation.pyis now only a thin compatibility launcher.docs/source/deployment/openshell.mdas the canonical operator guide for platform support, ownership, policy/config pairing, startup, acceptance, and troubleshooting.Reviewer Test Instructions
On macOS with Docker Desktop, this is a functional local-demo flow using explicit Landlock
best_effort:Open
http://localhost:3000, create two sessions, and start two concurrent deep-research jobs that request a CSV and PNG chart.While both jobs are running:
Expected behavior:
aiq=deep-researchand distinctaiq-job-idlabels.Run the automated macOS live suite with:
Expected result: all three live isolation/attestation, failure-cleanup/redaction, and shared-policy-mismatch tests pass.
A passing macOS run is functional demo evidence only. Production acceptance requires Linux, Docker, and a policy/config pairing that both require Landlock
hard_requirement, as documented indocs/source/deployment/openshell.md.Validation
Current-head scoped regression suite:
Result: 485 passed, 3 skipped in 6.58s. The three live tests were collected and skipped before optional OpenShell imports or gateway access because live testing was not enabled.
Lint, formatting, and shell syntax:
Result: all checks passed; 53 files already formatted; shell syntax passed.
Configuration validation:
Result: configuration valid.
Documentation:
Result: build succeeded. Sphinx reported two existing unresolved links in
docs/source/integration/agent-skills.md; neither warning originates from the OpenShell documentation.Live OpenShell validation recorded on macOS/Docker Desktop after the
0.0.80rebaseline:Linux/Docker acceptance with Landlock
hard_requirementhas not been run on this macOS host and remains the production acceptance gate.git commit -sor an equivalent sign-off.Where should reviewers start?
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pyscripts/check_openshell_readiness.pyandscripts/start_openshell_gateway.shtests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_live.pysrc/aiq_agent/agents/deep_researcher/custom_middleware.pydocs/source/deployment/openshell.mdRelated Issues
0.0.80: NVIDIA/OpenShell#2170Summary by CodeRabbit