feat(policy): add opt-in Tavily web-search preset for Deep Agents Code (#5621)

[abhi-0906]

Closes #5621
Fixes #5739

Adds the opt-in path for Tavily web search in the LangChain Deep Agents Code harness, which ships with Tavily and LangSmith denied by default.

• New maintained tavily policy preset (nemoclaw-blueprint/policies/presets/tavily.yaml) that opens egress to api.tavily.com:443 only, scoped to the python3/node/curl binaries (the harness's network calls are attributed to the sandbox python3 interpreter).
• The quickstart documents the enable flow (nemoclaw <name> policy-add tavily), supplying the Tavily credential at runtime (it is not baked into the managed config or image), and how to remove it.
• LangSmith tracing is documented as not yet supported: no preset opens api.smith.langchain.com and no supported mechanism injects LANGSMITH_API_KEY.
• A test asserts the preset declares api.tavily.com and that the Deep Agents Code default policy still denies both Tavily and LangSmith; the preset-name list is updated accordingly.

Signed-off-by: Abhimanyu Kumar abhimanyukumar7290@gmail.com

Summary by CodeRabbit

• New Features

  • Added an opt-in Tavily web search access policy preset to enable controlled outbound requests.

• Documentation

  • Expanded guidance for optional features: Tavily is now explicitly opt-in (including runtime API key handling and how to remove the policy).
  • Clarified that LangSmith tracing is not supported in the managed harness, and that no presets enable it.

• Tests

  • Added/updated tests to verify the Tavily preset includes the expected outbound domain and allowed execution paths, and to ensure default policy additions do not reference Tavily or LangSmith endpoints.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a new tavily preset, expands quickstart guidance for Tavily and LangSmith, and adds tests for preset discovery and expected policy contents.

Changes

Tavily opt-in preset, docs, and tests

Layer / File(s)	Summary
Tavily preset and coverage tests nemoclaw-blueprint/policies/presets/tavily.yaml, test/policies.test.ts, test/tavily-preset.test.ts	Defines the tavily preset with api.tavily.com:443 GET/POST allow rules and a python3*/node/curl binaries whitelist; tests verify it is listed, contains the expected host and binaries, and is absent from the DeepAgents default policy additions.
Tavily quickstart guidance docs/get-started/quickstart-langchain-deepagents-code.mdx	Documents Tavily as an opt-in sandbox preset, describes the allowed endpoint scope, requires TAVILY_API_KEY at runtime, notes process-wide egress for sandbox python3, and adds preset removal instructions.
LangSmith support note docs/get-started/quickstart-langchain-deepagents-code.mdx	States that LangSmith tracing is not supported in the managed harness, that start.sh forwards only non-secret toggles, and that no preset opens api.smith.langchain.com or injects LANGSMITH_API_KEY.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 I hop to Tavily, then hop back again,
With presets and tests, the paths stay plain.
LangSmith sits quiet, the docs make it clear,
A small new search gate appears this year.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately reflects the main change: adding an opt-in Tavily preset for Deep Agents Code.
Linked Issues check	✅ Passed	The PR matches #5621 by adding opt-in Tavily support, documenting LangSmith as unsupported, denying both by default, and adding tests.
Out of Scope Changes check	✅ Passed	The changes stay within the stated scope of Tavily/LangSmith policy, documentation, and test updates.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

docs/get-started/quickstart-langchain-deepagents-code.mdx (1)

59-59: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Fix sentence-per-line and colon punctuation style in the added prose.

Line 59 uses a colon between clauses (not to introduce a list), and Line 70 has multiple sentences on one source line. Split into one sentence per line and use periods unless a list follows.

As per path instructions, "Colons should only introduce a list" and "One sentence per line in source." As per coding guidelines, "Keep one sentence per line in Markdown and MDX source files."

Also applies to: 70-70

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/get-started/quickstart-langchain-deepagents-code.mdx` at line 59, In the
sentence beginning with "NemoClaw does not enable Tavily or LangSmith by default
for this harness," replace the colon that separates clauses with a period to
create two separate sentences, as colons should only introduce lists.
Additionally, review the content around this area (including line 70) and ensure
all prose follows the one-sentence-per-line rule by splitting any lines
containing multiple sentences into separate lines, with each sentence on its own
line.

Sources: Coding guidelines, Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/get-started/quickstart-langchain-deepagents-code.mdx`:
- Around line 65-67: The CLI command examples in the policy-add tavily commands
use the concrete alias nemoclaw instead of the placeholder $$nemoclaw. For
shared OpenClaw/Hermes documentation, replace all instances of nemoclaw with
$$nemoclaw in the command examples to ensure the rendered docs display the
correct command name per variant. This applies to the lines shown in the diff as
well as line 75.

In `@test/policies.test.ts`:
- Around line 182-184: The test checking the Tavily opt-in policy only verifies
api.tavily.com and Python path with expect(tavily).toContain() assertions, which
leaves it vulnerable to regressions in Node and Curl scoping. Add additional
expect(tavily).toContain() assertions after the existing Python path check to
also verify that the Tavily policy includes Node and Curl entries, ensuring the
full preset contract is enforced in the test.

---

Nitpick comments:
In `@docs/get-started/quickstart-langchain-deepagents-code.mdx`:
- Line 59: In the sentence beginning with "NemoClaw does not enable Tavily or
LangSmith by default for this harness," replace the colon that separates clauses
with a period to create two separate sentences, as colons should only introduce
lists. Additionally, review the content around this area (including line 70) and
ensure all prose follows the one-sentence-per-line rule by splitting any lines
containing multiple sentences into separate lines, with each sentence on its own
line.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 848de151-9528-406c-a482-d9ca4ba863ec

📥 Commits

Reviewing files that changed from the base of the PR and between a9f31e4 and d0b112d.

📒 Files selected for processing (3)

• docs/get-started/quickstart-langchain-deepagents-code.mdx
• nemoclaw-blueprint/policies/presets/tavily.yaml
• test/policies.test.ts

[abhi-0906]

CodeRabbit feedback addressed (commit 7502d43): added Node and Curl binary-scoping assertions to the Tavily opt-in test, so the test fails if any of the preset's binary scoping regresses.

The $$nemoclaw suggestion on the quickstart is not applicable here — that page is openclaw-only and not variant-generated, so literal nemoclaw is correct (rationale in the inline thread, which CodeRabbit acknowledged).

[wscurran]

✨ Thanks for adding the opt-in Tavily web-search preset that opens egress to api.tavily.com:443 for the Deep Agents Code harness. This proposes a way to enable web search via nemoclaw policy-add tavily while keeping Tavily and LangSmith denied by default in the default policy.

---

Related open issues:

• #5621 Define Deep Agents Code optional web search and tracing support

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/get-started/quickstart-langchain-deepagents-code.mdx`:
- Around line 59-72: Update the quickstart prose so colons are used only to
introduce lists, not as clause punctuation or to precede standalone command
blocks. In the section describing NemoClaw/Tavily, rephrase the sentence after
the harness mention to remove the trailing colon and adjust the “Preview the
endpoints…” and “Remove the access…” lead-ins so they flow without a colon,
while keeping the command examples under the existing `tavily` policy guidance
intact.
- Line 70: Split the multi-sentence text in the quickstart MDX docs so each
sentence is on its own source line, following the one-sentence-per-line style
guide; update the affected prose near the described `tavily`/runtime API key
guidance and any other matching sentence-packed line in the same section so the
formatting is consistent.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4c3fdbcf-0fe6-4c17-b67b-de6ecaf7f956

📥 Commits

Reviewing files that changed from the base of the PR and between e3da567 and 763e917.

📒 Files selected for processing (2)

• docs/get-started/quickstart-langchain-deepagents-code.mdx
• nemoclaw-blueprint/policies/presets/tavily.yaml

🚧 Files skipped from review as they are similar to previous changes (1)

• nemoclaw-blueprint/policies/presets/tavily.yaml

[coderabbitai]

🧹 Nitpick comments (1)

docs/get-started/quickstart-langchain-deepagents-code.mdx (1)

82-82: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove the emphasis from this routine status note.

not a supported integration is routine prose, so bolding it reads like LLM-style emphasis rather than a warning. LLM pattern detected.

Suggested edit

-LangSmith tracing is **not a supported integration** for this managed harness yet.
+LangSmith tracing is not a supported integration for this managed harness yet.

As per path instructions, "Unnecessary bold on routine instructions" should be flagged.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/get-started/quickstart-langchain-deepagents-code.mdx` at line 82, Remove
the bold emphasis from the routine status note in the quickstart markdown so it
reads as plain prose instead of a warning. Update the text in the relevant
documentation snippet to keep the message about LangSmith tracing and the
managed harness unchanged, but render “not a supported integration” without
emphasis. Locate the wording in the quickstart content and adjust only the
markdown styling, not the meaning.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/get-started/quickstart-langchain-deepagents-code.mdx`:
- Line 82: Remove the bold emphasis from the routine status note in the
quickstart markdown so it reads as plain prose instead of a warning. Update the
text in the relevant documentation snippet to keep the message about LangSmith
tracing and the managed harness unchanged, but render “not a supported
integration” without emphasis. Locate the wording in the quickstart content and
adjust only the markdown styling, not the meaning.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2492e51f-2fc2-4a23-a57a-0dce34c0ad45

📥 Commits

Reviewing files that changed from the base of the PR and between 763e917 and 303d69b.

📒 Files selected for processing (2)

• docs/get-started/quickstart-langchain-deepagents-code.mdx
• test/tavily-preset.test.ts

[cv]

Manual PR Review Advisor result

This PR Review Advisor analysis was run manually via workflow_dispatch, so the workflow did not post its usual sticky comment. Posting the advisor summary here because it produced actionable findings.

Run: https://github.com/NVIDIA/NemoClaw/actions/runs/28206227058

---

PR Review Advisor

The Tavily preset is opt-in and avoids default credential persistence, but its security boundary is broad and only string-tested.

Required before merge

• None.

Resolve or justify before merge

• Source-of-truth review needed: Tavily policy preset and Deep Agents Code documentation: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Verification hint: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Covered by the security finding: add parsed YAML tests that lock exact endpoint, enforcement, methods/paths, tunnel absence, and binary allowlist.
  • Evidence: Docs mention Python process-wide attribution, while tavily.yaml also allows Node and curl and GET/POST to /**; the new test does not assert exact shape.
• Tavily opt-in egress boundary is broad and not locked down by tests (nemoclaw-blueprint/policies/presets/tavily.yaml:16): The new preset fixes the host to api.tavily.com:443 with REST enforcement, but it permits GET and POST to "/**" and allows several process origins, including sandbox Python, Node, and curl. The docs justify the Python-wide attribution boundary for Deep Agents Code, but the changed files do not show why Node or curl need access to the credential-bearing Tavily API. The new test only checks for string containment, so it would still pass if the preset gained an extra host, switched to access: full/tls: skip, added methods, or further widened binaries.
  • Impact: A user who opts into Tavily would grant any matching Python, Node, or curl process broad exfiltration-capable egress to the Tavily API host. That is not an internal SSRF bypass, but it weakens least privilege for a sandbox network-policy preset and makes future accidental widening hard to catch.
  • Recommendation: Either narrow the preset to the minimum Tavily API paths and binaries required by Deep Agents Code, or add an in-repo justification for each broad element. In either case, replace the containment-only test with parsed YAML assertions for the exact host, port, protocol, enforcement, absence of L4/full-tunnel fields, allowed methods/paths, and exact binary allowlist.
  • Verification hint: Read nemoclaw-blueprint/policies/presets/tavily.yaml and confirm whether rules remain GET/POST /**, whether Node/curl remain in binaries, and whether test/tavily-preset.test.ts parses the YAML instead of using only toContain checks.
  • Missing regression test: Add a test such as parses tavily preset and permits only api.tavily.com:443 with REST enforcement, asserting exactly one endpoint, no access: full, no tls: skip, exact allowed methods/paths, and the exact justified binary list. If Node/curl stay, the test name or assertions should document that they are intentionally part of the supported boundary.
  • Evidence: tavily.yaml allows GET and POST path: "/**" and binaries /usr/bin/python3*, /usr/local/bin/python3*, /usr/local/bin/node, /usr/bin/node, and /usr/bin/curl; test/tavily-preset.test.ts only calls expect(content).toContain(...) for a subset of those strings.

In-scope improvements

• None.

Test follow-ups to resolve or justify

• Runtime validation — parses tavily preset and permits only api.tavily.com:443 with REST enforcement. The changed behavior is a sandbox/OpenShell network-policy preset. Static tests can lock the YAML boundary, but runtime validation is still useful to prove an applied preset reaches Tavily while default-denied integrations remain blocked.
• Runtime validation — rejects tavily preset widening to access: full, tls: skip, or additional hosts. The changed behavior is a sandbox/OpenShell network-policy preset. Static tests can lock the YAML boundary, but runtime validation is still useful to prove an applied preset reaches Tavily while default-denied integrations remain blocked.
• Runtime validation — locks tavily allowed methods and paths to the documented Tavily API surface. The changed behavior is a sandbox/OpenShell network-policy preset. Static tests can lock the YAML boundary, but runtime validation is still useful to prove an applied preset reaches Tavily while default-denied integrations remain blocked.
• Runtime validation — locks tavily binary allowlist to the justified Deep Agents Code interpreter boundary. The changed behavior is a sandbox/OpenShell network-policy preset. Static tests can lock the YAML boundary, but runtime validation is still useful to prove an applied preset reaches Tavily while default-denied integrations remain blocked.
• Runtime validation — after policy-add tavily, Python-originated Deep Agents Tavily traffic reaches api.tavily.com while LangSmith remains blocked without a separate policy. The changed behavior is a sandbox/OpenShell network-policy preset. Static tests can lock the YAML boundary, but runtime validation is still useful to prove an applied preset reaches Tavily while default-denied integrations remain blocked.
• Acceptance clause: Tests cover opt-in egress and credential handling for each supported optional integration. — add test evidence or identify existing coverage. Existing Deep Agents Code tests cover credential handling by asserting optional service secrets such as TAVILY_API_KEY, DEEPAGENTS_CODE_TAVILY_API_KEY, and LANGSMITH_API_KEY are not serialized. The new Tavily opt-in test only string-checks the preset and does not lock down exact endpoint, enforcement, method/path, L4 tunnel absence, or binary scope; this is covered by the finding.
• Acceptance clause: live validation that the applied preset actually reaches api.tavily.com through OpenShell (live e2e) — add test evidence or identify existing coverage. The linked issue comment identifies this as a maintainer/live-env action. This PR adds static preset coverage but no changed live OpenShell validation for applying the Tavily preset and reaching the host.
• Tavily policy preset and Deep Agents Code documentation — Covered by the security finding: add parsed YAML tests that lock exact endpoint, enforcement, methods/paths, tunnel absence, and binary allowlist.. Docs mention Python process-wide attribution, while tavily.yaml also allows Node and curl and GET/POST to /**; the new test does not assert exact shape.

What looks good

• The Tavily integration is opt-in rather than added to the Deep Agents Code default policy.
• The quickstart clearly documents that Tavily credentials are supplied at runtime and not baked into the managed config or image.
• LangSmith is explicitly documented as unsupported, avoiding an implied half-supported credential path.
• The PR uses a small focused test file and does not introduce new test runners or framework abstractions.