Skip to content

fix(policy): restore Tavily egress for managed Python #6134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
apurvvkumaria merged 3 commits into from
Jul 1, 2026
Merged

fix(policy): restore Tavily egress for managed Python #6134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d325e2c
fix(policy): restore Tavily egress for managed Python
apurvvkumaria Jul 1, 2026
9bb5072
test(policy): lock Tavily managed Python boundary
apurvvkumaria Jul 1, 2026
f2a161c
test(policy): cover Tavily project venv denial
apurvvkumaria Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions nemoclaw-blueprint/policies/presets/tavily.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ network_policies:
- allow: { method: GET, path: "/**" }
- allow: { method: POST, path: "/**" }
binaries:
# OpenShell attributes Deep Agents Code Tavily requests to this managed
# Python venv, which its strict Landlock policy mounts read-only.
- { path: /opt/venv/bin/python3* }
- { path: /usr/local/bin/node }
- { path: /usr/bin/node }
- { path: /usr/local/bin/curl }
Expand Down
3 changes: 3 additions & 0 deletions nemoclaw-blueprint/provider-profiles/tavily.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ endpoints:
access: read-write
enforcement: enforce
binaries:
# OpenShell attributes Deep Agents Code Tavily requests to this managed
# Python venv, which its strict Landlock policy mounts read-only.
- /opt/venv/bin/python3*
- /usr/local/bin/node
- /usr/bin/node
- /usr/local/bin/curl
Expand Down
28 changes: 27 additions & 1 deletion test/e2e/e2e-cloud-experimental/checks/09-deepagents-code-tavily-opt-in.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ SANDBOX_NAME="${SANDBOX_NAME:-${NEMOCLAW_SANDBOX_NAME:-e2e-cloud-onboard}}"
PREFIX="09-deepagents-code-tavily-opt-in"
REPO="${REPO:-$(pwd)}"
CLI="${NEMOCLAW_E2E_CLI:-${REPO}/bin/nemoclaw.js}"
PROJECT_VENV="/sandbox/.nemoclaw-e2e-project-venv"
PROJECT_PYTHON="${PROJECT_VENV}/bin/python3"

ok() { printf '%s\n' "${PREFIX}: OK ($*)"; }
info() { printf '%s\n' "${PREFIX}: $*"; }
Expand Down Expand Up @@ -93,13 +95,14 @@ PY

python_probe() {
local url="$1"
local python_bin="${2:-python3}"
local encoded remote_cmd
if [ -n "${NEMOCLAW_E2E_TAVILY_PROBE_FIXTURE+x}" ]; then
printf '%s\n' "$NEMOCLAW_E2E_TAVILY_PROBE_FIXTURE"
return 0
fi
encoded="$(python_probe_source | base64 | tr -d '\n')"
remote_cmd="python3 -c \"\$(printf '%s' ${encoded@Q} | base64 -d)\" ${url@Q}"
remote_cmd="${python_bin@Q} -c \"\$(printf '%s' ${encoded@Q} | base64 -d)\" ${url@Q}"
sandbox_exec "$remote_cmd"
}

Expand Down Expand Up @@ -167,5 +170,28 @@ else
fail_test "Tavily probe lacked reachability evidence after policy-add: $PROBE_OUTPUT"
fi

SYSTEM_PROBE_OUTPUT="$(python_probe "https://api.tavily.com/" "/usr/bin/python3" || true)"
if echo "$SYSTEM_PROBE_OUTPUT" | grep -q "BLOCKED:" && ! echo "$SYSTEM_PROBE_OUTPUT" | grep -q "REACHED:"; then
pass "system Python remains blocked from Tavily after policy-add"
elif echo "$SYSTEM_PROBE_OUTPUT" | grep -q "REACHED:"; then
fail_test "system Python reached Tavily unexpectedly after policy-add: $SYSTEM_PROBE_OUTPUT"
else
fail_test "system Python Tavily probe lacked denial evidence after policy-add: $SYSTEM_PROBE_OUTPUT"
fi

PROJECT_OUT="$(sandbox_exec "if ! test -x ${PROJECT_PYTHON@Q}; then python3 -m venv --copies ${PROJECT_VENV@Q}; fi; test -x ${PROJECT_PYTHON@Q} && readlink -f ${PROJECT_PYTHON@Q}" || true)"
if echo "$PROJECT_OUT" | grep -Fxq "$PROJECT_PYTHON"; then
PROJECT_PROBE_OUTPUT="$(python_probe "https://api.tavily.com/" "$PROJECT_PYTHON" || true)"
if echo "$PROJECT_PROBE_OUTPUT" | grep -q "BLOCKED:" && ! echo "$PROJECT_PROBE_OUTPUT" | grep -q "REACHED:"; then
pass "project venv Python under /sandbox remains blocked from Tavily after policy-add"
elif echo "$PROJECT_PROBE_OUTPUT" | grep -q "REACHED:"; then
fail_test "project venv Python reached Tavily unexpectedly after policy-add: $PROJECT_PROBE_OUTPUT"
else
fail_test "project venv Python Tavily probe lacked denial evidence after policy-add: $PROJECT_PROBE_OUTPUT"
fi
else
fail_test "project venv under /sandbox did not expose a usable python3 executable: $PROJECT_OUT"
fi
Comment on lines +182 to +194

@coderabbitai coderabbitai Bot Jul 1, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Fragile path-equality check via readlink -f.

readlink -f canonicalizes the full path, resolving any symlinks in the chain (e.g. venv python3 binaries are often themselves symlinks, or --copies may fall back to a symlink on some platforms/venv implementations). Comparing that canonical output against the literal, non-canonicalized $PROJECT_PYTHON string with grep -Fxq can spuriously fail even when the venv executable is perfectly usable, producing a flaky fail_test "project venv under /sandbox did not expose a usable python3 executable".

Prefer checking the exit status of test -x directly instead of string-comparing resolved paths.

🔧 Proposed fix 
-PROJECT_OUT="$(sandbox_exec "if ! test -x ${PROJECT_PYTHON@Q}; then python3 -m venv --copies ${PROJECT_VENV@Q}; fi; test -x ${PROJECT_PYTHON@Q} && readlink -f ${PROJECT_PYTHON@Q}" || true)"
-if echo "$PROJECT_OUT" | grep -Fxq "$PROJECT_PYTHON"; then
+PROJECT_OUT="$(sandbox_exec "if ! test -x ${PROJECT_PYTHON@Q}; then python3 -m venv --copies ${PROJECT_VENV@Q}; fi; test -x ${PROJECT_PYTHON@Q} && echo OK" || true)"
+if echo "$PROJECT_OUT" | grep -Fxq "OK"; then
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
PROJECT_OUT="$(sandbox_exec "if ! test -x ${PROJECT_PYTHON@Q}; then python3 -m venv --copies ${PROJECT_VENV@Q}; fi; test -x ${PROJECT_PYTHON@Q} && readlink -f ${PROJECT_PYTHON@Q}" || true)"
if echo "$PROJECT_OUT" | grep -Fxq "$PROJECT_PYTHON"; then
PROJECT_PROBE_OUTPUT="$(python_probe "https://api.tavily.com/" "$PROJECT_PYTHON" || true)"
if echo "$PROJECT_PROBE_OUTPUT" | grep -q "BLOCKED:" && ! echo "$PROJECT_PROBE_OUTPUT" | grep -q "REACHED:"; then
pass "project venv Python under /sandbox remains blocked from Tavily after policy-add"
elif echo "$PROJECT_PROBE_OUTPUT" | grep -q "REACHED:"; then
fail_test "project venv Python reached Tavily unexpectedly after policy-add: $PROJECT_PROBE_OUTPUT"
else
fail_test "project venv Python Tavily probe lacked denial evidence after policy-add: $PROJECT_PROBE_OUTPUT"
fi
else
fail_test "project venv under /sandbox did not expose a usable python3 executable: $PROJECT_OUT"
fi
PROJECT_OUT="$(sandbox_exec "if ! test -x ${PROJECT_PYTHON@Q}; then python3 -m venv --copies ${PROJECT_VENV@Q}; fi; test -x ${PROJECT_PYTHON@Q} && echo OK" || true)"
if echo "$PROJECT_OUT" | grep -Fxq "OK"; then
PROJECT_PROBE_OUTPUT="$(python_probe "https://api.tavily.com/" "$PROJECT_PYTHON" || true)"
if echo "$PROJECT_PROBE_OUTPUT" | grep -q "BLOCKED:" && ! echo "$PROJECT_PROBE_OUTPUT" | grep -q "REACHED:"; then
pass "project venv Python under /sandbox remains blocked from Tavily after policy-add"
elif echo "$PROJECT_PROBE_OUTPUT" | grep -q "REACHED:"; then
fail_test "project venv Python reached Tavily unexpectedly after policy-add: $PROJECT_PROBE_OUTPUT"
else
fail_test "project venv Python Tavily probe lacked denial evidence after policy-add: $PROJECT_PROBE_OUTPUT"
fi
else
fail_test "project venv under /sandbox did not expose a usable python3 executable: $PROJECT_OUT"
fi
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/e2e-cloud-experimental/checks/09-deepagents-code-tavily-opt-in.sh`
around lines 182 - 194, The `PROJECT_OUT` / `PROJECT_PYTHON` usability check is
brittle because `readlink -f` can return a canonicalized symlink target that
won’t string-match the original path, causing false failures in the
`sandbox_exec` probe. Update the logic around `PROJECT_OUT` in the deepagents
Tavily opt-in check to rely on `test -x`/shell exit status for the venv Python
executable instead of `grep -Fxq` path equality, and keep the existing
`python_probe` and `fail_test`/`pass` branches keyed off that executable check.


printf '%s\n' "${PREFIX}: $PASSED passed, $FAILED failed"
[ "$FAILED" -eq 0 ] || exit 1
10 changes: 9 additions & 1 deletion test/langchain-deepagents-code-image.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -520,10 +520,18 @@ describe("LangChain Deep Agents Code image contracts", () => {
expect(tavilyOptInCheck).toContain("https://api.tavily.com/");
expect(tavilyOptInCheck).toContain("python_probe_source");
expect(tavilyOptInCheck).toContain("base64 | tr -d");
expect(tavilyOptInCheck).toContain("python3 -c");
expect(tavilyOptInCheck).toContain("${python_bin@Q} -c");
expect(tavilyOptInCheck).toContain("NEMOCLAW_E2E_TAVILY_SELF_TEST");
expect(tavilyOptInCheck).toContain("/opt/venv/");
expect(tavilyOptInCheck).toContain("managed Deep Agents Code python can reach Tavily");
expect(tavilyOptInCheck).toContain('python_probe "https://api.tavily.com/" "/usr/bin/python3"');
expect(tavilyOptInCheck).toContain(
"system Python remains blocked from Tavily after policy-add",
);
expect(tavilyOptInCheck).toContain("/sandbox/.nemoclaw-e2e-project-venv");
expect(tavilyOptInCheck).toContain(
"project venv Python under /sandbox remains blocked from Tavily after policy-add",
);
expect(cloudExperimentalChecksForOnboarding("cloud-langchain-deepagents-code")).toEqual([
"test/e2e/e2e-cloud-experimental/checks/05-deepagents-code-landlock-readonly.sh",
"test/e2e/e2e-cloud-experimental/checks/06-deepagents-code-python-egress.sh",
Expand Down
1 change: 1 addition & 0 deletions test/tavily-preset.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ describe("tavily opt-in preset", () => {
},
]);
expect(policy?.binaries).toEqual([
{ path: "/opt/venv/bin/python3*" },
{ path: "/usr/local/bin/node" },
{ path: "/usr/bin/node" },
{ path: "/usr/local/bin/curl" },
Expand Down
26 changes: 26 additions & 0 deletions test/validate-blueprint.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,14 @@ const TAVILY_PROVIDER_PROFILE_PATH = new URL(
"../nemoclaw-blueprint/provider-profiles/tavily.yaml",
import.meta.url,
);
const TAVILY_POLICY_PRESET_PATH = new URL(
"../nemoclaw-blueprint/policies/presets/tavily.yaml",
import.meta.url,
);
const DEEPAGENTS_POLICY_PATH = new URL(
"../agents/langchain-deepagents-code/policy-additions.yaml",
import.meta.url,
);
const PERMISSIVE_POLICY_PATH = new URL(
"../nemoclaw-blueprint/policies/openclaw-sandbox-permissive.yaml",
import.meta.url,
Expand Down Expand Up @@ -87,6 +95,7 @@ type PolicyEntry = {

type SandboxPolicy = {
version?: number;
filesystem_policy?: { read_only?: string[] };
network_policies?: Record<string, PolicyEntry>;
};

Expand Down Expand Up @@ -479,6 +488,8 @@ describe("Brave Search provider profile", () => {

describe("Tavily Search provider profile", () => {
const profile = loadYaml<ProviderProfile>(TAVILY_PROVIDER_PROFILE_PATH);
const preset = loadYaml<PolicyPreset>(TAVILY_POLICY_PRESET_PATH);
const deepAgentsPolicy = loadYaml<SandboxPolicy>(DEEPAGENTS_POLICY_PATH);

it("routes TAVILY_API_KEY through a bearer authorization header", () => {
expect(profile.id).toBe("tavily");
Expand All @@ -505,12 +516,27 @@ describe("Tavily Search provider profile", () => {

it("limits the binary allowlist to runtimes the Tavily client actually uses", () => {
expect(profile.binaries).toEqual([
"/opt/venv/bin/python3*",
"/usr/local/bin/node",
"/usr/bin/node",
"/usr/local/bin/curl",
"/usr/bin/curl",
]);
});

it("keeps its binary allowlist aligned with the Tavily policy preset", () => {
const presetBinaries = preset.network_policies?.tavily?.binaries?.map(({ path }) => path);
expect(profile.binaries).toEqual(presetBinaries);
});

it("anchors managed Python access to Deep Agents Code's read-only venv", () => {
const managedPython = "/opt/venv/bin/python3*";
const managedInferenceBinaries = deepAgentsPolicy.network_policies?.managed_inference?.binaries;

expect(deepAgentsPolicy.filesystem_policy?.read_only).toContain("/opt/venv");
expect(managedInferenceBinaries).toContainEqual({ path: managedPython });
expect(profile.binaries).toContain(managedPython);
});
});

describe("permissive sandbox policy", () => {
Expand Down