test(e2e): enforce brew Git policy boundary by jyaunches · Pull Request #6586 · NVIDIA/NemoClaw

jyaunches · 2026-07-09T16:37:14Z

Summary

Align the live network-policy E2E with the least-privilege brew/github split introduced by #6541. The test now proves Homebrew works without granting Git egress, then proves the explicit github preset enables that same Git operation.

Related Issue

Refs #6502. Follow-up to #6541.

Changes

Require a non-timeout policy denial when system Git targets GitHub under brew alone.
Keep the Homebrew install proof before applying github, then require Git to succeed after the preset is added.
Add regression coverage that pins policy preset changes to the deterministic inference-policy / network-policy E2E floor.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Quality Gates

Tests added or updated for changed behavior
Existing tests cover changed behavior — justification:
Tests not applicable — justification:
Docs updated for user-facing behavior changes
Docs not applicable — justification: This corrects test coverage for the existing policy contract; CLI behavior, configuration, and user workflows are unchanged.
Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification:
Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
Targeted behavior tests pass for the current change set, or tests are marked not applicable above — npx vitest run --project integration test/pr-risk-plan.test.ts test/brew-github-policy-boundary.test.ts (24 passed); live E2E collection listed both network-policy tests; npm run typecheck passed.
Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
Quality Gates section completed with required justifications or waivers
No secrets, API keys, or credentials committed
npm run docs builds without warnings (doc changes only)
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: J. Yaunches jmyaunch@gmail.com

Summary by CodeRabbit

Bug Fixes
- Strengthened network access enforcement for the Brew preset, blocking direct GitHub Git repository access where it should not be allowed.
- Verified that GitHub Git access works only after the appropriate preset is applied, while preserving expected access to approved endpoints and installed tools.
Tests
- Expanded end-to-end coverage to check both denied and allowed Git operations under the relevant policy presets.

Signed-off-by: J. Yaunches <jmyaunch@gmail.com>

coderabbitai · 2026-07-09T16:37:58Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: adb370a7-a6cb-4554-86fe-f8e9bb0f980c

📥 Commits

Reviewing files that changed from the base of the PR and between 4970809 and 6ecdd8b.

📒 Files selected for processing (2)

test/e2e/live/network-policy.test.ts
test/pr-risk-plan.test.ts

📝 Walkthrough

Walkthrough

Updates the live network-policy e2e test to assert that git ls-remote against GitHub is denied under the brew preset (removing a prior allow-check) and succeeds under the github preset via a new probe. Adds a corresponding risk-plan test mapping for brew.yaml.

Changes

Network policy brew/github git egress test updates

Layer / File(s)	Summary
Brew preset denies GitHub git egress `test/e2e/live/network-policy.test.ts`	Adds a `sandboxBash` probe asserting `git ls-remote` to GitHub's brew.git fails (non-zero exit, denial text, no timeout) under the `brew` preset, and removes the prior inline success check that treated GitHub git access as allowed.
Github preset allows git egress and assertion cleanup `test/e2e/live/network-policy.test.ts`	Removes GitHub-specific markers from `brewProbe` output assertions, applies the `github` preset, and adds a `githubGitProbe` asserting `git ls-remote` succeeds with exit code 0 and a `GITHUB_GIT_OK` marker.
Risk plan mapping for brew preset `test/pr-risk-plan.test.ts`	Adds an `it.each` case mapping `nemoclaw-blueprint/policies/presets/brew.yaml` to the `inference-policy` family requiring `inference-routing` and `network-policy` jobs.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: area: ci, chore

Suggested reviewers: ericksoa

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: tightening the brew policy boundary for Git access in E2E tests.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/network-policy-brew-github-e2e

_{Comment @coderabbitai help to get the list of available commands.}

github-code-quality · 2026-07-09T16:38:34Z

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the fix/network-policy-b... branch remains at 77%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.

File	`main` 4970809	`fix/network-policy-b...` 6ecdd8b	+/-
`src/lib/actions...-add-restart.ts`	19%	14%	-5%
`src/lib/shields/index.ts`	60%	60%	0%
`src/lib/credentials/store.ts`	59%	61%	+2%
`src/lib/actions...ridge-policy.ts`	62%	64%	+2%
`src/lib/messagi.../persistence.ts`	92%	95%	+3%
`src/lib/actions...lution-probe.ts`	88%	94%	+6%
`src/lib/actions...e-validation.ts`	81%	90%	+9%
`src/lib/actions...x/mcp-bridge.ts`	35%	44%	+9%
`src/lib/agent/b...availability.ts`	63%	74%	+11%
`src/lib/sandbox...rsion-scheme.ts`	73%	100%	+27%

_{Updated July 09, 2026 16:47 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

github-actions · 2026-07-09T16:39:03Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: network-policy

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. No required E2E is recommended: this PR is tests-only and cannot affect runtime/user flows directly. The deterministic risk plan lists no required jobs.

Optional E2E

network-policy: Optional confidence check for the modified live network-policy test itself, especially the new brew-vs-github git egress assertions. Not required for merge because the PR is tests-only and the deterministic risk plan has no required jobs.

New E2E recommendations

None.

github-actions · 2026-07-09T16:39:04Z

E2E Target Recommendation

Required E2E targets: network-policy
Optional E2E targets: None

Dispatch required E2E targets:

gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=network-policy

Workflow run

Full E2E target advisor summary

E2E Target Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E targets

network-policy: Focused free-standing E2E job wired for changed live test test/e2e/live/network-policy.test.ts.
- Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field jobs=network-policy

Optional E2E targets

None.

Relevant changed files

test/e2e/live/network-policy.test.ts

github-actions · 2026-07-09T16:41:07Z

PR Review Advisor — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: No advisor follow-up required beyond maintainer review.
Open items: 0 required · 0 warnings · 0 suggestions · 0 test follow-ups

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

github-actions · 2026-07-09T16:41:32Z

PR Review Advisor (Nemotron Ultra) — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: No advisor follow-up required beyond maintainer review.
Open items: 0 required · 0 warnings · 0 suggestions · 0 test follow-ups
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

github-actions · 2026-07-09T16:47:13Z

E2E Target Results — ✅ All requested jobs passed

Run: 29034228621
Workflow ref: fix/network-policy-brew-github-e2e
Requested targets: (default — all supported)
Requested jobs: network-policy
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job	Result
network-policy	✅ success

## Summary  Align the live network-policy E2E with the least-privilege `brew`/`github` split introduced by NVIDIA#6541. The test now proves Homebrew works without granting Git egress, then proves the explicit `github` preset enables that same Git operation. ## Related Issue  Refs NVIDIA#6502. Follow-up to NVIDIA#6541. ## Changes  - Require a non-timeout policy denial when system Git targets GitHub under `brew` alone. - Keep the Homebrew install proof before applying `github`, then require Git to succeed after the preset is added. - Add regression coverage that pins policy preset changes to the deterministic `inference-policy` / `network-policy` E2E floor. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates  - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: This corrects test coverage for the existing policy contract; CLI behavior, configuration, and user workflows are unchanged. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification  - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npx vitest run --project integration test/pr-risk-plan.test.ts test/brew-github-policy-boundary.test.ts` (24 passed); live E2E collection listed both network-policy tests; `npm run typecheck` passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) ---  Signed-off-by: J. Yaunches <jmyaunch@gmail.com>  ## Summary by CodeRabbit * **Bug Fixes** * Strengthened network access enforcement for the Brew preset, blocking direct GitHub Git repository access where it should not be allowed. * Verified that GitHub Git access works only after the appropriate preset is applied, while preserving expected access to approved endpoints and installed tools. * **Tests** * Expanded end-to-end coverage to check both denied and allowed Git operations under the relevant policy presets.  Signed-off-by: J. Yaunches <jmyaunch@gmail.com>

test(e2e): enforce brew Git policy boundary

6ecdd8b

Signed-off-by: J. Yaunches <jmyaunch@gmail.com>

jyaunches added area: e2e End-to-end tests, nightly failures, or validation infrastructure area: policy Network policy, egress rules, presets, or sandbox policy v0.0.79 labels Jul 9, 2026

jyaunches self-assigned this Jul 9, 2026

cv approved these changes Jul 9, 2026

View reviewed changes

cv enabled auto-merge (squash) July 9, 2026 16:39

cv merged commit 4df9fc2 into main Jul 9, 2026
201 of 202 checks passed

cv deleted the fix/network-policy-brew-github-e2e branch July 9, 2026 16:47

Navigation Menu

Uh oh!

test(e2e): enforce brew Git policy boundary - #6586

test(e2e): enforce brew Git policy boundary#6586
cv merged 1 commit into
mainfrom
fix/network-policy-brew-github-e2e

jyaunches commented Jul 9, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 9, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

github-code-quality Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 9, 2026

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Uh oh!

github-actions Bot commented Jul 9, 2026

E2E Target Advisor

Required E2E targets

Optional E2E targets

Relevant changed files

Uh oh!

github-actions Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jyaunches commented Jul 9, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Type of Change

Quality Gates

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

github-code-quality Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

TypeScript / code-coverage/plugin

TypeScript / code-coverage/cli

Uh oh!

github-actions Bot commented Jul 9, 2026

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Uh oh!

github-actions Bot commented Jul 9, 2026

E2E Target Recommendation

E2E Target Advisor

Required E2E targets

Optional E2E targets

Relevant changed files

Uh oh!

github-actions Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor — No blocking findings

Uh oh!

github-actions Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor (Nemotron Ultra) — No blocking findings

Uh oh!

github-actions Bot commented Jul 9, 2026

E2E Target Results — ✅ All requested jobs passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jyaunches commented Jul 9, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 9, 2026 •

edited

Loading

github-code-quality Bot commented Jul 9, 2026 •

edited

Loading

github-actions Bot commented Jul 9, 2026 •

edited

Loading

github-actions Bot commented Jul 9, 2026 •

edited

Loading