Skip to content

ci: guard oversized test files - #4899

Closed
cv wants to merge 9 commits into
mainfrom
codex/test-size-guard
Closed

ci: guard oversized test files#4899
cv wants to merge 9 commits into
mainfrom
codex/test-size-guard

Conversation

@cv

@cv cv commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a test-file size budget so oversized test files cannot quietly grow again after the CLI split work. The guard mirrors the existing growth-budget style by grandfathering current oversized files at their present size while requiring future shrinkage to ratchet the budget down.

Related Issue

Related to #4892

Changes

  • Add ci/test-file-size-budget.json with a 1,500-line default ceiling and explicit legacy budgets for existing oversized test files.
  • Add scripts/check-test-file-size-budget.ts plus unit coverage for default limits, legacy growth prevention, ratcheting, stale budget entries, and line counting.
  • Wire npm run test-size:check into prek, PR static checks, and the data-only pull_request_target growth guardrail workflow.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Focused checks run:

  • npm run test-size:check
  • npx vitest run --project cli test/test-file-size-budget.test.ts
  • npm run typecheck:cli
  • npm run source-shape:check
  • umask 022 && npx prek run --files ci/test-file-size-budget.json scripts/check-test-file-size-budget.ts test/test-file-size-budget.test.ts package.json .pre-commit-config.yaml .github/workflows/pr.yaml .github/workflows/codebase-growth-guardrails.yaml
  • Commit hook and pre-push hook passed for the changed files; the first HTTPS push was rejected only because the OAuth token lacked workflow scope, then the SSH push succeeded.

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • Chores
    • Introduced automated test file-size guardrails enforced in CI and local hooks, added a configurable budget file and a new validation command to check test sizes.
  • Tests
    • Added unit, integration, contract and end-to-end tests validating budget parsing/enforcement, workflow integration, and proxy preflight behavior.
  • Documentation
    • Clarified proxy troubleshooting and NO_PROXY guidance to include additional local/container hostnames (including inference.local).

@cv cv self-assigned this Jun 7, 2026
@coderabbitai

coderabbitai Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cea7ac27-ef51-420c-a365-9b90e870b937

📥 Commits

Reviewing files that changed from the base of the PR and between 20bf994 and 9f35697.

📒 Files selected for processing (1)
  • test/pr-workflow-contract.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/pr-workflow-contract.test.ts

📝 Walkthrough

Walkthrough

Adds an enforced test-file-size budgeting system (script, JSON budget, tests, pre-commit/npm/CI wiring and an inline GitHub Actions validator enforcing monotonicity/legacy rules) and expands NO_PROXY augmentation to include container aliases and the managed hostname inference.local, with related tests and docs updates.

Changes

Test File Size Budget Enforcement

Layer / File(s) Summary
Test file size budget script
scripts/check-test-file-size-budget.ts
Core implementation: scans test/, src/, nemoclaw/src/ for test/spec files, counts lines, parses/validates ci/test-file-size-budget.json, evaluates violations (oversized, legacy-ratchet, stale-legacy), formats failure messages, and exits nonzero on violations.
Budget configuration
ci/test-file-size-budget.json
JSON budget: defaultMaxLines: 1500 with a legacyMaxLines map for per-file grandfathered limits.
Test suite validation
test/test-file-size-budget.test.ts
Vitest tests: verify countLines, parseBudget, oversized detection, legacy budget enforcement (no growth), legacy ratchet detection, and stale-legacy detection.
CI, npm, and pre-commit wiring
package.json, .pre-commit-config.yaml, .github/workflows/pr.yaml, .github/workflows/codebase-growth-guardrails.yaml, test/pr-workflow-contract.test.ts
Adds test-size:check npm script; pre-commit test-file-size-budget local hook; pr.yaml runs npm run test-size:check (and skips it in the main prek invocation); codebase-growth-guardrails.yaml gains permissions.contents: read and an inline Node.js validator that downloads changed test files and enforces budget/monotonicity rules; workflow contract tests updated accordingly.

NO_PROXY augmentation and proxy preflight

Layer / File(s) Summary
withLocalNoProxy implementation & docs
nemoclaw/src/lib/subprocess-env.ts, docs/reference/troubleshooting.mdx
Extends NO_PROXY augmentation to include inference.local, additional container/host aliases, and loopback/all-interfaces entries; updates docs to clarify host-side sandbox/create boundary and managed inference routing.
Proxy preflight checks & onboarding
src/lib/onboard/http-proxy-preflight.ts, src/lib/onboard/http-proxy-preflight.test.ts, test/onboard.test.ts
Require NO_PROXY to include localhost, 127.0.0.1, and inference.local to suppress warnings; update warning text and suggested NO_PROXY/no_proxy exports in tests and onboarding checks.
Unit tests and E2E
src/lib/subprocess-env.test.ts, test/credential-exposure.test.ts, test/host-proxy-inference-local-e2e.test.ts
Update and add unit and E2E tests verifying LOCAL_NO_PROXY expansion, correct injection of inference.local and host.containers.internal, safe .local handling, and host-side curl bypass behavior.

Sequence Diagram(s)

sequenceDiagram
  participant CI as GitHub Actions
  participant Validator as Inline Node.js validator
  participant Budget as ci/test-file-size-budget.json
  participant GHAPI as GitHub Contents API
  participant Head as PR head commit
  CI->>Validator: run "Require changed test files to stay within size budget" step
  Validator->>Budget: download budget at PR base SHA (fallback if missing)
  Validator->>Head: verify budget file retrievable at PR head if budget changed
  Validator->>GHAPI: fetch raw contents of each changed test file from PR head
  GHAPI-->>Validator: file contents
  Validator->>Validator: count lines, compare to default/legacy limits, check monotonicity
  Validator->>CI: report violations (fail) or PASS summary
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

area: ci, chore

Suggested reviewers

  • prekshivyas
  • cjagwani
  • ericksoa

Poem

🐰 I hop through diffs with linenoise small,
Counting tests so they don't sprawl,
I tuck old giants under legacy's wing,
Seed inference.local so curl can sing,
CI hums softly — the garden stays neat for all.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.75% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'ci: guard oversized test files' directly matches the main objective of the PR, which is to add a test-file size budget mechanism to prevent test files from growing unchecked. It is concise, specific, and clearly communicates the primary change.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/test-size-guard

Comment @coderabbitai help to get the list of available commands and usage tips.

@cv cv mentioned this pull request Jun 7, 2026
2 tasks
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • None. No E2E is recommended. The PR is limited to CI guardrails, pre-commit/static-check wiring, a test-size budget utility, and tests for that tooling. It does not change runtime code or assets that can affect installer/onboarding, sandbox lifecycle, credentials, security boundaries inside NemoClaw/OpenShell, network policy, inference routing, deployment, or real assistant user flows.

Optional E2E

  • None.

New E2E recommendations

  • None.

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

  • None. No changed files affect the scenario E2E workflows, scenario catalog metadata, expected-state contracts, validation suite catalog, scenario runtime, or scenario-specific suite/onboarding scripts under test/e2e-scenario/. The PR changes general CI guardrails, pre-commit configuration, package scripts, and unit tests outside the scenario E2E surface.

Optional scenario E2E

  • None.

Relevant changed files

  • None.

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 0 needs attention, 1 worth checking, 0 nice ideas
Top item: PR review advisor unavailable

Review findings

🛠️ Needs attention

  • None.

🔎 Worth checking

  • PR review advisor unavailable: The automated advisor could not complete: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt
    • Recommendation: Re-run the PR Review Advisor or perform a manual review.
    • Evidence: Could not parse JSON from PR review advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/pr-review-advisor/pr-review-advisor-raw-output.txt

🌱 Nice ideas

  • None.
Consider writing more tests for
  • **Mocked behavioral coverage** — Add or confirm behavioral tests with mocked filesystem/network/process boundaries.. Changed code has I/O, state, credentials, provider, or config behavior that should be covered with behavioral mocks: .pre-commit-config.yaml.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
ci/test-file-size-budget.json (1)

1-18: ⚠️ Potential issue | 🟠 Major

Align SPDX handling with strict JSON parsing for ci/test-file-size-budget.json

  • ci/test-file-size-budget.json is strict JSON consumed via JSON.parse in both scripts/check-test-file-size-budget.ts and .github/workflows/codebase-growth-guardrails.yaml, so adding comment-style SPDX headers (///#) would break parsing.
  • Current repo SPDX hook spdx-headers in .pre-commit-config.yaml does not run on .json files (only *.ts/*.py/*.sh), so this file’s missing SPDX header is not enforced; if JSON SPDX is still required by policy, update the checker to explicitly exempt this file or introduce a JSON-safe SPDX approach.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci/test-file-size-budget.json` around lines 1 - 18,
ci/test-file-size-budget.json is parsed with JSON.parse so comment-style SPDX
headers would break consumers; update the repo checks to avoid adding/removing
comment SPDX for this file by either (A) adding an explicit exclusion for
"ci/test-file-size-budget.json" in the spdx-headers hook configuration in
.pre-commit-config.yaml, or (B) altering the SPDX enforcement/checker to
recognize a JSON-safe SPDX mechanism (e.g., a top-level "spdx" property) or to
skip JSON files; ensure corresponding consumers in
scripts/check-test-file-size-budget.ts and the
.github/workflows/codebase-growth-guardrails.yaml continue to parse the file
with JSON.parse without expecting comment headers.

Source: Coding guidelines

.pre-commit-config.yaml (1)

1-3: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add SPDX header to this YAML config file.

Lines 1-3: this changed YAML file is missing the required SPDX copyright/license header.
As per coding guidelines, "**/*.{js,ts,tsx,jsx,sh,md,mdx,json,yaml,yml,css,svg} files must include SPDX headers."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.pre-commit-config.yaml around lines 1 - 3, This YAML config file (beginning
with the comment "# NemoClaw — prek hook configuration") is missing the required
SPDX header; add the SPDX header lines at the very top of
.pre-commit-config.yaml (e.g., an SPDX copyright text and an
SPDX-License-Identifier) so the file includes both SPDX-FileCopyrightText and
SPDX-License-Identifier entries before the existing comments.

Source: Coding guidelines

package.json (1)

1-1: ⚠️ Potential issue | 🟠 Major

SPDX headers are missing from package.json; apply the repo’s JSON $comment pattern or document an exception.

  • Root package.json contains no SPDX-FileCopyrightText / SPDX-License-Identifier header (and no JSON-safe $comment SPDX block), so the repo SPDX guideline for JSON files is unmet.
  • The repo already carries SPDX inside JSON via $comment in schemas/router-pool-config.schema.json and nemoclaw-blueprint/model-specific-setup/schema.json.
  • Add a top-level $comment to package.json with the required SPDX strings (keeps npm parsing intact), or make an explicit documented exception for package.json.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@package.json` at line 1, Add the SPDX metadata to package.json by inserting a
top-level JSON-safe $comment field containing the SPDX-FileCopyrightText and
SPDX-License-Identifier strings (matching the repo pattern used in
schemas/router-pool-config.schema.json and
nemoclaw-blueprint/model-specific-setup/schema.json); update package.json's root
object to include "$comment": "<SPDX-FileCopyrightText: ...>
<SPDX-License-Identifier: ...>" or, if you opt not to modify package.json, add a
short documented exception in the repo policy explaining why package.json is
exempt and reference that exception in the repo README or CONTRIBUTING files.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/codebase-growth-guardrails.yaml:
- Around line 123-124: The workflow's jq filter currently matches test/spec
files anywhere; update the jq selection used in the line containing --jq '.[] |
select((.status != "removed") and (.filename |
test("\\.(test|spec)\\.(ts|js|mts|mjs|cts|cjs)$"))) | [.filename, .raw_url] |
`@tsv`' to restrict matches to the same roots as the canonical checker
(scripts/check-test-file-size-budget.ts): only files with paths starting with
test/, src/, or nemoclaw/src/ and ending with .test.|.spec. + the existing
extensions; keep the status != "removed" check and output format intact but
change the regex to require the path prefix (e.g. ^(test|src|nemoclaw/src)/… )
so both enforcement layers target the same files.
- Around line 131-134: The gh api call that fetches
ci/test-file-size-budget.json using BASE_SHA can exit non-zero (404) and fail
the step; update the step around the gh api
"/repos/${REPO}/contents/ci/test-file-size-budget.json?ref=${BASE_SHA}" call so
it does not hard-fail: run the gh api and capture its exit status (or use a
conditional fallback), write decoded content to "$budget_file" only if the call
succeeded and produced content, otherwise create a safe default JSON into
"$budget_file" (or log and continue); ensure the logic references the same
BASE_SHA, gh api invocation and budget_file variable so missing-file cases use
the default instead of failing the workflow.

---

Outside diff comments:
In @.pre-commit-config.yaml:
- Around line 1-3: This YAML config file (beginning with the comment "# NemoClaw
— prek hook configuration") is missing the required SPDX header; add the SPDX
header lines at the very top of .pre-commit-config.yaml (e.g., an SPDX copyright
text and an SPDX-License-Identifier) so the file includes both
SPDX-FileCopyrightText and SPDX-License-Identifier entries before the existing
comments.

In `@ci/test-file-size-budget.json`:
- Around line 1-18: ci/test-file-size-budget.json is parsed with JSON.parse so
comment-style SPDX headers would break consumers; update the repo checks to
avoid adding/removing comment SPDX for this file by either (A) adding an
explicit exclusion for "ci/test-file-size-budget.json" in the spdx-headers hook
configuration in .pre-commit-config.yaml, or (B) altering the SPDX
enforcement/checker to recognize a JSON-safe SPDX mechanism (e.g., a top-level
"spdx" property) or to skip JSON files; ensure corresponding consumers in
scripts/check-test-file-size-budget.ts and the
.github/workflows/codebase-growth-guardrails.yaml continue to parse the file
with JSON.parse without expecting comment headers.

In `@package.json`:
- Line 1: Add the SPDX metadata to package.json by inserting a top-level
JSON-safe $comment field containing the SPDX-FileCopyrightText and
SPDX-License-Identifier strings (matching the repo pattern used in
schemas/router-pool-config.schema.json and
nemoclaw-blueprint/model-specific-setup/schema.json); update package.json's root
object to include "$comment": "<SPDX-FileCopyrightText: ...>
<SPDX-License-Identifier: ...>" or, if you opt not to modify package.json, add a
short documented exception in the repo policy explaining why package.json is
exempt and reference that exception in the repo README or CONTRIBUTING files.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4091819e-3dab-4763-bb6f-bd91b33caa97

📥 Commits

Reviewing files that changed from the base of the PR and between 6b05a85 and 028d5d3.

📒 Files selected for processing (7)
  • .github/workflows/codebase-growth-guardrails.yaml
  • .github/workflows/pr.yaml
  • .pre-commit-config.yaml
  • ci/test-file-size-budget.json
  • package.json
  • scripts/check-test-file-size-budget.ts
  • test/test-file-size-budget.test.ts

Comment thread .github/workflows/codebase-growth-guardrails.yaml Outdated
Comment thread .github/workflows/codebase-growth-guardrails.yaml Outdated
laitingsheng and others added 2 commits June 6, 2026 21:34
## Summary

Follow-up to PR #4331 (`fix(onboard): forward host proxy env into
sandbox`). #4331 started forwarding the host `HTTP_PROXY` /
`HTTPS_PROXY` / `NO_PROXY` into `openshell sandbox create -- env ...` so
OpenShell-internal traffic could traverse the host proxy when one is
configured. The forwarded `NO_PROXY` seed list did not yet include the
OpenShell-managed inference hostname (`inference.local`) or the rootless
container-host alias (`host.containers.internal`). When a host
`HTTP_PROXY` is active on macOS + Colima, OpenShell's L7 proxy chains
`inference.local` through the host proxy — which cannot reach an
OpenShell-internal hostname — and streaming chat completions stall until
the 120 s idle timeout fires (#4846).

This PR seeds `inference.local` and `host.containers.internal` into the
`withLocalNoProxy()` augmentation so the regression introduced by
#4331's host-proxy forwarding stops short of the managed inference
hostname.

## Related Issue
Fixes #4846 (regression source: #4331).

## Changes

- `src/lib/subprocess-env.ts` and `nemoclaw/src/lib/subprocess-env.ts`
(mirrored): append `inference.local` and `host.containers.internal` to
the seeded `NO_PROXY` host list in `withLocalNoProxy()`; JSDoc documents
the boundary (host-side subprocesses + `openshell sandbox create -- env
...`) and the removal condition.
- `src/lib/onboard/http-proxy-preflight.ts`: warning copy now names the
managed inference hostname and includes it in the suggested `export
NO_PROXY=` line for tools running outside the sandbox. Suppression now
also requires `inference.local` so users with the previous loopback-only
guidance still see the new export advice.
- `docs/reference/troubleshooting.mdx`: document the full set of host
names NemoClaw adds to `NO_PROXY` when a host HTTP proxy is detected.
- Regression tests in `src/lib/subprocess-env.test.ts` and
`src/lib/onboard/http-proxy-preflight.test.ts`, plus
`test/host-proxy-inference-local-e2e.test.ts` (curl-driven E2E proving
`inference.local` reaches a local listener directly when `HTTP_PROXY` is
set). `subprocess-env.test.ts` also adds negative tests proving
arbitrary `*.local` / `.local` hostnames are not added to the bypass and
that a caller-provided `.local` entry does not expand the bypass
surface. CLI / plugin sync test in `test/credential-exposure.test.ts`
updated to match the new host list.

## Scope and boundaries

- **`*.local` suffix.** Deliberately narrow: only exact
`inference.local` is seeded. The reported failure path is the single
OpenShell-managed inference hostname; widening to any `*.local` would
change the bypass surface beyond the reported repro without evidence of
other affected hostnames. A negative test (`subprocess-env.test.ts`)
pins this behaviour.
- **Sandbox-runtime `NO_PROXY` is unchanged.**
`scripts/nemoclaw-start.sh` continues to set
`NO_PROXY=localhost,127.0.0.1,::1,${PROXY_HOST}` inside the sandbox and
intentionally **does not** include `inference.local` there — OpenShell's
L7 proxy resolves that hostname internally, and bypassing it would force
a direct DNS lookup that does not resolve from inside the container.
This PR does not touch that runtime layer.
- **Fix surface is host-side env propagation.** `withLocalNoProxy()`
runs through `appendHostProxyEnvArgs()` into `openshell sandbox create
-- env ...`. The `NO_PROXY` entry is consumed by OpenShell at
sandbox-create time when it decides whether to chain its L7 proxy
through the host `HTTP_PROXY` for a given hostname. With
`inference.local` in `NO_PROXY` at sandbox-create time, OpenShell stops
tunneling that hostname through the host proxy (the reported `Proxy
connection error: Broken pipe (os error 32)` path).
- **E2E coverage.** `test/host-proxy-inference-local-e2e.test.ts`
exercises the env-construction primitive end-to-end via a real `curl`
spawn driven by `buildSubprocessEnv()` against a local listener bound to
`inference.local`. The full macOS + Colima sandbox path requires a macOS
runner and is left to scenario E2E (`ubuntu-repo-cloud-openclaw` covers
the closest cloud-onboard cross-section).

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [x] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Verification
- [x] `npx prek run --all-files` passes
- [x] `npm test` passes
- [x] Tests added or updated for new or changed behavior
- [x] No secrets, API keys, or credentials committed
- [x] Docs updated for user-facing behavior changes
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Proxy bypass now also excludes additional local hostnames
(inference.local, host.containers.internal, ::1, 0.0.0.0), preventing
local services and managed inference from being routed through user
proxies; preflight warning now requires inference.local to be present to
suppress warnings.

* **Documentation**
* Updated proxy-preflight guidance and warning text to show suggested
NO_PROXY/no_proxy exports including managed inference hosts and extra
local aliases.

* **Tests**
* Added/updated unit and e2e tests validating the expanded
NO_PROXY/no_proxy behavior and direct reachability of inference.local.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>

---------

Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
Co-authored-by: Carlos Villela <cvillela@nvidia.com>
@cv

cv commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator Author

Feedback fixes pushed in 9c304c9.

  • Hardened the trusted pull_request_target test-size guard to read PR-head files through the GitHub contents API using HEAD_REPO/HEAD_SHA, avoiding token-bearing requests to raw_url.
  • Added a base/head budget comparison so ci/test-file-size-budget.json cannot raise defaultMaxLines, raise legacy budgets, remove still-needed legacy entries, or add oversized new legacy allowances without failing the trusted guard.
  • Scoped the trusted changed-test filter to the canonical roots: test/, src/, and nemoclaw/src/.
  • Added a safe conservative fallback for rollout/backport bases that do not yet have ci/test-file-size-budget.json.
  • Updated test/pr-workflow-contract.test.ts to assert both the skipped pre-commit hook and the separate npm run test-size:check static step.
  • Added JSON/YAML-safe SPDX metadata for the files called out by CodeRabbit.

Local validation passed: npx prek run --files ... --skip test-cli --skip test-plugin, npx vitest run --project cli test/pr-workflow-contract.test.ts test/test-file-size-budget.test.ts, npm run test-size:check, npm run typecheck:cli, npm run source-shape:check, and git diff --check.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/codebase-growth-guardrails.yaml:
- Around line 244-254: The loop over headBudget.legacyMaxLines currently ignores
fetchHeadTextFile(file) returning null, so stale legacy entries slip through;
update the code in the loop (referencing headBudget.legacyMaxLines,
baseBudget.legacyMaxLines, fetchHeadTextFile, and violations) to push a
violation when fetchHeadTextFile(file) === null for a legacy entry that no
longer exists in the head (the same "stale-legacy-budget" semantics used by
scripts/check-test-file-size-budget.ts), and only run the countLines check when
text !== null so you don't call countLines on a missing file.
- Around line 127-145: The script currently records only .filename into
"$changed_files_file" and later greps for ci/test-file-size-budget.json purely
against filenames, so renames (which populate .previous_filename) are missed;
update the initial gh api call that writes to "$changed_files_file" to also
include .previous_filename (e.g., output TSV with status, filename,
previous_filename) and then change the grep/check logic that looks for the
budget file to consider either the current filename or previous_filename (so the
conditional that uses grep -q matches when either field equals
"ci/test-file-size-budget.json"); reference the existing variables
changed_files_file and the grep check when making these edits.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 056e75e3-6cbb-4ac1-b3d7-9cf2a62c3363

📥 Commits

Reviewing files that changed from the base of the PR and between 028d5d3 and 9c304c9.

📒 Files selected for processing (5)
  • .github/workflows/codebase-growth-guardrails.yaml
  • .pre-commit-config.yaml
  • ci/test-file-size-budget.json
  • package.json
  • test/pr-workflow-contract.test.ts
✅ Files skipped from review due to trivial changes (1)
  • ci/test-file-size-budget.json
🚧 Files skipped from review as they are similar to previous changes (2)
  • .pre-commit-config.yaml
  • package.json

Comment thread .github/workflows/codebase-growth-guardrails.yaml
Comment thread .github/workflows/codebase-growth-guardrails.yaml
@cv

cv commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up advisor fix pushed in 20bf994.

  • Closed the budget-file rename-away bypass by recording previous_filename from the pull-files API and treating either the current or previous path ci/test-file-size-budget.json as a budget policy change.
  • Kept the workflow fail-closed: if the budget path was changed, the PR head must still contain a parseable budget at the canonical path before base/head validation runs.
  • Matched local stale-legacy behavior in the trusted guard: a legacyMaxLines entry whose PR-head test file is missing now fails instead of being silently skipped.
  • Added workflow-contract coverage for the trusted boundary: no .raw_url, head repo/SHA contents API usage, rename detection, and stale legacy rejection.

Validation: focused prek file checks, npx vitest run --project cli test/pr-workflow-contract.test.ts test/test-file-size-budget.test.ts, npm run test-size:check, npm run typecheck:cli, git diff --check, and commit hooks all passed.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/pr-workflow-contract.test.ts (1)

154-156: ⚡ Quick win

Assert HEAD_SHA in the trusted-guard contract too.

Line 154 currently verifies HEAD_REPO but not HEAD_SHA. Add a HEAD_SHA assertion so this test catches regressions where the guard stops pinning content fetches to the PR head commit.

Suggested patch
     expect(guardRun).toContain("HEAD_REPO");
+    expect(guardRun).toContain("HEAD_SHA");
     expect(guardRun).not.toContain(".raw_url");

Based on learnings from the PR objectives/comments summary: the trusted-boundary contract is intended to enforce contents API usage with both head repo and head SHA.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/pr-workflow-contract.test.ts` around lines 154 - 156, Add an assertion
that the trusted-guard contract includes HEAD_SHA: update the test in
pr-workflow-contract.test.ts where guardRun is checked (the block with
expect(guardRun).toContain("HEAD_REPO") and
expect(guardRun).not.toContain(".raw_url")) to also assert
expect(guardRun).toContain("HEAD_SHA"); this will ensure the guard pins content
fetches to the PR head commit (use the same guardRun variable and assertion
style as the existing HEAD_REPO check).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/pr-workflow-contract.test.ts`:
- Around line 154-156: Add an assertion that the trusted-guard contract includes
HEAD_SHA: update the test in pr-workflow-contract.test.ts where guardRun is
checked (the block with expect(guardRun).toContain("HEAD_REPO") and
expect(guardRun).not.toContain(".raw_url")) to also assert
expect(guardRun).toContain("HEAD_SHA"); this will ensure the guard pins content
fetches to the PR head commit (use the same guardRun variable and assertion
style as the existing HEAD_REPO check).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5f1d749e-2064-4fcb-993d-90c77d09b042

📥 Commits

Reviewing files that changed from the base of the PR and between 67d820d and 20bf994.

📒 Files selected for processing (2)
  • .github/workflows/codebase-growth-guardrails.yaml
  • test/pr-workflow-contract.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/codebase-growth-guardrails.yaml

@cv cv mentioned this pull request Jun 7, 2026
12 tasks
@cv

cv commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator Author

Closing this noisy branch in favor of #4905.

#4905 keeps the same test-size guard behavior but starts from current main and trims the trusted workflow guard, reducing the GitHub review diff from 968 additions / 17 files to 520 additions / 8 files. The old branch could not be force-pushed cleanly because repository rules reject force-pushes, so replacing the PR is the cleanest review path.

@cv cv closed this Jun 7, 2026
cv added a commit that referenced this pull request Jun 7, 2026
## Summary
Adds a test-file size budget so new or touched test files cannot keep
growing into oversized catch-all suites. This is a trimmed replacement
for #4899 with a clean branch from current `main`, reducing the review
diff from ~1k lines to about 520 added lines.

## Related Issue
Related to #4892. Replaces #4899.

## Changes
- Add `scripts/check-test-file-size-budget.ts` and
`ci/test-file-size-budget.json` to enforce a 1,500-line default budget
with legacy ratchets for existing oversized tests.
- Wire `npm run test-size:check` into pre-commit and PR static checks.
- Add a compact trusted `pull_request_target` guard for changed test
files and budget-policy tampering without using PR-controlled code or
`raw_url` token fetches.
- Add focused Vitest coverage and workflow-contract assertions for the
new budget and trusted guard behavior.

## Type of Change
- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Verification
Focused local checks passed: `npx prek run --files ... --skip test-cli
--skip test-plugin`, `npx vitest run --project cli
test/pr-workflow-contract.test.ts test/test-file-size-budget.test.ts`,
`npm run test-size:check`, `npm run build:cli`, `npm run typecheck:cli`,
`npm run source-shape:check`, and `git diff --check`. Full local commit
hooks were attempted in the temporary clean worktree but the CLI
coverage hook hit/stuck on local coverage-process instability, so the
full matrix is left to CI.

- [ ] `npx prek run --all-files` passes
- [ ] `npm test` passes
- [x] Tests added or updated for new or changed behavior
- [x] No secrets, API keys, or credentials committed
- [ ] Docs updated for user-facing behavior changes
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: Carlos Villela <cvillela@nvidia.com>

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Introduced test file size budgeting system with configurable maximum
line count limits for test files.
* Legacy grandfathering support allows existing large test files to
remain within their own budgets while preventing further growth.
* Integrated automatic size validation into CI/CD workflow and
pre-commit hooks.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@wscurran wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants