Skip to content

fix(onboard): route runCaptureEx env through buildRunnerEnv to inject NO_PROXY (#2616) - #3801

Merged
cv merged 6 commits into
mainfrom
fix/2616-runCaptureEx-no-proxy
May 19, 2026
Merged

fix(onboard): route runCaptureEx env through buildRunnerEnv to inject NO_PROXY (#2616)#3801
cv merged 6 commits into
mainfrom
fix/2616-runCaptureEx-no-proxy

Conversation

@cjagwani

@cjagwani cjagwani commented May 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

The 2026-05-18 regression on #2616 reproduces because PR #2662's fix (withLocalNoProxy + buildSubprocessEnv wired into inference/ollama/proxy.ts) missed two paths that still hit localhost:11434/11435:

  1. runCaptureEx in src/lib/runner.ts:287 was using raw { ...process.env, ...extraEnv }, not buildRunnerEnv(extraEnv) like its sibling runCapture (line 234). validateOllamaModelrunCaptureExcurl http://localhost:11434/api/generate → inherits http_proxy=http://127.0.0.1:8118 with no NO_PROXY → Privoxy 500. This is the exact validation step in the reopen comment.
  2. Three spawn sites in inference/ollama/proxy.ts (streamed pull at :488, unloadOllamaModels ps + unload curls at :775 / :792) passed no env: option at all, defaulting to inheriting process.env.

Both routes now go through buildRunnerEnv / buildSubprocessEnv, which call withLocalNoProxy to inject NO_PROXY=localhost,127.0.0.1,host.docker.internal when any HTTP_PROXY is set.

Plus a preflight warning (per the reopen's reverify criteria) when the user's shell has HTTP_PROXY set without NO_PROXY=localhost,127.0.0.1 — so users see a clear hint before they trip the trap with any in-sandbox tool that respects HTTP_PROXY. Extracted to src/lib/onboard/http-proxy-preflight.ts to satisfy onboard-entrypoint-budget (onboard.ts is +1/-1 net).

Acceptance criteria mapping (from the 2026-05-18 reopen comment)

Clause Evidence Status
export http_proxy=http://127.0.0.1:8118/nemoclaw onboard → Option 7 succeeds (no Privoxy 500) runner.ts:287 swap to buildRunnerEnv(extraEnv) fixes the validation curl; regression test test/runner.test.ts #2616: runCaptureEx injects NO_PROXY=localhost,127.0.0.1 when http_proxy is set asserts the spawn env MET
Preflight mentions http_proxy if set New src/lib/onboard/http-proxy-preflight.ts, called from preflight at onboard.ts after the Docker check MET

Test plan

npm run build:cli
npx vitest run test/runner.test.ts
npx vitest run src/lib/inference/local.test.ts test/sandbox-connect-inference.test.ts test/ollama-tools-capability.test.ts test/ollama-proxy-recovery.test.ts

Results: 48/48 runner tests pass (1 new for #2616). 87/87 broader inference/Ollama tests still pass. Typecheck clean.

Notes for reviewers

  • onboard.ts net delta is +1/-1 (the helper is one require call, replacing a blank line). onboard-entrypoint-budget should pass.
  • The new module src/lib/onboard/http-proxy-preflight.ts accepts dependency injection (env, warn) so it stays unit-testable.
  • Sibling runCapture (the non-Ex variant) already uses buildRunnerEnv. This PR brings runCaptureEx in line with it. The two are now consistent.
  • Three other call sites in the codebase use the same { ...process.env, ...opts.env } anti-pattern (onboard.ts:4245, adapters/openshell/client.ts:124/152/183). Not on the Ollama validation path, deliberately out of scope for this PR. Worth a follow-up sweep.
  • No Privoxy required to verify locally — the regression test runs with mocked spawnSync and asserts the env handed to it.

Closes #2616

Summary by CodeRabbit

  • New Features

    • Added a preflight warning that detects host HTTP proxy settings which may not handle loopback and recommends NO_PROXY for localhost/127.0.0.1.
  • Bug Fixes

    • Prevented host proxy settings from affecting local service probes by sanitizing subprocess environments.
    • Redacted credentials when displaying proxy URLs.
  • Tests

    • Added regression and unit tests for proxy-bypass behavior and credential redaction.

Review Change Stack

… NO_PROXY (#2616)

PR #2662 added withLocalNoProxy() and wired buildSubprocessEnv into the
direct spawn sites in inference/ollama/proxy.ts (auth proxy, ollama pull),
but missed two paths that still hit localhost:

1. runCaptureEx in src/lib/runner.ts was still spawning with raw
   { ...process.env, ...extraEnv }. validateOllamaModel calls runCaptureEx
   to probe http://localhost:11434/api/generate, which is the exact step
   the 2026-05-18 reopen reproduces failing with "HTTP 500 Internal Privoxy
   Error" on macOS+Privoxy.

2. Three spawn sites in inference/ollama/proxy.ts (the streamed pull curl
   at :488, the two unloadOllamaModels curls at :775 and :792) passed no
   `env:` option, defaulting to inheriting process.env with the user's
   http_proxy intact.

Both routes now go through buildRunnerEnv / buildSubprocessEnv, which call
withLocalNoProxy and inject NO_PROXY=localhost,127.0.0.1,host.docker.internal
when any HTTP_PROXY is set.

Adds a preflight warning (per the reverify criteria on the reopen) when
the user's shell has HTTP_PROXY set without NO_PROXY=localhost — so users
see a clear hint before they trip the trap with any tool that respects
HTTP_PROXY. Extracted to src/lib/onboard/http-proxy-preflight.ts to stay
within the onboard-entrypoint-budget.

Acceptance (from the reopen comment):
- export http_proxy=http://127.0.0.1:8118/ → nemoclaw onboard → Option 7
  succeeds (no Privoxy 500). Covered by new regression test in
  test/runner.test.ts.
- Preflight mentions http_proxy when set. New helper module.

Closes #2616

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@coderabbitai

coderabbitai Bot commented May 19, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 79c9b093-9dcc-4f75-944b-800bc9475c28

📥 Commits

Reviewing files that changed from the base of the PR and between 238ca45 and 5533a79.

📒 Files selected for processing (1)
  • src/lib/onboard.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/onboard.ts

📝 Walkthrough

Walkthrough

Adds a proxy-preflight utility and tests, warns during onboard preflight when a host HTTP proxy misses loopback, routes runner subprocess env construction through buildRunnerEnv, and passes sanitized envs to Ollama curl subprocesses to avoid proxy interception.

Changes

HTTP Proxy Loopback Fix

Layer / File(s) Summary
HTTP proxy preflight detection utility
src/lib/onboard/http-proxy-preflight.ts, src/lib/onboard/http-proxy-preflight.test.ts
New module exports warnIfHostProxyMissesLoopback and redactProxyCredentials; unit tests verify credential redaction and warning behavior.
Onboard preflight integration
src/lib/onboard.ts
Preflight now calls warnIfHostProxyMissesLoopback() after Docker checks to warn users when a host HTTP proxy lacks loopback bypass.
Runner subprocess environment sanitization with regression test
src/lib/runner.ts, test/runner.test.ts
runCaptureEx uses buildRunnerEnv(extraEnv) for spawned subprocesses; regression test ensures loopback entries are injected into NO_PROXY/no_proxy when http_proxy is present.
Ollama proxy subprocess environment fixes
src/lib/inference/ollama/proxy.ts
pullOllamaModelViaHttp() and unloadOllamaModels() pass env: buildSubprocessEnv() to spawn/spawnSync calls so localhost daemon requests are not routed through host proxies.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

fix, observability

Suggested reviewers

  • cv

Poem

🐰 I nudged the NO_PROXY line,
Hid creds so curl won't find,
Localhost probes skip the route,
Daemons hum and logs stay mute,
Hooray — no proxy in the bind!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title clearly and specifically summarizes the main change: routing runCaptureEx environment through buildRunnerEnv to inject NO_PROXY, directly addressing the core fix for issue #2616.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/2616-runCaptureEx-no-proxy

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Comment @coderabbitai help to get the list of available commands and usage tips.

@cjagwani cjagwani added bug NV QA Bugs found by the NVIDIA QA Team UAT Issues flagged for User Acceptance Testing. provider: ollama Ollama local model provider behavior platform: macos Affects macOS, including Apple Silicon labels May 19, 2026
@cjagwani cjagwani self-assigned this May 19, 2026
@cjagwani
cjagwani requested a review from laitingsheng May 19, 2026 15:54

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/http-proxy-preflight.ts`:
- Line 24: The current warn call logs the raw proxyEnv value (warn(`Detected
proxy: ${proxyEnv}`)), which can expose credentials; update the logging to
redact credentials by parsing proxyEnv (via the URL constructor) and log only
safe parts such as protocol, hostname and port (or replace username/password
with "[REDACTED]") instead of the full string; locate the warn call that
references proxyEnv in src/lib/onboard/http-proxy-preflight.ts and replace the
raw interpolation with the redacted/protocol-host-port representation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 69219aa7-9d60-4008-a004-45ea4812682b

📥 Commits

Reviewing files that changed from the base of the PR and between c9479ea and 1d35fe5.

📒 Files selected for processing (5)
  • src/lib/inference/ollama/proxy.ts
  • src/lib/onboard.ts
  • src/lib/onboard/http-proxy-preflight.ts
  • src/lib/runner.ts
  • test/runner.test.ts

Comment thread src/lib/onboard/http-proxy-preflight.ts Outdated
@github-actions

github-actions Bot commented May 19, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: gpu-e2e, cloud-onboard-e2e, inference-routing-e2e
Optional E2E: gpu-double-onboard-e2e, ollama-proxy-e2e, onboard-inference-smoke-e2e

Dispatch hint: gpu-e2e,cloud-onboard-e2e,inference-routing-e2e

Auto-dispatched E2E: gpu-e2e, cloud-onboard-e2e, inference-routing-e2e via nightly-e2e.yaml at 5533a79306bf8e9b62ac95bbc95a2856368e831fnightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • gpu-e2e (high): Exercises the full local Ollama onboarding path on a GPU runner: install/onboard starts Ollama, pulls and validates a model via curl, starts the auth proxy, creates the sandbox, verifies proxy reachability, performs local inference, and destroys/uninstalls. This is the highest-signal existing coverage for the changed Ollama proxy and runCaptureEx environment behavior.
  • cloud-onboard-e2e (medium): Runs the real install/onboard flow with preflight checks, sandbox creation, policy setup, credential handling, and inference.local probing. The PR modifies onboard preflight behavior and user-facing proxy diagnostics, so a real onboard flow should block merge.
  • inference-routing-e2e (medium): Covers inference route validation, credential isolation, and curl/HTTP error classification through onboarding scenarios. The runner environment change affects curl probes used by inference validation and error paths.

Optional E2E

  • gpu-double-onboard-e2e (high): Adjacent Ollama coverage for re-onboard/token consistency. Useful because proxy.ts changed and local Ollama auth proxy lifecycle is involved, but the diff does not primarily change token persistence or re-onboard semantics.
  • ollama-proxy-e2e (medium): Host-side Ollama auth proxy E2E validates real Ollama, proxy auth, token persistence, restart/recovery, and container reachability without a full sandbox. It is useful adjacent coverage for proxy-related changes, though it may not directly exercise the modified proxy.ts subprocess env paths.
  • onboard-inference-smoke-e2e (medium): Regression coverage that asserts onboard does not report success until configured inference has served a real chat completion. Helpful for runner/inference validation changes but less direct than cloud-onboard and gpu-e2e.

New E2E recommendations

Dispatch hint

  • Workflow: nightly-e2e.yaml
  • jobs input: gpu-e2e,cloud-onboard-e2e,inference-routing-e2e

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26108909003
Target ref: 1d35fe5c4bc73f8fd7c4ca61cb7434566049e8db
Workflow ref: main
Requested jobs: gpu-e2e,cloud-onboard-e2e
Summary: 1 passed, 0 failed, 1 skipped

Job Result
cloud-onboard-e2e ✅ success
gpu-e2e ⏭️ skipped

CodeRabbit flagged the preflight warning on #3801: it logged the raw
HTTP_PROXY value, which leaks credentials when a user has
http_proxy=http://user:password@proxy:3128 in their shell (common in
corporate setups).

Adds redactProxyCredentials() that:
- Parses the URL and zeroes username/password if present
- Falls back to a regex over the //user:pass@ userinfo segment for
  non-URL-parseable strings

Plus a co-located test file covering: no proxy, NO_PROXY already bypasses
loopback, basic-auth redaction (user:pass), username-only redaction,
non-URL fallback, and the uppercase HTTP_PROXY variant.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cjagwani

Copy link
Copy Markdown
Collaborator Author

Addressed in 7a4bbac. redactProxyCredentials() parses the URL and zeroes username/password (falls back to a regex over the userinfo segment for non-URL-parseable strings). New co-located http-proxy-preflight.test.ts covers basic-auth, username-only, and non-URL fallback cases. 9/9 tests pass.

Comment thread src/lib/onboard/http-proxy-preflight.test.ts Fixed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/http-proxy-preflight.ts`:
- Around line 21-22: The current check using noProxyEnv and the regex treats
either "localhost" OR "127.0.0.1" as sufficient; change it to require both
loopback entries before suppressing the warning: parse noProxyEnv (the variable
noProxyEnv) into comma-separated tokens, normalize whitespace/lowercase, and
only return false when both "localhost" and "127.0.0.1" are present in the token
set (instead of the existing /(localhost|127\.0\.0\.1)/ test); otherwise keep
returning true so the warning is shown.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c158523d-72c1-42af-8a5e-95d23052fbb5

📥 Commits

Reviewing files that changed from the base of the PR and between 1d35fe5 and 7a4bbac.

📒 Files selected for processing (2)
  • src/lib/onboard/http-proxy-preflight.test.ts
  • src/lib/onboard/http-proxy-preflight.ts

Comment thread src/lib/onboard/http-proxy-preflight.ts Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26109647508
Target ref: 7a4bbac719224755ecc70eb0ee25b57d2f9d1860
Workflow ref: main
Requested jobs: cloud-onboard-e2e,gpu-e2e
Summary: 1 passed, 0 failed, 1 skipped

Job Result
cloud-onboard-e2e ✅ success
gpu-e2e ⏭️ skipped

cv and others added 3 commits May 19, 2026 16:25
…ort, function or class'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…press proxy preflight warning

HTTP libraries (curl, Node fetch, Python requests) match the literal
hostname against NO_PROXY. NO_PROXY=localhost alone still proxies
127.0.0.1 requests (and vice versa), so the preflight warning was
falsely suppressed when only one entry was present.

Switch the regex from OR to two separate checks AND'd together, and
add unit tests for both partial-NO_PROXY cases.

Addresses CodeRabbit major on #3801.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ⚠️ No requested jobs ran

Run: 26131528756
Target ref: 7b51967070fa76e9c26323a6bfe525196ab4ca26
Workflow ref: main
Requested jobs: gpu-e2e,cloud-onboard-e2e
Summary: 0 passed, 0 failed, 1 skipped

Job Result
cloud-onboard-e2e ⚠️ cancelled
gpu-e2e ⏭️ skipped

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26131631298
Target ref: 238ca4513d958b1dc44e721492c23aa844fc5788
Workflow ref: main
Requested jobs: gpu-e2e,cloud-onboard-e2e
Summary: 1 passed, 0 failed, 1 skipped

Job Result
cloud-onboard-e2e ✅ success
gpu-e2e ⏭️ skipped

@cjagwani
cjagwani requested a review from cv May 19, 2026 23:41
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26132120219
Target ref: 5533a79306bf8e9b62ac95bbc95a2856368e831f
Workflow ref: main
Requested jobs: gpu-e2e,cloud-onboard-e2e,inference-routing-e2e
Summary: 2 passed, 0 failed, 1 skipped

Job Result
cloud-onboard-e2e ✅ success
gpu-e2e ⏭️ skipped
inference-routing-e2e ✅ success

@cv
cv merged commit 2dc0942 into main May 19, 2026
28 checks passed
@wscurran wscurran added area: cli Command line interface, flags, terminal UX, or output area: local-models Local model providers, downloads, launch, or connectivity area: providers Inference provider integrations and provider behavior bug-fix PR fixes a bug or regression and removed priority: high labels Jun 3, 2026
@cv
cv deleted the fix/2616-runCaptureEx-no-proxy branch June 28, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Command line interface, flags, terminal UX, or output area: local-models Local model providers, downloads, launch, or connectivity area: providers Inference provider integrations and provider behavior bug-fix PR fixes a bug or regression NV QA Bugs found by the NVIDIA QA Team platform: macos Affects macOS, including Apple Silicon provider: ollama Ollama local model provider behavior UAT Issues flagged for User Acceptance Testing.

Projects

None yet

4 participants