test(proxy): drive the Copilot client cases through the built CLI

[murdore]

Summary

The three Copilot client cases in test/continuous-test-suite-proxy.ts imported src/cli/proxy-clients/{registry,copilot}.js directly, so a broken dist/cli/index.js could pass them. The post-apply note, the model-id default and the env-script content now come from what node dist/cli/index.js proxy start prints and writes in a throwaway HOME that holds a .copilot directory. A dist whose CLI stops doing any of those fails the cases.

Only the test suite changes: one file, no src/ change.

It sits on top of the change that rewrote Copilot's profile detection (#1905, merged) and leaves the profile-line table case that change added untouched.

Id	Thread	Result
T3885676104-a	#1593 (comment)	The Copilot positive paths run through the built CLI; the suite header no longer claims a throwaway HOME needs a launchd unit

What changed in the suite

• startProxyForTests now also points GROK_HOME at the throwaway HOME. proxy start configures every client it finds, the migrated cases now run that whole roster, and an inherited GROK_HOME would otherwise send the spawned proxy at a real Grok config directory.
• startProxyForTests takes an optional seed(home) callback that runs on the throwaway HOME before the child starts, and ProxyTestInstance gains output(), the tail of what the child printed. Existing callers are unaffected.
• startCopilotProxy(profile?) starts a proxy whose HOME holds .copilot (and a .zshrc when a profile is given) and waits for Restart Copilot CLI, which is printed right after the post-apply note. /health answers before the clients are configured, so waiting on health alone would race.
• Proxy clients: Copilot reports when its script is not sourced (name kept) spawns twice: with no profile the output must report the configured client and ask for the profile line; with the documented profile line it must report the configured client and not ask.
• Proxy clients: Copilot env script sets a model id (name kept) reads the script the built CLI wrote and checks the model id and its :- override form.
• Proxy clients: the built CLI writes a sourceable Copilot env script is new. It checks the three exports and the real http://127.0.0.1:<port>/v1 of the spawned proxy. It replaces Copilot emits a sourceable env script.
• Proxy clients: Copilot detects an absent CLI and restore removes the env script keeps the parts that stay direct calls: detect() with and without ~/.copilot, apply() writing the script, and restore() removing it.
• The three CLI cases are registered under proxy-infra because they spawn a proxy; the direct-call case stays under proxy-config.
• The header is corrected: a real server is needed, a throwaway HOME is not the obstacle (startProxyForTests sets NEUROLINK_PROXY_IGNORE_LAUNCHD=1), and the Copilot positive paths are listed as driven through the built CLI.

Every red assertion of the three old cases, and where it lives now, is mapped in the table below.

Old assertion	Now
writer reported a successful write	CLI case 1: Auto-configured Copilot CLI settings is in the output
plain success for a script no profile sources	CLI case 1, first spawn: the profile line is requested
still outstanding once sourced	CLI case 1, second spawn: the profile line is not requested
no model id; not overridable	CLI case 2
missing export (COPILOT_PROVIDER_TYPE, _BASE_URL, _API_KEY, the /v1 URL)	CLI case 3, with the real port
absent CLI not detected; present CLI detected; apply true; script exists; restore true; script gone	direct-call case, unchanged

The one behaviour that no longer exists is the closing restoreAllClients call of case 1: the throwaway HOME is deleted by stopProxyForTests, so nothing outside it is touched.

Tests

pnpm run test:proxy on the base before any edit: 183 passed, 7 skipped, 0 failed. After: 184 passed, 7 skipped (the same seven live-API cases), 0 failed. The extra pass is the old third case split in two.

There is no source fix to revert, so red proof is by mutation. A deliberately broken dist/cli was built from a mutated src/cli/proxy-clients/copilot.ts, then src/ was put back to the verified original (hash checked) without rebuilding, and each suite copy was run with only the Copilot cases selected:

Mutation in the built CLI	Old cases (import src/)	New CLI cases
set X: the post-apply note always null; the model id default unconditional; COPILOT_PROVIDER_TYPE renamed	all 4 pass	the three CLI cases each fail on the message for their own mutation; the direct-call cases pass
set Y: the sourced check always false	not run	case 1 fails on "still asked for the profile line once it was sourced"; the other four pass
none, good rebuild	all 4 pass	all 5 pass

The first row is the gap the thread described: the old cases stay green against a broken CLI. Each failure was read to confirm it is the intended assertion, and each prints a cross mark, not a skip. A one-assertion break-on-purpose in a scratch copy also prints a cross mark and exits non-zero.

Cost: case 1 (two spawns) 8 s wall including tsx start-up, case 2 4 s, case 3 3 s; all five Copilot cases run together in 6 s. The per-case bound is 180 s.

The suite was run again after the GROK_HOME line was added: 184 passed, 7 skipped.

Also run on this commit: pnpm run build, check, lint (0 errors; the warnings are in other files), check:tools-tests, check:deps, test:provider-structure, test:model-manifests.

Notes for review

• The three CLI cases skip, as every proxy-infra case does, when the suite detects a launchd-managed proxy on the machine. They spawn with NEUROLINK_PROXY_IGNORE_LAUNCHD=1 and do not need that guard, so this is an extra skip path on such a machine, not a coverage gain; it matches the neighbouring CLI: cases.
• The cases read the dist on disk, so pnpm run build:cli has to be current before test:proxy; a stale build fails them in a way that points at src.
• Assertion messages describe the discrepancy only; none quotes output or file content.
• The proxy suite has no dist-freshness guard (83 other suites call assertDistFresh). A dist older than src/ still runs here; the new cases fail only when the stale build behaves differently. Adding the guard is a separate decision, so it is not in this change.
• Case 1 relies on the base commit counting [ -f ~/.neurolink/copilot-env.sh ] && . ~/.neurolink/copilot-env.sh as a source command. It does, and the direct-call profile table already pins the same line.

Not done

• Restore on shutdown through the CLI: a proxy restores only on SIGINT or through the fail-open guard, which is racy to assert from outside. It stays a direct call.
• The sourced-profile detection table stays a direct-call case under the suite's determinism exception; moving it would cost one spawned proxy per profile text.

Could not verify

• The skip path of the three CLI cases with a launchd-managed proxy running (none was present, so they ran and passed).
• pnpm run test:providers-mocked was not run: this change touches only the proxy suite, which that job does not run.
• Linux and Windows. Only macOS was run, on a machine with a load average of 11 to 16.

Source thread: #1593 (comment)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Repository: juspay/neurolink/.coderabbit.yaml
• Review profile: CHILL
• Plan: Advanced
• Run ID: f0dde26b-e1d3-4312-8275-e3998d8260e9

📥 Commits

Reviewing files that changed from the base of the PR and between 89144ae and 9969c18.

📒 Files selected for processing (1)

• test/continuous-test-suite-proxy.ts

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

✅ Single Commit Policy - COMPLIANT

Status: Policy requirements met • 1 commit • Valid format • Ready for merge

📊 View validation details

📝 Commit Details

• Hash: 9969c18c2feb7b30d0a16c01013d7e9946ea854e
• Message: test(proxy): drive the Copilot client cases through the built CLI
• Author: Sachin Sharma

✅ Validation Results

• Single commit requirement met
• No merge commits in branch
• Semantic commit message format verified
• Ready for squash merge to release branch

---

🤖 Automated validation by NeuroLink Single Commit Enforcement

[github-actions]

🎉 This PR is included in version 12.47.2 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀