Skip to content

base red: toolpack MCP launchers take their secret store from main, so toolpack_mcp passes on macOS - #28898

Merged
lawrencecchen merged 4 commits into
feat-cmux-nextfrom
fix/chief-toolpack-mcp-store-injected
Oct 31, 2026
Merged

lawrencecchen merged 4 commits into
feat-cmux-nextfrom
fix/chief-toolpack-mcp-store-injected

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Base red on macOS: enabling_a_pack_adds_its_skills_and_servers_and_disabling_removes_them (Native/OptChat/optchat-chief/tests/toolpack_mcp.rs:219) fails on every macOS optchat-chief run since 2026-10-24 with secret:acme: this Chief has no Acme token. Linux CI passes it.

Cause: the test writes the token to a FileStore and then runs the optchat-chief pack mcp binary. The binary gets its store from toolpacks::store(), which on macOS is the login keychain (b4db5e6), so the binary never sees the token that the test wrote.

Fix: the store is passed at construction, with no settable or static hook.

  • serve_mcp and mcp_headers take the store as a parameter (StoreOf), and serve_mcp also takes its input and output.
  • main is the composition root. It passes toolpacks::store of the Chief home and the process's stdin and stdout, so the binary works as before.
  • The test calls serve_mcp in-process with the FileStore that it wrote. It still checks the same things: the secret reaches the server, tools/list is filtered, and a call to a tool that is not allowed is refused.

Takes back 0434546: feat fixed the same macOS red on 2026-10-31 with a debug-build env seam (OPTCHAT_CHIEF_TEST_FILE_SECRETS in toolpacks::store, test side 6b047a9). After the merge of feat, this PR keeps its own fix (the test passes its FileStore to serve_mcp), so nothing sets the variable. e854cfd deletes the seam, so no build reads a pack secret from a file because of an environment variable.

Testing

Focused command (fleet, macOS):

cmux-ci run --class isolated --script scripts/ci/cmux-tui-rust-check.sh --ref SHA --arg optchat-chief
  • Red on the base: feat 038bf00 fails at toolpack_mcp.rs:219 5 of 5 (steps 32985ab9, 3cfdf956, 367d52bc, aa89890f, 7d878a1d on aws-m4pro-3, -4, -6 and -8), and so does 3c94857 (step 52751e3eef0eaaade0c87d33).
  • Green at the head 5ce1432: step 101a2610ff026c4b726fd613 (aws-m4pro-3): enabling_a_pack_adds_its_skills_and_servers_and_disabling_removes_them ... ok; clippy and fmt pass. The step stays red only on the two base reds outside this PR, eval_upsell.rs:37 (base red: eval_upsell expects the Plus plan that d3340c16ef9 added to the paid-plans skill #28888) and acpmux_peer_moved.rs:187 (expired certificate on every host, in followups).

Observed / Not run

Refresh (merge of feat 19807ed, head 299cd21): fleet compile e477880e8bcb8a6d13b83850 passed (CmuxNext, app, cmux-tui). The Linux PR check Native/OptChat/optchat-chief passed clippy and cargo test at this head (toolpack_mcp 2 of 2). macOS at this head: fleet step c7ffd26ee150722f5b3191f9 (aws-m4pro-7) passed enabling_a_pack_adds_its_skills_and_servers_and_disabling_removes_them, clippy and fmt; the step is red only on the base red acpmux_peer_moved.rs:187 (expired clang fixture certificate; fixes #29072, #29362).

Observed: prepush-gate.py found no new violations. The pinned rustfmt formatted the three files.
Not run: no live check of a stdio MCP pack in a running Chief.

Changelog

none

…ain passes (toolpack_mcp red on macOS)

enabling_a_pack_adds_its_skills_and_servers_and_disabling_removes_them has
been red on every macOS run since 2026-10-24: the test wrote the Acme token
to a FileStore and ran the optchat-chief binary, whose `pack mcp` took the
store from toolpacks::store(), the login keychain on macOS
("secret:acme: this Chief has no Acme token"; tests/toolpack_mcp.rs:219).

serve_mcp and mcp_headers now take the store (StoreOf) and serve_mcp its
input and output as parameters; main, the composition root, passes the
Chief home's toolpacks::store and the process's stdin and stdout, so the
binary behaves as before. The test calls serve_mcp in-process with the
FileStore it wrote. No settable or static hook.

Red before: fleet optchat-chief runs at feat 038bf00 (step
32985ab9d258616a8e4bcaaf, aws-m4pro-4) and at 3c94857 (step
52751e3eef0eaaade0c87d33).
@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Base-red check at feat 038bf00, the optchat-chief mode, 5 runs: 32985ab9d258616a8e4bcaaf (aws-m4pro-4), 3cfdf95684c34f3b87d475e5 (aws-m4pro-3), 367d52bcee8418eeaa5d6884 (aws-m4pro-8), aa89890f02fcaf5775001de9 (aws-m4pro-6) and 7d878a1d076063225ec9dbfc (aws-m4pro-4).

All 5 runs fail the same 3 tests, each on its own assertion, with 1 failed test in each failed binary. No other test fails. Clippy and fmt pass in every run.

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (claude-opus independent reviewer): SHIP at 5ce1432

Correctness:

  • Native/OptChat/optchat-chief/src/main.rs:88-101: the binary behaves as before. toolpacks::store(&home, p) is the same store the old secret_of built. Input is BufReader<Stdin> (Send and 'static; StdinLock would not be Send for the reader thread), output is Stdout behind the same Arc<Mutex<_>>, and every write is still followed by flush(). The extra BufReader above Stdin's own buffer does not change line reading.
  • Native/OptChat/optchat-chief/src/toolpacks.rs:642-707: the reader thread gets a Box<dyn BufRead + Send> (implicitly 'static), so thread::spawn bounds hold. input.lines() uses the BufRead for Box<B> impl. EOF on input drops to_child, so the child sees EOF as before. Error texts, the -32601 refusal and the exit code are unchanged.
  • toolpacks.rs:660-663: when m.tools is empty, exec runs before input or output is touched, so the binary's exec path is the same. main builds the BufReader but reads nothing before exec, so no stdin bytes are lost.
  • Callers: git grep over Native and cmux-tui finds no other caller of serve_mcp, mcp_headers, secret_of or StoreOf, and no other crate depends on optchat_chief::toolpacks.
  • No settable or static hook: StoreOf is a borrowed &dyn Fn(&Pack) -> Box<dyn Store> parameter, and main is the only place that picks toolpacks::store.
  • Native/OptChat/optchat-chief/tests/toolpack_mcp.rs:220-255: the test still proves the three things: the secret from the FileStore the test wrote reaches the server env (key={SECRET} mode=fast), tools/list is filtered to acme_read, and the acme_delete call gets an error. The tools/list id is pushed before the line goes to the child, so the filter has no race.
  • clippy/rustfmt (read only, not run): boxed_local skips trait-object boxes, serve_mcp has 6 arguments (under too_many_arguments), the removed Command/Stdio imports have no other use in the test, and no new line goes over 100 columns except the byte-string literal at tests/toolpack_mcp.rs:224, which rustfmt cannot split.

Non-blocking follow-ups:

  • tests/toolpack_mcp.rs:231: no test runs optchat-chief pack mcp or pack mcp-headers through main now. On Linux the old test did, so the argv dispatch and the main.rs:88 wiring are not tested. A Linux-only (#[cfg(not(target_os = "macos"))]) binary run of the same three lines would restore that.
  • toolpacks.rs:636-641: the doc says serve_mcp speaks over input and output, but with no tool filter it execs and replaces the calling process, so it ignores both. An in-process caller (a test) with an unfiltered server would replace the test runner. Say this in the doc, or move the exec decision to main.
  • The PR body says the fleet run at the head (step 101a2610ff026c4b726fd613) is pending. Record its result before the ready-to-land label.

Not run by this reviewer: no build or test (read only).

Conflict in optchat-chief/tests/toolpack_mcp.rs: feat fixed the same macOS
red (6b047a9, 0434546) by a debug-build env seam that the spawned
binary reads. This PR fixes it by passing the store to serve_mcp, so the
test keeps this PR's in-process call with its FileStore.
…t passes its store to serve_mcp

0434546 added OPTCHAT_CHIEF_TEST_FILE_SECRETS so the spawned launcher
read the file store on macOS. After the merge of feat, the test calls
serve_mcp in-process with its own FileStore, so nothing sets the variable.
The seam was the only env-read path to a file store in toolpacks::store.
@lawrencecchen lawrencecchen removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Oct 31, 2026
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review e854cfd: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr28898 -f ref=fix/chief-toolpack-mcp-store-injected -f platform=macos -f backend_mode=local

Shipped in:

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (claude-refresh-review): SHIP at 299cd21
reviewer-session: c2808f2f-01cc-4abf-a40b-fbbc8defe1a3
author-session: a2912dc77d91aa151
perf claim: none

Compile: fleet job e477880e8bcb8a6d13b83850 at this exact head: passed (CmuxNext package with tests, app, cmux-tui). optchat-chief is its own Cargo workspace, so that job does not cover its test targets. The app build compiles only the optchat-chief binary (Xcode "Bundle optchat-chief" phase).

Checks read (gh pr checks 28898, head 299cd21): no red check. All non-skipped checks pass. Of note:

  • Native/OptChat/optchat-chief (rust workspaces run 41216711807, job 124582776930, Linux, headSha 299cd21): cargo clippy --locked --workspace --all-targets -- -D warnings passed, and cargo test --locked --workspace passed, with toolpack_mcp.rs 2 of 2 ok (enabling_a_pack_adds_its_skills_and_servers_and_disabling_removes_them ... ok). This also proves that the test file and main.rs compile.
  • cmux-next/compile, cmux-next fleet Release compile, cmux-next fast tier (cargo), risk evidence, merge freeze, ci-status: pass.
  • No macOS run at this head. The fleet refused the macOS optchat-chief test step for this branch (Mac test lane policy until 2026-11-05). The macOS green in the PR body is for 5ce1432, which is before the two merges. The macOS-specific code path (keychain store) did not change between that run and this head.

What I read: git show --cc 4d6e3f91fb1 (the resolution takes this PR's side of toolpack_mcp.rs exactly, so --cc shows no hunk), git show e854cfd0861, git show --cc 299cd21add2 (clean), and git diff origin/feat-cmux-next...299cd21add2 (3 files).

Findings

  1. main.rs is correct. It passes |p| toolpacks::store(&home, p) (the Chief home's store: keychain on macOS, else the 0600 file) to both serve_mcp and mcp_headers. serve_mcp gets BufReader<Stdin> and Stdout, so the binary behaves as before. The extra BufReader around Stdin is harmless, and the exec path (pack with no tool filter) still inherits the real fds.
  2. The seam removal is safe. git grep OPTCHAT_CHIEF_TEST_FILE_SECRETS at the head finds nothing. On feat, only toolpacks::store and the old spawned-binary test read or set it. A release build never had the seam. Now no environment variable can switch a debug build from the keychain to the file store.
  3. The test still checks what it should: the secret from the store reaches the server env (key=SECRET mode=fast), tools/list is filtered to acme_read, and acme_delete is refused. The order is deterministic. The reader thread writes the refusal before it drops the child's stdin, and the main loop ends only at the child's stdout EOF.
  4. Rust types are correct. Box<dyn BufRead + Send> is 'static and moves into the reader thread. Arc<Mutex<Box<dyn Write + Send>>> is Send and Sync. The closure with the &Pack annotation coerces to the higher-ranked StoreOf. clippy -D warnings --all-targets at this head confirms this.
  5. Small coverage loss, not a defect: the test no longer runs the optchat-chief pack mcp CLI wiring (argument parsing and the choice of store in main). That wiring is 10 simple lines.

Before ready-to-land (description, not code): the PR body is stale after the merges. Feat already fixed this red with the seam (6b047a9, 0434546, which landed today through the safe-push train with no PR number), and this PR now deletes that seam and replaces feat's version of the test. Please add a take-back line, for example Takes back 043454696ac (cx-71g32y, safe-push train, no PR): the store is now passed to serve_mcp, so the debug-build env seam has no user, and add a note that no macOS run exists at this head. The take-back rule can hold the PR until this line is in the description.

@lawrencecchen lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Oct 31, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

macOS run at head 299cd21 (agent a2912dc77d91aa151): fleet step c7ffd26ee150722f5b3191f9 on aws-m4pro-7, scripts/ci/cmux-tui-rust-check.sh optchat-chief (cargo test --no-fail-fast, clippy, fmt).

  • The test this PR fixes passes on macOS: tests/toolpack_mcp.rs enabling_a_pack_adds_its_skills_and_servers_and_disabling_removes_them ... ok. No gate-notice: FLAKY line is in the log.
  • clippy: pass. fmt: pass.
  • The step is red on one test only: tests/acpmux_peer_moved.rs:187 moved_peer_the_kernel_signature_of_a_peer_whose_executable_is_gone_is_read_and_checked ("its CMS signature does not verify (signer status 4, certificate result -2147409654)").

Classification: base finding, not this PR. That test fails on every macOS base since 2026-10-29 (5 of 5 runs on 4 hosts, CSSMERR_TP_CERT_EXPIRED: the Xcode clang test fixture has an expired leaf). Open fixes: #29072 and #29362. This PR does not change that test or the code it checks (its diff is only main.rs, toolpacks.rs and tests/toolpack_mcp.rs). The label stays.

@lawrencecchen
lawrencecchen merged commit db6fd41 into feat-cmux-next Oct 31, 2026
203 of 205 checks passed
@lawrencecchen
lawrencecchen deleted the fix/chief-toolpack-mcp-store-injected branch October 31, 2026 16:39

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 299cd21a Deployed Oct 31, 2026 by lawrencecchen via wait for same-key tree publisher #90463
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant