Skip to content

cmux mcp: agents open in their own workspace, Chief in the task's workspace; workspace evals - #29178

Open
lawrencecchen wants to merge 33 commits into
feat-cmux-nextfrom
feat-cmux-next-tool-evals-r17
Open

lawrencecchen wants to merge 33 commits into
feat-cmux-nextfrom
feat-cmux-next-tool-evals-r17

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Agents open their splits, files, diffs and pages in their own workspace, and Chief opens a task's things in that task's workspace (Lawrence 2026-11-02: "when there are a ton of workspaces, agents ... should open it in the same workspace ... careful with chief as well"). Also the r17 instruction changes (panes and columns are cmux objects; open names its pane). It contains #29080 until that lands.

  • Untargeted viewers go beside the caller. An MCP app_file_open, diff viewer or tab_new_browser call with no target went to the app's focused pane, which with many workspaces is often another workspace. It now goes through open's placement beside the caller, when the app places the caller in a pane (db021ca, c1ee0b7). A pathless diff names the chat's folder.
  • Chief: "workspace of the task". In a Chief turn (CMUX_CALLER_HOME_CHAT=1), the open and cmux_exec descriptions say that a request with a workspace's name in it, or about its project, agent, server or files, opens in that workspace in the background, and Chief names the workspace. Only a request with no workspace opens beside Chief. The rule is not in the instructions, which would pass 2048 characters with it (30a49d8, ec6f6b4, a82b64b, af0e5f6; the rule starts each description).
  • Evals. tests/tool-evals/cases/workspace.ndjson: 17 cases in a session of 6 workspaces where the person looks at another one (terminals, splits, files, diffs, sites, browser tabs, an asked-for other workspace, and Chief docked left). New checks stayed_in_workspace and new_panes. The stand-in app now opens an untargeted action in the focused pane and refuses unknown arguments, as the app does (it had hidden the leak).

Results (Claude CLI, MCP code mode, 2 samples per case, median per task)

Run Model Pass Leaks Tokens Seconds
Before (r17d, 14 cases) Opus 5.5 27/28 0 77k 18.6
Before (r17d, 14 cases) Sonnet 5.5 24/28 0 57k 10.8
Before (r17d, 3 Chief task cases) Opus 5.5 4/6 2 92k 27.8
Before (r17d, 3 Chief task cases) Sonnet 5.5 2/6 4 57k 17.7
After (r18e, 17 cases) Opus 5.5 34/34 0 78k 20.5
After (r18e, 17 cases) Sonnet 5.5 31/34 2 57k 12.9
After (r18f, 7 Chief cases) Opus 5.5 14/14 0 78k 14.9
After (r18g = head af0e5f6, 7 Chief cases) Opus 5.5 14/14 0 67k 16.7
After (r18f, 7 Chief cases) Sonnet 5.5 14/14 0 58k 12.8
After (r18g = head af0e5f6, 7 Chief cases) Sonnet 5.5 14/14 0 77k 13.8

Changelog

  • Changed: an agent's untargeted file, diff or page opens beside its chat in its own workspace, and Chief opens a task's things in that task's workspace.

Observed / Not run

  • Red (rbx cargo test --locked -p cmux-tui --lib -- <filter>): e7bda04 an_untargeted_viewer_from_a_chat_opens_beside_it failed; effd0f1 the Chief test failed; 8ced5fd chief_tools_open_in_the_workspace_of_the_task, an_untargeted_viewer_from_a_chat_opens_beside_it and an_untargeted_diff_names_the_chat_folder failed.
  • Red at 14b2987: chief_tools_open_in_the_workspace_of_the_task and an_untargeted_diff_names_the_chat_folder failed.
  • Green at af0e5f6 (and before at a82b64b): cli::mcp cli::beside 193 passed; --test cli caller_selector 4 passed; clippy -D warnings 0; cmux_surface classic and browser_engine passed.
  • Observed: the eval results above (stand-in app, daemons and MCP on rbx).
  • Not run: GPT 6 Astra and GPT 6.1 Sol (Codex through the subrouter fails, Canary blocked by unisolated Codex OAuth credential subrouter#640). A tagged app dogfood. The app's own placement for a plain terminal that runs cmux file open with no target (Swift, ControlRouter.placeBesideCaller) is not changed here.

…e, connected, supported" (honest answers that gave the screen text instead failed on the apostrophe form)
…pace and the last index are looked up) or just a workspace (its focused pane); the sheet names tab_move, workspace_create and terminal_input_keys (r13d: agents spent a docs step and two reads per move)
…efix or <session>: went to the daemon unresolved), a workspace alone moves to its active screen and pane (no focused pane in a background workspace), the index is past the end (the daemon clamps; a retry keeps its fingerprint), lookups run in the tab's session (review of 760ad56)
…ne_create (the daemon answered 'no current screen'); the sheet says workspace_create opens with a terminal pane a moved tab can go into (r14a: GPT 6 Astra made an empty workspace and spent 3 steps after it)
…, below, in a pane, next to logs, sticky column, other workspace, focus when asked); the stand-in app runs the app's open actions and records them; an 'opened' check grades kind, path, side and docked; case files are written on the daemon's host too. cmux mcp code mode: the sheet names app_file_open, app_browser_open_diff_viewer, app_tab_new_browser with target, and a sticky column (open-v1: no model found the file viewer)
… Tailscale SSH forwards the app socket as root 0600, so a relay of this user (ending with the MCP server) listens at the app path; every remote run before this saw no cmux app
…ed (right, left, below, above, here, a sticky column, a pane, a workspace) in one call: it splits or finds the column, docks a sticky one, and runs the app's open action into it (open-r16b: 2 to 4 steps an open, and no model found the sticky path)
…efuses any other non-word where (a model's where: {workspace} opened on the right); the sheet says how to make an existing column sticky. tool-evals: the stand-in app honors focus: true (open-r16d: both models asked to focus and the stand-in ignored it)
…chat's folder, sticky skips the agent chat's column, a refused open closes its split; stand-in app refuses as the app does

Review of round 16 found three failures the stand-in app hid: tab new-browser takes a tab
target, file open and the diff viewer refuse a relative path, and sticky could reuse the
agent chat's docked column. The stand-in now refuses the same two inputs the app refuses.
…o (by session name); a stop through a closed forward left 1,000+ daemons running on rbx
…y logs column"); open names its pane in the answer, and its description says how to open two side by side to compare

open-r16f: Sonnet read "make my logs column sticky" as a file search (2 of 2 samples, no cmux call);
Opus split, moved and closed tabs by hand for "open a.txt and b.txt to compare" (2 of 2 failed).
@lawrencecchen lawrencecchen added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Correctness review (subagent) of 373848c, and resolution

Blocking: none. The reviewer checked the string continuations and escapes, that no test pins the old wording, that pane is the target pane taken before the page→tab conversion, that success() always gives an object structuredContent, the edition-2024 borrows, and the test against the Fake backend.

Non-blocking

  • or_insert could keep an app's own pane field. Fixed in 00a36c4: insert, so the answer always names the target pane.
  • "side by side" did not match where: below, which stacks the files. Fixed in 00a36c4: the hint now says "both in view … one above the other in that column".
  • The added pane can take an answer about 45 bytes over MAX_RESULT_BYTES, because fit() runs first. Left as is: this is cosmetic, and the limit is a soft budget for the model's context.

@cursor

cursor Bot commented Oct 31, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

rbx verification of 373848c: cargo test -p cmux-tui MCP filter 158 passed, 4 passed; clippy -D warnings 0; cmux_surface classic 2 passed; browser_engine 4 passed. 00a36c4 (two-line review fix) is under test, and an open-case eval run on its build follows.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

View guided diff | Turn on auto-fix | Re-trigger cubic

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review af0e5f6: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr29178 -f ref=feat-cmux-next-tool-evals-r17 -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: red check: No literal state roots (per-build isolation S7, cx-is1kf.8) is not red on the base dfc60a4; do not merge until green; see lane-rules

@blacksmith-sh

blacksmith-sh Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Found 7 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmp1enlfd2c/summary-missing.json (live):/summary-missing.json (live): View Logs
/tmp/tmp28iut29w/summary-fail.json (live):/summary-fail.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
build_only must be a conjunctive exclusion on the helper, signing, and publication jobs
, and must not gate the unsigned app build/
build_only must be a conjunctive exclusion on the helper, signing, and publication jobs
, and must not gate the unsigned app build
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Chief cases start the MCP server with CMUX_CALLER_HOME_CHAT=1 and add three
cases where the request names a workspace's work (api tests, docs README,
infra dashboard). The cli surface no longer attaches the MCP server, and a
cli run that wrote cli.jsonl is not retried as untouched. Clearer prompts.
…lacks, as the app does

`cmux file open --pane X` sends args.pane. The app answers 'file.open has no
argument pane'; the stand-in opened the file in the focused pane and graded
it as a leak into another workspace.
- A caller the app does not place in a pane (unknown to it, or a workspace
  with no pane) keeps the app's own placement for an untargeted viewer.
- A pathless diff viewer opened beside the chat diffs the chat's folder; an
  empty or non-string path is not redirected.
- Chief's rule rides on the open and cmux_exec descriptions, and Chief's
  instructions stay within 2048 characters.
- Redirect an untargeted viewer only for a caller the app places in a pane;
  otherwise open's current would be the daemon's active pane.
- An in_progress open error carries the open's own key as idempotency_key,
  so a retry reconciles instead of opening a second split and viewer.
- A pathless diff names the chat's folder.
- Chief's rule moves from the instructions (2370 characters with it, past the
  2048 that Claude Code may keep) to the open and cmux_exec descriptions.
…rkspace's) and the argument to pass

Sonnet 5.5 as Chief made the api tests terminal in its own workspace with
the abstract rule (tool-evals chief-task-api, 0/2).
The in_progress error of open keeps the app stage's idempotency_key, as
#30752's open_splits_and_opens_in_one_call asserts; the retry key question
goes to its author. Clippy: drop the unneeded super:: qualification.
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Correctness review (subagent) of the merge resolution 0294309 and the new commits up to a4199ca, with the resolution at aec62b6:

  1. Redirect gate too wide (medium-high): fixed in c1ee0b7. An untargeted file open, diff viewer or tab new-browser call goes beside the caller only when the app places the caller in a pane (caller_scope::is_placed and a pane). A caller the app does not know, or a workspace with no pane, keeps the app's own placement. Regression: an_untargeted_viewer_from_a_chat_opens_beside_it (three unplaced callers).
  2. in_progress retry key (medium): not changed. The error of open carries the app stage's key as idempotency_key, and open_idempotency_key next to it. fix(mcp): preserve open viewer pane when the app acknowledgement is lost (SW-10) #30752's open_splits_and_opens_in_one_call asserts this. A retry with the stage key can start a second open. @teamleaderleo: should the error's idempotency_key be the open's own key? I left your contract as it is.
  3. Pathless diff picks the target pane's folder (medium): fixed in c1ee0b7. It now names the chat's folder (path: ".", which is made absolute against the chat). An empty or non-string path is not redirected. Regression: an_untargeted_diff_names_the_chat_folder.
  4. Chief instructions past 2048 characters (medium): fixed in c1ee0b7 and ec6f6b4. The Chief rule moved from the instructions to the open and cmux_exec descriptions, only in a Chief turn. Regression: chief_tools_open_in_the_workspace_of_the_task, which also checks the Chief instructions are 2048 characters or fewer.

Low findings: a non-bool focus reads as false (unchanged, same as open). An untargeted call with where, as_text or a blank browser tab is not redirected (unchanged, follow-up). Harness: the eval stand-in now refuses an argument the action schema lacks, as the app does (63a365a).

Merge resolution 0294309: the reviewer found it correct.

Verification on rbx (Linux, cmux-rust-builder-1):

  • Red: e7bda04 an_untargeted_viewer_from_a_chat_opens_beside_it failed; effd0f1 chief_instructions_open_in_the_workspace_of_the_task failed; 8ced5fd the three new regressions failed.
  • Green at aec62b6: cargo test -p cmux-tui --lib -- cli::mcp cli::beside 193 passed; --test cli caller_selector 4 passed; clippy -D warnings 0; cmux_surface classic and browser_engine passed.

…ces it there

Sonnet 5.5 as Chief opened "the infra dashboard" beside itself (tool-evals
chief-task-infra 0/2 on r18e); chief-task-api went from 0/2 to 2/2 with the
example of ec6f6b4.
@lawrencecchen lawrencecchen changed the title cmux mcp: panes and columns are cmux objects in the instructions; open names its pane and says how to compare two files cmux mcp: agents open in their own workspace, Chief in the task's workspace; workspace evals Nov 3, 2026
@lawrencecchen lawrencecchen removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 3, 2026
… open refuses keeps the action

Second review: cmux_exec's description is about 6 KB, so a client that cuts
it would drop a rule at its end. A file open with an idempotency_key of 129
to 200 characters worked as an action and open would refuse it.
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Second correctness review (subagent) of a4199ca..a82b64b, with the resolution at af0e5f6:

  1. Medium: Chief's rule at the end of the cmux_exec description (about 6 KB) could be cut by a client. Fixed in af0e5f6: the rule starts the open and cmux_exec descriptions. Regression: chief_tools_open_in_the_workspace_of_the_task asserts that the description starts with the rule.
  2. Low: a server whose allowed list has neither open nor cmux_exec gets no Chief rule. Not changed. Chief's server lists both.
  3. Low: a file open with an idempotency_key of 129 to 200 characters would be refused after the redirect. Fixed in af0e5f6: such a call keeps the action's own path. Regression: an_untargeted_diff_names_the_chat_folder (long key case).
  4. Low (test gap): path: "." is checked only in untargeted. Not changed. The reviewer confirmed that chat_path makes it absolute (open_tool.rs).

Checks the reviewer passed: both tools/list branches apply chief_rule; caller_has_pane matches caller_scope::apply; the stand-in refusal cannot reject a valid MCP call.

rbx: red at 14b2987 (chief_tools_open_in_the_workspace_of_the_task, an_untargeted_diff_names_the_chat_folder failed). Green at af0e5f6: cli::mcp cli::beside 193 passed; --test cli caller_selector 4 passed; clippy -D warnings 0; cmux_surface classic and browser_engine passed.

@lawrencecchen lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 3, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (independent-subagent): BLOCK at af0e5f6

The SHIP at 0e545f9 covered only the r17 instruction text and pane in open's answer. Everything below came after it: db021ca/c1ee0b7 (the untargeted-viewer interception), the Chief rule and the workspace evals.

Blocking: a second owner of beside-caller placement (db021ca, c1ee0b7; mcp.rs call, open_tool::untargeted)

  • The app already owns this placement. ControlRouter.placeBesideCaller (ControlCaller.swift) places openBrowser and openDiffViewer for acpmux agent and Chief callers, and it runs only when request.target == nil. Now the MCP server sends an explicit target for every caller that system.identify places in a pane, and that includes every acpmux agent chat and Chief. So the app owner is bypassed for exactly the callers it was written for, and there are two placement rules for the same actions.
  • This is a behavior regression in the real app:
    • openBrowser loses the same-kind rule (Leo 2026-10-31: a tab in the pane that already holds a browser, else a split below a lone pane, never a third column). open's right takes the geometric pane_neighbor_get pane and its shown tab, so a page can land in the terminal pane of the right column.
    • The app's besideCaller path is lost too: AgentBesidePlacement, the agent-tab marking, and newColumnBeside background placement. The app's besidePane also skips docked columns, and pane_neighbor_get does not.
  • The PR says the root gap stays open: "ControlRouter.placeBesideCaller ... is not changed here". That gap is file.open missing from besideCallerActions, and terminal-id callers not placed. So cmux file open from a terminal still leaks into the focused workspace. This is a partial fix in a second layer. Under the standing rule (no second owner, root-cause fixes, REVIEW-STANDARD), the fix belongs in placeBesideCaller: add file.open, and accept a terminal_id caller. Then the MCP interception can be removed, and the CLI and MCP share one path (cmux-shared-behavior).
  • Evidence gap: every eval ran against the stand-in app. The stand-in places an untargeted run in the focused pane for all callers, but the real app places agent and Chief callers beside the chat. So the evals measured the leak for a terminal caller only, and they do not show the real-app result for acpmux and Chief chats. No tagged app dogfood was run (the PR says so).

Not blocking

  • Focus: none of the new paths requests a focus change (focus is sent only when true; Chief's rule says "in the background"). I found no focus steal in the Rust code. I did not verify this in the real app.
  • The action contract does not apply here: this is the MCP server process, not an app action handler. The interception adds one extra system.identify round trip per untargeted viewer call, and nothing on the main thread waits for it.
  • The Chief rule (prefix on open and cmux_exec in a CMUX_CALLER_HOME_CHAT=1 turn) is prompt text only. It is gated correctly, and the test checks its position and the 2048-character limit on the instructions. OK.
  • The tests cover what they claim (the Rust unit tests, with red/green recorded in the body). The Swift side has no test, because nothing changed there.

CI on this head: no red is the PR's own

@lawrencecchen lawrencecchen removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 3, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Lead: I removed ready-to-land after the independent BLOCK at af0e5f6 (comment 6609251823). The MCP interception adds a second owner of 'beside the caller' placement next to ControlRouter.placeBesideCaller, which Lawrence's standing rule rejects, and it bypasses the app's browser rule for agent chats and Chief. The principled fix is in placeBesideCaller itself: add file open and accept terminal callers, so the CLI and MCP share one path. Then drop the MCP-side interception. The evals must run against the real app (a tagged build), not the stand-in. Whoever owns this PR: please take it from there.

This branch was previously deployed

3 inactive deployments
artifacts — af0e5f61 Deployed Nov 3, 2026 by lawrencecchen via publish tree-addressed cmux-tui #104585
sentry-dsyms — af0e5f61 Deployed Nov 3, 2026 by lawrencecchen via upload cmux-tui dSYMs to Sentry #104585
ci-routing-agent — af0e5f61 Deployed Nov 3, 2026 by lawrencecchen via wait for same-key tree publisher #104586
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants