Skip to content

fix(measure): cef-embedder.sh runs the fork's embedder tests on a writable copy of the CEF dist - #32084

Merged
lawrencecchen merged 2 commits into
feat-cmux-nextfrom
cp/cef-embedder-writable-dist
Nov 2, 2026
Merged

lawrencecchen merged 2 commits into
feat-cmux-nextfrom
cp/cef-embedder-writable-dist

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The fleet's CEF embedder step (scripts/measure/cef-embedder.sh) ran no case against a seeded CEF build. On 2026-11-02 steps 6d191ccbefe7ed7c11e80bad and 100ec79aad999f8be34bba21 (aws-m4pro-8, cef-154.0.34-cmux.42 arm64) printed cmux-embedder: 0/14 passed with no ==> case line, then the worker's cleanup failed with permission denied on the copied framework's locale.pak.

Cause: the worker seeds $CMUX_CI_CEF_DIR read-only. The fork's test-cmux-embedder.sh copies the framework into its test app and re-signs that copy, with stderr sent to /dev/null. So codesign failed on the read-only copy and the runner stopped under set -e before any case.

This blocks the CEF crash release cef-154.0.34-cmux.42. Its embedder case parent-dealloc-hidden is the test for the CmuxParentViewTracker abort (STATUS item 6061).

Change:

  • The runner runs on a writable copy of the dist (cp -R, then chmod -R u+w).
  • On macOS the wrapper signs the copy first, as the runner does, with stderr shown. A failure then names its cause.
  • A runner that stops before its first case is named in the summary.
  • The work dir is made writable on exit, so the worker's step cleanup can remove it.
  • Test seam: CMUX_EMBEDDER_INPUT_DIR takes the test-source archive from a local dir.

Testing

bash scripts/measure/tests/cef-embedder.test.sh uses a read-only dist and a fake runner that needs a writable framework copy, as the real one does.

  • Red at ed2acc9 (test only): cmux-embedder: 0/2 passed, then FAIL: a run on a read-only dist ran no case.
  • Green at 4d0b46e: cef-embedder: ok, 5 of 5 runs.

Observed / Not run

  • Observed: the red and the 5 greens above, on cmux-orchestrator-1 (Linux). The prepush gate found no new violations.
  • Not run: the macOS codesign branch is not covered by the test (no codesign on Linux). There is no fleet run of the fixed script yet: the controller runs this measurement script only at a commit on feat-cmux-next, so the real embedder runs (red on cmux.41, green on cmux.42) come after this lands.

Changelog

none


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Fixes the CEF embedder measurement script so it runs the fork's embedder tests against the seeded dist instead of stopping silently before any case.

  • Runs the runner on a writable copy of the dist (cp -R, chmod -R u+w), since the seeded $CMUX_CI_CEF_DIR is read-only and the runner re-signs its framework copy with stderr hidden, failing under set -e before any case ran.
  • On macOS, signs the framework copy first with stderr shown so a codesign failure names its cause.
  • Names a runner that stops before its first case in the summary instead of only printing 0/N passed.
  • Makes the work dir writable on exit so the worker's step cleanup can remove it (it previously failed with permission denied on locale.pak).
  • Adds CMUX_EMBEDDER_INPUT_DIR as a test seam to source the test archive from a local directory.

Adds scripts/measure/tests/cef-embedder.test.sh, which reproduces the read-only dist with a fake runner and covers both a full run and an early stop. The macOS codesign branch is not covered by the test (no codesign on Linux).

Written for commit 4d0b46e. Summary will update on new commits.

View guided diff Turn on auto-fix

… (red)

Fleet steps 6d191ccbefe7ed7c11e80bad and 100ec79aad999f8be34bba21 (2026-11-02,
aws-m4pro-8, cmux.42 arm64): 'cmux-embedder: 0/14 passed' with no '==> case'
line, then the worker's cleanup failed with 'permission denied' on the copied
framework's locale.pak. The fleet seeds $CMUX_CI_CEF_DIR read-only; the fork's
test-cmux-embedder.sh copies its framework into the test app and re-signs the
copy with stderr sent to /dev/null, so it stopped under set -e before any case.

scripts/measure/tests/cef-embedder.test.sh: a read-only dist and a fake runner
that needs a writable framework copy, as the real one does. Red: 0/2 passed.
CMUX_EMBEDDER_INPUT_DIR (test seam) takes the test-source archive from a local
dir instead of the controller store.
…py of the CEF dist and shows a codesign failure

- Copy $CMUX_CI_CEF_DIR to $WORK/cef (cp -R, chmod -R u+w): the fork's
  test-cmux-embedder.sh copies the framework into its test app and re-signs the
  copy, which failed on the fleet's read-only seed with its stderr sent to
  /dev/null (0/14 passed, no case ran: steps 6d191ccb, 100ec79a).
- On macOS sign the copy here first, the way the runner does, with stderr shown,
  so a failure says why.
- A runner that stops before its first case is named in the summary.
- The work dir is made writable on exit, so the worker's step cleanup can remove
  it (it failed with 'permission denied' on locale.pak).

Red: the previous commit. scripts/measure/tests/cef-embedder.test.sh: red 0/2 at
the previous commit, green here, 5 of 5 runs on cmux-orchestrator-1 (Linux;
the codesign branch runs only on macOS and is not covered by the test).
@lawrencecchen lawrencecchen added the risk:low Risk tier (docs/ci/risk-tiers.md) label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 4d0b46e: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr32084 -f ref=cp/cef-embedder-writable-dist -f platform=macos -f backend_mode=local

Shipped in:

The feat-cmux-next push run https://github.com/manaflow-ai/cmux/actions/runs/41425819578 at f22cb91 failed: cmux-next checks (god files, concurrency, crash safety, l10n).
Those jobs last passed at e836232. This pull request is one of 5 merged since: #28950, #32044, #32084, #31945, #32102.
If the failure is in your change, fix forward on feat-cmux-next. Pull requests run only the tiers their paths reach (docs/ci/cmux-next-tiers.md); label a batch PR full-ci to run them all before merging.

@github-actions github-actions Bot added risk:normal Risk tier (docs/ci/risk-tiers.md) and removed risk:low Risk tier (docs/ci/risk-tiers.md) labels Nov 2, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (subagent, correctness first): SHIP at 4d0b46e

Checked against the fork runner scripts/test-cmux-embedder.sh (cef-154.0.34-cmux.42 checkout). The runner writes into CEF_PATH only through its cp -R of the framework into the app and the ad hoc re-sign at lines 81-83. The compile step reads CEF_PATH and writes to $WORK/obj. So a u+w copy of the dist removes the cause.

  • cp -R of a read-only tree works with both BSD and GNU cp: each one makes a directory writable while it copies into it, then sets the source mode. That is why the runner's copy was read-only and why chmod -R u+w on $WORK/cef is the right fix. Symlinks are copied as symlinks.
  • EXIT trap: the exit status stays the same (checked: a die gives 2 and a false final (( )) gives 1, with the trap's || true). The trap runs on die after line 34. The chmod before rm -rf at line 31 is correct under set -e.
  • Preflight: it uses the same codesign --force --sign - arguments as the runner. die in the while ... < <(find -print0) loop runs in the main shell, so it exits 2, and the stderr of codesign is shown.
  • The "stopped before its first case" line matches the runner's ==> case output. The survivor rule (pgrep -f "$WORK/") does not change: no process that is still running holds $WORK/cef in its argv.
  • Seam: CMUX_EMBEDDER_INPUT_DIR still goes through the sha256 check, and hex is validated as 64 hex. It is safe if it is set by mistake.
  • Test: I ran it on Linux. It failed at ed2acc9 with 0/2 passed and FAIL: a run on a read-only dist ran no case, and passed 3 of 3 times at the head. Mutations: removing line 53 (chmod -R u+w "$WORK/cef") makes it red, and removing the early-stop echo makes it red.

Nits (none of these blocks the merge):

  • scripts/measure/cef-embedder.sh:34: no test covers the EXIT trap. Removing it still passes, because line 53 already makes the runner's copies writable (test line 45). It is belt and braces only.
  • scripts/measure/cef-embedder.sh:52: cp -R "$cef" copies all of CEF_PATH, which includes the Chromium Embedded Framework.dSYM.tar.xz archive that the runner does not use. Copying only the framework, include and libcef_dll, or using cp -Rc (an APFS clone) on Darwin, would save disk and time. The current size is acceptable in CI_JOB_DIR.
  • scripts/measure/cef-embedder.sh:59: the preflight is stricter than the runner. The runner's find -exec codesign \; ignores a failed dylib signature, but the preflight stops the step. I think this is better, but it is a change in behavior.
  • Not covered on Linux: the Darwin codesign branch (the PR says this).

@lawrencecchen lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026
@lawrencecchen
lawrencecchen merged commit 19a2a23 into feat-cmux-next Nov 2, 2026
167 of 181 checks passed
@lawrencecchen
lawrencecchen deleted the cp/cef-embedder-writable-dist branch November 2, 2026 10:00

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 4d0b46e1 Deployed Nov 2, 2026 by lawrencecchen via wait for same-key tree publisher #100517
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant