Skip to content

ci(nightly-next): notarize the app beside the DMG Mac's download, staple it there - #27518

Closed
teamleaderleo wants to merge 3 commits into
feat-cmux-nextfrom
dep-fleet--nightly-chain-staple-on-dmg-mac
Closed

teamleaderleo wants to merge 3 commits into
feat-cmux-nextfrom
dep-fleet--nightly-chain-staple-on-dmg-mac

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

On a pool-signed cmux NEXT nightly, signer-app signed the app and then waited 2.2-3.3 min for Apple before it packed and uploaded the stapled app. Only then could signer-dmg-image start, download that app over the office link (0.5-5 min) and check it. The Mac's download and checks waited behind Apple for no reason: Apple's ticket only has to be in the app before the DMG is built.

  • signer-app now packs and uploads the signed app right after signing (same artifact name, now signed-app.tar).
  • A new release-environment Linux job, signer-app-notary, checks that artifact against signer-app's digest and has Apple notarize those exact bytes with the unchanged release-signer/notarize-app.sh.
  • At the same time signer-dmg-image downloads the same bytes, starts the signed-app and verify-signed checks on copies, then staples Apple's ticket with Apple's own stapler (scripts/ci/nightly-steps/signer-staple-app.sh: stapler staple until Apple publishes the ticket for the app's CDHash, at most 25 min, then stapler validate), runs Apple's checks of the stapled app (signer-apple-checks.sh: codesign, stapler validate, spctl Notarized Developer ID, syspolicy_check, the copied helper) and builds the DMG from it as before.
  • signer-dmg needs signer-app-notary, so an app Apple rejected never reaches a signed DMG. The DMG is still signed, notarized and stapled on its own, and verify-final-dmg-nightly still requires the stapled app inside it.

What Apple notarizes is the exact signed bytes the Mac staples (one tar, one digest, checked on both sides). The Mac job holds no secret: stapling needs none. Nothing changes in build-nightly-app, prepare, publish or the macOS fallback signer.

Two smaller commits keep nightly.yml under tests/test_workflow_file_size.py's 490000-byte guard: the signed-app digest check and unpack moves into scripts/ci/nightly-steps/signer-unpack-app.sh, one step that both jobs share, and verify-final-dmg-nightly's inline "Verify final DMG artifact" block moves into scripts/ci/nightly-steps/verify-final-dmg-artifact.sh. That script runs from the workflow commit's tooling checkout and still calls the build tree's verify-dmg-artifact.sh. nightly.yml goes from 490342 bytes on base (already over the guard) to 489754.

On "notarize once": notarizing only the DMG would leave the app inside it without a stapled ticket. Today's checks require one (signer-apple-checks.sh's stapler validate and verify-dmg-artifact.sh --require-stapled-app), and it is what lets an app copied out of the DMG pass Gatekeeper offline. Dropping the app notarization would weaken those checks, so this PR keeps both notarizations and overlaps the work around the first one instead. The two Apple waits stay sequential because the app's staple changes the DMG's bytes.

Performance

Chain = build-nightly-app end to publish-nightly end. The before rows are the same five runs as #27508 (merged, f5082ea). The after column is filled from real nightly-next runs once both land.

metric before after delta harness or run link
signer-app (sign, notarize, pack, upload) 3.2-5.0 min pending (sign, pack, upload: expected about 1.2 min) expected -2 to -3.5 min before the Mac starts runs 40893555177, 40905688542, 40909458084, 40913618519, 40918131595
app notarization wait (Apple) 129-196 s unchanged, now beside the Mac's download and checks 0 same runs
signer-app end to signer-dmg-image end 3.0-5.5 min pending expected -0.5 to -2 min (download and checks overlap Apple) same runs
chain (build end to publish end) 16.5, 17.9, 17.7, 15.4, 15.9 min pending expected a further -0.5 to -2 min on top of #27508 same runs

Testing

  • python3 tests/test_nightly_signer_jobs.py: 25 tests pass, including the new wiring test (signer-app no longer notarizes; signer-app-notary checks the digest before it notarizes; signer-dmg needs it; the Mac staples before Apple's checks and the DMG; the background checks start before the staple) and SignerStapleApp (retries until the stub stapler finds the ticket, then validates; fails at the deadline without validating; fails on a staple that does not validate; refuses a missing app).
  • tests/test_nightly_step_coverage.py, tests/test_ci_signing_keychain_is_deleted_before_bundle_code.py, tests/test_supplychain_security.py, tests/test_nightly_next_recovery.py, tests/test_ci_nightly_off_blacksmith.py, tests/test_ci_runs_on_fallback.py, tests/test_ci_runs_on_values.py, tests/test_runs_on_labels.py pass. tests/test_ci_upload_artifact_retention.py fails only on the base's own cmux-tui.yml entry.

Observed / Not run

Observed: the tests above on Linux. Not run: Apple's stapler against a live submission (the stub models its "Record not found" exit before the ticket exists), and a real nightly-next run; the first runs after this lands fill the Performance table.

Changelog

none

🤖 Generated with Claude Code

Agent session: agent session

@teamleaderleo teamleaderleo added the risk:high Risk tier (docs/ci/risk-tiers.md) label Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 200e078: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr27518 -f ref=dep-fleet--nightly-chain-staple-on-dmg-mac -f platform=macos -f backend_mode=local

web and size checks: green

@blacksmith-sh

blacksmith-sh Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)/
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-2e0vx5 created for this PR by the cmux coordinator.

@teamleaderleo
teamleaderleo force-pushed the dep-fleet--nightly-chain-staple-on-dmg-mac branch from 9622445 to 6dbb837 Compare October 29, 2026 19:58
@cursor

cursor Bot commented Oct 29, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@teamleaderleo

teamleaderleo commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

teamleaderleo and others added 3 commits October 29, 2026 23:31
…ple it there

signer-app signed the app, then waited 2.2-3.3 min for Apple before it packed
and uploaded the stapled app, and only then could signer-dmg-image start its
0.5-5 min download over the office link and its checks.

signer-app now packs and uploads the signed app right after signing. A new
release-environment job, signer-app-notary, checks that artifact against
signer-app's digest and has Apple notarize those exact bytes
(release-signer/notarize-app.sh, unchanged). Meanwhile signer-dmg-image
downloads the same bytes, starts the signed-app and verify-signed checks on
copies, then staples Apple's ticket with Apple's stapler
(signer-staple-app.sh: retries `stapler staple` until Apple publishes the
ticket, up to 25 min, then `stapler validate`), runs Apple's checks of the
stapled app and builds the DMG from it as before. signer-dmg needs
signer-app-notary, so an app Apple rejected never reaches a signed DMG; the
DMG is still signed, notarized and stapled on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-a7306ef145c005ad24ab
…y.yml under its size guard

Moves the inline "Verify final DMG artifact" block into
scripts/ci/nightly-steps/verify-final-dmg-artifact.sh, run from the workflow commit's tooling
checkout (it still runs the build tree's verify-dmg-artifact.sh). nightly.yml: 491131 to 489642
bytes, under tests/test_workflow_file_size.py's 490000 (base was 490342).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-a7306ef145c005ad24ab
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high, next-review): SHIP at 6dbb837

No blocking findings. The changed scripts pass shell/Python syntax checks, and the artifact, notarization, stapling, verification, and fallback wiring is consistent.

@teamleaderleo
teamleaderleo force-pushed the dep-fleet--nightly-chain-staple-on-dmg-mac branch from 6dbb837 to 200e078 Compare October 29, 2026 23:38

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 6dbb837: an independent review passed this exact head (next-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high, next-review): SHIP at 200e078

No blocking findings.

Reviewed the signing dependencies, artifact digests, background checks, and stapling flow. Shell syntax checks pass. Live Apple notarization and the full workflow were not run.

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 200e078: an independent review passed this exact head (next-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Superseded by #28183, a backlog-drain carry that keeps this PR's commits and their authorship and resolves the base merge conflict. Closing this one in its favour.

teamleaderleo added a commit that referenced this pull request Oct 31, 2026
* ci(nightly-next): notarize the app beside the DMG Mac's download, staple it there

signer-app signed the app, then waited 2.2-3.3 min for Apple before it packed
and uploaded the stapled app, and only then could signer-dmg-image start its
0.5-5 min download over the office link and its checks.

signer-app now packs and uploads the signed app right after signing. A new
release-environment job, signer-app-notary, checks that artifact against
signer-app's digest and has Apple notarize those exact bytes
(release-signer/notarize-app.sh, unchanged). Meanwhile signer-dmg-image
downloads the same bytes, starts the signed-app and verify-signed checks on
copies, then staples Apple's ticket with Apple's stapler
(signer-staple-app.sh: retries `stapler staple` until Apple publishes the
ticket, up to 25 min, then `stapler validate`), runs Apple's checks of the
stapled app and builds the DMG from it as before. signer-dmg needs
signer-app-notary, so an app Apple rejected never reaches a signed DMG; the
DMG is still signed, notarized and stapled on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-a7306ef145c005ad24ab

* ci(nightly-next): one unpack script for the signed app; keep nightly.yml lean

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-a7306ef145c005ad24ab

* ci(nightly-next): verify-final's DMG check runs from a script, nightly.yml under its size guard

Moves the inline "Verify final DMG artifact" block into
scripts/ci/nightly-steps/verify-final-dmg-artifact.sh, run from the workflow commit's tooling
checkout (it still runs the build tree's verify-dmg-artifact.sh). nightly.yml: 491131 to 489642
bytes, under tests/test_workflow_file_size.py's 490000 (base was 490342).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-a7306ef145c005ad24ab

* Keep carried nightly workflow under size cap

Agent-Session: https://cmux.com/deeplink/agent?id=aid-2e040fd57fde5405f0d3

* Clean carried nightly coverage metadata

Agent-Session: https://cmux.com/deeplink/agent?id=aid-2e040fd57fde5405f0d3

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 200e0785 Deployed Oct 29, 2026 by teamleaderleo via cmux-tui tree preflight #78953
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant