Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 139 additions & 0 deletions docs/computer-use-process-restart-e2e.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# Computer Use Process-Restart E2E

## Goal

Prove that a Computer Use observation from one live application process cannot
authorize an action after the same canonical application path is relaunched as
a new process.

The expected lifetime follows the recovered Codex boundary:

```text
canonical application path
+ current live process instance
```

Conversation or transport continuity must not extend an observation across a
real target-process restart.

## Incident And Investigation

The first real-machine harness attempts exposed several independent test
assumptions before reaching the process boundary:

1. A fresh worktree had no workspace `node_modules`, so downstream packages
could not resolve `@maka/core`. Running `npm ci` established the same
dependency layout used by CI.
2. The ignored cua-driver artifact was absent in the new worktree. The launcher
now runs `prepare:cua-driver` and then verifies the pinned artifact before
starting the fixture.
3. WKWebView AX nodes can appear late and can be mirrored with duplicate labels
and different frames. The restart gate therefore uses the native
`CUA Lab Coordinate Target`; OOP behavior remains covered by the dedicated
guarded E2E.
4. Activating the fixture made the test pass but stole the user's focus.
`open -g` avoided activation but kept the app hidden from the driver's
on-screen window set. The final fixture mode uses LaunchServices background
launch plus `unhideWithoutActivation`, `moveToActiveSpace`, and
`orderFrontRegardless` so the window is on-screen without becoming the
frontmost application.
5. Runtime tool failures are projected through model-visible text rather than a
top-level `error` property. The gate verifies both the backend typed outcome
and the model-visible `target_missing` result.

These were harness defects or environment prerequisites. None dispatched an
action before the target-process identity gate was reached.

## Real Sequence

The launcher and harness keep one cua-driver backend and one Runtime tool
instance alive across five target restarts:

```text
repeat 5 times with one backend/Runtime instance:
-> observe current PID/window
-> bind coordinate action
-> terminate current app
-> launch same canonical app path
-> require globally new host PID and WebContent PID
-> attempt old observation
-> require target_missing, no dispatch, mutation 0 -> 0
-> clear session
-> observe new PID/window
-> execute fresh coordinate action
-> if visible, require px dispatch, target 0 -> 1, decoy 0 -> 0
-> if covered by the user's window, require target_occluded and mutation 0 -> 0
-> require cua-driver generations stable and restartAttempts == 0
```

The launcher owns sleep prevention, fixture cleanup, bounded child
termination, and private temporary handshake/report files. The synthetic app
is launched with `CUA_LAB_BACKGROUND=1`. In that mode the fixture orders its
window visible without activating the application or making it the user's
frontmost app. A continuous Swift sentinel allows the user to switch among
their own applications, type, and move the pointer, but fails immediately if
the synthetic fixture ever becomes frontmost or the screen locks. Cleanup
preserves the user's current application; restoration is only an emergency
path if the fixture itself stole focus.

Physical user pointer movement is allowed and reported as observation data.
When the user's window occludes the target, the stronger non-interference
proof is that the backend emits no dispatch and both target and decoy mutation
remain zero.

## Verified Result

The July 14, 2026 five-round no-focus soak proved:

```text
restart rounds: 5
distinct host PIDs: 6
distinct WebContent PIDs: 6

old observation per round:
outcome: target_missing
native dispatch: none
target mutation: 0 -> 0

fresh observation:
background px success: 4 rounds
fail_closed_occluded: 1 round
decoy mutation: always 0 -> 0

cua-driver service:
action generation: 1 throughout
capture generation: 0 throughout
restartAttempts: 0 throughout

desktop concurrency:
fixture became frontmost: never
user pointer moved: 151.2 logical points
user app switching: allowed
```

The occluded round proved:

```text
outcome: target_occluded
native dispatch: none
target mutation: 0 -> 0
decoy mutation: 0 -> 0
```

The test command is:

```bash
npm run e2e:computer-use-process-restart
```

## Remaining Boundary

This proves ordinary process restart isolation. It does not force the operating
system to reuse the old numeric PID. PID-reuse safety still requires either:

- a native atomic process-instance identity exposed by the executor; or
- a deterministic driver/host fixture that can substitute a new process under
the same PID-shaped identity.

The current fail-closed old-window lookup is sufficient for the observed real
restart, but it is not evidence for deliberate PID reuse.
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"typecheck": "npm run typecheck --workspaces --if-present",
"test": "npm run test:scripts && npm --workspace @maka/core test && npm --workspace @maka/storage test && npm --workspace @maka/runtime test && npm --workspace @maka/computer-use test && npm --workspace @maka/headless test && npm --workspace maka-agent test && npm --workspace @maka/ui test && npm --workspace @maka/desktop test",
"test:dist": "npm run test:scripts && npm exec -w @maka/core -- node --test \"dist/**/*.test.js\" && npm exec -w @maka/storage -- node --test \"dist/**/*.test.js\" && npm exec -w @maka/runtime -- node --test \"dist/**/*.test.js\" && npm exec -w @maka/computer-use -- node --test \"dist/**/*.test.js\" && npm exec -w @maka/headless -- node ../../scripts/run-headless-tests.mjs && npm exec -w maka-agent -- node --test \"dist/**/*.test.js\" && npm exec -w @maka/ui -- node --test \"dist/**/*.test.js\" && npm --workspace @maka/desktop run test:dist",
"test:scripts": "node --test scripts/run-headless-tests.test.mjs scripts/sync-model-metadata.test.mjs scripts/cua-driver-provenance.test.mjs scripts/cu-real-e2e-contract.test.mjs",
"test:scripts": "node --test scripts/run-headless-tests.test.mjs scripts/sync-model-metadata.test.mjs scripts/cua-driver-provenance.test.mjs scripts/cu-real-e2e-contract.test.mjs scripts/cu-process-restart-e2e-contract.test.mjs",
"dev": "npm --workspace @maka/desktop run dev:hmr --",
"dev:full": "npm run build && npm --workspace @maka/desktop run start",
"build": "npm --workspace @maka/core run build && npm --workspace @maka/storage run build && npm --workspace @maka/runtime run build && npm --workspace @maka/computer-use run build && npm --workspace @maka/headless run build && npm --workspace maka-agent run build && npm --workspace @maka/ui run build && npm --workspace @maka/desktop run build",
Expand All @@ -35,7 +35,8 @@
"prepare:cua-driver": "node scripts/prepare-cua-driver.mjs",
"check:cua-driver-artifact": "node scripts/check-cua-driver-bundle.mjs",
"e2e:computer-use-real": "node scripts/cu-real-e2e-launcher.mjs",
"e2e:computer-use-concurrent": "node scripts/cu-real-e2e-launcher.mjs --concurrent-user"
"e2e:computer-use-concurrent": "node scripts/cu-real-e2e-launcher.mjs --concurrent-user",
"e2e:computer-use-process-restart": "node scripts/cu-process-restart-e2e-launcher.mjs"
},
"devDependencies": {
"@types/node": "^25.0.0",
Expand Down
61 changes: 61 additions & 0 deletions scripts/cu-process-restart-e2e-contract.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import test from 'node:test';

const [packageJson, launcher, harness] = await Promise.all([
readFile(new URL('../package.json', import.meta.url), 'utf8'),
readFile(new URL('./cu-process-restart-e2e-launcher.mjs', import.meta.url), 'utf8'),
readFile(new URL('./cu-process-restart-e2e.mjs', import.meta.url), 'utf8'),
]);

test('process restart E2E owns the real fixture lifecycle', () => {
assert.match(packageJson, /e2e:computer-use-process-restart/);
assert.match(launcher, /runFixtureScript\('stop\.sh'\)/);
assert.match(launcher, /runFixtureScript\('launch\.sh', \{/);
assert.match(launcher, /prepare:cua-driver/);
assert.doesNotMatch(launcher, /activateFixture/);
assert.match(launcher, /CUA_LAB_BACKGROUND/);
assert.match(launcher, /finalFrontmost\?\.bundleIdentifier === FIXTURE_BUNDLE_ID/);
assert.match(launcher, /waitForRestartedState\([\s\S]*currentPID,[\s\S]*request\.oldWebContentPID/);
assert.match(launcher, /waitForInitialState\(\)/);
assert.match(launcher, /pointerBefore/);
assert.match(launcher, /Concurrent user pointer displacement observed/);
assert.match(launcher, /Synthetic fixture never became frontmost/);
assert.match(launcher, /--snapshot/);
assert.match(launcher, /restart-request-\$\{round\}\.json/);
assert.match(launcher, /restart-complete-\$\{round\}\.json/);
assert.match(launcher, /SOAK_ROUNDS = 5/);
assert.match(launcher, /--deny-frontmost-bundle/);
assert.match(launcher, /--concurrent-user',[\s\S]*'0'/);
assert.match(launcher, /for \(let round = 1; round <= SOAK_ROUNDS; round \+= 1\)/);
assert.match(launcher, /caffeinate', \['-dimsu'\]/);
});

test('old observation is rejected and fresh-process actions succeed or fail occluded', () => {
assert.match(harness, /old-observation-after-restart/);
assert.match(harness, /oldRunResult\?\.error !== 'target_missing'/);
assert.match(harness, /staleAttempt\.modelText/);
assert.match(harness, /newState\.coordinate\.clickCount !== 0/);
assert.match(harness, /fresh-process-coordinate-click/);
assert.match(harness, /observeUntilElement/);
assert.match(harness, /invalidApp: no visible window matched/);
assert.match(harness, /CUA Lab Coordinate Target/);
assert.match(harness, /candidateCount/);
assert.match(harness, /freshSucceeded/);
assert.match(harness, /freshOccluded/);
assert.match(harness, /fail_closed_occluded/);
assert.match(harness, /background_dispatch_succeeded/);
assert.match(harness, /currentPID === newPID/);
assert.match(harness, /currentWebContentPID === newWebContentPID/);
assert.match(harness, /for \(let round = 1; round <= soakRounds; round \+= 1\)/);
assert.match(harness, /seenHostPIDs/);
assert.match(harness, /seenWebContentPIDs/);
assert.match(harness, /serviceState/);
});

test('restart reports are private launcher-owned temporary files', () => {
assert.match(harness, /MAKA_CU_RESTART_TEMP_DIR/);
assert.match(harness, /relative\(resolve\(tmpdir\(\)\)/);
assert.match(harness, /flag: 'wx'/);
assert.match(launcher, /rm\(temporaryDirectory, \{ recursive: true, force: true \}\)/);
});
Loading
Loading