feat: support filesystem allowWrite with Cloud Hypervisor

.github/workflows/test-cloud-hypervisor.yml

@@ -19,6 +19,7 @@ on:
       - 'src/cloud-hypervisor-runtime-backend.ts'
       - 'src/cloud-hypervisor-runtime-backend.test.ts'
       - 'src/microvm/**'
+      - 'src/filesystem-policy.ts'
       - 'src/types/runtime-options.ts'
       - 'scripts/ci/cloud-hypervisor-*.sh'
       - 'docs/cloud-hypervisor-foundation.md'

docs/awf-config-spec.md

@@ -99,9 +99,14 @@ existing writable host mount. An empty list makes all non-internal host bind
 mounts read-only.
 
 AWF-owned agent log and session-state mounts and required virtual devices remain
-writable so the sandbox can operate. `filesystem.allowWrite` currently supports
-Docker and gVisor compose runtimes; AWF rejects it with unsupported microVM
-runtimes.
+writable so the sandbox can operate. `filesystem.allowWrite` is supported by the
+Docker and gVisor compose runtimes and by the Cloud Hypervisor microVM runtime,
+where it is enforced by the host mount tree that backs each virtio-fs export
+(see [docs/cloud-hypervisor-foundation.md](./cloud-hypervisor-foundation.md#runtime-integration)).
+Cloud Hypervisor has no always-writable internal mounts, so a policy narrows
+every export it publishes, including `/tmp/gh-aw` and the guest home directory
+at `/workspace/.awf-home`. AWF rejects `filesystem.allowWrite` with the sbx
+runtime and with Docker-in-Docker agent execution.
 
 ### 4.2 Cloud Hypervisor microVM preview
 

docs/awf-config.schema.json

@@ -83,7 +83,7 @@
             "type": "string",
             "pattern": "^/(?!\\.\\.(?:/|$))(?!.*(?:/)\\.\\.(?:/|$))[^*?\\[\\]{}]*$"
           },
-          "description": "Guest-visible absolute paths that may be writable. When present, existing writable host bind mounts are made read-only outside these paths. An empty array makes all non-internal host bind mounts read-only. Paths must exist and cannot use glob patterns. This option is currently supported by Docker and gVisor compose runtimes."
+          "description": "Guest-visible absolute paths that may be writable. When present, existing writable host bind mounts are made read-only outside these paths. An empty array makes all non-internal host bind mounts read-only. Paths must exist and cannot use glob patterns. This option is supported by the Docker and gVisor compose runtimes and by the Cloud Hypervisor microVM runtime; it is rejected with the sbx runtime and with Docker-in-Docker agent execution."
         }
       }
     },

docs/cloud-hypervisor-foundation.md

@@ -164,7 +164,7 @@ Temporary microVM workspace data lives under:
 With `--keep-containers`, AWF preserves this directory, the network namespace,
 and runtime diagnostics for investigation.
 
-### Write-policy planning (inert)
+### Write-policy planning
 
 [`src/cloud-hypervisor/filesystem-write-policy.ts`](../src/cloud-hypervisor/filesystem-write-policy.ts)
 plans how a `filesystem.allowWrite` allowlist would narrow validated exports. It
@@ -190,13 +190,11 @@ escape the export source.
 
 The planner only removes write access: it never widens a read-only export and
 never introduces a host path that an existing read-write export does not
-already cover. It is pure policy planning and is not yet wired into runtime
-execution — `filesystem.allowWrite` is still rejected for the Cloud Hypervisor
-runtime by [`src/filesystem-policy.ts`](../src/filesystem-policy.ts). The host
-side of that boundary — how a `hostRootMode: 'ro'` root with writable overlays is
-actually staged and enforced — is described in
-[Host mount-tree enforcement](#host-mount-tree-enforcement) below. The two layers
-are independent: neither is wired into runtime execution yet.
+already cover. It is pure policy planning; the host side of that boundary — how
+a `hostRootMode: 'ro'` root with writable overlays is actually staged and
+enforced — is described in
+[Host mount-tree enforcement](#host-mount-tree-enforcement) below, and
+[Runtime integration](#runtime-integration) describes how the two are joined.
 
 ## Host mount-tree enforcement
 
@@ -206,7 +204,7 @@ guest-side read-only mount is not a security boundary. The only trustworthy
 boundary is the host VFS.
 
 This is the host-side counterpart to
-[Write-policy planning](#write-policy-planning-inert): the planner decides which
+[Write-policy planning](#write-policy-planning): the planner decides which
 paths stay writable, and this layer stages a host mount tree that enforces it.
 
 `VirtiofsdManager.start()` therefore accepts an optional, strongly typed
@@ -313,6 +311,52 @@ require containment inside the export, but this residual setup-time TOCTOU windo
 cannot be closed without fd-based mount APIs that the current tooling does not
 expose.
 
+## Runtime integration
+
+[`src/cloud-hypervisor/filesystem-write-enforcement.ts`](../src/cloud-hypervisor/filesystem-write-enforcement.ts)
+is the only place where the planner and the host mount tree meet. The Cloud
+Hypervisor runtime backend resolves and validates its exports, then plans the
+policy in a dedicated `filesystem-write-policy` startup stage *before* the boot
+loop, so an invalid allowlist aborts the run before virtiofsd or the guest is
+ever launched, and before any retry can re-attempt it. The resulting
+`VirtiofsdMountEnforcement` is threaded through `createManager()` into
+`CloudHypervisorManager`, which forwards it to `VirtiofsdManager.start()`.
+
+No `internalTags` are passed to the planner. Cloud Hypervisor has no analogue of
+the Docker runtime's always-writable agent-log and session-state binds: every
+export it publishes is host-visible workspace or runner state, and marking one
+internal — `tmp-gh-aw` in particular — would defeat the narrowing that a policy
+such as `allowWrite: ["/tmp/gh-aw/agent"]` exists to express.
+
+The translation is total; there is no fallback path:
+
+| Planner disposition | Guest mount mode | Host staged root | Mount plan passed to virtiofsd |
+| --- | --- | --- | --- |
+| policy absent (`undefined`) | unchanged | unchanged | none — `start()` receives no enforcement argument at all, so behaviour is byte-identical to a run without a policy |
+| unrestricted / fully writable (`hostRootMode: 'rw'`) | `rw` | unchanged | none |
+| fully read-only (`hostRootMode: 'ro'`, no overlays) | `ro` | `ro` | plan with zero overlays |
+| selectively writable (`hostRootMode: 'ro'`, overlays) | `rw` | `ro` | plan with one overlay per allowed path |
+
+A read-only export with zero overlays still gets a plan rather than falling back
+to the legacy single `mount --bind` plus `remount,ro`. The staged tree is the
+only variant that recursively remounts carried-in submounts read-only and
+verifies the result against `/proc/self/mountinfo`, so a policy-narrowed export
+is always served by the stronger path.
+
+Because the host tree is the boundary, a selectively writable export is never
+mounted read-only guest-side. The guest mode is derived from the plan, so
+`validateCloudHypervisorExports()` accepts a read-only `workspace` export only
+when a mount plan for the `workspace` tag actually exists; a read-only workspace
+that nothing enforces is still rejected. Unknown plan tags remain fail-closed
+via `assertPlansMatchExports()`, and the planner's own validation is not
+duplicated here.
+
+One consequence is worth stating plainly: the guest `HOME` is
+`/workspace/.awf-home`, inside the workspace export. A policy that narrows
+`/workspace` — including an empty `allowWrite: []` — makes the agent's home
+directory read-only. That is the policy working as specified, not an oversight;
+add the home path to `allowWrite` if the workload needs it.
+
 ## Limitations
 
 The preview rejects configurations that weaken or conflict with its boundary,
@@ -349,6 +393,10 @@ The live job runs only when explicitly enabled by workflow dispatch or the
 - direct-egress, arbitrary-TCP, DNS, and metadata denial;
 - API proxy reachability and secret non-disclosure;
 - workspace persistence;
+- `filesystem.allowWrite` enforcement — an allowed directory and file write
+  persisting to the host, sibling/parent/create/truncate/rename/delete denial
+  outside the allowlist, an empty allowlist narrowing the whole workspace, and
+  a fail-closed abort on an allowlist entry that matches no export path;
 - exit-code, timeout, and cancellation behavior;
 - device assumptions;
 - partial-start and normal cleanup;

scripts/ci/cloud-hypervisor-ci-scripts.test.ts

@@ -97,6 +97,9 @@ describe('cloud-hypervisor-live-smoke.sh', () => {
       'api-proxy-reflect',
       'workspace-live-share',
       'runtime-cache-readonly',
+      'allow-write',
+      'allow-write-none',
+      'allow-write-invalid',
       'exit-code',
       'timeout-124',
       'partial-start-cleanup',
@@ -162,6 +165,24 @@ describe('cloud-hypervisor-live-smoke.sh', () => {
     expect(source).toMatch(/COMMON=\(\n(?:.*\n)*?\s*--network-isolation\n/);
   });
 
+  it('proves filesystem.allowWrite enforcement end to end, including fail-closed', () => {
+    const source = fs.readFileSync(smokePath, 'utf-8');
+    // Selective policy: host-visible persistence of an allowed write, plus
+    // sibling/parent/create/truncate/rename/delete denial outside the list.
+    expect(source).toContain('"allowWrite": ["/workspace/allowed", "/workspace/allowed-file.txt"]');
+    expect(source).toContain('test "$(cat "$allow_workspace/allowed/created.txt")" = guest-allowed');
+    expect(source).toContain('test "$(cat "$allow_workspace/blocked/file.txt")" = host');
+    expect(source).toContain('test ! -e "$allow_workspace/created-at-root.txt"');
+    expect(source).toContain('test ! -e "$allow_workspace/renamed.txt"');
+    // A selective export stays read-write guest-side; only the zero-overlay
+    // narrowing publishes the guest mount itself read-only.
+    expect(source).toContain('grep -q " /workspace/allowed virtiofs " /proc/mounts');
+    expect(source).toContain('grep -q " /workspace virtiofs ro," /proc/mounts');
+    // Unmatched allowlist entries abort the run instead of widening it.
+    expect(source).toContain('run_case allow-write-invalid 1');
+    expect(source).toContain("grep -q 'filesystem.allowWrite'");
+  });
+
   (shellcheckAvailable() ? it : it.skip)('has no shellcheck errors', () => {
     expect(() =>
       execFileSync('shellcheck', ['--severity=error', smokePath]),

scripts/ci/cloud-hypervisor-live-smoke.sh

@@ -280,6 +280,59 @@ run_case runtime-cache-readonly 0 \
   'grep -q cache-readable "$RUNNER_TEMP/gh-aw/awf-virtiofs-ro-probe" && ! printf changed > "$RUNNER_TEMP/gh-aw/awf-virtiofs-ro-probe"'
 test "$(cat "$RUNNER_TEMP/gh-aw/awf-virtiofs-ro-probe")" = cache-readable
 
+# filesystem.allowWrite enforcement (docs/cloud-hypervisor-foundation.md
+# "Runtime integration"). The host mount tree -- not the guest mount mode --
+# is the boundary, so a selective policy publishes /workspace to the guest
+# read-write while virtiofsd only ever sees a read-only staged root with
+# narrow read-write submounts bind-mounted back in.
+allow_workspace="$RUN_ROOT/allow-write/workspace"
+mkdir -p "$allow_workspace/allowed/nested" "$allow_workspace/blocked"
+printf 'host\n' >"$allow_workspace/allowed/removable.txt"
+printf 'host\n' >"$allow_workspace/allowed-file.txt"
+printf 'host\n' >"$allow_workspace/blocked/file.txt"
+printf 'host\n' >"$allow_workspace/rename-me.txt"
+cat >"$RUN_ROOT/allow-write.json" <<'JSON'
+{
+  "filesystem": {
+    "allowWrite": ["/workspace/allowed", "/workspace/allowed-file.txt"]
+  }
+}
+JSON
+run_case allow-write 0 \
+  'printf guest-allowed > /workspace/allowed/created.txt && mkdir -p /workspace/allowed/nested/deeper && printf deep > /workspace/allowed/nested/deeper/file.txt && rm /workspace/allowed/removable.txt && printf guest-file > /workspace/allowed-file.txt && grep -q " /workspace/allowed virtiofs " /proc/mounts && grep -q host-input /workspace/input.txt && ! printf blocked > /workspace/blocked/file.txt && ! printf blocked > /workspace/created-at-root.txt && ! mkdir /workspace/blocked-dir && ! : > /workspace/input.txt && ! mv /workspace/rename-me.txt /workspace/renamed.txt && ! rm -f /workspace/blocked/file.txt' \
+  --config "$RUN_ROOT/allow-write.json"
+test "$(cat "$allow_workspace/allowed/created.txt")" = guest-allowed
+test "$(cat "$allow_workspace/allowed/nested/deeper/file.txt")" = deep
+test "$(cat "$allow_workspace/allowed-file.txt")" = guest-file
+test ! -e "$allow_workspace/allowed/removable.txt"
+test "$(cat "$allow_workspace/blocked/file.txt")" = host
+test "$(cat "$allow_workspace/input.txt")" = host-input
+test -f "$allow_workspace/rename-me.txt"
+test ! -e "$allow_workspace/renamed.txt"
+test ! -e "$allow_workspace/created-at-root.txt"
+test ! -e "$allow_workspace/blocked-dir"
+
+# An empty allowlist narrows every writable export with zero overlays: the
+# guest mount itself is published read-only on top of the read-only staged
+# host root.
+cat >"$RUN_ROOT/allow-write-none.json" <<'JSON'
+{ "filesystem": { "allowWrite": [] } }
+JSON
+run_case allow-write-none 0 \
+  'grep -q host-input /workspace/input.txt && grep -q " /workspace virtiofs ro," /proc/mounts && ! printf blocked > /workspace/input.txt && ! printf blocked > /workspace/created.txt' \
+  --config "$RUN_ROOT/allow-write-none.json"
+test "$(cat "$RUN_ROOT/allow-write-none/workspace/input.txt")" = host-input
+test ! -e "$RUN_ROOT/allow-write-none/workspace/created.txt"
+
+# Fail-closed: an allowlist entry that matches no export path must abort the
+# run before any guest boots, and must leave no staged mount residue.
+cat >"$RUN_ROOT/allow-write-invalid.json" <<'JSON'
+{ "filesystem": { "allowWrite": ["/workspace/does-not-exist"] } }
+JSON
+run_case allow-write-invalid 1 'true' \
+  --config "$RUN_ROOT/allow-write-invalid.json"
+grep -q 'filesystem.allowWrite' "$RUN_ROOT/allow-write-invalid/stderr.log"
+
 run_case exit-code 37 'exit 37'
 run_case timeout-124 124 'sleep 90' --agent-timeout 1
 

src/awf-config-schema.json

@@ -83,7 +83,7 @@
             "type": "string",
             "pattern": "^/(?!\\.\\.(?:/|$))(?!.*(?:/)\\.\\.(?:/|$))[^*?\\[\\]{}]*$"
           },
-          "description": "Guest-visible absolute paths that may be writable. When present, existing writable host bind mounts are made read-only outside these paths. An empty array makes all non-internal host bind mounts read-only. Paths must exist and cannot use glob patterns. This option is currently supported by Docker and gVisor compose runtimes."
+          "description": "Guest-visible absolute paths that may be writable. When present, existing writable host bind mounts are made read-only outside these paths. An empty array makes all non-internal host bind mounts read-only. Paths must exist and cannot use glob patterns. This option is supported by the Docker and gVisor compose runtimes and by the Cloud Hypervisor microVM runtime; it is rejected with the sbx runtime and with Docker-in-Docker agent execution."
         }
       }
     },

src/cloud-hypervisor-runtime-backend.test.ts

@@ -1,3 +1,6 @@
+import { promises as fsPromises } from 'fs';
+import * as os from 'os';
+import * as path from 'path';
 import { PassThrough } from 'stream';
 import type { WrapperConfig } from './types';
 import * as hostEligibility from './cloud-hypervisor/host-eligibility';
@@ -226,6 +229,81 @@ describe('Cloud Hypervisor runtime backend', () => {
     }
   });
 
+  it('plans filesystem.allowWrite before any boot attempt and passes it to the manager', async () => {
+    const directory = await fsPromises.realpath(
+      await fsPromises.mkdtemp(path.join(os.tmpdir(), 'ch-backend-write-')),
+    );
+    try {
+      const workspaceSource = path.join(directory, 'workspace');
+      const ghAwSource = path.join(directory, 'gh-aw');
+      await fsPromises.mkdir(path.join(ghAwSource, 'agent'), { recursive: true });
+      await fsPromises.mkdir(workspaceSource, { recursive: true });
+      const resolved = [
+        { tag: 'workspace', source: workspaceSource, target: '/workspace', mode: 'rw' as const },
+        { tag: 'tmp-gh-aw', source: ghAwSource, target: '/tmp/gh-aw', mode: 'rw' as const },
+      ];
+      const { deps } = harness({ resolveExports: jest.fn().mockResolvedValue(resolved) });
+      const backend = new CloudHypervisorRuntimeBackend(
+        config({ filesystemAllowWrite: ['/tmp/gh-aw/agent'] }),
+        deps,
+      );
+
+      await backend.start('/tmp/awf', ['github.com']);
+
+      expect(deps.createManager).toHaveBeenCalledWith(
+        expect.anything(),
+        '/tmp/awf',
+        expect.anything(),
+        [
+          { ...resolved[0], mode: 'ro' },
+          { ...resolved[1], mode: 'rw' },
+        ],
+        { uid: 1000, gid: 1000 },
+        {
+          plans: [
+            { tag: 'workspace', writableOverlays: [] },
+            {
+              tag: 'tmp-gh-aw',
+              writableOverlays: [{
+                source: path.join(ghAwSource, 'agent'),
+                destination: path.join(ghAwSource, 'agent'),
+                kind: 'directory',
+              }],
+            },
+          ],
+        },
+      );
+    } finally {
+      await fsPromises.rm(directory, { recursive: true, force: true });
+    }
+  });
+
+  it('passes no enforcement argument when filesystem.allowWrite is undefined', async () => {
+    const { deps } = harness();
+    const backend = new CloudHypervisorRuntimeBackend(config(), deps);
+
+    await backend.start('/tmp/awf', ['github.com']);
+
+    const call = (deps.createManager as jest.Mock).mock.calls[0];
+    expect(call[3]).toEqual([
+      { tag: 'workspace', source: '/workspace-host', target: '/workspace', mode: 'rw' },
+    ]);
+    expect(call[5]).toBeUndefined();
+    expect(call).toHaveLength(6);
+  });
+
+  it('fails closed on an unmatched allowlist path before creating a manager', async () => {
+    const { deps } = harness();
+    const backend = new CloudHypervisorRuntimeBackend(
+      config({ filesystemAllowWrite: ['/workspace/does-not-exist'] }),
+      deps,
+    );
+
+    await expect(backend.start('/tmp/awf', ['github.com']))
+      .rejects.toThrow('filesystem.allowWrite path is not an existing path within a writable');
+    expect(deps.createManager).not.toHaveBeenCalled();
+  });
+
   it('starts infrastructure, revalidates it, boots and probes before execution', async () => {
     const { order, manager, deps, stdin } = harness();
     const backend = new CloudHypervisorRuntimeBackend(config(), deps);