Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1020,7 +1020,7 @@ jobs:
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version: '1.25.0'
cache-dependency-path: guest/firecracker-supervisor/go.mod
cache-dependency-path: guest/microvm-supervisor/go.mod

- name: Install guest build prerequisites
run: |
Expand Down
14 changes: 6 additions & 8 deletions .github/workflows/test-cloud-hypervisor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
- '.github/workflows/test-cloud-hypervisor.yml'
- 'guest/cloud-hypervisor/**'
- 'containers/build-tools/**'
- 'guest/firecracker-supervisor/**'
- 'guest/microvm-supervisor/**'
- 'src/cloud-hypervisor/**'
- 'src/cloud-hypervisor-runtime-backend.ts'
- 'src/cloud-hypervisor-runtime-backend.test.ts'
Expand Down Expand Up @@ -46,13 +46,11 @@ jobs:
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version: '1.25.0'
cache-dependency-path: guest/firecracker-supervisor/go.mod
cache-dependency-path: guest/microvm-supervisor/go.mod

- name: Run guest supervisor unit tests
working-directory: guest/firecracker-supervisor
# guest/firecracker-supervisor is shared, unmodified, between the
# Firecracker and Cloud Hypervisor backends (see build.sh above).
# Running its unit tests here (not just building it) catches
working-directory: guest/microvm-supervisor
# Running unit tests here (not just building it) catches
# defects like an incorrect syscall.Mount() fstype before they
# only surface as a guest kernel panic during the live-KVM job
# below, which is much slower to diagnose.
Expand Down Expand Up @@ -255,10 +253,10 @@ jobs:
set -euo pipefail
while read -r namespace _; do
case "$namespace" in
awffc-*) sudo ip netns delete "$namespace" ;;
awfvm-*) sudo ip netns delete "$namespace" ;;
esac
done < <(sudo ip netns list)
if sudo ip netns list | grep -q '^awffc-'; then
if sudo ip netns list | grep -q '^awfvm-'; then
echo "::error::Cloud Hypervisor namespace residue remains after cleanup"
exit 1
fi
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,4 @@ __pycache__/
# Local `go build`/`go vet` output for the guest supervisor module
# (built binary shares its directory's module name with no extension,
# easy to accidentally leave behind after a local build/test cycle)
/guest/firecracker-supervisor/firecracker-supervisor
/guest/microvm-supervisor/microvm-supervisor
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,6 @@ See [GitHub Actions](docs/github_actions.md) for advanced setup and `awf logs su
- [Diagnosing AWF failures](docs/diagnosing-awf-failures.md) — use the Self-Hosted Runner Doctor agent to triage self-hosted/ARC/GHES/GHEC failures
- [Auth Doctor Updater workflow](.github/workflows/auth-doctor-updater.md) — daily/manual audit that opens bounded PRs with evidence-backed authentication and API-proxy documentation corrections
- [Image verification](docs/image-verification.md) — cosign signature verification
- [Firecracker integration (preview)](docs/firecracker-integration.md) — Firecracker v1.16.1 microVM backend: explicit opt-in, Linux/KVM only, macOS/Windows unsupported, operator-managed artifacts with mandatory SHA-256 digests, fail-closed egress, mandatory API proxy credential isolation
- [Cloud Hypervisor integration (preview)](docs/cloud-hypervisor-foundation.md) — Cloud Hypervisor v53.0 microVM backend: explicit opt-in, GitHub-hosted Ubuntu x86_64 KVM runners only, operator-managed artifacts with mandatory SHA-256 digests, Landlock/seccomp-confined launcher in place of a jailer, fail-closed egress, mandatory API proxy credential isolation

## Development
Expand Down
56 changes: 11 additions & 45 deletions docs/INTEGRATION-TESTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,44 +193,12 @@ Each document provides per-test-case analysis with plain-language descriptions,
- **[CI & Smoke Tests](test-analysis/ci-smoke.md)** — All 27 CI/smoke/build-test workflows analyzed
- **[Test Infrastructure](test-analysis/test-infra.md)** — Runner architecture, batch pattern, cleanup strategy, limitations

## Firecracker preview integration tests

The dedicated Firecracker CI workflow is disabled. The deterministic artifact
build and live KVM smoke scripts remain available for explicit local validation,
but they are not run by pull request, push, schedule, or manual Actions events.

The live smoke/security suite verifies all five SHA-256 digests before running.
Its preflight requires usable KVM and fails closed if `/dev/kvm` or another
required host capability is unavailable.

Live assertions (see `scripts/ci/firecracker-live-smoke.sh`):

| Case | What it proves |
|------|---------------|
| `allowed-https` | Allowed domains reach the internet through Squid |
| `blocked-domain` | Non-allowlisted domains are blocked |
| `direct-egress` | Bypassing proxy env vars does not enable direct egress |
| `arbitrary-tcp` | Raw TCP to arbitrary IPs is blocked |
| `dns-denial` | Direct DNS (8.8.8.8:53) is blocked from the guest |
| `metadata-denial` | EC2/GCP/Azure instance metadata IP (`169.254.169.254`) is unreachable |
| `api-proxy-reflect` | API proxy `/reflect` reachable; secret sentinel not present in output |
| `workspace-copyback` | Guest file writes, permission changes, and symlinks survive copy-back |
| `exit-code` | Agent exit code propagates faithfully (37 → 37) |
| `timeout-124` | Timed-out agent exits 124 |
| `partial-start-cleanup` | Corrupt rootfs causes clean failure; no namespace residue |
| `cancellation` | `SIGTERM` cleans up network namespace; exits 143 |
| `keep` | `--keep-containers` preserves jail/namespace/images; all diagnostics ≤1 MiB |

After every case, the suite asserts no `awffc-*` namespaces or Firecracker
interface residue remain. See [Firecracker integration (preview)](../docs/firecracker-integration.md#part-14--ci-workflow)
for the current automation status and local validation details.

## Cloud Hypervisor preview integration tests

The Cloud Hypervisor backend has its own separate CI workflow
(`test-cloud-hypervisor.yml`), based on the retained Firecracker test
conventions but scoped to Cloud Hypervisor paths and **GitHub-hosted Ubuntu
x86_64 runners only** (self-hosted runners are explicitly rejected).
(`test-cloud-hypervisor.yml`), scoped to Cloud Hypervisor paths and
**GitHub-hosted Ubuntu x86_64 runners only**. Self-hosted runners are explicitly
rejected.

**Trigger:** `workflow_dispatch`, or pull request open/synchronize/reopen/label
scoped to `guest/cloud-hypervisor/**`, `src/cloud-hypervisor/**`,
Expand All @@ -239,8 +207,8 @@ scoped to `guest/cloud-hypervisor/**`, `src/cloud-hypervisor/**`,
schedule.

**Build job** (`ubuntu-24.04`): Builds deterministic guest artifacts — Cloud
Hypervisor v53.0 binary, the same pinned Linux 6.1.141 kernel config
Firecracker uses, BusyBox 1.36.1 rootfs, and the shared AWF guest supervisor —
Hypervisor v53.0 binary, the pinned Linux 6.1.141 kernel config, BusyBox 1.36.1
rootfs, and the shared AWF guest supervisor —
from pinned, SHA-256 verified sources. Attests provenance. Uploads as a
7-day workflow artifact (`cloud-hypervisor-test-x86_64`).

Expand All @@ -250,9 +218,8 @@ SHA-256 digests plus GitHub-hosted-only host eligibility (`GITHUB_ACTIONS`,
live smoke/security suite. The preflight requires usable KVM and fails closed
if `/dev/kvm` or another required host capability is unavailable.

Live assertions (see `scripts/ci/cloud-hypervisor-live-smoke.sh`) reproduce
Firecracker's full 13-case contract verbatim, plus two Cloud Hypervisor-only
cases:
Live assertions (see `scripts/ci/cloud-hypervisor-live-smoke.sh`) cover the
following behavior:

| Case | What it proves |
|------|---------------|
Expand All @@ -272,10 +239,9 @@ cases:
| `keep` | `--keep-containers` preserves namespace/run-directory; diagnostics ≤1 MiB |
| `security-assertions` **(CH-only)** | Live jailer-replacement boundary: non-root uid, `CapEff` limited to `CAP_NET_ADMIN` alone, `no_new_privs`, active seccomp filter, per-run cgroup membership/bounded memory, `landlock_enable` + exactly-minimal disk/net/vsock topology via `vm.info` |

After every case, the suite asserts no `awffc-*` namespaces, `fch*`/`fcn*`/`fct*`
interfaces (shared naming with Firecracker), `awf-cloud-hypervisor` cgroup
entries, or `cloud-hypervisor` processes remain. The secret sentinel
(`awf-cloud-hypervisor-real-secret-do-not-expose`, distinct from
Firecracker's) is scanned for in the same way. See
After every case, the suite asserts no `awfvm-*` namespaces,
`vmh*`/`vmn*`/`vmt*` interfaces, `awf-cloud-hypervisor` cgroup entries, or
`cloud-hypervisor` processes remain. The suite also scans output for the secret
sentinel (`awf-cloud-hypervisor-real-secret-do-not-expose`). See
[Cloud Hypervisor integration (preview)](../docs/cloud-hypervisor-foundation.md#part-14--ci-workflow)
for the full CI workflow specification.
43 changes: 19 additions & 24 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,27 +240,22 @@ Use `--keep-containers` to preserve containers and files after execution for deb
- `js-yaml`: YAML generation for Docker Compose config
- TypeScript 5.x, compiled to ES2020 CommonJS

## Firecracker microVM runtime (preview)

When `--container-runtime firecracker --firecracker-preview` is supplied, AWF
uses a substantially different execution architecture:

- The **agent container is replaced** by a Firecracker microVM — a hardware-isolated
virtual machine with its own Linux kernel.
- The Squid proxy and API proxy **remain as Docker Compose containers** on the host.
- The workspace is **not bind-mounted**; it is copied into a bounded ext4 image
(`workspace.ext4`) before boot and copied back after the agent exits.
There is no live filesystem passthrough (no virtiofs).
- A **dedicated network namespace** (`awffc-<runId>`) isolates the VM's network.
nftables rules inside the namespace allow only Squid and the API proxy; all
other guest outbound connections are denied.
- The **API proxy is mandatory**; provider credentials are never passed as guest
environment variables and an explicit assertion enforces this.
- **TTY, Docker-in-Docker, topology peers, enclaves, extra volume mounts, and
remote Docker hosts all fail closed** in this preview.
- **Linux/KVM only** — macOS and Windows are permanently unsupported.
CI specifically supports GitHub-hosted x64 `ubuntu-24.04`; KVM remains
mandatory, and hosts without usable `/dev/kvm` access fail closed.

See [Firecracker integration (preview)](./firecracker-integration.md) for the
full architecture, trust model, and operator guide.
## Cloud Hypervisor microVM runtime (preview)

With `--container-runtime cloud-hypervisor --cloud-hypervisor-preview`, AWF
runs the agent in a hardware-isolated microVM:

- Squid and the API proxy remain Docker Compose services on the host.
- A sandboxed `virtiofsd` exports the workspace read-write to `/workspace`.
- A dedicated `awfvm-<runId>` network namespace contains `vmh*`, `vmn*`, and
`vmt*` veth/TAP interfaces. nftables permits access only to Squid and the API
proxy.
- The mandatory API proxy keeps provider credentials out of the guest
environment.
- The VMM runs as a non-root identity with `no_new_privs`, a minimal capability
set, Landlock filesystem rules, seccomp, and explicit cgroup v2 limits.
- The preview supports only GitHub-hosted Ubuntu x86_64 KVM runners and fails
closed on unsupported hosts or missing artifacts.

See [Cloud Hypervisor integration](./cloud-hypervisor-foundation.md) for the
complete architecture, trust model, and operator guide.
48 changes: 6 additions & 42 deletions docs/awf-config-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,6 @@ following top-level properties. All are OPTIONAL:
| `apiProxy` | object | API proxy sidecar configuration |
| `security` | object | Security and isolation settings |
| `container` | object | Container and Docker settings |
| `firecracker` | object | Firecracker v1.16.1 control-plane preview settings |
| `cloudHypervisor` | object | Cloud Hypervisor v53.0 microVM preview settings (see §4.1) |
| `chroot` | object | Chroot execution overrides for split-filesystem ARC/DinD runners |
| `dind` | object | Bootstrap helpers for ARC/DinD split runner/daemon filesystems |
Expand All @@ -93,18 +92,18 @@ normatively by `docs/awf-config.schema.json`.

The `cloudHypervisor` surface pins Cloud Hypervisor v53.0 artifacts and
digests (binary, PCI-capable guest kernel, rootfs, and the shared AWF guest
supervisor) and, like Firecracker, requires explicit
`--cloud-hypervisor-preview` opt-in plus `container.containerRuntime:
supervisor) and requires explicit `--cloud-hypervisor-preview` opt-in plus
`container.containerRuntime:
"cloud-hypervisor"` to execute a workload. Supported host target is
GitHub-hosted Ubuntu `x86_64` runners with KVM only — self-hosted and
non-Ubuntu/non-x86_64 hosts are rejected explicitly by
[`src/cloud-hypervisor/host-eligibility.ts`](../src/cloud-hypervisor/host-eligibility.ts),
unlike Firecracker's preview which permits self-hosted hosts. See
with no fallback to another runtime. See
[`src/cloud-hypervisor/preflight.ts`](../src/cloud-hypervisor/preflight.ts)
for the artifact/host trust-check module,
[`src/cloud-hypervisor/launcher.ts`](../src/cloud-hypervisor/launcher.ts)
for the secure host launcher (network-namespace join, privilege drop, and
Landlock-based filesystem confinement in place of Firecracker's jailer),
for the secure host launcher (network-namespace join, privilege drop, Landlock
filesystem confinement, and seccomp),
[`src/cloud-hypervisor/manager.ts`](../src/cloud-hypervisor/manager.ts) for
the VM lifecycle, and [`guest/cloud-hypervisor/`](../guest/cloud-hypervisor/)
for the guest artifact build/verification pipeline. See
Expand Down Expand Up @@ -223,21 +222,7 @@ AWF settings MAY be supplied via config files, including stdin (`--config -`).
- `container.dockerHostPathPrefix` → `--docker-host-path-prefix`
- `container.runnerToolCachePath` → *(config-only; checked first for optional read-only runner tool cache mount, before `RUNNER_TOOL_CACHE` and `/home/runner/work/_tool` auto-detection)*
- `container.mounts[]` → `-v, --mount` *(repeatable; each array entry maps to one Docker volume mount in `/host_path:/container_path[:ro|rw]` format (both paths must be absolute; host path must exist); in chroot mode, container paths are automatically prefixed with `/host`)*
- `container.containerRuntime` → `--container-runtime` *(user-facing runtime name: `"gvisor"` for OCI runtime in compose, `"sbx"` for Docker sbx microVM, `"firecracker"` for the explicit Firecracker v1.16.1 workload preview, or `"cloud-hypervisor"` for the explicit Cloud Hypervisor v53.0 workload preview (GitHub-hosted Ubuntu x86_64 KVM runners only; see §4.1). For gvisor: translates to `"runsc"`, injects `extra_hosts` for DNS workaround. For sbx, Firecracker, and Cloud Hypervisor: infrastructure stays in Compose while the primary agent runs in a microVM.)*
- `firecracker.previewEnabled` → `--firecracker-preview`
- `firecracker.firecrackerBinary` → `--firecracker-binary`
- `firecracker.jailerBinary` → `--firecracker-jailer-binary`
- `firecracker.kernelPath` → `--firecracker-kernel`
- `firecracker.rootfsPath` → `--firecracker-rootfs`
- `firecracker.supervisorPath` → `--firecracker-supervisor`
- `firecracker.vcpuCount` → `--firecracker-vcpus`
- `firecracker.memoryMib` → `--firecracker-memory-mib`
- `firecracker.apiTimeoutMs` → `--firecracker-api-timeout-ms`
- `firecracker.sha256.firecracker` → `--firecracker-binary-sha256`
- `firecracker.sha256.jailer` → `--firecracker-jailer-sha256`
- `firecracker.sha256.kernel` → `--firecracker-kernel-sha256`
- `firecracker.sha256.rootfs` → `--firecracker-rootfs-sha256`
- `firecracker.sha256.supervisor` → `--firecracker-supervisor-sha256`
- `container.containerRuntime` → `--container-runtime` *(user-facing runtime name: `"gvisor"` for an OCI runtime in Compose, `"sbx"` for a Docker sbx microVM, or `"cloud-hypervisor"` for the explicit Cloud Hypervisor v53.0 workload preview (GitHub-hosted Ubuntu x86_64 KVM runners only; see §4.1). gVisor translates to `"runsc"` and injects `extra_hosts` for its DNS workaround. For sbx and Cloud Hypervisor, infrastructure stays in Compose while the primary agent runs in a microVM.)*
- `cloudHypervisor.previewEnabled` → `--cloud-hypervisor-preview` *(requires `container.containerRuntime: "cloud-hypervisor"` and a GitHub-hosted Ubuntu x86_64 KVM runner to execute a workload)*
- `cloudHypervisor.cloudHypervisorBinary` → `--cloud-hypervisor-binary`
- `cloudHypervisor.kernelPath` → `--cloud-hypervisor-kernel`
Expand Down Expand Up @@ -300,27 +285,6 @@ AWF settings MAY be supplied via config files, including stdin (`--config -`).

When `container.dockerHostPathPrefix` points at a daemon-visible shared `/tmp` path, the implementation stages the invoking CLI binary together with `/etc/passwd`, `/etc/group`, and the generated chroot `/etc/hosts` under that shared path so chroot mode can bootstrap on split-filesystem ARC/DinD hosts.

The `firecracker` surface is an explicit workload preview pinned to Firecracker
v1.16.1 on Linux/KVM (`x86_64` or `aarch64`). It requires strict network
isolation, a local Unix-socket Docker daemon, the matching jailer, and explicit
SHA-256 digests for Firecracker, jailer, kernel, rootfs, and the AWF guest
supervisor. AWF starts Compose infrastructure only, attaches the jailed
microVM to the proven internal bridge, and executes through vsock. Host access,
DinD, extra mounts, TTY, topology peers, and enclaves fail closed in this
preview. Selecting `firecracker` never falls back to another runtime.

**macOS and Windows are permanently unsupported.** CI specifically supports
GitHub-hosted x64 `ubuntu-24.04`; KVM remains mandatory, and hosts without usable
`/dev/kvm` access fail closed. The API proxy is mandatory; provider credentials
are never passed as guest environment variables. No auto-download of artifacts;
all five artifact paths and their SHA-256 digests are required on every invocation.
The `firecracker-test-x86_64` x86_64 test/preview artifacts can be built
explicitly from the repository, but are not built or published by GitHub
Actions. They are not production defaults and are never auto-downloaded. See
[Firecracker integration (preview)](./firecracker-integration.md) for the
complete operator guide, trust model, workspace semantics, local validation,
and troubleshooting reference.

When DinD is detected, AWF preserves the detected `DOCKER_HOST` value for the agent environment (including MCP servers) so DinD-aware tooling can reach the correct daemon without manual workflow env overrides.

`security.allowHostPorts` (`--allow-host-ports`) is accepted together with
Expand Down
Loading
Loading