Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/self-hosted-runner-doctor.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 1 addition & 3 deletions .github/workflows/self-hosted-runner-doctor.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ Prefer the narrowest match. Examples:
- DNS resolution fails in `--network-isolation` mode specifically on GKE/ARC using NodeLocal DNSCache (resolver `169.254.20.10`), even though the resolver is reachable → B17 update (github/gh-aw-firewall#7188 — filtering is now reachability-probed, not category-blanket; upgrade AWF)
- Azure CLI / ADO MCP auth failures with `~/.azure` missing inside AWF sandbox, or `AZURE_CONFIG_DIR`/`ADO_MCP_AUTH_TOKEN` empty inside agent despite being set on the runner → B18 (`.azure` was in `home.forbiddenSubdirs` and auth env vars were not forwarded; fixed in github/gh-aw-firewall#6690 — note: pre-AWF `az login` is not inherited; agent must perform OIDC re-login inside the sandbox, see github/gh-aw-firewall#6686)
- `[WARN] Could not fix squid log permissions: ... Operation not permitted` appears as the last log after an unrelated primary AWF startup failure → B19 (benign rootless chmod cleanup error not demoted, obscuring the real failure; fixed in github/gh-aw-firewall#6939)
- `ENETUNREACH ... :18443` (or default bridge gateway IP) from `awf-cli-proxy` in `--network-isolation` mode → B20 (known unresolved; #7066 dual-home approach was reverted because it granted general external egress from `cli-proxy`)
- `ENETUNREACH ... :18443` (or default bridge gateway IP) from `awf-cli-proxy` in `--network-isolation` mode → B20 (fixed in github/gh-aw-firewall#7338 via credential-free `cli-proxy-egress` relay attached to `awf-ext`; upgrade AWF)
- `unable to create native thread` / `Cannot create worker GC thread` inside the AWF agent from concurrent JVM builds → B21 (Docker `pids_limit` hardcoded to 1000 with no visibility/configurability; fixed in github/gh-aw-firewall#7150 with `--pids-limit`/`container.pidsLimit` plus `mount_host_cgroupfs()`)
- Strict-security workflow (no `--legacy-security`) cannot reach a GitHub Actions `services:` raw-protocol port via `--enable-host-access` → B22 (strict topology has neither a `host.docker.internal` route nor a host-access bypass, and raw clients cannot use Squid; AWF needs a supported topology path and gh-aw must derive service ports)
- `spawn /usr/local/bin/copilot ENOENT` specifically on a tool-cache **hit** (distinguish from the general PATH-resolution note already covered elsewhere) → B23 (gh-aw's cache-hit path skips the `/usr/local/bin/copilot` wrapper install while the compiler harness always spawns that hardcoded path; AWF's chroot-side gap is fixed in github/gh-aw-firewall#7245; only the upstream installer/harness mismatch in github/gh-aw-firewall#7130 remains open)
Expand Down Expand Up @@ -180,8 +180,6 @@ B18 / github/gh-aw-firewall#6686, github/gh-aw-firewall#6690 — Azure CLI, Azur

B19 / github/gh-aw#50384, github/gh-aw-firewall#6934, github/gh-aw-firewall#6939 — AWF fails on a primary error (e.g. topology-peer attach failure: `Failed to connect container "awmg-mcpg" to network "awf-net": ... No such container`), but the last prominent diagnostic consumers see is instead a large `[WARN] Could not fix squid log permissions: Error: Command failed with exit code 1: chmod -R a+rX ... Operation not permitted` dump from best-effort cleanup, obscuring the real root cause. `preserveDirectory()` in `src/artifact-preservation.ts` ran a direct host-side `chmod -R a+rX` on Squid-owned log directories during cleanup and logged the full `execa` error object at `warn` level whenever it failed with `EPERM`/`EACCES` — even though this is an expected, benign outcome on rootless runners. The existing Docker-based rootless permission repair (`fixArtifactPermissionsForRootless()`) already classified these as benign debug output, but the direct chmod path did not reuse that classifier. **Fixed in AWF (PR github/gh-aw-firewall#6939, merged 2026-08-04):** `preserveDirectory()` now reuses the same benign-permission-error classifier as `fixArtifactPermissionsForRootless()`, demoting expected `EPERM`/`EACCES` chmod cleanup failures to concise `debug` output while still surfacing unexpected cleanup failures as warnings. This preserves the primary startup error as the last prominent diagnostic. Upgrade AWF to include github/gh-aw-firewall#6939.

B20 / github/gh-aw-firewall#7063, github/gh-aw-firewall#7066 — On `ubuntu-latest`/GitHub-hosted or plain self-hosted runners (no Tailscale/custom routing) in `--network-isolation` mode, `awf-cli-proxy` never becomes healthy because tcp-tunnel dials `ENETUNREACH 172.17.0.1:18443` against `host.docker.internal`, exhausting the DIFC liveness probe before the agent starts. `awf-net` is `internal: true` with no outbound route. Squid and api-proxy were already dual-homed onto `awf-ext`, but `cli-proxy` was attached only to `awf-net` even though it uses `host.docker.internal` to reach the external DIFC proxy. **Known unresolved:** the dual-home attempt in github/gh-aw-firewall#7066 was reverted because attaching `cli-proxy` to `awf-ext` grants general external egress from that sidecar. A constrained host-reachability fix has not merged yet, so upgrading to #7066 does not resolve this mode.

B21 / github/gh-aw-firewall#7148, github/gh-aw-firewall#7150 — Concurrent JVM-heavy builds (javac, Android manifest merger) inside the AWF agent container fail with `unable to create native thread` / `Cannot create worker GC thread`, and `/sys/fs/cgroup` inside the sandbox exposes no `pids.max`/`pids.current` (`ulimit -u` reports unlimited), so JVM tools have no way to discover or size against the real process/thread ceiling. AWF hardcoded Docker's `pids_limit` to 1000 with no visibility or configurability. **Fixed in AWF (PR github/gh-aw-firewall#7150, merged 2026-08-09):** a new `--pids-limit <n>` CLI flag (default 1000, matching prior behavior) with `container.pidsLimit` config-file support is plumbed through `cli-options.ts` → `validators/log-and-limits.ts` (`parsePidsLimit`) → `build-config.ts` → `services/agent-service.ts`. `containers/agent/entrypoint.sh` adds `mount_host_cgroupfs()` (best-effort) to bind-mount the container's delegated `/sys/fs/cgroup` read-only onto `/host/sys/fs/cgroup` so `pids.max`/`pids.current` are visible inside chroot. Upgrade AWF to include github/gh-aw-firewall#7150; raise the ceiling with `--pids-limit 4000` (or similar) for concurrent JVM builds.

B22 / github/gh-aw-firewall#7149, github/gh-aw-firewall#7152, github/gh-aw-firewall#7132 — Strict-security (`--network-isolation`, no `--legacy-security`) workflows cannot directly reach a GitHub Actions `services:` container that uses a raw protocol (for example Postgres on 5432) through `--enable-host-access` / `--allow-host-ports`. Strict topology omits the agent's `host.docker.internal` mapping and host-access iptables bypass, while raw clients such as `psql` cannot use Squid's HTTP `CONNECT` protocol. **Known unresolved:** AWF needs a supported strict-topology route for raw service protocols, and gh-aw must derive the required service ports. Until then, run direct service clients outside strict topology (for example with `--legacy-security`) or use a separately verified tunnel.
Expand Down
Loading