From 32fadf3cc7b3a1e2b4bb41ee6007bf9fe111b2a7 Mon Sep 17 00:00:00 2001 From: Alex Smith Date: Thu, 6 Aug 2026 21:29:40 -0400 Subject: [PATCH] ci: stop claiming egress-block on jobs that do not enforce it harden-runner has never enforced `egress-policy: block` in this repo. It arms block mode by reading an Actions cache entry, GitHub made that cache read-only for untrusted triggers, and an unguarded `new URL()` on the resulting miss fails open to `audit` with a single `core.info` line. Root cause, evidence and fix are in #487. Comments only. No policy value, permission, allowlist or logic changes, so behaviour is identical before and after. What changes is that the files stop asserting a control that is not running: - claude-repro.yml named egress-block as one of the two *real* controls behind I1, in the same comment that carefully demotes the credential check to a backstop. I1 rests on two legs, not three. - ai-scan.yml claimed "harden-runner blocks egress" in its security header. - .github/CLAUDE.md rule 10 and the I1 summary described block as the resting state for new jobs without noting that block is inert here. - ai-triage.yml's FOLLOW-UP asked for an audit-to-block flip that is now known to be a no-op. Replaced with the measured endpoint gap: a real session reaches nine hosts and the allowlist is missing three of them, so the flip would have broken triage at the Claude CLI download rather than hardening it. The allowlists are left as they are and the declared policies are unchanged, so closing #487 needs no edit here. Found while carrying out #456's audit-to-block obligation. The fail-closed assertion that would catch a future silent downgrade (`jq -e '.egress_policy == "block"' /home/agent/agent.json`) is deliberately not in this commit: it fails every AI run until enforcement is restored, so it has to land with the fix in #487. Refs #487, #456 --- .github/CLAUDE.md | 27 +++++++++++++++++++++-- .github/workflows/ai-scan.yml | 22 ++++++++++++++----- .github/workflows/ai-triage.yml | 18 ++++++++++----- .github/workflows/claude-repro.yml | 35 +++++++++++++++++++----------- 4 files changed, 76 insertions(+), 26 deletions(-) diff --git a/.github/CLAUDE.md b/.github/CLAUDE.md index cdbb5970e..6426a191c 100644 --- a/.github/CLAUDE.md +++ b/.github/CLAUDE.md @@ -67,6 +67,12 @@ regardless of how convenient it is. execute. Note the direction of the guarantee — it raises the cost of exfiltration, it is not a proof. The credential-shape check in `Collect draft` (literal, base64, and hex forms) is a backstop for the same reason, and is documented as one. + + I1 used to be described as resting on three legs, the third being harden-runner's egress + block. It rests on two. **No job in this repository enforces egress** — `block` silently + degrades to `audit` (#487) — so anywhere you are tempted to trade away a tool restriction + because "egress is blocked anyway", the trade is against nothing. + - **I2 — no untrusted or model-authored free text reaches a re-trigger-capable identity.** Comments posted with `GITHUB_TOKEN` do not emit workflow events. Comments posted with a PAT **do**. So a drafted reproduction is sanitized deterministically and posted via @@ -188,12 +194,29 @@ non-trivial parsing in `scripts/*.mjs` with a `node --test` sibling — root `pn Two parsers are still inline and tracked in #454: the publish sanitizer and the scan-verdict parser. -### 10. `harden-runner` on new jobs starts at `block` +### 10. `harden-runner` on new jobs starts at `block`, and `block` does not currently mean block New jobs ship with `egress-policy: block`. An **existing** live job may be introduced at `audit` first, because flipping straight to block risks breaking it if the action's runtime egress needs an un-allow-listed host — but that is a temporary state that owes a follow-up issue, not a -resting place. `ai-triage.yml` is currently the one job at `audit`. +resting place. + +**Nothing in this repository is enforcing egress today.** harden-runner arms block mode by +reading an Actions cache entry; GitHub made that cache read-only for untrusted triggers +(`issues`, `issue_comment`, `pull_request` — i.e. every workflow here), and harden-runner fails +**open** to `audit` on the resulting miss, announcing it with a single `core.info` line. Tracked +in #487 with the fix and the measured endpoint list. + +Two things follow, and both matter more than the policy value in the YAML: + +- Never cite egress-block as a control in a comment, a PR description, or a review argument + until #487 closes. It is the exact failure the checklist below ends on. +- The allowlists are stale as well as unenforced: a real session reaches three hosts none of + them list. Widening an allowlist is still a security change (rule 2), and #487 is where the + measured set lives. + +Verify rather than assume, on any run: harden-runner logs `Switching egress-policy to audit +mode` in its pre-step, and its post-step prints the effective `EgressPolicy:`. ## Review checklist for a workflow change diff --git a/.github/workflows/ai-scan.yml b/.github/workflows/ai-scan.yml index ddd084208..25fb6fd76 100644 --- a/.github/workflows/ai-scan.yml +++ b/.github/workflows/ai-scan.yml @@ -26,9 +26,16 @@ name: AI Security Scan # (CodeRabbit, Copilot) run on their own triggers and never see this label. # # SECURITY: the scan session holds NO PAT and NO write tools — only read-only -# `gh view`/`gh diff` — and harden-runner blocks egress. Its reasoning IS the -# detection logic, so `show_full_output` is deliberately OFF and only a coarse -# category (never free-text) is surfaced, to avoid an evasion oracle. +# `gh view`/`gh diff`. Its reasoning IS the detection logic, so +# `show_full_output` is deliberately OFF and only a coarse category (never +# free-text) is surfaced, to avoid an evasion oracle. +# +# EGRESS IS NOT ENFORCED HERE, despite the `egress-policy: block` below. +# harden-runner silently degrades it to `audit` on every run in this repo +# (#487): arming block needs an Actions cache entry, GitHub made that cache +# read-only for untrusted triggers, and the resulting miss fails OPEN. So the +# tool allowlist is the only thing confining this session. Do not count +# egress-block as a second control until #487 closes. # # Be precise about where "read-only" comes from: the job's GITHUB_TOKEN is # WRITE-scoped (issues/pull-requests), because the gate charges the budget marker @@ -99,9 +106,12 @@ jobs: pull-requests: write # apply needs-security-review on PRs steps: # Defense-in-depth: the session's only tools are read-only gh (which reaches - # only GitHub) — it cannot open arbitrary sockets — but egress-block bounds - # the OAuth-token surface regardless. Allowlist is additive to harden-runner's - # GitHub Actions baseline; tighten from the first run's report if unused. + # only GitHub), so it cannot open arbitrary sockets. Block mode would bound + # the OAuth-token surface regardless, but it is not in effect (#487, see the + # header). Left declared so the intent stays recorded and closing #487 needs + # no change here. The allowlist is additive to harden-runner's GitHub Actions + # baseline and is NOT currently complete: a real session reaches three hosts + # it does not list (#487 has the measured set). - name: Harden runner uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 with: diff --git a/.github/workflows/ai-triage.yml b/.github/workflows/ai-triage.yml index ac8f2ab64..7e456687a 100644 --- a/.github/workflows/ai-triage.yml +++ b/.github/workflows/ai-triage.yml @@ -164,12 +164,20 @@ jobs: # defense-in-depth on the PAT / OAuth-token surface. # # AUDIT (not block) deliberately: this is a live, working workflow, and the - # Claude action's full runtime egress is not cleanly documented. Audit mode + # Claude action's full runtime egress was not cleanly documented. Audit mode # reports every endpoint the run touches without blocking any, so it cannot - # break triage. FOLLOW-UP: after one real run, read harden-runner's endpoint - # report and, once the list below is confirmed complete, flip - # `egress-policy` to `block`. The new jobs (claude-repro.yml, ai-scan.yml) - # already run in block mode — they are new and canary-tested before relied on. + # break triage. + # + # That caution was justified. A real run measured nine hosts (#487 has the + # table), and the list below is missing three of them — claude.ai, + # downloads.claude.ai and the Claude CLI's telemetry endpoint — so flipping + # to block would have broken triage at the CLI download. + # + # Do NOT flip this to `block` expecting an effect. claude-repro.yml and + # ai-scan.yml both declare block and neither enforces it: harden-runner + # degrades block to audit on every run in this repo (#487). Until that is + # fixed, `audit` here is an honest label for what all three jobs do, and the + # flip is a no-op that would only make this comment wrong again. - name: Harden runner uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 with: diff --git a/.github/workflows/claude-repro.yml b/.github/workflows/claude-repro.yml index 5fe9b358c..607b0524b 100644 --- a/.github/workflows/claude-repro.yml +++ b/.github/workflows/claude-repro.yml @@ -26,11 +26,16 @@ name: Claude Repro # # Be precise about what that buys: it raises the cost of the naive path and # converts it into a failed build, but an encoding check cannot be complete -# (chunk it, reverse it, rot13 it). I1's real strength is the combination — -# no execution primitive, harden-runner egress block, and the token never -# sharing a job with the publish step — with the credential check as the -# last line rather than the first. Do not weaken any of the others on the -# grounds that the check exists. +# (chunk it, reverse it, rot13 it). I1's real strength is the combination of +# no execution primitive and the token never sharing a job with the publish +# step, with the credential check as the last line rather than the first. Do +# not weaken either of those on the grounds that the check exists. +# +# That list used to name harden-runner's egress block as a third leg. It is +# not one, and never has been: the policy declared below silently degrades +# to audit on every run in this repo (#487). I1 rests on two controls today, +# not three. Both are real and both still hold, which is why #487 is a +# defense-in-depth regression rather than an open exfil path. # I2 No attacker- or Claude-authored free text reaches a comment un-neutralized # AND with a re-trigger-capable identity. The `publish` job deterministically # defangs machine markers / @mentions in the drafted test, then posts it via @@ -122,10 +127,13 @@ jobs: result: ${{ steps.draft.outputs.result }} test: ${{ steps.draft.outputs.test }} steps: - # Egress-block is defense-in-depth here (Claude has no tool that opens a - # socket). The allowlist covers the Claude action's runtime + gh; it is - # additive to harden-runner's built-in GitHub Actions baseline. Tighten from - # the first run's network report if any host proves unused. + # Egress-block would be defense-in-depth here (Claude has no tool that opens + # a socket), but it is not in effect: harden-runner degrades `block` to + # `audit` on every run in this repo (#487). Left declared so the intent stays + # recorded and closing #487 needs no change here. The allowlist is additive + # to harden-runner's GitHub Actions baseline and is NOT currently complete: + # a real session reaches three hosts it does not list (#487 has the set), so + # it must be widened before enforcement is switched on. - name: Harden runner uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 with: @@ -268,10 +276,11 @@ jobs: # Literal, plus the two encodings a drafter would reach for first. # This raises the cost of exfil; it is NOT a proof. An encoding check # can never be complete — chunked, reversed, rot13 or arithmetic - # encodings all defeat it. The real controls are the tool restriction - # (no Bash/gh/Task) and harden-runner's egress block; this catches the - # naive path and turns it into a failed build instead of a public - # comment. + # encodings all defeat it. The real control is the tool restriction + # (no Bash/gh/Task); this catches the naive path and turns it into a + # failed build instead of a public comment. Egress-block was named + # here as the second real control and it is not in effect (#487), + # which leaves the tool restriction carrying that weight alone. for secret in "$OAUTH_TOKEN" "$JOB_TOKEN"; do [ -n "$secret" ] || continue B64=$(printf '%s' "$secret" | base64 | tr -d '\n=')