diff --git a/.github/CLAUDE.md b/.github/CLAUDE.md index 8687162e4..853e85739 100644 --- a/.github/CLAUDE.md +++ b/.github/CLAUDE.md @@ -73,9 +73,13 @@ regardless of how convenient it is. `claude-pr-loop` (`fix`/`verify`), `claude-review`, `claude` and `bestaxbot-reply` all check out a branch and run repository code in the same job as that token, and they must — fixing and reviewing code is the job. What holds them is a different, weaker set: the tool allowlist, the - protected-path deny rules, the trusted-labeler gates, and (once #578 lands) an enforced egress - policy. Saying I1 covers those jobs would be the exact overstatement the checklist at the - bottom of this file ends on. It covers the pipeline that was designed around it. + protected-path deny rules, the trusted-labeler gates, and an enforced egress policy — which as + of #578 covers `claude`, `claude-implement` and `bestaxbot-reply`, while `claude-review` and + `claude-pr-loop`'s `fix`/`verify` are still at `audit` and so have that last leg only in name. + Check the rule 10 inventory before citing it for a specific job rather than reading this + sentence as a group claim. Saying I1 covers those jobs would be the exact overstatement the + checklist at the bottom of this file ends on. It covers the pipeline that was designed around + it. harden-runner's egress block enforces again as of #487 — `block` used to degrade silently to `audit`, and now enforces and is asserted (rule 10). **Do not promote it to a third leg of @@ -337,13 +341,13 @@ Copy the full version from any block job — the three shapes above are each loa - **The `-e` check comes first** because harden-runner has deliberate paths that install nothing and still exit 0 (a StepSecurity outage, the `skip-harden-runner` repo property, a container or - slim runner). Without it a vendor outage fails every block job at once — thirteen of them as of + slim runner). Without it a vendor outage fails every block job at once — sixteen of them as of the inventory below, not the seven this line said before the `ai-scan`/`ai-triage` job splits — with a bare `jq: could not open file`. It still fails, because a job holding a credential must not run unprotected, but it says why. Do not hand-maintain that number: it is the count of block-mode jobs, and it has been wrong once already. Derive it, and grep for the **key** rather than the string — `grep -rn "^ *egress-policy: block$"`. A plain `grep egress-policy: block` - returns fourteen: `deploy-worker.yml` names the policy in a comment, which is the same way a + returns seventeen: `deploy-worker.yml` names the policy in a comment, which is the same way a `node -e` in a comment misclassifies a job below. - **The status check polls** rather than testing once. The pre-step waits only for the file to _exist_ and gives up after ~9s, while the agent resolves every allow-listed host before writing @@ -474,11 +478,14 @@ jobs (`ci.yml`, `deploy.yml`, `test-deploy.yml`, `visual-regression.yml`, `story - **Enforcing and asserted** — all three `ai-scan` jobs (`gate`, `scan`, `label`), all four `ai-triage` jobs (`gate`, `triage`, `publish`, `cleanup`), `claude-repro` (`author` **only**), `deploy-worker` (`deploy`), `supply-chain` (`consumer-sbom` and `sign-sbom`), - `security-txt-expiry` (`check`), `auto-close-duplicates` (`auto-close`). Thirteen jobs; the - command below is the check. -- **Audit, deliberately, pending a measured allowlist** — `claude`, `claude-implement`, - `claude-pr-loop` (`fix` and `verify`), `claude-review`, `bestaxbot-reply`. These run repo code - with a model token; their block flip is the follow-up this rule owes, tracked in #578. + `security-txt-expiry` (`check`), `auto-close-duplicates` (`auto-close`), `claude` (`claude`), + `claude-implement` (`implement`), `bestaxbot-reply` (`respond`). Sixteen jobs; the command + below is the check. +- **Audit, deliberately, pending a measured allowlist** — `claude-pr-loop` (`fix` and `verify`) + and `claude-review`. These run repo code with a model token; their block flip is the remainder + of the follow-up this rule owes, tracked in #578. `fix` and `review` are measured and waiting + only on their own PR; `verify` is the one job with no measurement yet, because it runs only + when a deep review files a blocking inline finding (#593). - **No harden-runner at all**, and these are two different groups — do not merge them into one "API-only" line, which understates the second: - _Genuinely API-only_ — no checkout, no repository code: `on-slop`, `auto-label-claude-prs`, @@ -536,6 +543,48 @@ allow-listed host (see I1). Widening an allowlist remains a security change unde Verify rather than assume, on any run: the assertion step passes, and harden-runner's post-step prints the effective `EgressPolicy:`. +#### Measuring an allowlist + +An audit-mode run's endpoints are in the **run log**, not only the StepSecurity web UI — the +`Post Harden runner` step dumps the agent log, which records every DNS resolution and connection. +So this is a command rather than a transcription exercise, and it is the answer to "where does +the list come from" wherever this rule says a list must be measured: + +```bash +gh run view --log \ + | grep -oE 'domain: [a-z0-9.-]+\., pid' \ + | sed 's/domain: //; s/\., pid//' | sort -u +``` + +Two things about reading its output, both learned assembling the #578 lists: + +- **Runner infrastructure appears in that log and must never be allow-listed.** + `results-receiver.actions.githubusercontent.com`, + `productionresultssa.blob.core.windows.net`, + `run-actions--azure-*.actions.githubusercontent.com` and `hosted-compute-*.githubapp.com` + are the runner talking to its own control plane. The blob host cannot be pinned even in + principle — its name rotates per run (sa3, sa6, sa7, sa9, sa11 and sa13 across six runs). Run + 33221210633 is the evidence that omitting them is right: `auto-close-duplicates` at `block` + with the five-host list observed only `api.github.com` and `github.com`, and completed all + fifteen of its API calls under the firewall. + + **Leaving them out costs nothing, and that is measured rather than assumed.** harden-runner + does not gate the runner's own control-plane traffic, so the Actions cache keeps working: + run 33286967625 ran `claude-review` at `block` with neither the blob host nor + `results-receiver` allow-listed, and its `setup-node` step restored the pnpm cache + successfully before `pnpm install --frozen-lockfile` completed. Worth stating plainly + because the first revision of this bullet predicted a cache miss and called it an accepted + cost. That was a guess, it was wrong, and the run is what showed it. If you catch yourself + writing down what a policy will cost, go and read a run instead. + +- **One run is not a measurement.** Vendor telemetry samples, so a host can be absent from four + runs and present in the fifth — `telemetry.vercel.com` (turbo) showed up exactly once across + the six #578 runs. Prefer denying telemetry at the source (`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC`, + `TURBO_TELEMETRY_DISABLED`) over allow-listing a host you would rather not talk to; that is the + standing call here, and it also keeps the next audit report clean for whoever reads it. + +Then resolve every host before it goes in the list, per the rule above. + ## Review checklist for a workflow change - [ ] Does any allowlist grow? If yes, name the credential in that job and justify each entry. diff --git a/.github/workflows/bestaxbot-reply.yml b/.github/workflows/bestaxbot-reply.yml index 51ac6f5e1..da1054e42 100644 --- a/.github/workflows/bestaxbot-reply.yml +++ b/.github/workflows/bestaxbot-reply.yml @@ -77,18 +77,100 @@ jobs: issues: write # issue-comment replies id-token: write # OIDC exchange for the subscription token steps: - # AUDIT, not block, and rule 10's "existing live job" clause is why: this - # job checks out code, installs dependencies and runs a Claude session - # holding both bestaxbot's PAT and the OAuth token, and none of that egress - # has ever been measured. A guessed allowlist would break replies on the - # first miss; audit mode reports every endpoint the run touches without - # refusing any, which is what produces the list a block flip needs. That - # flip is the follow-up rule 10 owes (#578) — audit is a starting point - # here, not a resting place. + # BLOCK, with the allowlist MEASURED rather than guessed (#578). This job's + # audit run was 33231488110; all six runs the flip is built on are recorded + # on that issue, each read out of its own `Post Harden runner` log rather + # than transcribed from the StepSecurity UI. + # + # The eight hosts are the set already enforcing on ai-scan/scan, + # ai-triage/triage and claude-repro/author, and the measurement came out a + # strict subset of it: objects.githubusercontent.com, + # release-assets.githubusercontent.com and nodejs.org never appeared, because + # the Node 24 toolcache hit on every run. All three stay for the + # toolcache-miss path, and they are not interchangeable: the first two carry + # the actions/node-versions manifest and its release assets, while nodejs.org + # is setup-node v7 own fallback when that lookup misses. + # + # nodejs.org is the one host NOT in the Claude-session set copied above, and + # the reason is worth stating so it does not get trimmed back out: none of + # ai-scan/scan, ai-triage/triage or claude-repro/author runs setup-node at + # all, so that set never needed it. This job does. Every block job in this + # repo that runs setup-node allows it (auto-close-duplicates, + # security-txt-expiry, deploy-worker, supply-chain/consumer-sbom). + # + # Two observed hosts are deliberately absent. telemetry.vercel.com is turbo's + # telemetry, denied via TURBO_TELEMETRY_DISABLED below rather than + # allow-listed — the same call this repo already makes for the CLI's Datadog + # host. The Actions cache blob host cannot be pinned at all: its name rotates + # per run (productionresultssa3/6/7/9/11/13 across six runs). It does not need + # to be. harden-runner does not gate the runner own control-plane traffic, and + # run 33286967625 is the proof rather than the assumption: claude-review at + # block, with neither that host nor results-receiver.actions.githubusercontent.com + # allow-listed, restored its pnpm cache and completed pnpm install + # --frozen-lockfile. An earlier revision of this comment predicted a cache miss + # here; it was wrong, and measuring it is what showed that. + # + # Resolve any host before adding it here. An unresolvable entry is not inert: + # the agent reverts the firewall while the pre-step still exits 0, which is + # how statsig.anthropic.com silently disabled the policy on #577 (rule 10). - name: Harden runner uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 with: - egress-policy: audit + egress-policy: block + allowed-endpoints: > + api.anthropic.com:443 + claude.ai:443 + downloads.claude.ai:443 + api.github.com:443 + github.com:443 + objects.githubusercontent.com:443 + release-assets.githubusercontent.com:443 + registry.npmjs.org:443 + nodejs.org:443 + + # Asserts the policy the pre-step decided (agent.json) AND that the agent + # actually came up (agent.status, written only once the firewall rules are + # installed). Either alone is a false pass: #487 was the first kind, and an + # unresolvable host in an allowlist is the second. + # + # Scope it honestly: this proves enforcement was armed AT THIS STEP. It does + # NOT prove the policy stays armed — the agent writes `Initialized` and then + # serves, and a later runtime error makes it revert the firewall with both + # files still reading exactly as they do here. Rule 10 in .github/CLAUDE.md + # carries the mechanism and what to check when a session looks wrong. + # + # Placement is rule 10's default (immediately after harden-runner): nothing + # `always()`-guarded keys off this job's outputs, and the job is not itself + # a fail-closed path, so neither placement exception applies. + # + # Linux-only paths (macOS uses /opt/step-security); all runners are ubuntu-latest. + - name: Assert egress policy is enforced + run: | + set -uo pipefail + # harden-runner has several deliberate "install nothing, exit 0" paths + # (StepSecurity unavailable, the skip-harden-runner repo property, a + # container or slim runner). None of them writes agent.json, so name + # that case rather than leaving a bare jq "could not open file". + if [ ! -e /home/agent/agent.json ]; then + echo "::error::harden-runner installed no agent, so nothing is enforcing egress. Usual causes: a StepSecurity outage (its pre-step returns green without installing), the skip-harden-runner repo property, or a container/slim runner. This job holds bestaxbot's PAT and a model token, and will not run unprotected (#487)." + exit 1 + fi + if ! jq -e '.egress_policy == "block"' /home/agent/agent.json; then + echo "::error::effective egress policy is not block — harden-runner downgraded it (#487)." + exit 1 + fi + # Poll rather than test once: the pre-step waits only for the file to + # EXIST and gives up after ~9s, while the agent resolves every + # allow-listed host before writing its status, so a cold resolver or a + # long list can leave the file absent or still empty here. + for _ in $(seq 1 30); do + if grep -q '^Initialized' /home/agent/agent.status 2>/dev/null; then + exit 0 + fi + sleep 1 + done + echo "::error::agent.json reads block but the agent never reported Initialized — the firewall is not up. Check the agent log for 'Reverted changes' (usually an unresolvable allow-listed host)." + exit 1 # Layer 2 (authoritative): author_association can be COLLABORATOR with # only read access. Re-check the LIVE role and require write/triage+. @@ -175,10 +257,17 @@ jobs: # it (#487). It also covers feature-flag evaluation, which is why # statsig.anthropic.com is gone. # - # On this `audit` job the point is the MEASUREMENT: audit mode exists to - # produce the endpoint list a block flip will use (#578), and leaving - # telemetry on would put the Datadog host in that report, inviting whoever - # reads it to allow-list a host we have just decided to deny. + # This job now enforces `block` (#578), so the denial is doing two jobs at + # once and both matter. The allowlist above refuses the Datadog host, and + # this setting stops the CLI trying to reach it, which keeps a denied call + # out of the run rather than merely blocked. Leaving telemetry on and + # relying on the allowlist alone would fill the logs with refused traffic + # and invite the next reader to "fix" it by allow-listing the host. + # + # It also earned the allowlist: this was set during the audit phase + # precisely so the endpoint list the flip was built from reflected the + # intended end state, rather than recording a host we had already decided + # to deny. # # A step `env:` rather than the action's `settings:` input, and here that # is load-bearing rather than tidiness: `settings:` is written to the @@ -195,6 +284,12 @@ jobs: # `with:`. env: CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: '1' + # Same call, second vendor: turbo phones telemetry.vercel.com from the + # pnpm commands this session runs. It appeared in only one of the six + # audit runs because turbo samples, which is precisely the intermittent + # that reddens a block job weeks after a flip. Denied, not allow-listed + # (#578). + TURBO_TELEMETRY_DISABLED: '1' uses: anthropics/claude-code-action@51c47629174b5dbca955cf93690fd39c8a41dadf # v1 with: # Respond as the bestaxbot machine account so replies/commits come @@ -202,7 +297,7 @@ jobs: github_token: ${{ secrets.AI_LOOP_PAT }} ssh_signing_key: ${{ secrets.AI_LOOP_SSH_SIGNING_KEY }} bot_name: bestaxbot - bot_id: "300268469" + bot_id: '300268469' claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} use_commit_signing: true # Per-turn tool calls + denials in the log (the artifact blob host is diff --git a/.github/workflows/claude-implement.yml b/.github/workflows/claude-implement.yml index 9443f1491..d088a800d 100644 --- a/.github/workflows/claude-implement.yml +++ b/.github/workflows/claude-implement.yml @@ -44,19 +44,108 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 60 steps: - # AUDIT, not block, and rule 10's "existing live job" clause is why: this - # job checks out a branch, installs dependencies and runs a Claude session - # with a model token, and none of that egress has ever been measured. A - # guessed allowlist would break the job on its first miss; audit mode - # reports every endpoint the run touches without refusing any, which is - # what produces the list a block flip needs. That flip is the follow-up - # rule 10 owes (#578) — audit is a starting point here, not a resting - # place. Six jobs run repo code alongside the model token (rule 10 lists - # them); this is one of them, not the only one. + # BLOCK, with the allowlist MEASURED rather than guessed (#578). This job's + # audit run was 33230068020; all six runs the flip is built on are recorded + # on that issue, each read out of its own `Post Harden runner` log rather + # than transcribed from the StepSecurity UI. + # + # The eight hosts are the set already enforcing on ai-scan/scan, + # ai-triage/triage and claude-repro/author, and the measurement came out a + # strict subset of it: objects.githubusercontent.com, + # release-assets.githubusercontent.com and nodejs.org never appeared, because + # the Node 24 toolcache hit on every run. All three stay for the + # toolcache-miss path, and they are not interchangeable: the first two carry + # the actions/node-versions manifest and its release assets, while nodejs.org + # is setup-node v7 own fallback when that lookup misses. + # + # nodejs.org is the one host NOT in the Claude-session set copied above, and + # the reason is worth stating so it does not get trimmed back out: none of + # ai-scan/scan, ai-triage/triage or claude-repro/author runs setup-node at + # all, so that set never needed it. This job does. Every block job in this + # repo that runs setup-node allows it (auto-close-duplicates, + # security-txt-expiry, deploy-worker, supply-chain/consumer-sbom). + # + # Three observed hosts are deliberately absent. telemetry.vercel.com is + # turbo's telemetry, denied via TURBO_TELEMETRY_DISABLED below rather than + # allow-listed — the same call this repo already makes for the CLI's Datadog + # host. The Actions cache blob host cannot be pinned at all: its name rotates + # per run (productionresultssa3/6/7/9/11/13 across six runs). It does not need + # to be. harden-runner does not gate the runner own control-plane traffic, and + # run 33286967625 is the proof rather than the assumption: claude-review at + # block, with neither that host nor results-receiver.actions.githubusercontent.com + # allow-listed, restored its pnpm cache and completed pnpm install + # --frozen-lockfile. An earlier revision of this comment predicted a cache miss + # here; it was wrong, and measuring it is what showed that. + # + # The third is specific to this job, the only one of the six that touched it: + # private-user-images.githubusercontent.com, the session fetching an image + # embedded in an issue body. The cost of leaving it out is real and small — a + # `claude-fix` issue carrying a screenshot degrades, because the session + # cannot read the image. The reason is that on a public repo it is the one + # candidate host serving attacker-supplied bytes into a job holding + # bestaxbot's PAT. Revisit if a run actually needs it, not before. + # + # Resolve any host before adding it here. An unresolvable entry is not inert: + # the agent reverts the firewall while the pre-step still exits 0, which is + # how statsig.anthropic.com silently disabled the policy on #577 (rule 10). - name: Harden runner uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 with: - egress-policy: audit + egress-policy: block + allowed-endpoints: > + api.anthropic.com:443 + claude.ai:443 + downloads.claude.ai:443 + api.github.com:443 + github.com:443 + objects.githubusercontent.com:443 + release-assets.githubusercontent.com:443 + registry.npmjs.org:443 + nodejs.org:443 + + # Asserts the policy the pre-step decided (agent.json) AND that the agent + # actually came up (agent.status, written only once the firewall rules are + # installed). Either alone is a false pass: #487 was the first kind, and an + # unresolvable host in an allowlist is the second. + # + # Scope it honestly: this proves enforcement was armed AT THIS STEP. It does + # NOT prove the policy stays armed — the agent writes `Initialized` and then + # serves, and a later runtime error makes it revert the firewall with both + # files still reading exactly as they do here. Rule 10 in .github/CLAUDE.md + # carries the mechanism and what to check when a session looks wrong. + # + # Placement is rule 10's default (immediately after harden-runner): nothing + # `always()`-guarded keys off this job's outputs, and the job is not itself + # a fail-closed path, so neither placement exception applies. + # + # Linux-only paths (macOS uses /opt/step-security); all runners are ubuntu-latest. + - name: Assert egress policy is enforced + run: | + set -uo pipefail + # harden-runner has several deliberate "install nothing, exit 0" paths + # (StepSecurity unavailable, the skip-harden-runner repo property, a + # container or slim runner). None of them writes agent.json, so name + # that case rather than leaving a bare jq "could not open file". + if [ ! -e /home/agent/agent.json ]; then + echo "::error::harden-runner installed no agent, so nothing is enforcing egress. Usual causes: a StepSecurity outage (its pre-step returns green without installing), the skip-harden-runner repo property, or a container/slim runner. This job holds bestaxbot's PAT and a model token, and will not run unprotected (#487)." + exit 1 + fi + if ! jq -e '.egress_policy == "block"' /home/agent/agent.json; then + echo "::error::effective egress policy is not block — harden-runner downgraded it (#487)." + exit 1 + fi + # Poll rather than test once: the pre-step waits only for the file to + # EXIST and gives up after ~9s, while the agent resolves every + # allow-listed host before writing its status, so a cold resolver or a + # long list can leave the file absent or still empty here. + for _ in $(seq 1 30); do + if grep -q '^Initialized' /home/agent/agent.status 2>/dev/null; then + exit 0 + fi + sleep 1 + done + echo "::error::agent.json reads block but the agent never reported Initialized — the firewall is not up. Check the agent log for 'Reverted changes' (usually an unresolvable allow-listed host)." + exit 1 # Anyone with triage access or higher may start the loop; everyone # else (including issue authors whose template applied the label) is @@ -109,10 +198,17 @@ jobs: # it (#487). It also covers feature-flag evaluation, which is why # statsig.anthropic.com is gone. # - # On this `audit` job the point is the MEASUREMENT: audit mode exists to - # produce the endpoint list a block flip will use (#578), and leaving - # telemetry on would put the Datadog host in that report, inviting whoever - # reads it to allow-list a host we have just decided to deny. + # This job now enforces `block` (#578), so the denial is doing two jobs at + # once and both matter. The allowlist above refuses the Datadog host, and + # this setting stops the CLI trying to reach it, which keeps a denied call + # out of the run rather than merely blocked. Leaving telemetry on and + # relying on the allowlist alone would fill the logs with refused traffic + # and invite the next reader to "fix" it by allow-listing the host. + # + # It also earned the allowlist: this was set during the audit phase + # precisely so the endpoint list the flip was built from reflected the + # intended end state, rather than recording a host we had already decided + # to deny. # # A step `env:` rather than the action's `settings:` input. `settings:` is # written to the USER-level ~/.claude/settings.json, which a project-level @@ -125,6 +221,12 @@ jobs: # into their class. Do not move this back under `with:`. env: CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: '1' + # Same call, second vendor: turbo phones telemetry.vercel.com from the + # pnpm commands this session runs. This is the job that measured it — + # once, in run 33230068020, because turbo samples, which is precisely + # the intermittent that reddens a block job weeks after a flip. Denied, + # not allow-listed (#578). + TURBO_TELEMETRY_DISABLED: '1' uses: anthropics/claude-code-action@51c47629174b5dbca955cf93690fd39c8a41dadf # v1 with: # Run GitHub operations as the bestaxbot machine account (like Bun's @@ -139,7 +241,7 @@ jobs: # precedence over use_commit_signing. ssh_signing_key: ${{ secrets.AI_LOOP_SSH_SIGNING_KEY }} bot_name: bestaxbot - bot_id: "300268469" + bot_id: '300268469' claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} track_progress: true use_commit_signing: true diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index c52bced2c..a43f2c3f9 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -61,17 +61,105 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 30 steps: - # AUDIT, not block, and rule 10's "existing live job" clause is why: this - # job installs dependencies and runs a Claude session with a model token, - # and none of that egress has ever been measured. A guessed allowlist here - # would break the job on its first miss; audit mode reports every endpoint - # the run touches without refusing any, which is what produces the list a - # block flip needs. That flip is the follow-up rule 10 owes (#578) — - # audit is a starting point here, not a resting place. + # BLOCK, with the allowlist MEASURED rather than guessed (#578). This job's + # audit run was 33231511797; all six runs the flip is built on are recorded + # on that issue, each read out of its own `Post Harden runner` log rather + # than transcribed from the StepSecurity UI. + # + # Worth stating because #578 predicted the opposite: this job has no + # `Install dependencies` step, and its endpoint set still came out identical + # to the jobs that do. `pnpm/action-setup` pulls pnpm from registry.npmjs.org + # either way, so the missing install step narrows nothing. + # + # The eight hosts are the set already enforcing on ai-scan/scan, + # ai-triage/triage and claude-repro/author, and the measurement came out a + # strict subset of it: objects.githubusercontent.com, + # release-assets.githubusercontent.com and nodejs.org never appeared, because + # the Node 24 toolcache hit on every run. All three stay for the + # toolcache-miss path, and they are not interchangeable: the first two carry + # the actions/node-versions manifest and its release assets, while nodejs.org + # is setup-node v7 own fallback when that lookup misses. + # + # nodejs.org is the one host NOT in the Claude-session set copied above, and + # the reason is worth stating so it does not get trimmed back out: none of + # ai-scan/scan, ai-triage/triage or claude-repro/author runs setup-node at + # all, so that set never needed it. This job does. Every block job in this + # repo that runs setup-node allows it (auto-close-duplicates, + # security-txt-expiry, deploy-worker, supply-chain/consumer-sbom). + # + # Two observed hosts are deliberately absent. telemetry.vercel.com is turbo's + # telemetry, denied via TURBO_TELEMETRY_DISABLED below rather than + # allow-listed — the same call this repo already makes for the CLI's Datadog + # host. The Actions cache blob host cannot be pinned at all: its name rotates + # per run (productionresultssa3/6/7/9/11/13 across six runs). It does not need + # to be. harden-runner does not gate the runner own control-plane traffic, and + # run 33286967625 is the proof rather than the assumption: claude-review at + # block, with neither that host nor results-receiver.actions.githubusercontent.com + # allow-listed, restored its pnpm cache and completed pnpm install + # --frozen-lockfile. An earlier revision of this comment predicted a cache miss + # here; it was wrong, and measuring it is what showed that. + # + # Resolve any host before adding it here. An unresolvable entry is not inert: + # the agent reverts the firewall while the pre-step still exits 0, which is + # how statsig.anthropic.com silently disabled the policy on #577 (rule 10). - name: Harden runner uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 with: - egress-policy: audit + egress-policy: block + allowed-endpoints: > + api.anthropic.com:443 + claude.ai:443 + downloads.claude.ai:443 + api.github.com:443 + github.com:443 + objects.githubusercontent.com:443 + release-assets.githubusercontent.com:443 + registry.npmjs.org:443 + nodejs.org:443 + + # Asserts the policy the pre-step decided (agent.json) AND that the agent + # actually came up (agent.status, written only once the firewall rules are + # installed). Either alone is a false pass: #487 was the first kind, and an + # unresolvable host in an allowlist is the second. + # + # Scope it honestly: this proves enforcement was armed AT THIS STEP. It does + # NOT prove the policy stays armed — the agent writes `Initialized` and then + # serves, and a later runtime error makes it revert the firewall with both + # files still reading exactly as they do here. Rule 10 in .github/CLAUDE.md + # carries the mechanism and what to check when a session looks wrong. + # + # Placement is rule 10's default (immediately after harden-runner): nothing + # `always()`-guarded keys off this job's outputs, and the job is not itself + # a fail-closed path, so neither placement exception applies. + # + # Linux-only paths (macOS uses /opt/step-security); all runners are ubuntu-latest. + - name: Assert egress policy is enforced + run: | + set -uo pipefail + # harden-runner has several deliberate "install nothing, exit 0" paths + # (StepSecurity unavailable, the skip-harden-runner repo property, a + # container or slim runner). None of them writes agent.json, so name + # that case rather than leaving a bare jq "could not open file". + if [ ! -e /home/agent/agent.json ]; then + echo "::error::harden-runner installed no agent, so nothing is enforcing egress. Usual causes: a StepSecurity outage (its pre-step returns green without installing), the skip-harden-runner repo property, or a container/slim runner. This job holds a model token and runs as claude[bot] with contents/issues/pull-requests write, and will not run unprotected (#487)." + exit 1 + fi + if ! jq -e '.egress_policy == "block"' /home/agent/agent.json; then + echo "::error::effective egress policy is not block — harden-runner downgraded it (#487)." + exit 1 + fi + # Poll rather than test once: the pre-step waits only for the file to + # EXIST and gives up after ~9s, while the agent resolves every + # allow-listed host before writing its status, so a cold resolver or a + # long list can leave the file absent or still empty here. + for _ in $(seq 1 30); do + if grep -q '^Initialized' /home/agent/agent.status 2>/dev/null; then + exit 0 + fi + sleep 1 + done + echo "::error::agent.json reads block but the agent never reported Initialized — the firewall is not up. Check the agent log for 'Reverted changes' (usually an unresolvable allow-listed host)." + exit 1 # Layer 2 (authoritative): author_association can be COLLABORATOR with # only read access. Re-check the LIVE role and require write/triage+. @@ -121,10 +209,17 @@ jobs: # it (#487). It also covers feature-flag evaluation, which is why # statsig.anthropic.com is gone. # - # On this `audit` job the point is the MEASUREMENT: audit mode exists to - # produce the endpoint list a block flip will use (#578), and leaving - # telemetry on would put the Datadog host in that report, inviting whoever - # reads it to allow-list a host we have just decided to deny. + # This job now enforces `block` (#578), so the denial is doing two jobs at + # once and both matter. The allowlist above refuses the Datadog host, and + # this setting stops the CLI trying to reach it, which keeps a denied call + # out of the run rather than merely blocked. Leaving telemetry on and + # relying on the allowlist alone would fill the logs with refused traffic + # and invite the next reader to "fix" it by allow-listing the host. + # + # It also earned the allowlist: this was set during the audit phase + # precisely so the endpoint list the flip was built from reflected the + # intended end state, rather than recording a host we had already decided + # to deny. # # A step `env:` rather than the action's `settings:` input, and here that # is load-bearing rather than tidiness: `settings:` is written to the @@ -141,6 +236,12 @@ jobs: # `with:`. env: CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: '1' + # Same call, second vendor: turbo phones telemetry.vercel.com from the + # pnpm commands this session runs. It appeared in only one of the six + # audit runs because turbo samples, which is precisely the intermittent + # that reddens a block job weeks after a flip. Denied, not allow-listed + # (#578). + TURBO_TELEMETRY_DISABLED: '1' uses: anthropics/claude-code-action@51c47629174b5dbca955cf93690fd39c8a41dadf # v1 with: # OAuth token from a Claude Pro/Max subscription (flat cost, no