Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7f55d95
docs: one home per fact, writing rules, and the Copilot section cleanup
allxsmith Sep 7, 2026
79df662
ci: enforce the fragile-prose rules with a conformance check
allxsmith Sep 7, 2026
9a64e97
docs: keep the guide and the allowlist comment inside the rules they …
allxsmith Sep 7, 2026
5486ee5
ci: test the fragile-prose walk on a real tree and count members too
allxsmith Sep 7, 2026
50a165e
ci: count magnitude words too, and drop the absolute the contract had…
allxsmith Sep 7, 2026
039d6bd
ci: catch counts three words from their noun and parenthesised line refs
allxsmith Sep 7, 2026
d22532b
ci: let the comment mask span newlines so it cannot half-match
allxsmith Sep 7, 2026
64182b2
ci: widen the scanner's nouns and paths, and stop promising reviewers…
allxsmith Sep 7, 2026
c3442d4
ci: match counts across wrapped lines and inside markdown, and say wh…
allxsmith Sep 7, 2026
bb2b6bf
ci: cover more inventory nouns, mask MDX comments, and walk .yaml wor…
allxsmith Sep 7, 2026
02135d6
ci: catch the all-N back-reference, and cite one control in the allow…
allxsmith Sep 7, 2026
2361c4a
ci: give each finding a remedy that would fix it, and say which field…
allxsmith Sep 7, 2026
2258697
ci: carry multiline comment state for the MDX spelling too
allxsmith Sep 7, 2026
c258f45
ci: catch a tally standing alone, and read the ticket exemption acros…
allxsmith Sep 7, 2026
d78136a
ci: hold the marker to its reason, and make comment state survive cod…
allxsmith Sep 7, 2026
2441baf
ci: accept the legacy HTML comment terminator
allxsmith Sep 7, 2026
0022360
ci: read the marker's reason inside its own comment
allxsmith Sep 7, 2026
43b9dab
ci: cover a count wrapped across several narrow lines, and scope the …
allxsmith Sep 7, 2026
67bdec4
ci: reach the whole wrap window, wrapped locators, and only exempt in…
allxsmith Sep 7, 2026
8614fa4
ci: count an approximate number, stop the window at a block, scope th…
allxsmith Sep 8, 2026
3b8e893
ci: detect an approximate count on its own, and counts past four digits
allxsmith Sep 8, 2026
25decdb
docs: describe the verify fallback by what it can establish, not by c…
allxsmith Sep 8, 2026
ad57f1a
ci: count a determiner-led back-reference too
allxsmith Sep 8, 2026
68a4ed5
ci: count a number word carrying the verb, and read grouped numbers w…
allxsmith Sep 8, 2026
96e0afb
ci: read the ticket exemption from masked text
allxsmith Sep 8, 2026
10e9352
ci: unwrap reference-style links as well as inline ones
allxsmith Sep 8, 2026
a667eb2
docs: name the branch that reads the reviewer payload
allxsmith Sep 8, 2026
bafe5d2
ci: count an other-N back-reference, and keep code review on .github …
allxsmith Sep 8, 2026
e05eb78
ci: skip reference-link definitions, count error inventories, drop tw…
allxsmith Sep 8, 2026
5234ca6
ci: carry code-span state across lines, and stop capping line numbers
allxsmith Sep 8, 2026
601200a
ci: close a code span only on a run of its own length
allxsmith Sep 8, 2026
06ad8e9
ci: mask only real HTTP statuses, and strip code before comments
allxsmith Sep 8, 2026
1d262ac
ci: parse code spans and comments in one interleaved pass
allxsmith Sep 8, 2026
ac12be5
ci: accept the legacy terminator in the per-line comment mask too
allxsmith Sep 8, 2026
74210a2
test: build the masking fixture as one string
allxsmith Sep 8, 2026
cc06a50
ci: close an MDX comment whose brace is on another line, and restore …
allxsmith Sep 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,19 @@ reviews:
instructions: >
Docusaurus documentation site. Check MDX/Markdown correctness, working
internal links, and that documented APIs match the library.
- path: '.github/workflows/**'
instructions: >
Review the workflow logic normally. Their COMMENTS are reviewed for
false claims about triggers, actors, or credentials only: do not
request added qualifiers, counts, run ids, or dates in a comment;
suggest deleting an overstated clause instead. Scripts and composite
actions elsewhere under .github/ get ordinary correctness and security
review.
- path: '.github/CLAUDE.md'
instructions: >
The workflow security contract. Flag a claim that is false about
triggers, actors, or credentials; do not request added qualifiers,
counts, run ids, or dates.
- path: '**/*.{ts,tsx}'
instructions: >
Follow the repo ESLint + Prettier config (Prettier 3.9.4). Prefer
Expand Down
89 changes: 58 additions & 31 deletions .github/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -336,18 +336,17 @@ resting place.
exit 1
```

Copy the full version from any block job — the three shapes above are each load-bearing:
Copy the full version from any block job — the shapes above are each load-bearing:

- **The `-e` check comes first** because harden-runner has deliberate paths that install nothing
and still exit 0 (a StepSecurity outage, the `skip-harden-runner` repo property, a container or
slim runner). Without it a vendor outage fails every block job at once — nineteen of them as of
the inventory below, not the seven this line said before the `ai-scan`/`ai-triage` job splits —
with a bare `jq: could not open file`. It still fails, because a job holding a credential must
not run unprotected, but it says why. Do not hand-maintain that number: it is the count of
block-mode jobs, and it has been wrong once already. Derive it, and grep for the **key** rather
than the string — `grep -rn "^ *egress-policy: block$"`. A plain `grep egress-policy: block`
returns twenty: `deploy-worker.yml` names the policy in a comment, which is the same way a
`node -e` in a comment misclassifies a job below.
slim runner). Without it a vendor outage fails every block job at once, with a bare
`jq: could not open file`. It still fails, because a job holding a credential must not run
unprotected, but it says why. Do not write down how many block jobs that is: an earlier version
of this line did, and it was wrong before the `ai-scan`/`ai-triage` job splits. Derive it, and
grep for the **key** rather than the string — `grep -rn "^ *egress-policy: block$"`. A plain
`grep egress-policy: block` counts one more: `deploy-worker.yml` names the policy in a comment,
which is the same way a `node -e` in a comment misclassifies a job below.
- **The status check polls** rather than testing once. The pre-step waits only for the file to
_exist_ and gives up after ~9s, while the agent resolves every allow-listed host before writing
its status, so a cold resolver or a long list can leave it absent or empty at this point.
Expand Down Expand Up @@ -424,12 +423,12 @@ revert is the agent's; the pre-step only makes one visible by printing `agent.lo
debugging a red assertion, `Reverted changes` is in the agent's output, not harden-runner's.

**What the pair does NOT prove is that the policy stays armed.** In `step-security/agent`, the
sequence is `writeStatus("Initialized")` (agent.go:307) and then a serve loop whose
`case e := <-errc:` (:313) calls `RevertChanges` (:315). So any runtime error after
sequence is `writeStatus("Initialized")` in `agent.go` and then a serve loop whose
error branch calls `RevertChanges`. So any runtime error after
initialization tears the firewall down while `agent.status` still reads `Initialized` and
`agent.json` still reads `block` — **both assertion lines pass, nothing is enforcing.**
Separately, `refreshDNSEntries` re-resolves every 30s and on failure only logs
`failed to insert new ipaddress in firewall` (:357), so an allow-listed host whose IPs rotate can
`failed to insert new ipaddress in firewall`, so an allow-listed host whose IPs rotate can
quietly stop being reachable.

Two further false-pass modes are known and not covered, both latent for this repo but worth
Expand Down Expand Up @@ -474,22 +473,22 @@ and the jobs adjacent to them, per job, not every job in the directory** — the
jobs (`ci.yml`, `deploy.yml`, `test-deploy.yml`, `visual-regression.yml`, `story-screenshots.yml`,
`scorecard.yml`, `dependency-review.yml`) carry no harden-runner and are out of scope here.

- **Enforcing and asserted** — all three `ai-scan` jobs (`gate`, `scan`, `label`), all four
`ai-triage` jobs (`gate`, `triage`, `publish`, `cleanup`), `claude-repro` (`author` **only**),
- **Enforcing and asserted** — every `ai-scan` job (`gate`, `scan`, `label`), every
`ai-triage` job (`gate`, `triage`, `publish`, `cleanup`), `claude-repro` (`author` **only**),
`deploy-worker` (`deploy`), `supply-chain` (`consumer-sbom` and `sign-sbom`),
`security-txt-expiry` (`check`), `auto-close-duplicates` (`auto-close`), `claude` (`claude`),
`claude-implement` (`implement`), `bestaxbot-reply` (`respond`), `claude-review` (`review`),
`claude-pr-loop` (`fix` and `verify`). Nineteen jobs; the command below is the check.
`claude-pr-loop` (`fix` and `verify`). The command below lists them.
- **Audit, deliberately, pending a measured allowlist** — none, as of #578. That issue closed the
group by measuring all six of its members instead of guessing for them. All six now **use** the
same nine-host allowlist, and the gap between that and what the measurement produced is the part
worth keeping: the runs surfaced six application hosts, a strict subset of the eight the
Claude-session jobs above already used, and `nodejs.org` appeared in no run at all. It is on the
list anyway, and that ninth host is why "measured" is not the same as "complete". None
group by measuring every member instead of guessing for them. They all now **use** the same
allowlist, and the gap between that and what the measurement produced is the part worth
keeping: the runs surfaced a strict subset of the application hosts the Claude-session jobs
above already used, and `nodejs.org` appeared in no run at all. It is on the list anyway, and
that unmeasured host is why "measured" is not the same as "complete". None
of `ai-scan`/`scan`, `ai-triage`/`triage` or `claude-repro`/`author` runs `actions/setup-node`,
so their eight-host list never needed it; all six of #578's jobs do run it, and setup-node falls
so their list never needed it; all six of #578's jobs do run it, and setup-node falls
back to `nodejs.org/dist` when the toolcache misses and the `actions/node-versions` lookup fails.
Six audit runs could not surface that, because the toolcache hit every time and an unexercised
The audit runs could not surface that, because the toolcache hit every time and an unexercised
path leaves no endpoint in any log. Review caught it, not measurement. Keep the group here rather
than deleting it: audit remains the correct starting point for an **existing** live job whose
egress has never been observed, and the recipe under this rule is what keeps it a short state
Expand All @@ -508,8 +507,8 @@ jobs (`ci.yml`, `deploy.yml`, `test-deploy.yml`, `visual-regression.yml`, `story
`verify-provenance` (installs published packages, runs verification scripts). Calling any of
these API-only — as this list did twice — understates the unmonitored execution and egress
surface in the one inventory meant to state it precisely. `auto-close-duplicates` was the
fifth here until it was given an enforced policy; it is now in the first group, and it is the
only one of the five that has moved.
fifth here until it was given an enforced policy; it is now in the first group, and it is
the only one that has moved.

Do not maintain any of this by hand; it has now been wrong three times, and each time the error
moved a code-executing job into the harmless-looking group. Derive it:
Expand Down Expand Up @@ -537,7 +536,7 @@ are not "API-only" in the harmless sense either.

`claude-repro` deserves the emphasis: only its `author` job is hardened. **`publish` — the job
that runs the sanitizer over attacker-influenced text and holds `issues: write` — has no egress
policy at all.** Anyone citing "claude-repro enforces egress" is overstating it by three jobs.
policy at all.** Anyone citing "claude-repro enforces egress" is overstating it by every job but `author`.

Keep this inventory correct when you add a job, and qualify per job rather than per workflow. Its
first version omitted `bestaxbot-reply` and named `claude-repro` unqualified — a table that
Expand Down Expand Up @@ -572,11 +571,11 @@ Two things about reading its output, both learned assembling the #578 lists:
`productionresultssa<N>.blob.core.windows.net`,
`run-actions-<N>-azure-*.actions.githubusercontent.com` and `hosted-compute-*.githubapp.com`
are the runner talking to its own control plane. The blob host cannot be pinned even in
principle — its name rotates per run (sa3, sa6, sa7, sa9, sa11, sa13 and sa17 so far, and the
list keeps growing every time anyone looks). Run
principle — its name rotates per run (`productionresultssa<N>`, and N varies between runs).
Run
33221210633 is the evidence that omitting them is right: `auto-close-duplicates` at `block`
with the five-host list observed only `api.github.com` and `github.com`, and completed all
fifteen of its API calls under the firewall.
with its list observed only `api.github.com` and `github.com`, and completed every one of
its API calls under the firewall.

**Leaving them out costs nothing, and that is measured rather than assumed.** harden-runner
does not gate the runner's own control-plane traffic, so the Actions cache keeps working:
Expand All @@ -587,8 +586,8 @@ Two things about reading its output, both learned assembling the #578 lists:
cost. That was a guess, it was wrong, and the run is what showed it. If you catch yourself
writing down what a policy will cost, go and read a run instead.

- **One run is not a measurement.** Vendor telemetry samples, so a host can be absent from four
runs and present in the fifth — `telemetry.vercel.com` (turbo) showed up exactly once across
- **One run is not a measurement.** Vendor telemetry samples, so a host can be absent from a
run and present in the next — `telemetry.vercel.com` (turbo) showed up exactly once across
the six #578 runs. Prefer denying telemetry at the source (`CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC`,
`TURBO_TELEMETRY_DISABLED`) over allow-listing a host you would rather not talk to; that is the
standing call here, and it also keeps the next audit report clean for whoever reads it.
Expand Down Expand Up @@ -618,6 +617,34 @@ reviewers never see it, a session called "read-only" when its token was write-sc
encoding check described as a proof when it is a backstop. A comment that overstates its control
is worse than no comment, because the next reader stops checking.

### How to be exact: claim less

Exact means claiming less, not explaining more. The way to make a security comment true is to
delete the clause that overstates it, not to add the qualifiers that would make it true.

- A security comment names the control and, when the control is a third party's, cites where
it lives, in one line: `# matched against the run actor by check-human-actor in
anthropics/claude-code-action at the pinned SHA`. Do not restate the third party's
internals; the restatement rots the moment the pin moves, and nobody re-reads it then.
- A count lives in a command, never in a sentence. Write the grep that produces it (rule 10's
`egress-policy: block` grep is the model). A hand-maintained number is wrong within months,
and this file has been caught doing exactly that.
- Evidence (dates, tallies, measurements) goes on the issue and is linked. A comment says what
the mechanism is; the issue says how we know. A run id may stand in a workflow comment as the
receipt for the flip it justified, and nowhere in a guide.
- One home per fact. A mechanism is explained once, where it is configured; every other
mention is a pointer (`# same list as fix; rationale there`). Two copies are two things to
keep equal, and a reviewer will find the day they differ.
- Claim the case in front of you. "Every", "only", "always" and "exactly one" are one
counterexample from false; "the review arm" survives, "every arm" does not.
- Never cite a line number in prose. Lines move on the next edit; a step `id`, a job name, a
rule number, or a heading does not.
- A PR body says what changed and why. State an invariant as the command that checks it
(`grep -c` the pin), not as today's count of what it found.

The `fragile-prose` conformance check enforces the count, run-id, and line-reference rules;
mark a deliberate exception with `bestax:count-ok` on the same line and say why.

## Where the rest is documented

- **Design and operation of the AI loop, and the repository-variable table** — the
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ai-scan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ name: AI Security Scan
# scheduled (runner outage, cancellation between jobs) leaves quota spent for a
# session that never ran. Pre-split, a job that never started charged nothing.
#
# A second, accepted: three named jobs are three public check runs, so `scan`'s
# A second, accepted: each named job is a public check run, so `scan`'s
# conclusion (`skipped` when the gate declined, `success` when a session ran)
# is now readable without log access. That leaks WHETHER an item was scanned —
# and since the gate fails open, a budget-exhausted day is exactly the window
Expand Down Expand Up @@ -560,7 +560,7 @@ jobs:
#
# The parsing lives in scripts/parse-scan-verdict.mjs (rule 9; the
# fail-closed contract and its rationale are in that file's header, the
# seven-shape trace in its test sibling). The script always exits 0 and
# shape-by-shape trace in its test sibling). The script always exits 0 and
# prints exactly `clean` or `flagged <category>`; ANY other outcome — node
# missing because the checkout failed under `always()`, a crashed script,
# garbled output — yields an empty `verdict`, which the label job's enum
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/ai-triage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ name: AI Triage
# skip, zero usage. The counter is read-modify-write, so its correctness
# DEPENDS on the global single-flight concurrency group below — do not
# re-scope the group per item, and note the job split did not change that:
# `concurrency` is workflow-level, so it still serializes all four jobs.
# `concurrency` is workflow-level, so it still serializes every job.
#
# AI_TRIAGE_AUTOCLOSE (`on` / `dry-run` / unset ⇒ `off`) is passed to the
# RENDERER, not to the session: when active, a dedupe comment that names a
Expand Down Expand Up @@ -160,8 +160,8 @@ permissions: {}
# third run arrives while one is in progress and one is pending, the PENDING
# one is CANCELLED. So in a burst, only the first and last items get triaged,
# and the ones in between leave no trace at all — cancelled, not skipped, so
# there is no gate log and no budget charge to explain the gap. The four-job
# split widens the in-progress window from one runner spin-up to four, which
# there is no gate log and no budget charge to explain the gap. The per-job
# split widens the in-progress window from one runner spin-up to one per job, which
# makes the burst that loses items smaller. `cancel-in-progress: false` only
# stops a NEW run killing a RUNNING one; it does not protect the queue.
#
Expand Down Expand Up @@ -940,7 +940,7 @@ jobs:
NUMBER: ${{ needs.gate.outputs.number }}
# The item-type pin binds this job too. Enforcing it only in the render
# job put the guarantee in the job that cannot write, while this one
# holds the PAT and invokes all three commands.
# holds the PAT and invokes the commands.
IS_PR: ${{ needs.gate.outputs.is_pr }}
TRIGGER: ${{ github.event.action }}
DEDUPE_BODY: ${{ needs.triage.outputs.dedupe_body }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/auto-close-duplicates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
# fail-open, where block was already observed arming on the nearest
# sibling (security-txt-expiry.yml, run 31952287389).
#
# The list is this job's needs, and it is deliberately the same five hosts
# The list is this job's needs, and it is deliberately the same hosts
# that sibling allows: checkout and setup-node reach GitHub and nodejs.org,
# and scripts/auto-close-duplicates.mjs is dependency-free (node: builtins
# + global fetch) against api.github.com and nothing else. Adding no new
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/bestaxbot-reply.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,15 @@ jobs:
id-token: write # OIDC exchange for the subscription token
steps:
# BLOCK, with the allowlist MEASURED rather than guessed (#578). This job's
# audit run was 33231488110; all six runs the flip is built on are recorded
# audit run was 33231488110; the runs the flip is built on are recorded
# on that issue, each read out of its own `Post Harden runner` log rather
# than transcribed from the StepSecurity UI.
#
# The eight hosts are the set already enforcing on ai-scan/scan,
# These hosts are the set already enforcing on ai-scan/scan,
# ai-triage/triage and claude-repro/author, and the measurement came out a
# strict subset of it: objects.githubusercontent.com,
# release-assets.githubusercontent.com and nodejs.org never appeared, because
# the Node 24 toolcache hit on every run. All three stay for the
# the Node 24 toolcache hit on every run. They stay for the
# toolcache-miss path, and they are not interchangeable: the first two carry
# the actions/node-versions manifest and its release assets, while nodejs.org
# is setup-node v7 own fallback when that lookup misses.
Expand All @@ -102,7 +102,7 @@ jobs:
# telemetry, denied via TURBO_TELEMETRY_DISABLED below rather than
# allow-listed — the same call this repo already makes for the CLI's Datadog
# host. The Actions cache blob host cannot be pinned at all: its name rotates
# per run (productionresultssa3/6/7/9/11/13 across six runs). It does not need
# per run (productionresultssa<N>, a different N per run). It does not need
# to be. harden-runner does not gate the runner own control-plane traffic, and
# run 33286967625 is the proof rather than the assumption: claude-review at
# block, with neither that host nor results-receiver.actions.githubusercontent.com
Expand Down Expand Up @@ -285,7 +285,7 @@ jobs:
env:
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: '1'
# Same call, second vendor: turbo phones telemetry.vercel.com from the
# pnpm commands this session runs. It appeared in only one of the six
# pnpm commands this session runs. It appeared in only one of the
# audit runs because turbo samples, which is precisely the intermittent
# that reddens a block job weeks after a flip. Denied, not allow-listed
# (#578).
Expand Down
Loading
Loading