Repository navigation
Add an on-demand Claude security review and a model-free flag on risky paths - #895
seathatflowsinourveins wants to merge 12 commits into
Conversation
…y paths security-review-flag.yml runs on pull requests that touch workflows, hooks, credential tooling, the credential inventory or the paper-trading blueprints and only writes a notice with the exact dispatch command; it runs no model, checks out nothing and holds no scope. claude-security-review.yml is that review: a manual dispatch from main for one pull request head, with the same read-only fence, bounds and accounting as the on-demand pull request review and a security-review prompt built from the action's own example and this repository's risk classes. anthropics/claude-code-security-review is not used (API key only, retired default model, writes a PR review, unpinned installs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Dependency limit exceeded — report not shown. This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report. Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard. Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account. |
The review job now passes `--effort max` in claude_args. Under the command center's effort mapping of 2026-10-08, judgment work (designated reads, adjudication, pull request and security reviews, audits) runs at `max`, and every job records its level and the reason. The level has to be in claude_args because `--restricted` ignores the settings files that would otherwise carry it, and Opus 5.5 runs at `medium` on the API when a request leaves effort unset. The bounds test now requires the flag (removing it fails the test). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bounds step failed the job unless the result's num_turns was at most the --max-turns value. On Claude Code 2.1.295 num_turns counts transcript messages, tool results included: a local run with #894's prompt and claude_args (api-actions LR, 2026-10-08) made 12 API requests under --max-turns 12 and reported num_turns 57. The step now counts distinct assistant message ids in the execution file and checks that count against 12; num_turns stays in the record and the job summary as "messages". The test fixture emits two messages per turn id, as the client does, and a new test passes a 12-turn run that reports 57 messages and fails a 13-turn one. Three mutants (bound on num_turns, count forced to 0, count without de-duplicating ids) each fail the tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main moved to aba02ec (#889 and others), and manifests/evidence.json conflicted, so no pull_request workflow ran on the branch. The manifest is resolved by the hot-file protocol: main's copy, then this branch's 8 changed files re-registered with host_receipts.register_file, followed by component_matrix.py --write and new_host_grand_list.py --write; validate.py exits 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kflows The pre-cue read of the sibling harness-audit PR found gaps that the review workflow shares; this commit carries the same fixes here. Bounds step: an allow-list tool check (Read, Glob and Grep), a session start record without its tool or MCP lists fails, a run without result text fails, and the step names every bound it finds unmet. The review step takes the last non-empty result, cuts on a character boundary and says when the review was cut. Tests: the step tests read the workflow with the policy test's own loader when PyYAML is absent, so they no longer skip; new tests cover the new bounds, the named failures, the character-safe cap and the last non-empty result (32 tests). Record: a section names every behaviour of both new workflows; the evidence-class line states the fence receipt's flags instead of "the shared flag set" and says this workflow has not run; the 2026-10-03 cooldown waiver is cited; the workflow header says the job holds no write scope beyond the OIDC token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sweep of the W4 drafts for the classes #894's pre-cue toolkit read found, applied here: the empty paths array is expanded safely on bash before 4.4; a review step that succeeds without an execution file fails the job; display_report and track_progress are asserted; the record states that a public run's summary is public and when a finding belongs on the local route instead; ci-least-privilege lists the review job and the flag workflow in its tables and names each exempt job (in place). 34 tests; actionlint and zizmor report nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ding findings The pre-cue toolkit read of 459777d left findings standing at df440f4: - The record's link to the pull request review's record, and its receipt citation, now say that #894 (and #892) add those files and land first. - The docs bullet names #894 as where the shared pin, federation, fence and bounds are defined. - The binding step's unused id `bind` is removed; nothing referred to it. - The record's by-name list now names what df440f4 added: the final execution-file step, the bash 3.2 expansion and the Visibility section. It also names this PR's edits outside the record. - The test count, stale at 32, is 34. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the record stand alone - R4: the guard binds the ACTIONS_STEP_DEBUG and ACTIONS_RUNNER_DEBUG secrets and variables as booleans, the secret first, and refuses either. A test pins both bindings, and six weakened copies each fail it. - The publish step writes the review with `jq -j`, so a 60,000-byte review is no longer announced as cut. A boundary test covers it. - From the J8 micro read of fbe30ad: the record and the docs bullet no longer rest on #894 being present. - They state the action pin, the federation variables, the fence flags and the bounds themselves. - They name the pull request review's record by plain path, not by link. - They cite receipt.json from this PR's own tree, a byte-identical copy of the one #892 and #894 add. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e the audit budget to $5 R5, quality and correctness only. The numbers step counts every tool-list entry that is not a string as a forbidden tool, named "non-string tool entry", instead of dropping it (GPT designated read of #895, P2). A new test pins the whole claude_args list and the whole --settings JSON. The client budget moves from $3 to $5 with the turn limit at 20, from the command center's measurement-based caps: the one measured run used 10 turns and $2.03. The shape tests now read the workflow with the policy test's strict loader, so none skips without PyYAML. The record gains a dated R5 section, the cap derivation, the shared agent-mode origin-URL sentence and the follow-ups line, and withdraws the earlier dry estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tries and raise the caps to 30 turns and $5 R5, quality and correctness only. A model-free step removes every symbolic link from the pull request head right after its checkout, so a link to a process environment file or to ../.git/config cannot carry credentials into the job summary (GPT designated read of #909, P2; the command center's security read). The numbers step counts every tool-list entry that is not a string as a forbidden tool, named "non-string tool entry", instead of dropping it (GPT designated read of this PR at f5f1fd2, P2). New tests pin the whole claude_args list and the whole --settings JSON. The caps follow the command center's measurement-based derivation: 30 turns and a $5 client budget. The record gains a dated R5 section, corrects its evidence class, withdraws the earlier dry cost estimate and states which tests run without PyYAML. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tries and raise the caps to 30 turns and $5 R5, quality and correctness only. A model-free step removes every symbolic link from the pull request head right after its checkout, with tests on planted links and on a link-free head. The numbers step counts every tool-list entry that is not a string as a forbidden tool, named "non-string tool entry", instead of dropping it (GPT designated read of #895, P2). New tests pin the whole claude_args list, the whole --settings JSON and the exact if: conditions of the publish and numbers steps. The caps follow the command center's measurement-based derivation: 30 turns and a $5 client budget, with the prompt and the numbers step's bounds following. The record gains a dated R5 section, withdraws the earlier dry cost estimate and states which tests run without PyYAML; the R4 guard lines are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…thin-bound budget stops, and drop --max-turns R6. The cost bound becomes the budget times the measured overrun factor 1.10 ($5.50) (command center read of 5903cee, P2); a budget stop at or under it publishes and is named, and publishing is gated on the numbers step's own outcome. --max-turns 30 leaves claude_args: the pinned action fails a success whose num_turns exceeds it, and on Claude Code 2.1.295 num_turns counts transcript messages (57 for 12 requests); the numbers step's check of 1 to 30 assistant turns stays. Both exact pins compare canonical JSON. The record gains the shared cost paragraph and an upstream-issue draft (not filed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o 30 minutes R7. With --max-turns removed in R6, the numbers step's bound of 30 assistant turns is the turn limit. At the one measured pace of this shape (LR: 12 turns in 512,670 ms, Opus 5.5 at max) 30 turns take about 21.4 minutes before checkout and setup, so a 20-minute job timeout would cancel a run inside the approved bounds before execution_file exists. A new test pins 30, and a weakened copy setting it back to 20 fails it; #892 and #894 use 30. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…894) ### Scope - What this PR changes, in one or two sentences: adds `claude-pr-review.yml`, an on-demand, read-only review of one same-repository pull request head by `anthropics/claude-code-action` v1.0.247. A maintainer dispatches it from `main` with the pull request number and the exact head commit; the review goes to the job summary and nothing is posted to the pull request. It runs only while the repository variable `CLAUDE_PR_REVIEW_ENABLED` is `true`, so merging this starts nothing. - Effort (added 2026-10-08 at head a6d2379): `--effort max` in `claude_args`, because judgment work (designated reads, adjudication, pull request and security reviews, audits) runs at `max` under the command center's effort mapping, and every job records its level and reason. The level has to be in `claude_args`: `--restricted` ignores the settings files that would otherwise carry it, and Opus 5.5 runs at `medium` on the API when a request leaves effort unset. The bounds test now requires the flag; removing it fails the test. A loopback dry run of the installed 2.1.295 client with this workflow's `claude_args` sent `output_config.effort: "max"` on every request. - Fixes from the pre-cue read of the sibling #892 (2026-10-09, head d6f8909): the record names every behaviour of this new workflow: the trigger; the slug, owner, first-attempt and variable guards; the timeout and grants; the guard and binding exits; the head-as-data checkout and diff limits; the explicit action inputs `display_report` and `track_progress` (defaults in the pinned `action.yml`); and the 14-day usage artifact. The bounds step now uses an allow-list, fails on a start record without lists or on an empty result, and names each unmet bound. The review cut is character-safe and says when it cut. The step tests no longer skip without PyYAML. The 2026-10-03 cooldown waiver is cited. - Fixes from this PR's own pre-cue toolkit read (J8 on d6f8909, head 8e64120): - the empty paths array is now expanded safely on bash before 4.4; - a success without an execution file now fails the job; - `display_report` and `track_progress` are asserted; - the artifact name and the action-pin sentence are corrected, and the parity receipt's dependency on #892 is stated; - `claude-pr-review.yml:review` is now in both grant tables of ci-least-privilege; - the record states that a public run's summary is public. - Pre-cue toolkit read of this PR (J8 on d6f8909, 2026-10-09; `pr-test-analyzer` and `silent-failure-hunter`): every finding at confidence 80 or above, and its status at head 41c9c15. | Finding | Status at 41c9c15 | Fixing commit and evidence | | --- | --- | --- | | `pr-test-analyzer` 1 (90): the record names the wrong artifact | fixed | 8e64120: the record names `claude-pr-review-usage-<run id>-<attempt>`, the upload's name; 41c9c15 adds `test_the_usage_artifact_is_named_for_the_run_and_its_attempt` | | `silent-failure-hunter` 2 (88): the record and the workflow disagree on the artifact | fixed | the same two commits | | `pr-test-analyzer` 2 (85): `docs/github-automation.md` claims the harness audit's action pin | fixed | 8e64120: the bullet says the two pins differ until #892 lands | | `silent-failure-hunter` 1 (92): the same false claim (pin and federation variables) | fixed | 8e64120, as above; the four federation variables are the same | | `pr-test-analyzer` 3 (85): the headline evidence file is not in this PR | fixed | 8e64120 states that #892 adds it and lands first; 41c9c15 cites its `LR` entry field by field | | `silent-failure-hunter` 3 (80): "#897" and "two blocking findings" have no source in that receipt | fixed | #892 adds `target_pull_request`, `target_head`, `report_blocking_findings` and `report_sha256` to the `LR` entry (a marked 2026-10-09 amendment from the run's harness receipt); 41c9c15 cites them, calls the verdict the model's own, and states that the prompt and `claude_args` are unchanged from the run's workflow commit a6d2379 to that head (R5 later raises the two caps and the prompt's turn sentence; see R5 below) | - R4, debug logging set as a repository secret or variable (2026-10-09; the cc-native-practice lane's L3 read against `anthropics/claude-code-action` v1.0.247 practice): the guard binds `(secrets.ACTIONS_STEP_DEBUG || vars.ACTIONS_STEP_DEBUG) == 'true'` and `(secrets.ACTIONS_RUNNER_DEBUG || vars.ACTIONS_RUNNER_DEBUG) == 'true'` as booleans and refuses either before any token is requested; a test pins both bindings, and six weakened copies each fail at least one test. This is hardening: the action step already pins `ACTIONS_STEP_DEBUG: 'false'`, which is what the action's full-output switch reads. zizmor's auditor persona reports `secrets-outside-env` on the two reads; CI runs the regular persona with `--no-ignores`, which reports nothing. The R4 lines are byte-identical in #892, #894, #895, #896 and #909; R5 leaves them unchanged here. - The report notice at the cap (2026-10-09; the J8 micro read of #892): the publish step wrote with `jq -r`, which adds a newline, so a 60,000-byte report was announced as cut. It writes with `jq -j` now; a test publishes exactly 60,000 and 60,001 bytes, and a copy that writes with `jq -r` fails it. R5 adds the 59,999-byte case. - From the J8 micro read of 41c9c15 (2026-10-09; 4 of 6 listed findings fixed, PTA-3 and SFH-3 partly because the receipt the record cites was not in this tree): this PR now carries a byte-identical copy of `local-parity-receipt.json` (sha256 `ba498b82…`, the file #892 adds), so the citation no longer depends on which PR lands first, and the docs bullet states the action pin and the federation variables instead of comparing with the harness audit on `main`. - R5 (2026-10-09, head d1bcb6c, limited by the command center to correctness and test quality) adds a model-free step, `Remove symbolic links from the pull request head`, right after the head checkout: `set -euo pipefail; find pr-head -path pr-head/.git -prune -o -type l -exec rm -f {} +`. It raises the approved caps from measurement: `--max-turns` 12→30 and `--max-budget-usd` 3→5. The prompt's "You have at most 30 turns", the numbers step's bounds (1 to 30 assistant turns, at most $5, with matching messages), the header comment and `docs/github-automation.md` follow. The numbers step now counts each non-string tool-list entry as a forbidden tool named `non-string tool entry` instead of dropping it. The R4 guard lines, the deny list and the `github_token` input are kept exactly as at a1bc3bc. The test module goes from 33 to 41 tests. Eight are new: the symlink step's position and exact text; its behaviour on planted links to `/proc/self/environ`, `../.git/config` and a subdirectory, plus a re-run on a link-free head; a 59,999-byte report published whole; object, null and number tool entries refused and named; the prompt stating the `--max-turns` value; and the caps at 30 turns and $5 and just above. Changed tests pin the full `claude_args` list and `--settings` JSON exactly, assert the exact `if:` strings of the publish and numbers steps, run the guard with production's debug-off values (false/false/empty), and check that the 59,999-byte prefix before a cut character survives. All 28 weakened copies of the workflow fail at least one test with PyYAML present, as the hosted validate job has it; without PyYAML 15 of 28 do, because the pins sit in the PyYAML-only shape tests. The decision record gains a dated R5 section and names the R4 guard change in its what-changed paragraph. It corrects "Read denies for every `.git` directory" to "for the files under every `.git` directory". It restates the cap derivation and the expected cost per run, and withdraws the dry $0.70–$1.30 estimate, which was below the measured first-read cost. It adds the shared follow-up line, the shared `origin` URL sentence and two action sources. `manifests/evidence.json` is rehashed for the four changed files. - Sources of the R5 items: the command center's security read of a1bc3bc (the symbolic links, the closed flag list, the exact `if:` strings, the guard's debug-off values, the 59,999-byte prefix, and the stale guard sentence in the record); the command center's caps decision of 2026-10-09; the GPT designated read of #895, P2 (non-string tool entries, applied to all five W4 workflows); the cc-native-practice lane's third audit, P3-1 (the shared `origin` URL sentence). - Caps derivation: `LR`, a first read of #897 with this workflow's prompt and `claude_args` on the installed client, used all 12 of its 12 assistant turns and cost $2.23 ($1.87 by the client's estimate). So the need is above 12, and 2.5 times that floor gives 30 turns; twice $2.23 is about $4.5, rounded up to $5. Four delta re-reads (LR2 to LR5) used 5, 6, 7 and 6 turns and cost $1.75, $1.35, $1.54 and $0.73. A run is expected to cost about $1.9 to $2.3 for a first read and $0.7 to $1.8 for a delta re-read. - Filed as follow-ups before any enabling variable is set, not in R5: the security hardening from the 2026-10-09 read (debug-value widening, the tool-list shape apart from the non-string entry above, extra deny rules, token-source and guard-step assertions); the rest of F-02 (a non-empty tool list, checking the `tool_use` names); F-16 (the pins that run only with PyYAML). - Not run in R5: a fence smoke with a planted symbolic link, which needs a model run; any hosted run. On the first run, watch whether the action appends an `--mcp-config` that `--strict-mcp-config` still honours (the read's G3); the numbers step fails the run on any MCP server. - Base commit: `aba02ec3` (merge base at a1bc3bc and at d1bcb6c, which builds on a1bc3bc; as read at a1bc3bc, main had moved to `ebe6c223` with #890, #900, #897 and #907; only `manifests/evidence.json` overlaps, and the one rebase happens at landing) - Lane: `lane:foundation` - Owned paths touched: `.github/workflows/claude-pr-review.yml` (new), `tests/test_claude_pr_review_workflow.py` (new), `tests/test_workflow_policy.py` (one inventory entry, one exemption), `tests/test_workflow_security_coverage.py` (coverage set and one zizmor test), `docs/decisions/2026-10-08-claude-actions-pr-review.md` (new), `docs/decisions/2026-10-04-ci-least-privilege.md` (dated addendum only), `docs/github-automation.md` (current-practice bullet), `evidence/artifacts/claude-actions-fence-smoke-20261008/receipt.json` (new), `evidence/artifacts/claude-actions-fence-smoke-20261008/local-parity-receipt.json` (new; byte-identical to the copy #892 adds), `manifests/evidence.json` (registration, last commit). ### R6 (2026-10-09), at bd10ba3 - **Cost bound:** the budget × 1.10 = $5.50, from four measured J8 runs on a $5 budget ($5.46, $5.33, $5.16, $5.0007), re-derived after the first three hosted runs. - A budget stop at or under it publishes the text written before the stop, with a line naming the stop. - Publishing is gated on `!cancelled() && steps.numbers.outcome == 'success'`, because the action fails its own step on any budget stop. - **`--max-turns 30` removed:** the pinned action (`base-action/src/run-claude-sdk.ts:241-250` at 2dca132f, since 6ef6450f) fails a success whose `num_turns` exceeds it. - On Claude Code 2.1.295, `num_turns` counts transcript messages: 57 for 12 requests under `--max-turns 12`. That is a code reading plus a local measurement, not a hosted run. - The numbers step's 30-assistant-turn bound and the prompt's "at most 30 turns" stay. - **Job timeout:** 20 → 30 minutes. 30 turns at the measured pace (12 turns in 512,670 ms) take about 21.4 minutes before setup. #892 uses 30. - **Settings pins:** both compare canonical JSON, so `1` no longer passes for `true` (J8 R5 micro, N1, confidence 70). - **Record:** - the shared cost paragraph; - an upstream-issue draft (not filed); - the seven changed test expectations the J8 R5 micro found undeclared (U1–U7). - **Local checks at bd10ba3:** - `validate.py` and the manifest check pass (10,906 files); - six modules: 262 tests, OK; - this module: 44 tests (16 skip without PyYAML); - actionlint and zizmor are clean; - 39 of 39 weakened copies fail a test with PyYAML. - **Hosted checks at bd10ba3:** not read yet. ### SOTA sources - anthropics/claude-code-action, v1.0.247 at `2dca132ff0e0c4094ce6048b422c6915a071210b` (<https://github.com/anthropics/claude-code-action/tree/2dca132ff0e0c4094ce6048b422c6915a071210b>): `docs/security.md` ("Do not check out an untrusted ref into the workspace root before this action"; base ref at the root and head ref in a subdirectory), `docs/setup.md` (federation inputs; "a static credential takes precedence and federation will not be used"), `action.yml` (inputs and the `execution_file` output), `base-action/src/parse-sdk-options.ts` (argument parsing with `shell-quote`, default setting sources, debug forcing full output), `src/github/operations/git-config.ts` (git authentication in the root checkout; lines 129-134 write the job token into the checkout's `origin` URL when `allowed_non_write_users` is unset) and, new in R5, `src/modes/agent/index.ts` (lines 52-61, where agent mode calls that configuration). R5 read both files at v1.0.246 (`38c80c1`); the command center's compare of v1.0.246 and v1.0.247 changes only the bundled Claude Code version. - Claude Code 2.1.295, the version the pin installs, `claude --help`: `--restricted` ("removes the built-in tools that run commands or code ... and ignores user, project and local settings files ... Also confines the file tools to the working directories"), `--permission-prompts none` ("anything that would prompt is denied automatically"), `--tools`, `--settings`, `--strict-mcp-config`, `--max-budget-usd` and, for the R5 caps, `--max-turns`. The help does not say whether a symbolic link's target is resolved, which is why R5 removes the links before the model runs. - GNU findutils `find` 4.10.0 as installed (`find --version`), new in R5: `find pr-head -path pr-head/.git -prune -o -type l -exec rm -f {} +`, measured by the step test on planted links and on a link-free head (exit 0, no `rm` started). The runner image's `find` was not measured. - GitHub, Contexts reference, `runner.debug` (set only while debug logging is on, so the guard's `RUNNER_DEBUG_SIGNAL` is empty on a normal run), new in R5 as the basis of the guard tests' debug-off values; taken from the command center's read of a1bc3bc, not re-read in R5 (no network). - GitHub, OpenID Connect reference, "Filtering for pull_request events" and "Filtering for a specific branch" (<https://docs.github.com/en/actions/reference/security/oidc>); GitHub Security Lab, Preventing pwn requests (<https://securitylab.github.com/research/github-actions-preventing-pwn-requests/>). - Anthropic documentation read 2026-10-08: Workload identity federation (<https://platform.claude.com/docs/en/manage-claude/workload-identity-federation>); How Claude Code uses prompt caching (<https://code.claude.com/docs/en/prompt-caching>); Pricing (<https://platform.claude.com/docs/en/about-claude/pricing>). ### Evidence-class table | Claim | Evidence class | Command / receipt | | --- | --- | --- | | With the fence flags (`--restricted`, `--tools`/`--allowedTools` Read,Glob,Grep, `--strict-mcp-config`, `--permission-prompts none`, `--settings` with hooks off, `claudeMdExcludes` and three deny rules; Haiku 5.5, 10 turns, $0.50, as the receipt lists; this workflow adds `--setting-sources user`, `--add-dir`, `--effort max`, four deny rules and `blockReadsOutsideWorkingDirectories`, a key in the 2.1.295 build that the receipt does not exercise) the session's tools are Glob, Grep and Read; reads outside the tree and of `.git/config` are denied; planted hooks, instruction files and a skill under `pr-head` have no effect | `native_proven` on the installed 2.1.295 client (not through the action, not on a GitHub runner) | `evidence/artifacts/claude-actions-fence-smoke-20261008/receipt.json`, three runs | | The action's own parser turns this workflow's `claude_args` into those flags and that JSON | `local_integration` (at a1bc3bc only) | the parser's three helpers replayed with `shell-quote` 1.10.0 on the workflow text at a1bc3bc; not re-run at d1bcb6c (no network to install `shell-quote`). R5 changes two plain values (`12` to `30`, `3` to `5`), and `test_claude_has_three_read_tools_and_fixed_bounds` asserts the whole list, split per line with `shlex`, and the `--settings` JSON | | The guard, binding, symbolic-link, diff, numbers and review steps behave as described | `synthetic` | `tests/test_claude_pr_review_workflow.py`, 41 tests at d1bcb6c (the 26 step tests run without PyYAML too; the 15 shape tests need it); each step's shell is taken from the workflow file and executed against a local stand-in for `gh`, a local git repository or a planted `pr-head` tree | | Symbolic links under `pr-head` are removed before the model runs, `pr-head/.git` is left as the checkout wrote it, and a link-free head passes | `synthetic` | `test_symbolic_links_in_the_head_are_removed_and_nothing_else` (links to `/proc/self/environ`, `../.git/config` and `../..`; GNU find 4.10.0); no fence smoke with a planted link has run on the client | | The tests detect a weakened workflow | `synthetic` | 20 mutated copies on 2026-10-08 (that day's bounds), 20 failed at least one test; on 2026-10-09 three turn-bound mutants each failed, and the new bounds have their own tests; at 41c9c15 three artifact-name mutants each pass the 30 earlier tests and fail the new name test; at d1bcb6c 28 mutated copies (the symbolic-link step, the flag and settings pins, the `if:` strings, the guard's debug-off values, the report cap, the non-string tool entries, the caps and the prompt) each fail at least one test with PyYAML 6.0.3 present, as the hosted validate job has it; without PyYAML 15 of 28 do (the other 13 are pins in the shape tests; follow-up F-16) | | This workflow's own prompt and `claude_args` review a real pull request within bounds at `--effort max` on Opus 5.5 | `local_integration` (installed client, billed to a second key; not through the action) | `evidence/artifacts/claude-actions-fence-smoke-20261008/local-parity-receipt.json`, carried byte-identical in this PR and in #892 (sha256 `ba498b82…`), its `LR` entry: workflow commit a6d2379, whose prompt and `claude_args` are unchanged at d1bcb6c apart from the two caps R5 raised (`--max-turns` 12 to 30, `--max-budget-usd` 3 to 5) and the prompt's turn sentence; #897 at `target_head` 24821c5; 12 of 12 assistant turns; $1.868667 of the then $3 budget; tools Glob/Grep/Read; no MCP; `success`; `report_blocking_findings: 2`, the report's own verdict line, unchecked (`report_sha256` c49bee04, a 2026-10-09 amendment) | | The caps, 30 assistant turns and $5, bound a runaway at about twice the measured need; a run is expected to cost about $1.9 to $2.3 for a first read and $0.7 to $1.8 for a delta re-read | `local_integration` (installed 2.1.295 client, billed to the second key; not through the action) | the `LR` entry above (12 of 12 turns, client estimate $1.868667); the $2.23 figure and the delta re-reads LR2 to LR5 (5, 6, 7 and 6 turns; $1.75, $1.35, $1.54 and $0.73) come from the command center's caps decision of 2026-10-09 and are not in this repository | | In agent mode the action writes the job token into the root checkout's `origin` URL, so `persist-credentials: false` does not keep it out of `.git/config`; the deny rules keep the model from reading it | `source_review` for the code path; the denial of `.git/config` reads is the fence receipt's (first row) | `src/github/operations/git-config.ts:129-134` and `src/modes/agent/index.ts:52-61`, read at v1.0.246 (`38c80c1`; the command center's compare shows only the bundled client version changing at v1.0.247); the record's shared sentence also cites two 2026-10-09 probes on the installed 2.1.295, not reproduced here | | The workflow is well formed and has no audit finding | `local_integration` | at d1bcb6c: `actionlint` 1.17.0 exit 0; `zizmor` 1.30.1 offline, regular (`--no-ignores`) and pedantic, no findings (2 suppressed in each: `secrets-outside-env` on the R4 secret reads, as at a1bc3bc) | | A hosted run authenticates, stays in bounds and reads the cache | not established | no hosted run is part of this PR | ### Local commands run ``` $ python3 scripts/validate.py # on the R5 tree, committed as d1bcb6c {"components": 70, "hashed_files": 10906, "profiles": 4, "receipts": 239, "status": "passed"} exit 0 $ python3 scripts/evidence_manifest.py --check {"files": 10906, "status": "passed"} exit 0 $ uv run --quiet --no-project --python 3.12 --with pyyaml==6.0.3 python -m unittest tests.test_claude_pr_review_workflow tests.test_workflow_policy tests.test_workflow_security_coverage tests.test_workflow_hardening tests.test_github_automation_practice tests.test_adoption_docs_consistency Ran 259 tests ... OK (skipped=2) exit 0 $ uv run --quiet --no-project --python 3.12 python -m unittest tests.test_claude_pr_review_workflow # without PyYAML Ran 41 tests ... OK (skipped=15) exit 0 $ actionlint .github/workflows/claude-pr-review.yml exit 0 $ zizmor --offline --no-config --persona=pedantic .github/workflows/claude-pr-review.yml No findings to report. Good job! (2 suppressed) exit 0 $ zizmor --offline --no-config --no-ignores --persona regular .github/workflows/claude-pr-review.yml No findings to report. Good job! (2 suppressed) exit 0 $ uv run --quiet --no-project --python 3.12 --with pyyaml==6.0.3 python -B <scratchpad>/r5_pr894_mutants.py . a1bc3bc # outside the repository 28 mutants, 28 caught, survivors: [] exit 0 $ uv run --quiet --no-project --python 3.12 python -B <scratchpad>/r5_pr894_mutants.py . a1bc3bc # without PyYAML 28 mutants, 15 caught exit 1 (expected: the 13 survivors are pins in the PyYAML-only shape tests; follow-up F-16) ``` Hosted checks at d1bcb6c: not read yet. ### Decision record `docs/decisions/2026-10-08-claude-actions-pr-review.md` (why a manual dispatch and no pull request trigger, how the head is handled, the model's fence and what was checked at runtime, cost of one run with the measured runs and the cap derivation, changed test contracts, alternatives, what would overturn it), with a dated R5 section: the symbolic-link step, the caps from measurement, the non-string tool entries, the closed flag list and the boundary tests, the mutants, and the follow-ups. `docs/decisions/2026-10-04-ci-least-privilege.md` gets a dated addendum naming the second job on the federation rule. ### Host evidence Not applicable: no file under `evidence/hosts/` changes. ### Checklist - [x] New/changed GitHub Actions are pinned to a full commit SHA with a version comment (no floating tags). - [x] New/changed workflows declare top-level `permissions: {}` and grant each job only what it needs (`contents: read`, or an explicitly justified addition). - [x] No secrets are printed, logged or committed; no new required secret was added without a documented owner. - [x] No new paid hosting, subscription or billing surface was introduced. - [x] Peer-owned untracked files and worktrees were preserved (not deleted, moved or overwritten). Activation is a separate step after landing: an owner sets `CLAUDE_PR_REVIEW_ENABLED` to `true`, then dispatches the workflow with a pull request number and head commit. Runs authenticate by workload identity federation and spend from the Console organization the four `ANTHROPIC_*` repository variables name; no Anthropic key is stored in GitHub. Each run is capped at 30 assistant turns and a $5 client budget, the owner's per-run spend limit on that organization. This PR and #892 both edit the write-grant inventory and the evidence manifest, so the second to land rebases once. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
…e the audit budget to $5 R5, quality and correctness only. The numbers step counts every tool-list entry that is not a string as a forbidden tool, named "non-string tool entry", instead of dropping it (GPT designated read of #895, P2). A new test pins the whole claude_args list and the whole --settings JSON. The client budget moves from $3 to $5 with the turn limit at 20, from the command center's measurement-based caps: the one measured run used 10 turns and $2.03. The shape tests now read the workflow with the policy test's strict loader, so none skips without PyYAML. The record gains a dated R5 section, the cap derivation, the shared agent-mode origin-URL sentence and the follow-ups line, and withdraws the earlier dry estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) * Bound the harness audit and move it to claude-code-action v1.0.247 The audit job ran with 60 turns, no budget, a `gh issue create` shell tool and `issues: write`. It now gets Read, Glob and Grep, 20 turns and a $3 client budget, holds only `contents: read` and `id-token: write`, and runs only while CLAUDE_HARNESS_AUDIT_ENABLED is 'true'. A model-free step keeps the run's token and cost numbers and fails the job when a bound or the cache-read check is unmet; a second copies the report to the job summary, escaped and capped. The pin moves from v1.0.245 to v1.0.247 (two one-commit releases that bump the bundled Claude Code to 2.1.295). Federation stays the only authentication. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Harden the bounded audit after an independent workflow audit Claude Code now runs under --restricted with --permission-prompts none, and the deny rules travel in --settings, because --restricted ignores the settings file the action's `settings` input writes. The job runs on a first attempt only (a re-run would spend again), for the owner as actor and triggering actor. The numbers step also reads the session's own tool list and fails on a shell, write, web, subagent or MCP tool, writes its record atomically and replaces names that are not plain identifiers. Step debugging is pinned off in the action step's environment. The flag set was run on the installed 2.1.295 client against a planted tree; the receipt is under evidence/artifacts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Set the harness audit's effort level explicitly The audit job now passes `--effort max` in claude_args. Under the command center's effort mapping of 2026-10-08, judgment work (designated reads, adjudication, pull request and security reviews, audits) runs at `max`, and every job records its level and the reason. The level has to be in claude_args because `--restricted` ignores the settings files that would otherwise carry it, and Opus 5.5 runs at `medium` on the API when a request leaves effort unset. A loopback dry run of the installed 2.1.295 client with this workflow's claude_args sent `output_config.effort: "max"` on every request. The bounds test now requires the flag (removing it fails the test). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Bound the harness audit's turns by assistant turns, not num_turns The bounds step failed the job unless the result's num_turns was at most the --max-turns value. On Claude Code 2.1.295 num_turns counts transcript messages, tool results included: a local run with #894's prompt and claude_args (api-actions LR, 2026-10-08) made 12 API requests under --max-turns 12 and reported num_turns 57. The step now counts distinct assistant message ids in the execution file and checks that count against 20; num_turns stays in the record and the job summary as "messages". The test fixture emits two messages per turn id, as the client does, and a new test passes a 20-turn run that reports 57 messages and fails a 21-turn one. Three mutants (bound on num_turns, count forced to 0, count without de-duplicating ids) each fail the tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix the harness audit's bounds and declare every change it makes A pre-cue read by Anthropic's pr-review-toolkit agents (J8, on 4d0a14d) found gaps in this PR; this commit fixes them. Bounds step: the tool check is an allow-list (any tool outside Read, Glob and Grep fails, not only the nine named ones); a session start record without its tool or MCP lists fails instead of passing as empty; a run without result text fails; and the step names every bound it finds unmet instead of a bare exit 1. The report step takes the last non-empty result, cuts on a character boundary (iconv -c, whose exit 1 for a cut character no longer fails the step) and says when the report was cut. Tests: the step tests read the workflow with the policy test's own loader when PyYAML is absent, so they no longer skip; new tests cover an unknown tool, a start record without lists, an empty result, the named failures, the character-safe cap and its notice, and the last non-empty result (21 tests). Record: every behaviour change against main is named (timeout 45 to 30, the slug, first-attempt, variable and owner-only guards, the guard step's exit 2, the display name and prompt, no issue, the two explicit action inputs verified in the pinned action.yml, the usage artifact); the fence receipt's flags are stated instead of "the same flag set"; the cooldown waiver of 2026-10-03 is cited; and a numbers-only receipt of local runs of this workflow's own prompt and claude_args at max effort (10 of 20 turns, about $2.03 of $3) and of the num_turns measurement is added. ci-least-privilege's stale issues-write lines are fixed in place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Apply the pre-cue read's fix classes to the harness audit The sweep of the W4 drafts for the classes #894's pre-cue toolkit read found, applied here: an audit step that succeeds without an execution file now fails the job instead of leaving it green with nothing checked; display_report and track_progress are asserted; the record states that a public run's summary is public. The workflow has no paths input, so the bash 3.2 empty-array class does not apply. 22 tests; actionlint and zizmor report nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the harness audit's partly fixed pre-cue findings The pre-cue toolkit read of 4d0a14d left six findings only partly fixed at 89d05dd, and the test count was stale again. - The record now lists every flag difference between the fence receipt and this workflow's claude_args. The receipt's runs set claudeMdExcludes, so they do not show that --restricted alone keeps instruction files out, and native_proven is limited to the receipt's own flag set. - The .git deny rules: a 2026-10-09 local subscription probe measured Read and Grep into .git as refused. Whether the empty Glob results come from the rules is pending a control run. Whether blockReadsOutsideWorkingDirectories binds Glob and Grep stays not established. - The numbers step records the result's subtype through its name filter, in the usage record, the summary table and the failure message, so a turn-limit stop and a budget stop read apart. Two new tests; seven weakened copies are each caught. - The by-name list now names the execution-file step 89d05dd added, and the other inputs and messages it had left out. - The docs bullet adds the slug, first-attempt and owner-only guards. The record says why the triggering-actor clause stays. - The source list adds action.yml's lines for track_progress, display_report and show_full_output. - The test count is now 24. - local-parity-receipt.json's LR entry gains its target pull request #897 at 24821c5, its report's two blocking findings and the report's sha256. This is a marked 2026-10-09 amendment from the run's harness receipt, and #894's record cites it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Record that the deny rules bind Glob and Grep, measured against a control A control run of the 2026-10-09 probe, with the same fence flags and the deny rules removed, lists the .git files, finds both markers, returns the .env marker through Grep, and reads every marker. With the rules, the same calls returned nothing or were refused. So the empty Glob and Grep results come from the deny rules, not from the tools skipping hidden paths. The record now shows probe against control step by step, and the bounds table and evidence classes say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Bind repository debug settings in the guard, fix the cap notice, keep the probe receipt - R4, from the cc-native-practice lane's L3 read against claude-code-action v1.0.247 practice: the guard binds (secrets.ACTIONS_STEP_DEBUG || vars.ACTIONS_STEP_DEBUG) == 'true' and the same for ACTIONS_RUNNER_DEBUG. Only a boolean reaches the shell, and the guard refuses either setting. A test pins both bindings, and six weakened copies each fail it. - From the J8 micro read of 61494a3: the publish step wrote the report with `jq -r`, which adds a newline, so a 60,000-byte report was announced as cut. It now uses `jq -j`, and a boundary test covers exactly 60,000 and 60,001 bytes. The step comment now says it publishes the last non-empty result. - The deny-rule probe and its control are kept as deny-probe-receipt.json: flags, client, model and authentication, every call and its outcome, and each stream's sha256. - The record states that the workflow is unchanged from e48de2e to the merge base aba02ec, and no longer says the documentation exceeds the runs for the deny-rule question. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refuse non-string tool entries, pin the client flags exactly and raise the audit budget to $5 R5, quality and correctness only. The numbers step counts every tool-list entry that is not a string as a forbidden tool, named "non-string tool entry", instead of dropping it (GPT designated read of #895, P2). A new test pins the whole claude_args list and the whole --settings JSON. The client budget moves from $3 to $5 with the turn limit at 20, from the command center's measurement-based caps: the one measured run used 10 turns and $2.03. The shape tests now read the workflow with the policy test's strict loader, so none skips without PyYAML. The record gains a dated R5 section, the cap derivation, the shared agent-mode origin-URL sentence and the follow-ups line, and withdraws the earlier dry estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Bound the audit's cost at the budget times 1.10, publish within-bound budget stops, and drop --max-turns R6. The cost bound becomes the budget times the measured overrun factor 1.10 ($5.50); a budget stop at or under it publishes and is named, and the publish step runs on the bounds step's own outcome because the action fails its step on any budget stop. --max-turns 20 leaves claude_args: the pinned action fails a success whose num_turns exceeds it, and on Claude Code 2.1.295 num_turns counts transcript messages (57 for 12 requests); the bounds step's check of 1 to 20 assistant turns stays. The exact --settings pin compares canonical JSON, so 1 no longer passes for true. The record gains the shared cost paragraph and an upstream-issue draft (not filed). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix the harness audit's J8 R5-R6 findings: exemption scope, budget-stop note, stale turn text The budget-stop exemption is tested to waive only the success and result-text checks (a stop that breaks any other bound fails, named). The budget-stop note moves to the publish step, so it is written only once every bound passed. Record and workflow comments no longer describe a client turn limit or an error_max_turns stop. N5-N9 are dispositioned in the record's R7 section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Pin the harness audit's action inputs and budget-stop floor that R7 claimed were held The GPT read of 0cf13fc found R7's N8/N9 disposition overstated the tests. The action step's complete input set is now compared, and budget stops at $0.01, $1 and $4.99 must pass, so an added input or a cost floor fails. The record corrects the disposition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Add the R8 source resolution for the harness audit's input-set and budget-floor pins The record names the pinned upstream sources the two R8 tests rely on (the action's input declarations at 2dca132f, the Agent SDK 0.3.295 result type, Python 3.13 unittest assertEqual and subTest), states that the 10-input set and the absent cost floor are this repository's acceptance policy over an unchanged workflow, and labels the mutation results synthetic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Publish a budget-stopped audit's written text and keep instruction files out of the audit session R10, the two Codex P2 threads at b41b969: - harness-audit.yml:231: after a budget stop without result text, the bounds and publish steps use the text the model wrote until then, as claude-pr-review.yml does; every other subtype still gets result text only. - harness-audit.yml:125: the audit step sets CLAUDE_CODE_DISABLE_CLAUDE_MDS=1 (documented); a measured 2.1.295 probe shows project, imported, rules and nested instruction files load only without both --restricted and --setting-sources user, and not at all with the switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: seathatflowsinourveins <234074349+seathatflowsinourveins@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
Parked: Claude review inside GitHub is retired (owner decision 2026-10-10, ~17:15Z); see #981 and docs/decisions/2026-10-10-retire-in-github-claude-review.md. No further pushes. |
|
Closing as superseded, with the owner's OK (CC, 2026-10-11). It adds .github/workflows/claude-security-review.yml, which runs the Claude action. main's tests.test_workflow_hardening.InGitHubClaudeReviewRetiredTests now forbids any workflow that runs it. Claude review inside GitHub was retired by #981 (8bffa2c, docs/decisions/2026-10-10-retire-in-github-claude-review.md). The Anthropic API refuses the action's entry point on Console keys (CC correction #141; the action never had a successful run). AI reviews now run locally, and GitHub enforces them as required checks. The local route is #953 (the review worker). Nothing is lost: this PR can be reopened, and any model-free part of it can come back as a new PR under the local-review route. Evidence: the CC's PR triage (research-cc-role-20261008/pr-triage-20261011.md, class CLOSE). |
Scope
What this PR changes, in one or two sentences: adds
security-review-flag.yml, which on pull requests touching workflows,scripts/hooks/,tools/credentials/, the credential inventory or the paper-trading blueprints only writes a notice with the exact dispatch command (no model, no checkout, no token scope), andclaude-security-review.yml, an on-demand, read-only security review of one same-repository pull request head byanthropics/claude-code-actionv1.0.247, bounded at 30 assistant turns and a $5 client budget. The review runs only whileCLAUDE_SECURITY_REVIEW_ENABLEDistrue, so merging this starts no model run.Effort (added 2026-10-08 at head 7a131b5):
--effort maxinclaude_args, because judgment work (designated reads, adjudication, pull request and security reviews, audits) runs atmaxunder the command center's effort mapping, and every job records its level and reason. The level has to be inclaude_args:--restrictedignores the settings files that would otherwise carry it, and Opus 5.5 runs atmediumon the API when a request leaves effort unset. The bounds test now requires the flag; removing it fails the test.Fixes from the pre-cue read of the sibling Bound the harness audit and move it to claude-code-action v1.0.247 #892 (2026-10-09, head 459777d): the record names every behaviour of both new workflows, including the slug, owner, first-attempt and variable guards, the timeout and grants, the guard and binding exits, the explicit action inputs
display_reportandtrack_progress(defaults in the pinnedaction.yml) and the 14-day usage artifact. The bounds step now uses an allow-list, fails on a start record without lists or on an empty result, and names each unmet bound. The review cut is character-safe and says when it cut. The step tests no longer skip without PyYAML. The evidence line states the fence receipt's flags. The 2026-10-03 cooldown waiver is cited.Fixes at df440f4 (not described here until now): a final step, "Require the run's execution file", fails a success without an execution file; the optional paths expand safely under
set -uon bash before 4.4; the record gains a Visibility section (a public run's summary is public);ci-least-privilege.mdis edited in place (the write-grant and per-workflow table rows, and the exemption sentence) as well as given an addendum.Pre-cue toolkit read of this PR (J8 on 459777d, 2026-10-09;
pr-test-analyzerandsilent-failure-hunter): every finding at confidence 80 or above, and its status at head fbe30ad.pr-test-analyzer(90): the record links Add an on-demand, read-only pull request review by the Claude action #894's record, absent from this treesilent-failure-hunter(90): the same dangling referencepr-test-analyzer(85): thedocs/github-automation.mdbullet describes an absent workflowpr-test-analyzer(90): the cited fence receipt is absentreceipt.jsonis added by Bound the harness audit and move it to claude-code-action v1.0.247 #892 and Add an on-demand, read-only pull request review by the Claude action #894, which land first; this description already said so, and fbe30ad says it in the record toosilent-failure-hunter(90): the same receiptpr-test-analyzer(90): the binding step'sid: bindis unusedsilent-failure-hunter(90): the description does not match the headci-least-privilege.mdeditsR4, debug logging set as a repository secret or variable (2026-10-09; the cc-native-practice lane's L3 read against
anthropics/claude-code-actionv1.0.247 practice): the guard binds(secrets.ACTIONS_STEP_DEBUG || vars.ACTIONS_STEP_DEBUG) == 'true'and(secrets.ACTIONS_RUNNER_DEBUG || vars.ACTIONS_RUNNER_DEBUG) == 'true'as booleans and refuses either before any token is requested; a test pins both bindings, and six weakened copies each fail at least one test. This is hardening: the action step already pinsACTIONS_STEP_DEBUG: 'false', which is what the action's full-output switch reads. zizmor's auditor persona reportssecrets-outside-envon the two reads; CI runs the regular persona with--no-ignores, which reports nothing. The R4 lines are byte-identical in Bound the harness audit and move it to claude-code-action v1.0.247 #892, Add an on-demand, read-only pull request review by the Claude action #894, Add an on-demand Claude security review and a model-free flag on risky paths #895, Add a weekly lane triage: the model proposes, a model-free job applies #896 and Add a pre-cue read by Anthropic's pr-review-toolkit agents in Actions #909, and R5 leaves them unchanged.The report notice at the cap (2026-10-09; the J8 micro read of Bound the harness audit and move it to claude-code-action v1.0.247 #892): the publish step wrote with
jq -r, which adds a newline, so a 60,000-byte report was announced as cut. It writes withjq -jnow; a test publishes exactly 60,000 and 60,001 bytes, and a copy that writes withjq -rfails it.From the J8 micro read of fbe30ad (2026-10-09;
id: bindfixed, the cross-PR items partly fixed because they rested on Add an on-demand, read-only pull request review by the Claude action #894 being present): the record and the docs bullet now stand alone. They state the action pin, the federation variables, the fence flags and the bounds themselves; name the pull request review's record by plain path instead of a link; and citereceipt.jsonfrom this PR's own tree, a byte-identical copy of the one Bound the harness audit and move it to claude-code-action v1.0.247 #892 and Add an on-demand, read-only pull request review by the Claude action #894 add.R5 (2026-10-09, at 5903cee): the symbolic-link step, exact pins, non-string tool entries, the caps from measurement and the corrected "what has run"; see the R5 section below.
Base commit:
aba02ec3(merge base at 5903cee, as at f5f1fd2; at the last local fetch main is at861c475e, twelve commits on, through Scope research Hindsight skills and make wiring checks fail closed #835; onlymanifests/evidence.jsonoverlaps, and the one rebase happens at landing)Lane:
lane:foundationOwned paths touched:
.github/workflows/claude-security-review.yml(new),.github/workflows/security-review-flag.yml(new),tests/test_claude_security_review_workflow.py(new),tests/test_workflow_policy.py(one inventory entry, one exemption),tests/test_workflow_security_coverage.py(coverage set and two zizmor tests),docs/decisions/2026-10-08-claude-actions-security-review.md(new),docs/decisions/2026-10-04-ci-least-privilege.md(two table rows and the exemption sentence edited in place, and a dated addendum),docs/github-automation.md(current-practice bullet),manifests/evidence.json(registration, last commit). R5 touches the review workflow, its test module, the record, thedocs/github-automation.mdbullet andmanifests/evidence.json.R5 (2026-10-09), at 5903cee
R5 makes the changes asked for by the command center's security read and the GPT designated read of f5f1fd2, and by the command center's caps from measurement, all on 2026-10-09. It is limited to quality and correctness.
claude-security-review.yml:find pr-head -path pr-head/.git -prune -o -type l -exec rm -f {} +underset -euo pipefail. A link in the head to/proc/self/environor../.git/configcould otherwise carry what it points at into the review and the public job summary.--restrictedconfines the file tools to the working directories, andclaude --helpon 2.1.295 does not say whether a link's target is resolved before that check or the deny rules apply. The step's comment says why it is there.{"name":"Bash"},nullor17pass every bound in a synthetic transcript. Each such entry is now a forbidden tool, namednon-string tool entryinusage.json, in the summary's tools column and in the bounds message; string entries are checked against Read, Glob and Grep as before, and the step's comment names the new case.--max-turnsgoes from 12 to 30 and--max-budget-usdfrom 3 to 5. The numbers step's bounds become 1 to 30 assistant turns and at most $5, the prompt says "at most 30 turns", and the opening comment carries the new values. A cap bounds a runaway; it never trims a normal run. This review has no runs of its own yet and takes the caps of the pull request review (Add an on-demand, read-only pull request review by the Claude action #894), which has the same fence, model and effort with a different prompt. Add an on-demand, read-only pull request review by the Claude action #894's prompt andclaude_argsreading one pull request in full used 12 of 12 assistant turns and cost $2.23, so 12 is a floor: 2.5 times it gives 30 turns, and twice $2.23 is about $4.5, rounded up to $5. Four delta re-reads took 5 to 7 turns and $0.73 to $1.75. Expected cost per run: about $2, by that shape, not yet measured.tests/test_claude_security_review_workflow.py(36 to 42 tests):test_symbolic_links_in_the_head_are_removed_and_nothing_elsetest_the_symbolic_link_step_has_no_condition_and_a_fixed_scripttest_claude_args_are_exactly_the_reviewed_listtest_settings_are_exactly_the_reviewed_jsontest_a_tool_entry_that_is_not_a_string_is_a_forbidden_tool, which refuses{"name":"Bash"},nulland17test_the_caps_hold_at_the_bound_and_fail_just_above_it, where 30 turns and $5 pass and 31 turns and $5.01 failtest_steps_run_in_the_order_the_binding_depends_onrequires the new step right after the head checkout.test_no_step_executes_anything_from_the_pull_request_headexempts that step by name, pins its script, and checksworking-directoryon every step.test_claude_has_three_read_tools_and_fixed_boundschecks 30 and 5, and the prompt's turn count.test_an_unmet_bound_fails_after_the_numbers_were_kept,test_the_step_names_every_unmet_boundandtest_the_turn_bound_counts_assistant_turns_not_transcript_messagesnow go just past the caps (31 turns, $5.01 or $5.50).STRIP,STRIP_RUN,RUNNER_TEMP_STAND_IN,SETTINGS,CLAUDE_ARGSandclaude_args().docs/decisions/2026-10-08-claude-actions-security-review.md:claude_argsbullet.max.--setting-sources user, in two of its three runs. The workflow's model-free steps have run locally under the unit tests. The action step's own configuration has not run, locally or hosted.docs/github-automation.md: the "Security review on demand" bullet now says 30 turns and a $5 client budget.manifests/evidence.json: the four changed rows are rehashed.security-review-flag.yml: unchanged.originURL with the job token (src/github/operations/git-config.ts:129-134, called fromsrc/modes/agent/index.ts:52-61at2dca132f), sopersist-credentials: falsedoes not keep the token out of.git/config; the deny rulesRead(./.git/**)andRead(./**/.git/**)are what keep the model from reading it, as two 2026-10-09 probes on the installed 2.1.295 measured."tool_usenames remain.--setting-sources userbeside--restrictedis pinned as it stands, pending Add a weekly lane triage: the model proposes, a model-free job applies #896's ruling on that pair.R6 (2026-10-09), at e57d8eb
!cancelled() && steps.numbers.outcome == 'success'.--max-turns 30removed: the pinned action (base-action/src/run-claude-sdk.ts:241-250at 2dca132f, since 6ef6450f) fails a success whosenum_turnsexceeds it. On Claude Code 2.1.295num_turnscounts transcript messages: 57 for 12 requests under--max-turns 12. That is a code reading plus a local measurement, not a hosted run. The numbers step's 1-to-30 assistant-turn check and the prompt's "at most 30 turns" stay.validate.pyand the manifest check pass (10,906 files);R7 (2026-10-09), at 02b80c6
--max-turnsremoved in R6, the numbers step's 30-assistant-turn bound is the turn limit, and 30 turns at the measured pace (12 turns in 512,670 ms) take about 21.4 minutes before setup. A new test pins 30, and a weakened copy set back to 20 fails it. Bound the harness audit and move it to claude-code-action v1.0.247 #892 and Add an on-demand, read-only pull request review by the Claude action #894 use 30.validate.pyand the manifest check pass; 264 tests across six modules, OK; this module, 45 tests; actionlint and zizmor are clean; 36 of 36 weakened copies fail a test with PyYAML.SOTA sources
2dca132ff0e0c4094ce6048b422c6915a071210b(https://github.com/anthropics/claude-code-action/tree/2dca132ff0e0c4094ce6048b422c6915a071210b):docs/solutions.md("Security Review" example, the source of the general checklist),docs/security.md("Do not check out an untrusted ref into the workspace root before this action"; base ref at the root and head ref in a subdirectory),docs/setup.md(federation inputs; "a static credential takes precedence and federation will not be used"),action.yml(inputs and theexecution_fileoutput),base-action/src/parse-sdk-options.ts(argument parsing withshell-quote, default setting sources, debug forcing full output),src/github/operations/git-config.ts(git authentication in the root checkout; lines 129-134 rewrite theoriginURL with the job token),src/modes/agent/index.ts(lines 52-61: agent mode calls that git configuration). R5 read the last two at v1.0.246 (38c80c1); per the command center's compare, v1.0.247 changes only the bundled Claude Code version, so neither file differs at the pin.claude --help:--restricted("removes the built-in tools that run commands or code ... and ignores user, project and local settings files ... Also confines the file tools to the working directories (--add-dir included)"; the help says nothing about symbolic links, read 2026-10-09),--permission-prompts none("anything that would prompt is denied automatically"),--tools,--settings,--strict-mcp-config,--max-budget-usd.find4.10.0, as installed where the R5 tests ran:-path ... -prune,-type land-exec ... {} +, the semantics the symbolic-link step relies on.0c6a49f1fa56a1d472575da86a94dbc1edb78eda, its only pin (no release or tag; last commit 2026-02-11), read and not adopted:action.yml(API key only),claudecode/constants.py(defaultclaude-opus-4-1-20250805),scripts/comment-pr-findings.js(submits a PR review),README.md("not hardened against prompt injection attacks"); Anthropic, Model deprecations (https://platform.claude.com/docs/en/about-claude/model-deprecations): that model retired 2026-08-05.Evidence-class table
--restricted,--tools/--allowedToolsRead,Glob,Grep,--strict-mcp-config,--permission-prompts none,--settingswith hooks off and three deny rules; Haiku 5.5, 10 turns, $0.50, as the receipt lists; two of its three runs also passed--setting-sources,run2-userwithuser) give a read-only session (tools Glob, Grep, Read; reads outside the tree and of.git/configdenied; planted hooks and instruction files without effect)native_provenon the installed 2.1.295 client (not through the action, not on a GitHub runner)evidence/artifacts/claude-actions-fence-smoke-20261008/receipt.json, carried byte-identical by this PR, #892 and #894--add-dir,--effort max,blockReadsOutsideWorkingDirectoriesand four more deny rules; Opus 5.5, 30 turns, $5; through the action) holdssynthetictests/test_claude_security_review_workflow.py, 42 tests; each step's shell is taken from the workflow file and executed. Without PyYAML the 19 shape and flag tests skip and the 23 step tests runpr-head/are gone before the model starts, while regular files andpr-head/.gitstaysynthetictest_symbolic_links_in_the_head_are_removed_and_nothing_else(links to/proc/self/environ, to../.git/configand from a subdirectory);test_the_symbolic_link_step_has_no_condition_and_a_fixed_scriptsynthetictest_a_tool_entry_that_is_not_a_string_is_a_forbidden_tool:{"name":"Bash"},nulland17each exit 1 withnon-string tool entrysynthetictest_the_caps_hold_at_the_bound_and_fail_just_above_itclaude_argson the installed 2.1.295 client, not through the action; applied to this workflow by task shape, not measured on itsyntheticlocal_integrationactionlint1.17.0 exit 0;zizmor1.30.1 offline, regular (--no-ignores) and pedantic, no findings (2 suppressed), at R5Local commands run
Hosted checks at 5903cee: not read yet.
Decision record
docs/decisions/2026-10-08-claude-actions-security-review.md(why notanthropics/claude-code-security-review, why the flag and the review are separate, the prompt, the caps and their derivation under "Cost of one run", changed test contracts, alternatives, what would overturn it, and the dated R4 and R5 sections).docs/decisions/2026-10-04-ci-least-privilege.mdlists the review job in its write-grant table and both workflows in its per-workflow table, names each exempt job in its federation-exemption sentence, and gets a dated addendum naming this job on the federation rule.Host evidence
Not applicable: no file under
evidence/hosts/changes.Checklist
version comment (no floating tags).
permissions: {}and grant each jobonly what it needs (
contents: read, or an explicitly justified addition).added without a documented owner.
moved or overwritten).
Activation is a separate step after landing: an owner sets
CLAUDE_SECURITY_REVIEW_ENABLEDtotrue; the flag then names the dispatch command on each matching pull request. Runs authenticate by workload identity federation and spend from the Console organization the fourANTHROPIC_*repository variables name; no Anthropic key is stored in GitHub. Each run is bounded by the $5 client budget and 30 assistant turns, the owner's per-run spend limit once the variable is set. The review workflow and its tests share their guard and accounting steps with #894 (on-demand pull request review); this PR, #892 and #894 all edit the write-grant inventory and the evidence manifest, so each later one rebases once.🤖 Generated with Claude Code