Bound the harness audit and move it to claude-code-action v1.0.247

[seathatflowsinourveins]

Scope

• What this PR changes, in one or two sentences: harness-audit.yml moves from anthropics/claude-code-action v1.0.245 to v1.0.247 and becomes a bounded read-only job (Read, Glob and Grep under --restricted; at most 20 assistant turns, checked after the run; a $5 client budget; no write scope beyond the OIDC token; the report in the job summary from a model-free step; the run's token and cost numbers kept). It runs only while the repository variable CLAUDE_HARNESS_AUDIT_ENABLED is true, and only on the first attempt of a run, so merging this starts nothing.

• Effort (added 2026-10-08 at head a2f2570): --effort max in claude_args, because judgment work (designated reads, adjudication, pull request and security reviews, audits) runs at max under the command center's effort mapping, and every job records its level and reason. The level has to be in claude_args: --restricted ignores the settings files that would otherwise carry it, and Opus 5.5 runs at medium on the API when a request leaves effort unset. The bounds test now requires the flag; removing it fails the test. A loopback dry run of the installed 2.1.295 client with this workflow's claude_args sent output_config.effort: "max" on every request.

• Fixes from the pre-cue read (2026-10-09, head 5891534): a pre-cue read by Anthropic's pr-review-toolkit agents found six behaviour changes this description and the record did not name, and four overstated claims. The record's new section "Behaviour changes in this PR" now names every change against main: the timeout 45 to 30, the slug, first-attempt, variable and owner-only guards, the guard step's exit 2, the display name and prompt, no issue, the two explicit action inputs display_report and track_progress (defaults, declared in the pinned action.yml), and the 14-day usage artifact. The bounds step now uses an allow-list, fails on a start record without lists or on an empty result, and names each unmet bound. The report cut is character-safe and says when it cut. The step tests no longer skip without PyYAML. The stale issues: write lines in ci-least-privilege are fixed in place, and the 2026-10-03 cooldown waiver is cited.

• Pre-cue toolkit read of this PR (J8 on 4d0a14d, 2026-10-09; pr-test-analyzer and silent-failure-hunter): every finding at confidence 80 or above, and its status at head 725d25c.

  Finding	Status at 725d25c	Fixing commit and evidence
  pr-test-analyzer F1 (80): the runtime receipt is not "the same flag set", and one result is confounded	fixed	5891534 dropped "the same flag set"; 725d25c tables every flag difference, says the receipt's claudeMdExcludes means it does not show that --restricted alone keeps instruction files out, and limits native_proven to the receipt's flag set
  silent-failure-hunter 1 (85): the receipt does not test the flag set the workflow ships	fixed	725d25c, as F1; for the .git deny rules a local subscription probe (2026-10-09) measured Read and Grep into .git as refused, and Glob and Grep are measured as bound by the deny rules against a control run
  pr-test-analyzer F3 (90): "17 tests" is wrong	fixed	725d25c: 24, from Ran 24 tests (it had gone stale again at 89d05dd)
  pr-test-analyzer F4 (80): the description's evidence and metadata do not describe this head	fixed (description)	this description: base, checks and counts at 725d25c
  silent-failure-hunter 9 (90): description inaccuracies	fixed	this description, and the record's test count in 725d25c
  silent-failure-hunter 4 (85): the acceptance check fails with no explanation	fixed	5891534 names each failed bound; 725d25c adds the result subtype to the record, the summary and the message, so a turn-limit stop and a budget stop read apart (two new tests, seven mutants caught)
  silent-failure-hunter 8 (90): display_report and track_progress undeclared	fixed	5891534 (record), 89d05dd (test), 725d25c (action.yml lines in the sources)
  silent-failure-hunter 10 (95): timeout-minutes 45 to 30 undeclared	fixed	5891534 (record and description)
  silent-failure-hunter 11 (85): the job if gained two undeclared clauses	fixed	5891534 (record and description); 725d25c adds the slug, first-attempt and owner-only guards to docs/github-automation.md and says why the triggering-actor clause stays

  725d25c also names the execution-file step 89d05dd added (undeclared until now), and adds the LR entry fields Add an on-demand, read-only pull request review by the Claude action #894 cites (Make the credential guard hook fail closed #897 at 24821c5, two blocking findings, the report's sha256) as a marked 2026-10-09 amendment of local-parity-receipt.json.

• R4, debug logging set as a repository secret or variable (2026-10-09; the cc-native-practice lane's L3 read against anthropics/claude-code-action v1.0.247 practice): the guard binds (secrets.ACTIONS_STEP_DEBUG || vars.ACTIONS_STEP_DEBUG) == 'true' and (secrets.ACTIONS_RUNNER_DEBUG || vars.ACTIONS_RUNNER_DEBUG) == 'true' as booleans and refuses either before any token is requested; a test pins both bindings, and six weakened copies each fail at least one test. This is hardening: the action step already pins ACTIONS_STEP_DEBUG: 'false', which is what the action's full-output switch reads. zizmor's auditor persona reports secrets-outside-env on the two reads; CI runs the regular persona with --no-ignores, which reports nothing. The R4 lines are byte-identical in Bound the harness audit and move it to claude-code-action v1.0.247 #892, Add an on-demand, read-only pull request review by the Claude action #894, Add an on-demand Claude security review and a model-free flag on risky paths #895, Add a weekly lane triage: the model proposes, a model-free job applies #896 and Add a pre-cue read by Anthropic's pr-review-toolkit agents in Actions #909.

• The report notice at the cap (2026-10-09; the J8 micro read of Bound the harness audit and move it to claude-code-action v1.0.247 #892): the publish step wrote with jq -r, which adds a newline, so a 60,000-byte report was announced as cut. It writes with jq -j now; a test publishes exactly 60,000 and 60,001 bytes, and a copy that writes with jq -r fails it.

• From the J8 micro read of 61494a3 (2026-10-09; 7 of 9 listed findings fixed, F4 and SFH 9 partly, both about this description, which this refresh makes current): the deny-rule probe and its control are kept as deny-probe-receipt.json (flags, client, model, authentication, every call and its outcome, each stream's sha256), so the Glob and Grep finding no longer rests on runs outside the repository; the record states that the workflow is unchanged from e48de2e6 to the merge base aba02ec3; and the step comment now says the last non-empty result text is published.

• R5 (2026-10-09, head e5ff29f), quality and correctness plus the approved caps, from the command center's security read of 0be47ac, a designated GPT read of Add an on-demand Claude security review and a model-free flag on risky paths #895 (P2) and the command center's caps from measurement. The step "Keep the run's numbers and check the bounds" now binds the session's whole tool list as $listed and adds one non-string tool entry to forbidden_tools for each entry that is not a string. A list holding {"name":"Bash"}, null or 17 therefore fails with "Bounds not met: tools outside Read, Glob and Grep: non-string tool entry" instead of passing (GPT designated read of Add an on-demand Claude security review and a model-free flag on risky paths #895, P2). The client budget moves from $3 to $5 while the turn limit stays 20. That covers --max-budget-usd 5, the numbers step's cost check .total_cost_usd > 5 with the message "client cost estimate USD, above 5", the workflow header comment and the Harness audit entry of docs/github-automation.md. The caps follow the command center's measurement-based caps: the one measured run used 10 turns and $2.03; twice that is 20 turns and about $4.1, rounded up to $5; a run is expected to cost about $2. HarnessAuditShapeTests no longer skips without PyYAML: it reads the workflow with the policy test's strict loader. A new test, test_claude_args_and_the_settings_json_are_pinned_exactly, pins the whole claude_args token list and the whole --settings JSON. test_the_cap_never_splits_a_character_and_a_short_report_has_no_notice now requires exit 0 and the report published inside <pre>. test_the_turn_bound_counts_assistant_turns_not_transcript_messages now requires a run with no assistant message to fail with "0 assistant turns, outside 1 to 20". Two more tests are new: test_a_non_string_tool_entry_is_a_forbidden_tool and test_the_cost_bound_is_the_five_dollar_budget ($5 passes, $5.01 fails, named). The budget values in test_claude_has_three_read_tools_and_fixed_bounds, test_an_unmet_bound_fails_after_the_numbers_were_kept and test_the_step_names_every_unmet_bound follow the $5 budget. The module runs 29 tests with and without PyYAML, none skipped. The record gains a dated R5 section. It names every change, lists 17 weakened workflow copies and a control with the tests each one fails, and notes that the symbolic-link strip does not apply because no pull request head is checked out. It also carries the shared agent-mode origin URL sentence, the gap-5 note (binding the federation rule to job_workflow_ref is an owner Console decision outside this PR) and the follow-ups line. The record also gains the cap derivation under Cost of one run, $5 throughout its current-state text, a description of forbidden_tools in the by-name list, the test count 29 and the no-PyYAML run in Evidence class. The evidence manifest rows for the four changed files are rehashed. The guard step is unchanged from 0be47ac7. The record now gives the shape class's test count as ten (the nine that skipped plus the pin test), and withdraws its earlier dry estimate of $0.90 to $1.10 a run, which was below the measured $2.03 of a run of this shape; the measurement replaces it, and the Effort sentence that called the estimate a floor now points to the withdrawal. Security hardening from the 2026-10-09 read (debug-value widening, tool-list shape, extra deny rules, token-source and guard-step assertions) is filed as follow-ups before any enabling variable is set.

• Base commit: aba02ec3 (the merge base at 0be47ac, on which R5's changes were made; by 0be47ac main had moved to ebe6c223 with Split readiness G6 and record G5 closure criteria #890, docs(trading): revise R3 census to 2018-2023 #900, Make the credential guard hook fail closed #897 and Record G2 STARTED at 2026-10-08T23:02:18Z #907, and against the local origin/main ref, 861c475e, the only path both sides change is still manifests/evidence.json; the one rebase happens at landing)

• Lane: lane:foundation

• Owned paths touched: .github/workflows/harness-audit.yml, tests/test_claude_harness_audit_bounds.py (new), tests/test_workflow_policy.py (one inventory entry), docs/decisions/2026-10-08-claude-actions-harness-audit-bounds.md (new), docs/decisions/2026-10-04-ci-least-privilege.md (dated addendum only), docs/github-automation.md (current-practice bullet), evidence/artifacts/claude-actions-fence-smoke-20261008/receipt.json (new; the same file as in Add an on-demand, read-only pull request review by the Claude action #894), evidence/artifacts/claude-actions-fence-smoke-20261008/local-parity-receipt.json (new), evidence/artifacts/claude-actions-fence-smoke-20261008/deny-probe-receipt.json (new), manifests/evidence.json (registration; R5 rehashes the rows of the four files it changes).

R6 (2026-10-09), at bf34373

• Cost bound: the budget × 1.10 = $5.50, from four measured J8 runs on a $5 budget ($5.46, $5.33, $5.16, $5.0007), re-derived after the first three hosted runs. A budget stop at or under it publishes and is named. The publish step runs on !cancelled() && steps.bounds.outcome == 'success', because the action fails its own step on any budget stop.
• --max-turns 20 removed: the pinned action (base-action/src/run-claude-sdk.ts:241-250 at 2dca132f, since 6ef6450f) fails a success whose num_turns exceeds it. On Claude Code 2.1.295 num_turns counts transcript messages: 57 for 12 requests under --max-turns 12. That is a code reading plus a local measurement, not a hosted run, and upstream has no fix as of 2026-10-09. The bounds step's 1-to-20 assistant-turn check stays.
• Settings pin: compares canonical JSON, so 1 no longer passes for true.
• Record: the shared cost paragraph, and an upstream-issue draft (not filed).
• Local checks at bf34373:
  • validate.py and the manifest check pass (10,907 files);
  • six modules, 247 tests, OK;
  • this module, 30 tests with and without PyYAML;
  • actionlint and zizmor are clean;
  • every R6 weakened copy fails a test.
• Hosted checks at bf34373: not read yet.

SOTA sources

• anthropics/claude-code-action, v1.0.247 at 2dca132ff0e0c4094ce6048b422c6915a071210b (https://github.com/anthropics/claude-code-action/tree/2dca132ff0e0c4094ce6048b422c6915a071210b): action.yml (federation inputs, settings, claude_args, show_full_output, output execution_file), docs/setup.md ("a static credential takes precedence and federation will not be used"), base-action/src/parse-sdk-options.ts (full output is forced on when ACTIONS_STEP_DEBUG is true), base-action/src/setup-claude-code-settings.ts (a pre-existing settings file is logged), base-action/src/run-claude-sdk.ts (console output drops token counts).
• R5's citations at the same pin (2dca132f): src/github/operations/git-config.ts lines 129-134 (without allowed_non_write_users, the checkout's origin URL is rewritten to carry the job token), src/modes/agent/index.ts lines 52-61 (agent mode runs that git setup when neither commit-signing input is set) and src/modes/detector.ts lines 79-80 (a schedule or dispatch run falls through to agent mode). These three were read in the v1.0.246 tree (38c80c1a32cdbc0c6f34042464a0669d89363dde); the v1.0.246-to-v1.0.247 compare touches none of them.
• Release path read with gh api repos/anthropics/claude-code-action/compare/...: v1.0.245 to v1.0.246 and v1.0.246 to v1.0.247 are one commit each ("bump Claude Code to 2.1.294 / 2.1.295 and Agent SDK to 0.3.294 / 0.3.295"); the root action.yml is in neither diff.
• Claude Code 2.1.295 (the version the pin installs), claude --help: --restricted, --permission-prompts, --tools, --allowedTools, --max-budget-usd, --setting-sources, --strict-mcp-config, --add-dir, --settings; --max-turns is accepted by the same build's argument parser. anthropics/claude-code CHANGELOG.md, 2.1.295: "Fixed --tools and --restricted not applying to built-in tools that register after launch" (the reason for the newest pin); 2.1.259 (--permission-prompts none); 2.1.248 (--restricted).
• Anthropic documentation read 2026-10-08: How Claude Code uses prompt caching, "Cache lifetime" (https://code.claude.com/docs/en/prompt-caching); Pricing (https://platform.claude.com/docs/en/about-claude/pricing); Claude Code fast mode (https://code.claude.com/docs/en/fast-mode).
• GitHub, OpenID Connect reference, subject of a run on a branch and the job_workflow_ref claim (R5's note on the federation rule) (https://docs.github.com/en/actions/reference/security/oidc).

Evidence-class table

Claim	Evidence class	Command / receipt
With the fence flags (--restricted, --tools/--allowedTools Read,Glob,Grep, --strict-mcp-config, --permission-prompts none, --settings with hooks off and three deny rules; Haiku 5.5, 10 turns, $0.50, as the receipt lists; this workflow adds --setting-sources user, --add-dir, --effort max, three more deny rules and blockReadsOutsideWorkingDirectories, and does not set the receipt's claudeMdExcludes; the record tables every difference) the session's tools are Glob, Grep and Read; reads outside the tree and of .git/config are denied; planted hooks have no effect; because of claudeMdExcludes the receipt does not show that --restricted alone keeps instruction files out	native_proven for the receipt's own flag set on the installed 2.1.295 client (not this workflow's flags, not through the action, not on a GitHub runner)	evidence/artifacts/claude-actions-fence-smoke-20261008/receipt.json, three runs
The action's own parser turns this workflow's claude_args into those flags and that JSON	local_integration	the parser's three helpers replayed with shell-quote 1.10.0 on the workflow text
The guard, numbers and report steps behave as described on synthetic inputs	synthetic	tests/test_claude_harness_audit_bounds.py, 29 tests at e5ff29f, all of which run without PyYAML too (none skipped); the steps' shell is taken from the workflow file and executed
The tests detect a weakened workflow	synthetic	16 mutated copies on 2026-10-08 (that day's bounds), 16 failed at least one test; on 2026-10-09 three turn-bound mutants each failed, and the new bounds have their own tests; at 725d25c seven result-subtype mutants each failed at least one test; at e5ff29f 17 weakened copies (exact pins, the short report, the lower turn bound, non-string tool entries, the $5 cap) each failed at least one test with and without PyYAML, and an unmodified control failed none
A non-string entry in the session's tool list fails the bounds, named (it passed at 0be47ac)	synthetic	the numbers step's shell on synthetic execution files listing Read, Glob, Grep and {"name":"Bash"}, null or 17: at 0be47ac exit 0 with forbidden_tools empty; at e5ff29f exit 1 with "Bounds not met: tools outside Read, Glob and Grep: non-string tool entry" (test_a_non_string_tool_entry_is_a_forbidden_tool)
This workflow's own prompt and claude_args stay within bounds at --effort max on Opus 5.5	local_integration (installed client, billed to a second key; not through the action)	evidence/artifacts/claude-actions-fence-smoke-20261008/local-parity-receipt.json: 10 of 20 assistant turns, about $2.03 of the $3 budget it ran with, tools Glob/Grep/Read, no MCP; the same file records num_turns 57 for a 12-request run
The $5 budget is about twice the one measured run ($2.03, rounded up) and the 20-turn limit twice its 10 turns; a run is expected to cost about $2	local_integration (one run, as above)	local-parity-receipt.json; the command center's caps from measurement (2026-10-09); the record's Cost of one run, which also withdraws the earlier dry estimate of $0.90 to $1.10
In agent mode the action writes the job token into the checkout's origin URL, so persist-credentials: false does not keep it out of .git/config; the .git deny rules are what keep the model from reading it	source_review (the URL rewrite); local_integration (the deny rules, two local probe runs on 2.1.295, probe and control)	the R5 sources above; evidence/artifacts/claude-actions-fence-smoke-20261008/deny-probe-receipt.json
The workflow is well formed and has no audit finding	local_integration	at e5ff29f: actionlint 1.17.0 exit 0; zizmor 1.30.1 offline, pedantic and regular with --no-ignores, no findings (2 suppressed)
The repository's policy tests accept the change	local_integration	unittest run below
Pin, flags and settings keys exist upstream	source_review	the sources above; settings keys disableAllHooks, blockReadsOutsideWorkingDirectories found in the 2.1.295 build
A hosted run authenticates, stays in bounds and reads the cache	not established	no hosted run is part of this PR; run 37739403956 failed at the token exchange before this change

Local commands run

# at e5ff29fd: run on the R5 tree before it was committed
$ python3 scripts/validate.py
{"components": 70, "hashed_files": 10907, "profiles": 4, "receipts": 239, "status": "passed"}   exit 0
$ python3 scripts/evidence_manifest.py --check
{"files": 10907, "status": "passed"}   exit 0
$ uv run --quiet --no-project --python 3.12 --with pyyaml==6.0.3 python -m unittest tests.test_claude_harness_audit_bounds tests.test_workflow_policy tests.test_workflow_security_coverage tests.test_workflow_hardening tests.test_github_automation_practice tests.test_adoption_docs_consistency
Ran 246 tests ... OK (skipped=2)   exit 0
$ uv run --quiet --no-project --python 3.12 python -m unittest tests.test_claude_harness_audit_bounds   # no PyYAML
Ran 29 tests ... OK   exit 0
$ actionlint .github/workflows/harness-audit.yml
exit 0
$ zizmor --offline --no-config --persona=pedantic .github/workflows/harness-audit.yml
No findings to report. Good job! (2 suppressed)   exit 0
$ zizmor --offline --no-config --no-ignores --persona regular .github/workflows/harness-audit.yml
No findings to report. Good job! (2 suppressed)   exit 0
$ uv run --quiet --no-project --python 3.12 [--with pyyaml==6.0.3] python mutants.py   # scratch script outside the repository
control: 0 failing; each of 17 weakened copies fails at least one test, with and without PyYAML   exit 0

The two skips are outside this PR's module: test_workflow_hardening's hash-frozen exemptions test (no workflow is hash-frozen) and test_adoption_docs_consistency's release-notes cell check (no profile's coverage differs).

A wider run of the first commit before its rebase (base e48de2e6, the 66 test modules that read workflows, the automation doc or the evidence manifest, plus the new module): 4,493 tests, 2 failures, both in tests/test_token_e2e_grader.py, a file this PR does not touch. F27c_KeysOnRealContent.test_keys_at_the_pinned_revision failed on git clone --local across filesystems (the temporary directory was on tmpfs) and passes with the temporary directory on the repository's filesystem. F19_Mutants.test_M31_user_base_dropped_from_the_allowlist fails the same way on a clean checkout of main at 9f7b38b2 in the same interpreter environment. Hosted checks at e5ff29f: not read yet.

Decision record

docs/decisions/2026-10-08-claude-actions-harness-audit-bounds.md (what was wrong, the pin, each bound and its source, cost of one run with the cap derivation, changed test contract, alternatives, what would overturn it, and a dated R5 section that names every R5 change). docs/decisions/2026-10-04-ci-least-privilege.md gets a dated addendum: the audit job no longer holds issues: write.

Host evidence

Not applicable: no file under evidence/hosts/ changes.

Checklist

• New/changed GitHub Actions are pinned to a full commit SHA with a
  version comment (no floating tags).
• New/changed workflows declare top-level permissions: {} and grant each job
  only what it needs (contents: read, or an explicitly justified addition).
• No secrets are printed, logged or committed; no new required secret was
  added without a documented owner.
• No new paid hosting, subscription or billing surface was introduced.
• Peer-owned untracked files and worktrees were preserved (not deleted,
  moved or overwritten).

Activation is a separate step after landing: an owner sets CLAUDE_HARNESS_AUDIT_ENABLED to true. Runs authenticate by workload identity federation and spend from the Console organization the four ANTHROPIC_* repository variables name; no Anthropic key is stored in GitHub. Once enabled, each weekly schedule tick and each owner dispatch is one run, capped at $5 by the client's estimate and expected to cost about $2.

R7 (2026-10-09, head 0cf13fc)

Fixes from the J8 micro read of the R5 and R6 delta (0be47ac7..bf34373d):

• N1: a test pins the budget-stop exemption to the success and result-text checks only. A $5.20 stop that breaks the tool, MCP, turn, session or cache bound fails, named.
• N3: the budget-stop note moves to the publish step, so it is never written without a report.
• N2 and N4: record and comment text no longer describes a client turn limit.

N5 is a proposed follow-up; N6 is a corrected count; N7-N9 are dispositioned. 32 tests; the read's mutant and three more weakened copies each fail; validate rc 0; zizmor pedantic and actionlint rc 0.

🤖 Generated with Claude Code

[socket-security]

Dependency limit exceeded — report not shown.

This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report.

Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard.

Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-09T23:53:11.794226Z	b41b969	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b41b969a27

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".