Skip to content

AN-13 M4/M5: two-family lanes; M4 kept, M5 split after blind adjudication; judge-dispatch anti-pattern row - #460

Merged
seathatflowsinourveins merged 8 commits into
mainfrom
claude/an13-m4-m5-20260928
Sep 28, 2026
Merged

seathatflowsinourveins merged 8 commits into
mainfrom
claude/an13-m4-m5-20260928

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

  • What this PR changes: it records the second-family lanes for AN-13's single-family findings M4 and M5.
    • M4: both lanes reject the report's rewrite of the vendored verification-before-completion skill, so it gets
      no edit. The lanes did not test the skills trial's immediate-removal rule
      (2026-09-25-skills-trial-and-usage.md:284-286); the record says so, leaves that question open and makes a
      two-family finding of conflict an overturn condition.
    • M5 (blueprints/convergence-practice/application-delivery/AGENTS.md:15-18): both lanes amend it, but with
      different texts. One blind adjudication, in both orders by both families, split, so the lines are unchanged and
      both texts are recorded.
    • Anti-pattern row: this round built its judge audit from another round's audit instead of the lane-A template
      that the log names, and dispatched before that template's checks ran. Both Claude judgments were void on false
      positives the template already covers; the outcome does not depend on them.
  • Base commit: 3058b237.
  • Lane: lane:foundation. The paths are docs/ and evidence/artifacts/, plus the manifest re-registration,
    which does not make a PR shared (docs/lanes.md).
  • Owned paths touched:
    • docs/decisions/2026-09-28-an13-m4-m5.md (new);
    • evidence/artifacts/prompt-audit-an13-m45-20260928/ (new, 62 files);
    • one row in docs/harness-defaults.md, placed where no open PR edits the log;
    • manifests/evidence.json, re-registered in the last commit.

SOTA sources

  • M4, the pinned upstream skill (kept): obra/superpowers@8ca22dba skills/verification-before-completion/SKILL.md.
    • Its sha256 is 2befe7fc…, the pin's skill_md_sha256 in adoption/skills/manifest.json.
    • Upstream main served the same blob on 2026-09-28.
  • M4, upstream's latest change to that file: obra/superpowers commit 3be5aad3dd24
    (2026-07-24). It dropped the persuasion sections and kept the Iron Law, the gate function and the rationalization
    table: "This is the eval-gated class … evals on this branch decide".
  • M4, the concern's support: Anthropic, Prompting Claude Opus 5, "Task scope and over-verification",
    retrieved 2026-09-28.
  • M5, both lanes' prompting sources:
  • Method:
    • the convergence rule, docs/decisions/2026-09-27-prompt-audit-resolution.md:54-60 ("Rule"). The rule that a
      void judgment does not count is from the lane-A template's outcome-actions.md:9-12 and
      tools/sota-convergence/README.md:1503-1505;
    • the packaged GPT-6 runner, tools/sota-convergence/landscape-sweep/codex_call.sh (codex-cli 0.157.1,
      gpt-6-astra, effort max);
    • the blind-adjudicator agent, .claude/agents/blind-adjudicator.md;
    • the lane-A judge-audit template the log names, evidence/artifacts/prompt-audit-20260927/lane-a/final/adjudication/
      (its self-test and root scan, run afterwards against this round's audit);
    • the anti-pattern log rule in docs/harness-defaults.md, "Record a correction the same turn".

Evidence-class table

Claim Evidence class Command / receipt
Both lanes returned reject for M4 (0.91, 0.87) and amend for M5 (0.97, 0.74), with the two texts quoted in the record native_proven (runs on this host, records retained) round1/returns/, round1/runner/
M5's adjudication: two valid GPT-6 judgments for different texts; both Claude judgments void; outcome split native_proven (runs), local_integration (audit and tally) adjudication/{gpt6,claude}.{AB,BA}.json, m45-mapping.json, audit-m45.json, tally-m45.json
Both Claude voids are false positives: "Codex lane" in the adjudicator's own definition, and evidence-reviewer in a listing of the export's agent files local_integration adjudication/audit-m45-windows.json (each hit's window, host details replaced)
The lane-A template's two checks, applied to this round's audit and patterns without the template's own scripts, both fail (the new row's check) local_integration (reproduction after the fact, repeated byte for byte) python3 -B scripts/reproduce_template_checks_m45.py <clone> 9f8db582 <empty dir> <out> exits 0; adjudication/template-checks-m45.json
The audit reproduces, and voids planted accesses. It is not the pre-dispatch control: it has no clean-run criterion, so it cannot catch false positives local_integration (after the judges) adjudication/audit-control-m45.json
The Claude lane and both Claude judges called only Read, Glob, Grep and one advisor each local_integration (transcript counts) tool-counts.json
The packet and the audit's scripts are unchanged since freezing local_integration sha256sum -c packet.sha256 in round1/; sha256sum -c frozen-sha256.txt in scripts/
M4's reasons (the pin, upstream's choice, the pinned preload, the scheduled comparison) source_review decision record, "M4"
No model behavior changed or was measured — (nothing claimed) decision record, "Limitations and residuals"

Local commands run

On a35194e8 (the repair round, a wording fix and the M4 limitation, re-registered on 3058b237). The full suite
ran on 4341285d. After it, c94c87aa and 968e6f9a change only prose, the reproduction script's docstring and
its output's note; e3791344 and a35194e8 re-register.

$ python3 -m unittest tests.test_adoption_docs_consistency        # after the row's rewrite
Ran 37 tests … OK (skipped=1)
$ python3 scripts/validate.py                                      # after re-registration
{"components": 69, "hashed_files": 7726, "profiles": 4, "receipts": 163, "status": "passed"}
$ bash ci_local_fast.sh <worktree> <logs>                          # validate.yml's local steps, in CI order
workflows-sha256, workflows-contracts, zizmor, validate, host-receipts, release-due, validate-catalogs,
validate-foundation, landscape, audit-reports, validate-convergence, build-verdicts, component-matrix,
grand-list, verdict-flip, gap-crosswalk, gap-wave-ledger, build-ecosystem, skill-contract: all exit 0
$ python3 -m unittest                                              # full suite, on 4341285d
Ran 6889 tests in 750.071s; FAILED (failures=1, errors=1, skipped=765)
  ERROR: tests.test_order_throughput.CapacityRunTests.test_cli_refuses_live_base_url_from_env_file_without_network
  FAIL: tests.test_gpt6_family_tiering_20260926.RunnerTests.test_sigterm_records_the_running_call_as_interrupted_without_counting_it
# the same two as on the first head; both fail identically when run alone on a clean detached checkout of
# 3058b237 (unmodified main): this host has a /tmp/.git directory, and both tests refuse temp paths inside a repository
$ python3 -B scripts/reproduce_template_checks_m45.py <worktree> 9f8db582 <empty dir> <out>
{"payloads_void": [true, true], "gpt6_clean": true, "path_hits": 8, "text_hits": 242, "reproduced": true}
exit 0; this run wrote adjudication/template-checks-m45.json, and a repeat of the earlier version was byte-identical
$ (cd evidence/artifacts/prompt-audit-an13-m45-20260928/round1 && sha256sum -c packet.sha256)
packet.md: OK
$ (cd evidence/artifacts/prompt-audit-an13-m45-20260928/scripts && sha256sum -c frozen-sha256.txt)
void_patterns_m45.py: OK / audit_m45.py: OK / make_m5_adjudication.py: OK
$ pre-push scans: added lines and messages for UUIDs, home paths, the host user name, emails, session scratch,
  secret-like strings and agent ids: 0 each; validate.py's PRIVATE_CONTENT patterns: 0 hits in the 64 files;
  client-configuration text in the package: one hit, `.mcp.json` named as a scan category in the review brief,
  with no configuration quoted

Decision record

docs/decisions/2026-09-28-an13-m4-m5.md. It gives:

  • the decision per item, both M5 texts verbatim, the method and checks;
  • the audit's false positives, the skipped template and its checks run afterwards;
  • usage, with each provider's counters separate;
  • alternatives, overturn conditions and limitations.

Review

One review per family ran on the local head 31ab17f7, before the first push. One repair round followed: 297709c6,
re-registered in 4341285d. A wording fix found before merge followed: c94c87aa, re-registered in e3791344. Then a gap raised after the
reviews: 968e6f9a, re-registered in a35194e8. Each finding and what happened to it is in
review/README.md, with both prompts, returns
and usage.

  • GPT-6: codex exec --json -s read-only -m gpt-6-astra -c model_reasoning_effort="max" -c web_search="live",
    14:48:54Z to 15:03:54Z. Changes-needed: four should-fix findings and two nits, all on the record's wording. They
    covered a lane attribution, the rule's citation, the freeze-time evidence, what the judges' inputs held, a receipt
    citation and a checksum invocation. It confirmed the outcomes, the recomputed tally, the other repository
    citations, the upstream pin and commit, the usage figures, the sent hashes and the registration.

  • Claude: one evidence-reviewer (Opus 5.5, effort max), given the command results. Changes-needed: six
    should-fix findings and three nits. The two main ones:

    • the round adapted X9 round 3's audit instead of the lane-A template that the log names, which already covers both
      false positives;
    • the row's "Where enforced" cited a control that cannot fail on this mistake.

    It confirmed the outcomes, that the outcome does not depend on the voids, the figures, the inputs' symmetry,
    privacy and the registration.

  • Repairs: every finding was fixed. The row now names the template, and its check is the template reproduction
    above. A check before merge found that the row and the record said the template's checks were run; the
    reproduction applies them without running the template's own scripts, and the wording now says so. The hit windows and tool counts are published. My own check after the repair found one more defect: the
    record cited the void rule to lines that do not state it. It now cites that rule's own sources.

  • The M4 gap: the packet did not include the skills trial's rule that removes a trial skill whose instructions
    conflict with CLAUDE.md or AGENTS.md, and neither return mentions it. The record now says M4 covers the report's
    rewrite only, and adds a two-family finding of conflict as an overturn condition.

  • Residuals: no second review round ran, so the repairs had no independent review. The adjudication packet is
    kept as sent, with its wrong receipt.json:88 premise; the record's limitations explain it.

Host evidence

Not applicable: nothing under evidence/hosts/ changes.

Checklist

  • New/changed GitHub Actions are pinned to a full commit SHA with a
    version comment (no floating tags). No Actions change.
  • New/changed workflows declare top-level permissions: contents: read
    (or a narrower, explicitly justified addition). No workflow change.
  • No secrets are printed, logged or committed; no new required secret was
    added without a documented owner.
  • No new paid hosting, subscription or billing surface was introduced.
  • Peer-owned untracked files and worktrees were preserved (not deleted,
    moved or overwritten).

🤖 Generated with Claude Code

Scout and others added 2 commits September 28, 2026 10:46
M4 (the vendored verification-before-completion skill's wording): GPT-6
and Claude both reject the report's rewrite, so it gets no edit. The pin,
upstream's own 3be5aad3 choice and the reviewed preload stand; the
project-side remedy stays with the scheduled 2026-10-25 comparison.

M5 (application-delivery/AGENTS.md:15-18): both lanes amend with
different texts. The blind adjudication, in both orders by both
families, splits, so the lines stay unchanged and both texts are
recorded.

The adjudication's void-pattern control ran after the judges rather
than before dispatch; the new anti-pattern row records that, and the
outcome does not depend on the two false-positive voids.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Takes main's manifest at 3058b23 and registers the 48 files this
branch adds or changes (docs/lanes.md registration procedure);
component_matrix and new_host_grand_list are unchanged; validate.py
passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Scout and others added 6 commits September 28, 2026 11:38
Fix the record's rule citation (void rule cited to its own sources), the
hash-check attribution, the freeze-time evidence, the judges' inputs, the
receipt.json:88 premise and the packet's wrong summary, and the checksum
invocation. The anti-pattern row now names the skipped lane-A template, and
reproduce_template_checks_m45.py shows both template checks fail on this
round's audit and patterns. Publish the hit windows, the Claude runs' tool
counts and both reviews with each finding's disposition.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The reproduction applies the lane-A template's self-test payloads and a root
scan like the template's to this round's audit and patterns; it does not run
audit_selftest_final.py or root_scan_final.py. Say so in the row, the record,
both READMEs and the script, and regenerate its output (only the note
changes). Note that the reviewed head stays reachable from refs/pull/460/head
after the squash merge.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The skills trial removes a trial skill at once if its instructions conflict
with CLAUDE.md/AGENTS.md (2026-09-25-skills-trial-and-usage.md:284-286). The
M4 packet did not include that rule and neither return mentions it. Record the
gap as a limitation, note that the 2026-10-25 deferral does not answer it, and
add a two-family finding of conflict as an overturn condition.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@seathatflowsinourveins
seathatflowsinourveins merged commit 8d8f79c into main Sep 28, 2026
26 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/an13-m4-m5-20260928 branch September 28, 2026 16:26
seathatflowsinourveins pushed a commit that referenced this pull request Sep 28, 2026
#460's record now says the M4 lanes did not test the skills trial's rule that
removes a trial skill whose instructions conflict with CLAUDE.md or AGENTS.md,
and leaves that question open. Say so where this record points to it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins pushed a commit that referenced this pull request Sep 28, 2026
…dence.json

Main moved to 8d8f79c (#460) during repair r2. The rebase took main's
manifests/evidence.json at each of its three stops, the only conflicting
file, and dropped the now-empty 534af74. This commit re-registers the
PR's 13 hash-registered files at the tip. validation.json records the new
base, the merged docs/harness-defaults.md input hash and the checks rerun
on the rebased tree; research.md names the second rebase.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins added a commit that referenced this pull request Sep 28, 2026
…CLAUDE.md, 4 of 4 blind judgments; dashboard manifest bound (#444)

* AGENTS.md: scope role dispatch to new or ad-hoc stages; current-state holdout wording

Two items of the 2026-09-27 prompt audit, resolved by cross-family convergence
(docs/decisions/2026-09-27-prompt-audit-resolution.md, lane:foundation PR):

- Role dispatch applies to each new or ad-hoc workflow agent() stage; the saved
  scripts vendored in examples/claude-native/workflows keep their reviewed routing,
  byte-identical to agent-lab, as that README states in its opening paragraph and the
  Workflow contract's Dispatch by role bullet (F1).
- The inspected 2021 control segment: "has been inspected, so no experiment may
  present it as a fresh untouched holdout" (F4).

o200k (gpt-tokenizer 3.4.0): AGENTS.md 2,374 -> 2,401 tokens (+27).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* X9 second round: blind two-family re-judgment splits again; CLAUDE.md unchanged

After round 1 split 2 and 2 on X9 (CLAUDE.md:3-4), a second round added:
- the installed client's own refusals of /context and /mcp through the Skill
  tool;
- an executed comparison of the three texts, all headless claude -p runs
  through promptfoo 0.123.1: K1-K3 as 27 preregistered runs on Opus 5.5, and
  K4 as 9 runs on Sonnet 5;
- 38 dated sources, plus 5 recorded searches that found nothing.

Attempt 1 was void, because its judges could read this record and the
coordinator's work directory. Attempt 2 was final. Its judges read a plain
export of ba1700a and judged in both orders: blind-adjudicator agents and the
packaged GPT-6 runner. A void audit, hashed before any return was read, found
no voiding hit.

Both GPT-6 judgments chose the GPT-6 lane's text, and both Claude judgments
chose the Claude lane's text, so under the unanimity rule nothing is applied.
All eight judgments across both rounds reject the current line. The addendum
records both positions and the comparison that would decide it.

Review:
- One round, by GPT-6 and Claude reviewers, then one repair round:
  - disclose that K4 ran on Sonnet 5 while the packets said Opus 5.5;
  - make check_orders.py exit 1 on a difference, and add retally.py;
  - correct the counts;
  - add receipts (run windows, settings file names per run, the K4
    skill-list sentences, the isolation receipt, a source supplement);
  - fix the wording.
- A separate re-check found all 13 items fixed and two new defects: an
  overstated "keeps only" claim and an overstated account of K4's origin.
  Both are fixed. The recovered first check_orders.py is kept to show its
  flaw.
- A pre-handoff review found that the published packets, prompts and K4 files
  quoted answer heads containing client configuration (an enabled-plugin
  entry, status-line wiring, plugin install records). Every published copy
  now withholds them. attempt2/sent-sha256.json keeps the hashes of the files
  as sent. This commit squashes the earlier ones, so no commit on the branch
  carries that text.

docs/harness-defaults.md gains three dated anti-pattern rows:
- a model scope taken from the preregistration instead of each run's
  recorded model;
- a check that reported a mismatch and exited 0;
- judge inputs published with answer text that quotes client configuration.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Prompt audit round 3: X9 applied to CLAUDE.md; lane A's S1 and X5b in AGENTS.md; AN-13 receipt

- CLAUDE.md lines 3-8 (X9): round 3 compared the two texts on the
  measure the second round named (native /mcp and /context refusal in
  print mode, with a promptfoo-graded, preregistered comparison) and the
  blind adjudication chose the GPT-6 lane's text in all four judgments.
- AGENTS.md:3 (X5b): the operator's 2026-09-28 top-rule paragraph, byte
  for byte; AGENTS.md:37 (S1): build a PR description from the pull
  request template. Both converged in lane A (agree / agree). X5a and X5c
  land in #458.
- AN-13: the headless /doctor prompt-audit run the 2026-09-28 community
  sweep left open. Facts of both runs in
  evidence/artifacts/prompt-audit-20260927/an13/ (the report text is not
  published); no edit made from it; each finding's disposition is in its
  README and in the decision record.
- Decision record and x9-round3/ evidence (preregistration, isolation
  receipt, fixtures, judges, usage).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Prompt audit: print-mode anti-pattern row (AN-13), line-7 co-change with X5b, round-3 token receipts

- docs/harness-defaults.md: a dated row for running a headless skill
  that starts background agents under print mode's default 600 s wait
  (AN-13's first run: four Explore agents stopped by the client, an
  interim note instead of the report); the rerun with
  CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 returned the report
  (code.claude.com/docs/en/env-vars).
- docs/harness-defaults.md:7 restates AGENTS.md's new top-rule heading
  (X5b), disclosed in the decision record.
- controls/token-counts-r3-{f508ffb,base,after}.txt: the o200k counts
  the record cites for CLAUDE.md (30 -> 96 at f508ffb; 96 -> 162 at
  fb14ded) and AGENTS.md (2,374 -> 2,484 at fb14ded).
- The record cites validate.yml's heading rule at both revisions and
  notes the Codex-lane host step done on this workstation after #458.
- x9-round3/README.md: the scan note names every kind of match (names,
  not values), including one judgment's quote of the untested cases.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Grand dashboard: the evidence manifest gets its own read bound

This pull request's registrations take manifests/evidence.json past the
2,000,000-byte bound progress.py applied to every source (1,987,653 bytes
on main at fb14ded; 2,017,827 with this branch), so the snapshot refused
it and eight dashboard tests failed. The snapshot reads only the
manifest's receipt count: SOURCE_MAX_BYTES gives the manifest 8,000,000
bytes and every other source keeps MAX_BYTES. read(root, relative) keeps
its signature (the tests patch it). The new test fails on the unchanged
reader (controls/dashboard-bound-before.txt) and passes with 16 others
(dashboard-bound-after.txt). Overturn: when the manifest nears the new
bound, count receipts from a smaller source instead of raising it again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Prompt audit round 3: review repairs, the audit's control, the Codex lane host receipt

One review round on 0779e91a (GPT-6 codex exec and a Claude evidence-reviewer, both
changes-needed); every finding was in the record or the evidence text:
- the record: --max-turns 10 with one 11-turn result; every builder change after the
  freeze; the applied X9 text ties with the line it replaced on the preregistered
  metrics; the runs used ba1700a's files; X5b's wording and the tests that read
  AGENTS.md; the advisor calls; M4 is project-side; a review subsection;
- x9-round3: audit_control_r3.py and judges/audit-control.json (clean copies reproduce
  the published verdicts; one planted access per judgment voids all four); the README
  lists every builder change, the judge-action flag and the tested configuration;
- codex-worker-lane-host-20260928: the 12:14-12:15Z dry run, apply and proof, sanitized
  as #406's were, and a recheck with exit codes and times (dry run: block in place;
  proof 7 of 7);
- review-444: both prompts, returns, usage and each finding's disposition.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Prompt audit: map the reviewed commits to the rebased ones

The branch was rebased onto 9f8db58 (#400) with every patch unchanged (git range-diff
shows each pair as =). review-444/README.md says 0779e91a, the reviewed head, was never
pushed and was 7e5eddcd plus a registration commit, and gives the rebased SHAs of
9e036e7d and 7e5eddcd; the decision record points there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Prompt audit: #446 carried the dashboard bound; map reviewed commits by subject

After the rebase onto 3058b23, the dashboard commit keeps only its
record subsection and the two receipts: #446 landed the same
progress.py and test hunks (two-file patch-id 8f693abd4c601190). The
review README now names the reviewed commits by subject, since each
rebase changes their SHAs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* AN-13: point M4 and M5 to their 2026-09-28 second-family lanes

M4: both lanes reject the rewrite (no edit; the preload stays with the
2026-10-25 comparison). M5: the lanes amended with different texts and
the blind adjudication split (unchanged). Both are recorded in
docs/decisions/2026-09-28-an13-m4-m5.md, a separate lane:foundation PR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* AN-13 M4 pointer: the trial's removal rule is open (#460)

#460's record now says the M4 lanes did not test the skills trial's rule that
removes a trial skill whose instructions conflict with CLAUDE.md or AGENTS.md,
and leaves that question open. Say so where this record points to it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Register the instruction files, the decision record, the anti-pattern rows and the prompt-audit evidence

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins pushed a commit that referenced this pull request Sep 28, 2026
research.md and validation.json record the second rebase, onto 8d8f79c
(#460), the merged docs/harness-defaults.md input hash and the checks rerun
on that tree. The manifests/evidence.json re-registration that was part of
this commit was dropped when the branch was rebased onto c1581fa, and is
redone at the tip.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins pushed a commit that referenced this pull request Sep 28, 2026
research.md and validation.json record the second rebase, onto 8d8f79c
(#460), the merged docs/harness-defaults.md input hash and the checks rerun
on that tree. The manifests/evidence.json re-registration that was part of
this commit was dropped when the branch was rebased onto c1581fa, and is
redone at the tip.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins pushed a commit that referenced this pull request Sep 28, 2026
research.md and validation.json record the second rebase, onto 8d8f79c
(#460), the merged docs/harness-defaults.md input hash and the checks rerun
on that tree. The manifests/evidence.json re-registration that was part of
this commit was dropped when the branch was rebased onto c1581fa, and is
redone at the tip.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins added a commit that referenced this pull request Sep 28, 2026
…l manifest (#429)

* Runtime-worker skills program: project-scoped installs and a 138-skill pinned manifest

GPT-6 (astra, max) build. install_skills.py gains --project-dir and --agent
targets with the default global behaviour unchanged, project lock identity and
pinned source-tree checks, and scoped rollback. The runtime-worker manifest pins
138 skills (77 OpenHands extensions v0.25.0, all 15 superpowers lifecycle
skills, 28 existing adoption entries by reference) with a report-only freshness
checker and scheduled workflow. Not installed into any worker yet; invocation
and pruning are measured after install.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Runtime-worker skills: repair round for the independent Claude review

GPT-6 (astra, max) repair of all seven findings; the blocker (#407's scoped
global rollback) was already resolved at the rebase and is kept byte for
byte. Project rollback now removes exactly the add targets and checks the
canonical folder, project lock entry and Claude link independently,
reporting a retained copy as an error instead of widening the deletion
(skills@7407f389 src/remove.ts:209-333). A project .claude/skills/<name>
that is not a symlink to the canonical folder is local-modified unless
--force (installer.ts:254-264). Every selected source/ref is fetched before
any add; an unavailable gh is reported as unverified, not as a mismatch.
The pruning rule records Claude Code's skill-listing budget and treats
dropped descriptions as unknown. 52 tests pass; five fail-first runs are
recorded in validation.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* CI fix: cover the skills freshness workflow and classify the repair values

Found and fixed through the upstream gh-fix-ci skill (openai/skills@49f948fa)
on the GPT-6 lane, then reviewed by the coordinator:
- tests/test_workflow_security_coverage.py lists
  runtime-worker-skills-freshness.yml; the workflow itself already passes the
  hardening, pinning, permissions and zizmor checks.
- blind_checkout: the review disposition "fixed" joins the closed-vocabulary
  labels (as in #416), and the superseded rollback assertion's disposition is
  data.
- harness-defaults anti-pattern row: run acceptance with TMPDIR outside every
  repository.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test_install_skills: compare resolved paths on both sides for macOS temp dirs

On macOS the temporary directory sits under /var, a symlink to /private/var,
so local.resolve() returned the /private/var form while the expected path kept
the /var form. validate-macos failed 8 subtests of the two new project-install
tests (run 36349458060). Resolve the expected path too, as
tests/adaptive_paper_hermetic.py already does with os.path.realpath.

Reproduced on Linux with TMPDIR set to a symlinked directory: 8 failures
before, OK after; the module passes 38 tests with a plain or symlinked TMPDIR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Runtime-worker skills repair r2: exclude main's gated security-audit, take reused gates from main

Coordinator direction on the #429 blocker (option b): runtime workers must
not bypass a gate set on main.

- Contract: every skill in adoption/skills/manifest.json is either reused
  by reuse_ref (pins equal to main's, gates not restated) or listed in
  `excluded` with an adoption_ref, a reason and an overturn condition.
  security-audit (#448, 8315274: Codex off, name-only pending the M5c
  bake-off against /security-review) is excluded, and a test fails as soon
  as main's gate on it changes. A synthetic future main skill in neither
  set fails the contract (negative control).
- Gates: install_skills.py resolves each reuse_ref when it reads a
  manifest, refuses an entry whose pin drifted or that restates
  codex_enabled/claude_listing, and takes both gates from main.
  --print-codex-config on the runtime manifest now disables the 16 reused
  skills main keeps off for Codex; before, it printed none. The README
  states that claude_listing is a Claude Code skillOverrides setting that
  the OpenHands SDK does not read (OpenHands/software-agent-sdk@fcc102a6
  skill.py:1123-1130, 182-185, 278-284, 528-538; same blob at da28c773).
- --only naming a pruned project skill fails as pruned, not unknown.
- The README documents that --check-only needs the pinned skills binary,
  and gh sign-in and network in project mode.
- Counts: 138 selected skills from 12 sources (unchanged); 28 of main's 29
  adoption skills reused; 8 exclusions (7 OpenHands, 1 adoption).
- validation.json records the round (fail-first: 9 tests, failures=7,
  errors=2; after the fix: 9 OK) and refreshes every input hash; the
  anti-pattern log gains the dropped-gates row; the 8 changed files are
  re-registered in manifests/evidence.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Runtime-worker skills: narrow the OpenHands loader claim and the security-audit overturn (R2-6)

The README said OpenHands withholds a skill from the model only through
SKILL.md disable-model-invocation, and research.md said the SDK reads only
SKILL.md frontmatter. load_user_skills also loads OpenHands' own enabled
installed skills (skill.py:936-973 at fcc102a6). Both now say the loader
reads skill directories, SKILL.md frontmatter and its installed-skill enable
state, never Claude Code settings or a skillOverrides key. The
available_skills citation moves from lines 182-185 to 183-186.

The security-audit overturn now names a promotion as codex_enabled true or
claude_listing on; any other change to main's gate still fails the
exclusion test and needs a new review. The test message says so.

validation.json records R2-6, the corrected sources, the red excerpt in log
order, refreshed input hashes and the rerun checks; the five changed files
are re-registered in manifests/evidence.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Runtime-worker skills records: note the rebase onto main 8d8f79c

research.md and validation.json record the second rebase, onto 8d8f79c
(#460), the merged docs/harness-defaults.md input hash and the checks rerun
on that tree. The manifests/evidence.json re-registration that was part of
this commit was dropped when the branch was rebased onto c1581fa, and is
redone at the tip.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Runtime-worker skills repair r3: containment and pin checks before add, project scope, name-keyed reuse

Two independent reviews of 8efdc35, a GPT-6 cross-family review
(codex_call.sh, gpt-6-astra max) and a headless Claude session review,
returned changes-needed with no blocker or high finding. Each code item
failed first against the unchanged code:

- GPT6-C1: refuse a project whose .agents/skills, .claude/skills or
  skills-lock.json passes through a symlink, whose selected skill paths
  resolve outside it, or that is --home, before any skills or gh call.
- CL-D1: compare every selected pinned tree with its tree_sha in the
  preflight and refuse before any add; the native in-use guard keeps a
  rolled-back copy while a detected agent such as Codex reads it.
- CL-D3: the runtime manifest carries "scope": "project" and is refused
  without --project-dir; pruned entries are never installed in either mode.
- CL-D6: reuse_ref and adoption_ref name the adoption manifest and match
  by skill name.
- CL-D7: runtime_skill_freshness.py reads the CLI version and ref from
  the manifest's cli pin.
- GPT6-C5: offline zizmor analysis of the freshness workflow, with
  floating-tag, persist-credentials and permissions controls.

Docs: the rendered OpenHands skill list omits <location>; disabled_skills
and the loading flags are its controls; third-party instruction files load
from the work directory and Git root, so the example worker projects move
outside the checkout; rollback retention is certain when Codex is detected;
skills-review-r1.json is labelled unpublished.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Runtime-worker skills repair r4: exclude verification-before-completion with main (M4, #464)

Main c0966da (#464) moved verification-before-completion from the
adoption skills to its excluded list under the skills-trial conflict
rule M4. On the rebased tree the unchanged coverage contract and
installer refused the runtime manifest's reuse_ref for it (67 tests,
6 failures, 2 errors).

- Exclude the skill in the runtime manifest with its reviewed pin, the
  M4 and AGENTS.md:16 reason, and the overturn "main re-admits it"; it
  carries no adoption_ref because main no longer lists it in skills.
- New test_verification_before_completion_stays_excluded_until_main_readmits_it
  and the revised superpowers lifecycle test failed first; a source-count
  test ties sources and the research table to the selected skills.
- Recount: 137 skills from 12 sources (superpowers 14), 27 reused
  (16 Codex-disabled on main), 9 exclusions, 15 empty cells (new
  github-ci-fix/extraction-caller gap), 1,020,955 bytes and 36,739
  description characters.
- validation.json: repair_r3.not_relayed becomes the five confirmed
  CONFIRM ids; repair_r4 records the rebase, sources, fail-first and checks.
- State Codex detection precisely ($CODEX_HOME, ~/.codex when unset, or
  /etc/codex) in research.md, the harness-defaults row and a code comment.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Rebase onto main c0966da: re-register the PR's files in manifests/evidence.json

Main's evidence.json with the 13 PR files registered at their current
sha256 and byte size (8 added, 5 updated); no other entry changed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant