Repository navigation
Skills trial: remove verification-before-completion under the conflict rule (M4), #381 Amendment 3 - #464
Merged
seathatflowsinourveins merged 3 commits intoSep 28, 2026
Conversation
…t rule (M4), #381 Amendment 3 The pinned SKILL.md (obra/superpowers@8ca22dba) says "Previous run" is not sufficient and requires a fresh full re-run "in this message"; AGENTS.md:16 says to reuse passing evidence whose inputs still match. The trial's in-force rule (skills-trial :284-286) removes a trial skill whose instructions conflict with AGENTS.md once read in full. Findings: the trial owner's full read, the practice-sweep reviewer, and GPT-6 (gpt-6-astra, max) "VERDICT: conflict"; decision record docs/decisions/2026-09-28-delegated-decisions.md (#462). - manifest row moved to excluded with a dated reason; budgets recomputed (Claude 7,409 -> 7,184, Codex 3,054 -> 2,829); template skillOverrides key removed - isolated-builder (three copies): preload and body line removed; token-lanes-block.builder.md gains the default block's evidence sentence (addendum records why #456's removal no longer applies) - landscape-sweep templates and TEMPLATE_SKILLS drop the skill; workflow suites pin the new preload set - #381 token-E2E preregistration Amendment 3 (pre-execution, authorised by the Gate A owner): new isolated-builder role-body row and RUNBOOK seal, history rows kept, tests updated Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e protocol Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…umes the removed skill Independent verification of #464 (low defect): the reason still read as if verification-before-completion were installed. The exclusion stands on the fp-check duplication and the Socket Warn. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins
deleted the
claude/remove-verification-before-completion-20260928
branch
September 28, 2026 18:29
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
…on with main (M4, #464) Main c0966da (#464) moved verification-before-completion from the adoption skills to its excluded list under the skills-trial conflict rule M4. On the rebased tree the unchanged coverage contract and installer refused the runtime manifest's reuse_ref for it (67 tests, 6 failures, 2 errors). - Exclude the skill in the runtime manifest with its reviewed pin, the M4 and AGENTS.md:16 reason, and the overturn "main re-admits it"; it carries no adoption_ref because main no longer lists it in skills. - New test_verification_before_completion_stays_excluded_until_main_readmits_it and the revised superpowers lifecycle test failed first; a source-count test ties sources and the research table to the selected skills. - Recount: 137 skills from 12 sources (superpowers 14), 27 reused (16 Codex-disabled on main), 9 exclusions, 15 empty cells (new github-ci-fix/extraction-caller gap), 1,020,955 bytes and 36,739 description characters. - validation.json: repair_r3.not_relayed becomes the five confirmed CONFIRM ids; repair_r4 records the rebase, sources, fail-first and checks. - State Codex detection precisely ($CODEX_HOME, ~/.codex when unset, or /etc/codex) in research.md, the harness-defaults row and a code comment. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 28, 2026
…on with main (M4, #464) Main c0966da (#464) moved verification-before-completion from the adoption skills to its excluded list under the skills-trial conflict rule M4. On the rebased tree the unchanged coverage contract and installer refused the runtime manifest's reuse_ref for it (67 tests, 6 failures, 2 errors). - Exclude the skill in the runtime manifest with its reviewed pin, the M4 and AGENTS.md:16 reason, and the overturn "main re-admits it"; it carries no adoption_ref because main no longer lists it in skills. - New test_verification_before_completion_stays_excluded_until_main_readmits_it and the revised superpowers lifecycle test failed first; a source-count test ties sources and the research table to the selected skills. - Recount: 137 skills from 12 sources (superpowers 14), 27 reused (16 Codex-disabled on main), 9 exclusions, 15 empty cells (new github-ci-fix/extraction-caller gap), 1,020,955 bytes and 36,739 description characters. - validation.json: repair_r3.not_relayed becomes the five confirmed CONFIRM ids; repair_r4 records the rebase, sources, fail-first and checks. - State Codex detection precisely ($CODEX_HOME, ~/.codex when unset, or /etc/codex) in research.md, the harness-defaults row and a code comment. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 28, 2026
…l manifest (#429) * Runtime-worker skills program: project-scoped installs and a 138-skill pinned manifest GPT-6 (astra, max) build. install_skills.py gains --project-dir and --agent targets with the default global behaviour unchanged, project lock identity and pinned source-tree checks, and scoped rollback. The runtime-worker manifest pins 138 skills (77 OpenHands extensions v0.25.0, all 15 superpowers lifecycle skills, 28 existing adoption entries by reference) with a report-only freshness checker and scheduled workflow. Not installed into any worker yet; invocation and pruning are measured after install. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Runtime-worker skills: repair round for the independent Claude review GPT-6 (astra, max) repair of all seven findings; the blocker (#407's scoped global rollback) was already resolved at the rebase and is kept byte for byte. Project rollback now removes exactly the add targets and checks the canonical folder, project lock entry and Claude link independently, reporting a retained copy as an error instead of widening the deletion (skills@7407f389 src/remove.ts:209-333). A project .claude/skills/<name> that is not a symlink to the canonical folder is local-modified unless --force (installer.ts:254-264). Every selected source/ref is fetched before any add; an unavailable gh is reported as unverified, not as a mismatch. The pruning rule records Claude Code's skill-listing budget and treats dropped descriptions as unknown. 52 tests pass; five fail-first runs are recorded in validation.json. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * CI fix: cover the skills freshness workflow and classify the repair values Found and fixed through the upstream gh-fix-ci skill (openai/skills@49f948fa) on the GPT-6 lane, then reviewed by the coordinator: - tests/test_workflow_security_coverage.py lists runtime-worker-skills-freshness.yml; the workflow itself already passes the hardening, pinning, permissions and zizmor checks. - blind_checkout: the review disposition "fixed" joins the closed-vocabulary labels (as in #416), and the superseded rollback assertion's disposition is data. - harness-defaults anti-pattern row: run acceptance with TMPDIR outside every repository. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test_install_skills: compare resolved paths on both sides for macOS temp dirs On macOS the temporary directory sits under /var, a symlink to /private/var, so local.resolve() returned the /private/var form while the expected path kept the /var form. validate-macos failed 8 subtests of the two new project-install tests (run 36349458060). Resolve the expected path too, as tests/adaptive_paper_hermetic.py already does with os.path.realpath. Reproduced on Linux with TMPDIR set to a symlinked directory: 8 failures before, OK after; the module passes 38 tests with a plain or symlinked TMPDIR. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Runtime-worker skills repair r2: exclude main's gated security-audit, take reused gates from main Coordinator direction on the #429 blocker (option b): runtime workers must not bypass a gate set on main. - Contract: every skill in adoption/skills/manifest.json is either reused by reuse_ref (pins equal to main's, gates not restated) or listed in `excluded` with an adoption_ref, a reason and an overturn condition. security-audit (#448, 8315274: Codex off, name-only pending the M5c bake-off against /security-review) is excluded, and a test fails as soon as main's gate on it changes. A synthetic future main skill in neither set fails the contract (negative control). - Gates: install_skills.py resolves each reuse_ref when it reads a manifest, refuses an entry whose pin drifted or that restates codex_enabled/claude_listing, and takes both gates from main. --print-codex-config on the runtime manifest now disables the 16 reused skills main keeps off for Codex; before, it printed none. The README states that claude_listing is a Claude Code skillOverrides setting that the OpenHands SDK does not read (OpenHands/software-agent-sdk@fcc102a6 skill.py:1123-1130, 182-185, 278-284, 528-538; same blob at da28c773). - --only naming a pruned project skill fails as pruned, not unknown. - The README documents that --check-only needs the pinned skills binary, and gh sign-in and network in project mode. - Counts: 138 selected skills from 12 sources (unchanged); 28 of main's 29 adoption skills reused; 8 exclusions (7 OpenHands, 1 adoption). - validation.json records the round (fail-first: 9 tests, failures=7, errors=2; after the fix: 9 OK) and refreshes every input hash; the anti-pattern log gains the dropped-gates row; the 8 changed files are re-registered in manifests/evidence.json. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Runtime-worker skills: narrow the OpenHands loader claim and the security-audit overturn (R2-6) The README said OpenHands withholds a skill from the model only through SKILL.md disable-model-invocation, and research.md said the SDK reads only SKILL.md frontmatter. load_user_skills also loads OpenHands' own enabled installed skills (skill.py:936-973 at fcc102a6). Both now say the loader reads skill directories, SKILL.md frontmatter and its installed-skill enable state, never Claude Code settings or a skillOverrides key. The available_skills citation moves from lines 182-185 to 183-186. The security-audit overturn now names a promotion as codex_enabled true or claude_listing on; any other change to main's gate still fails the exclusion test and needs a new review. The test message says so. validation.json records R2-6, the corrected sources, the red excerpt in log order, refreshed input hashes and the rerun checks; the five changed files are re-registered in manifests/evidence.json. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Runtime-worker skills records: note the rebase onto main 8d8f79c research.md and validation.json record the second rebase, onto 8d8f79c (#460), the merged docs/harness-defaults.md input hash and the checks rerun on that tree. The manifests/evidence.json re-registration that was part of this commit was dropped when the branch was rebased onto c1581fa, and is redone at the tip. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Runtime-worker skills repair r3: containment and pin checks before add, project scope, name-keyed reuse Two independent reviews of 8efdc35, a GPT-6 cross-family review (codex_call.sh, gpt-6-astra max) and a headless Claude session review, returned changes-needed with no blocker or high finding. Each code item failed first against the unchanged code: - GPT6-C1: refuse a project whose .agents/skills, .claude/skills or skills-lock.json passes through a symlink, whose selected skill paths resolve outside it, or that is --home, before any skills or gh call. - CL-D1: compare every selected pinned tree with its tree_sha in the preflight and refuse before any add; the native in-use guard keeps a rolled-back copy while a detected agent such as Codex reads it. - CL-D3: the runtime manifest carries "scope": "project" and is refused without --project-dir; pruned entries are never installed in either mode. - CL-D6: reuse_ref and adoption_ref name the adoption manifest and match by skill name. - CL-D7: runtime_skill_freshness.py reads the CLI version and ref from the manifest's cli pin. - GPT6-C5: offline zizmor analysis of the freshness workflow, with floating-tag, persist-credentials and permissions controls. Docs: the rendered OpenHands skill list omits <location>; disabled_skills and the loading flags are its controls; third-party instruction files load from the work directory and Git root, so the example worker projects move outside the checkout; rollback retention is certain when Codex is detected; skills-review-r1.json is labelled unpublished. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Runtime-worker skills repair r4: exclude verification-before-completion with main (M4, #464) Main c0966da (#464) moved verification-before-completion from the adoption skills to its excluded list under the skills-trial conflict rule M4. On the rebased tree the unchanged coverage contract and installer refused the runtime manifest's reuse_ref for it (67 tests, 6 failures, 2 errors). - Exclude the skill in the runtime manifest with its reviewed pin, the M4 and AGENTS.md:16 reason, and the overturn "main re-admits it"; it carries no adoption_ref because main no longer lists it in skills. - New test_verification_before_completion_stays_excluded_until_main_readmits_it and the revised superpowers lifecycle test failed first; a source-count test ties sources and the research table to the selected skills. - Recount: 137 skills from 12 sources (superpowers 14), 27 reused (16 Codex-disabled on main), 9 exclusions, 15 empty cells (new github-ci-fix/extraction-caller gap), 1,020,955 bytes and 36,739 description characters. - validation.json: repair_r3.not_relayed becomes the five confirmed CONFIRM ids; repair_r4 records the rebase, sources, fail-first and checks. - State Codex detection precisely ($CODEX_HOME, ~/.codex when unset, or /etc/codex) in research.md, the harness-defaults row and a code comment. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Rebase onto main c0966da: re-register the PR's files in manifests/evidence.json Main's evidence.json with the 13 PR files registered at their current sha256 and byte size (8 added, 5 updated); no other entry changed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Scout <scout@local> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 28, 2026
Merged
Open
Grand dashboard: integrity-lane rows after #476 (actionlint M2, CodeQL M4, skills-trial window)
#478
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR removes the trial skill
verification-before-completion(obra/superpowers@8ca22dba9a94f28898bbce59f2537ff4d87c747d) under the trial's in-force conflict rule,docs/decisions/2026-09-25-skills-trial-and-usage.md:284-286: "gives instructions that conflict with CLAUDE.md/AGENTS.md once actually read in full → remove it immediately".The conflict:
SKILL.mdputs "Previous run" under Not Sufficient for "Tests pass" (L42). It also says "If you haven't run the verification command in this message, you cannot claim it passes" (L20) and "Execute the FULL command (fresh, complete)" (L28).AGENTS.md:16says "Reuse passing evidence when its inputs still match and run only checks needed for a concrete gap".Findings. They are recorded in
docs/decisions/2026-09-28-delegated-decisions.md(#462) andevidence/artifacts/delegated-decisions-20260928/:gpt-6-astraat max, read-only): "VERDICT: conflict", disposition "remove now".Findings 1 and 2 are not shown to be independent.
coordination.mdrecords that one reading was shared before the GPT-6 verdict.Changes
excluded, with a dated reason that quotes both sides. The budgets are recomputed: Claude from 7,409 to 7,184, and Codex from 3,054 to 2,829. The template'sskillOverrideskey is removed.isolated-builder(all three copies, byte-identical, sha2560f8e0834…): the preload and the body line that named the skill are removed.adoption/hooks/claude/token-lanes-block.builder.md: gains the default block's evidence sentence, so the builder keeps an evidence rule. The token-lanes addendum records why Token-lanes carrier: evidence sentence replaces the verification-skill line (AN-15, after #447) #456's removal no longer applies. The new block sha256 isc81a91c4ffb2fa906feb403d5a11bd294b6cf2db45795d80f9acd662c5004973.TEMPLATE_SKILLS: drop the skill. "Evidence before any verdict" stays as plain text, and the prompt hash is updated.['context-mode:context-mode'].isolated-builder.mdrole-body row0f8e0834…and a new RUNBOOK seal135340b6…;d022295aand Amendment 2 rows kept as history;tests/test_token_e2e_preregistration.pypins both.Host steps after merge
These run on this host only, and a receipt comes in a follow-up:
DISABLE_TELEMETRY=1 skills remove verification-before-completion -g -y -a claude-code codex, through the pinned skills 1.7.0 CLI.-alimits the removal to the two agents the installer adds it for (install_skills.pyL60-62). Without it,removetargets every known agent.python3 tools/adoption/install_claude_profile.py --only guard --only agents.python3 scripts/skills_status.pyread-back.Review
Gate A owner (native-agent-stack-a9) reviewed Amendment 3 at
d98ff7d0and approved it with no blocking findings. It re-derived the role-body and seal hashes and re-ran the preregistration, token-lanes and install-profile tests (109 OK), both node suites andvalidate.py.GPT-6 cross-family review (gpt-6-astra, max, read-only):
VERDICT: clean. Usage: 1,207,847 input tokens (1,094,784 of them cached) and 15,607 output.Independent
stack-verifier: it confirmed the tests, all hashes, the budget recompute (7,184 and 2,829, by both the test predicate and the named method) and the registration (26 of 26). A scratch-HOMEinstall_claude_profile.py --home ... --only guard --only agentsshowed:context-mode:context-mode;It found two low defects:
29ed7d98: thepost-patch-validationexclusion reason still assumed the removed skill.Checks
These are our integration checks, run after merging
origin/mainatc61e6657:Ran 350 tests,OK (skipped=7).test-envelope.mjs: 254 of 254 pass.test-contract-mutations.mjs: 74 of 74 pass.python3 scripts/validate.py: passed.SOTA sources
obra/superpowers@8ca22dba9a94f28898bbce59f2537ff4d87c747d,skills/verification-before-completion/SKILL.md(sha2562befe7fc…), read in full on 2026-09-28. Upstreammainhas no newer commit touching the skill: the last is3be5aad3, from 2026-07-24.skillsCLI 1.7.0:remove <name> -g -y -a <agents>.🤖 Generated with Claude Code