Runtime-worker skills program: project-scoped installs and a 137-skill manifest

[seathatflowsinourveins]

Scope

• What this PR changes: a skills program for runtime workers.
  • tools/adoption/install_skills.py gains project-scoped installs (--project-dir, --agent, --only) and --check-only.
  • Project installs refuse to run in these cases, before any skills or gh call:
    • the project skills directory is a symlink, or resolves outside --project-dir;
    • a pinned tree does not match the manifest's tree_sha;
    • the manifest's "scope": "project" is used without --project-dir.
  • After an add, a mismatch rolls back with the native CLI. Pruned entries are never installed, and --only <pruned> reports "pruned".
  • Global behaviour is unchanged and keeps install_skills.py: scope the failed-hash rollback to the installed agents (#405) #407's scoped -a claude-code codex rollback.
• The runtime-worker manifest pins 137 skills from 12 sources:
  • It reuses 27 of main's 28 adoption skills by name. Each reused skill takes main's codex_enabled and claude_listing gates when the manifest is read. A reused entry whose pin drifted from main, or that restates a gate, is refused before any install. --print-codex-config disables the 16 reused skills main disables.
  • 9 exclusions: the 7 OpenHands ones, plus two gated by main:
    • security-audit (Skills trial: cloudflare/security-audit-skill name-only for LLM-native automation; stale-upstream flags; sandbox-profile open gate #448), until main promotes it after M5c;
    • verification-before-completion (Skills trial: remove verification-before-completion under the conflict rule (M4), #381 Amendment 3 #464, M4 conflict with AGENTS.md:16), until main re-admits it.
  • A report-only freshness checker runs on a schedule; its expected CLI version comes from the manifest's pin.
• Base: main c3f15cd2.
• Lane: lane:foundation.
• Owned paths touched:
  • blueprints/runtime-workers/skills/*
  • tools/adoption/install_skills.py, tools/adoption/runtime_skill_freshness.py
  • tests/test_install_skills.py, tests/test_runtime_worker_skills.py, tests/test_workflow_security_coverage.py
  • .github/workflows/runtime-worker-skills-freshness.yml
  • tools/sota-convergence/blind_checkout.py (classification of repair dispositions)
  • nine anti-pattern rows in docs/harness-defaults.md
  • hash re-registration in manifests/evidence.json, taken from main and re-registered per docs/lanes.md

SOTA sources

• Installer semantics: vercel-labs/skills v1.7.0 @ 7407f389.

  • src/add.ts L1775-1810: canonical folder vs copy mode.
  • src/installer.ts L128-131, L151-158, L193-200 and L388: agent directories and canonical placement.
  • src/remove.ts L207-215 and L293-331: project alias removal, and the shared data a targeted remove keeps.
  • src/agents.ts L10, L158, L224-232 and L815: agent target paths and Codex detection ($CODEX_HOME, else ~/.codex, or /etc/codex).
  • src/skill-lock.ts: skillFolderHash is the source tree hash.
  • The npm tarball is skills@1.7.0, with the sha512 integrity recorded in the manifest.

• OpenHands loader (software-agent-sdk @ fcc102a6):

  • openhands-sdk/openhands/sdk/skills/skill.py: folder and SKILL.md frontmatter loading, its own installed-skill state, third-party instruction files at L346-352 and L1091-1121, and the renderer at L1524.
  • context/agent_context.py L93-135 and L171: project-loading flags and disabled_skills.
  • invoke_skill rejects disable-model-invocation skills (da28c773 invoke_skill.py L109).

• Skill sources: each is pinned to a commit resolved with gh api repos/<owner>/<repo>/commits/<sha>, and each skill carries its own repo@commit path:line merit citation in the manifest.

  Source	Commit	Selected
  OpenHands/extensions (v0.25.0)	bea7a20c	77
  obra/superpowers	8ca22dba	14
  mattpocock/skills	c55ee460	12
  anthropics/skills	33375500	9
  trailofbits/skills	0cc1c73a	9
  openai/skills	49f948fa	6
  vercel-labs/agent-skills	063bee94	4
  affaan-m/ECC	2b6e8397	2
  assafelovic/gpt-researcher	0957c301	1
  typesafe-ai/skills	65a39f39	1
  vercel-labs/agent-browser	d01253d9	1
  vercel-labs/skills	7407f389	1

Evidence-class table

Claim	Evidence class	Command / receipt
Project install, check-only, containment and tree refusals, scope enforcement, pruned handling and scoped rollback behave as specified	synthetic (fake skills and gh CLI subprocess fixtures)	python3 -m unittest tests.test_install_skills tests.test_runtime_worker_skills; the fail-first records for rounds r1-r4 are in blueprints/runtime-workers/skills/validation.json
Every main adoption skill is reused or explicitly excluded; gates are inherited, and drift is refused	synthetic plus in-memory negative controls	the contract tests; putting verification-before-completion back into main's skills in memory returns exactly ['verification-before-completion: neither reused nor excluded']
The freshness workflow is least-privilege and SHA-pinned	integration test	tests.test_workflow_security_coverage analyses its contents; floating the checkout pin to @v7 turns it red
Each selected skill exists at its pinned commit and path	source_review	manifest repo@commit path:line citations; gh api tree listings
Skills installed into worker workspaces, discovered, invoked, and costing a measured amount of prompt	not claimed	next step: native install into worker workspaces, then invoke-rate measurement and pruning

Local commands run (on this head, rebased onto c3f15cd2)

$ python3 -m unittest tests.test_runtime_worker_skills tests.test_install_skills tests.test_workflow_security_coverage tests.test_workflow_hardening
OK (skipped=2)
$ python3 scripts/validate.py
{"components": 69, "hashed_files": 7927, "profiles": 4, "receipts": 164, "status": "passed"}
$ python3 scripts/host_receipts.py validate
{"receipts": 182, "status": "passed"}

The builder's r4 acceptance on 46176c29 (rebased onto c0966da2) also passed: the three PR modules (79 tests), test_blind_checkout (40), test_pre_push_gate (16), and four docs readers.

Review

• Round 1: an independent Claude review, repaired by GPT-6 (aa0d1ee7).
• Round 2 (8efdc35d), two independent reviews; verdicts are in the comment above:
  • GPT-6 cross-family (Codex CLI 0.157.1, gpt-6-astra, max, read-only): changes-needed, with one medium finding (C1: a symlinked project skills directory could reach global state).
  • Headless Claude session: changes-needed, with one medium finding (D1: the pinned-tree check ran only after add).
  • Neither found a blocker or high.
• Repair r3 (8961e1e1): fixes every finding.
  • Code: C1 and D1, D3 scope marker, D6 reuse by name, D7 pinned CLI version, C5 workflow content assertions.
  • Docs: C6 loader and native controls, D2 third-party instruction files and git-root search, D4 rollback retention, D5 unpublished review relabelled.
  • C2-C4, C7 and C1-manifest-pins were confirmations.
• Repair r4 (f231d274): adapts to main's Skills trial: remove verification-before-completion under the conflict rule (M4), #381 Amendment 3 #464 by excluding verification-before-completion. It was checked by red→green and a re-admission control, but not independently reviewed.
• CI: one validate-macos run on r3 failed in tests.test_adoption_status (a SIGHUP subtest timing out). That test is unrelated to this PR, and the job passed on rerun.

Residuals

• No native skills CLI install and no worker end-to-end run; all installer evidence uses fake executables.
• New gap: the github-ci-fix / extraction-caller cell is empty, because its only selection was verification-before-completion.
• Claude Code workers must apply claude_listing through their own skillOverrides; the installer writes no Claude settings.
• One pinned-tree mismatch fails the whole project run before any add, including --check-only --json.
• Dated artifacts are left as history: freshness-20260927.json, the r1 discovery line and pins_verified (138).
• Record fix: an r3 record wrongly listed GPT-6's confirmations as "not relayed". r4 relabels them as confirmed. The cause was the coordinator relaying only non-confirm findings.

Status

Ready to merge after CI on this head. Nothing is installed on any host yet. #425 (OpenHands runtime worker) builds on this.

🤖 Generated with Claude Code

[seathatflowsinourveins]

Independent reviews of 8efdc35d

GPT-6 cross-family review (Codex CLI codex-cli 0.157.1, model gpt-6-astra, effort max, read-only sandbox, via tools/sota-convergence/landscape-sweep/codex_call.sh). Overall: changes-needed

id	verdict	severity	location	reason
C1-project-isolation	refute	medium	tools/adoption/install_skills.py:146	Changes are needed because project scope does not enforce filesystem containment. If /.agents/skills links to ~/.agents/skills, matching global SKILL.md bytes and an absent project lock allow installation without --force. The pinned CLI recreates that global directory; failed verification can then delete it when no remaining agent retains it, leaving the global lock stale. The in-memory probe confirmed the wrapper accepts this path. See upstream installation and removal. Reject targets whose resolved parents escape the project.
C1-manifest-pins	confirm	info	blueprints/runtime-workers/skills/manifest.json:4513	Confirmed 138 distinct skills from 12 sources. All 138 path/tree_sha pairs matched independently retrieved, untruncated upstream Git trees at the recorded revisions. With separate project/global directories, in-memory rollback probes preserved pre-existing global files for all three supported agent targets.
C2-adoption-coverage	confirm	info	tests/test_runtime_worker_skills.py:121	All 29 adoption skills are accounted for: 28 reused and security-audit explicitly excluded. Adding an uncovered adoption skill failed the contract probe. Changing security-audit's gate to either promotion or another listing mode failed its dedicated test.
C3-reuse-gates	confirm	info	tools/adoption/install_skills.py:129	Drifted pins, invalid references, and either restated gate were rejected before installer or network calls. Both gates followed altered adoption values at read time. --print-codex-config produced exactly the expected 16 disabled entries.
C4-pruned-selection	confirm	info	tools/adoption/install_skills.py:456	In project mode, selecting a pruned skill returned exit 1 with 'pruned --only', without invoking the installer. This guarantee is explicitly limited to project mode.
C5-workflow-test-coverage	refute	low	tests/test_workflow_security_coverage.py:64	This addition checks only the workflow filename; the module never analyzes its contents. Replacing an action SHA with a floating tag leaves this module green. Add an _analyze assertion. The workflow itself has contents:read, SHA-pinned actions, disabled credential persistence, and no pull_request trigger; direct offline zizmor returned no findings. Repository-wide CI scanning and separate pin assertions mitigate the coverage gap.
C6-loader-documentation	amend	low	blueprints/runtime-workers/skills/README.md:87	The folder, frontmatter, installed-state, and non-Claude-settings claims hold. However, the promised catalog location field is absent: the pinned renderer deliberately emits only name and description, directing invocation through invoke_skill. A consumer cannot obtain the documented location from that catalog. Cite the actual renderer, rather than its stale class docstring.
C6-native-controls	amend	low	blueprints/runtime-workers/skills/README.md:90	The two-control description is incomplete. The pinned SDK also provides AgentContext.disabled_skills and source-loading flags. Installed-skill state does not disable a separately discovered project-directory copy; the native deny-list can exclude it without modifying pinned files. See AgentContext.
C7-validation-evidence	confirm	info	blueprints/runtime-workers/skills/validation.json:124	All nine recorded input hashes match the head, and retained test counts match current discovery. scripts/validate.py passed; all 18 runtime-manifest tests passed when loaded directly. The requested combined unittest command failed before collection because the sandbox denied temporary files even under /var/tmp, so historical full-suite exit codes were not independently reconfirmed. No native installer was run, and the checkout remains clean.

GPT-6 overall: changes-needed

Headless Claude session review (separate claude -p session, Opus, read-only tools, no Agent/Workflow). Overall: changes-needed

id	verdict	severity	location	reason
C1	confirm	info	blueprints/runtime-workers/skills/manifest.json; tools/adoption/install_skills.py:342-344	Pins hold. The manifest has 138 skills from 12 sources: OpenHands/extensions 77, obra/superpowers 15, 28 reused, plus targeted additions. I re-fetched every pin read-only with gh api. All 12 pinned commits resolve, and all 138 directory tree SHAs, SKILL.md sha256 values and byte counts match. Every frontmatter name equals its manifest name, with no duplicates, and upstream's sanitizeName leaves them unchanged. The CLI derives the --skill filter, the canonical folder and the lock key from the frontmatter name (skills@7407f389 src/skills.ts:339-347; installer.ts:316-317; add.ts:1962-1964). With an explicit ref, tryBlobInstall returns null (blob.ts:567), so the lock's skillPath is <path>/SKILL.md as the wrapper expects. Global copies are safe: upstream remove.ts:258-270 without -g builds only cwd paths. My probe put a global copy under --home, then ran a project install whose tree did not match. The wrapper ran remove shared -y -a universal claude-code in the project and left the global folder, the ~/.claude link and the global lock entry in place.
C2	confirm	info	tests/test_runtime_worker_skills.py:28-59,147-155	adoption_contract_problems passes on the real manifests: 28 reused, security-audit excluded at #/skills/28 with a reason and an overturn condition. The negative control fails as expected on a new main skill that is in neither set, and on double claims, restated gates and bare exclusions. The security-audit test fails on any change to main's (codex_enabled, claude_listing) = (False, 'name-only'), including a promotion, so it acts as a review trigger as README:96-101 states. Main's adoption manifest is unchanged between 8d8f79c and b9eb62d. Note that exclusion is enforced only by this test. My probe added a reuse_ref to #/skills/28 with matching pins, and install_skills.py accepted it (--print-codex-config exited 0).
C3	confirm	info	tools/adoption/install_skills.py:104-137,369-376	Each probe on a /var/tmp copy of the runtime manifest exited 1 before any skills-bin call: tree_sha drift ("differs from the adoption pin in ['tree_sha']"), codex_enabled restated with main's own value, claude_listing restated as null, and a reuse_ref moved to another index (all seven pin keys reported). A bad prefix gave 'names no adoption skill'. --print-codex-config emits exactly 16 enabled=false tables, which equals the reused skills main disables. Minor: the installer compares 7 pin keys, while the test compares 10 (it adds skill_md_bytes, description_chars and upstream_disable_model_invocation). A leading-zero pointer ('#/skills/024') is also accepted.
C4	confirm	info	tools/adoption/install_skills.py:445-461	The claim holds as scoped. In project mode, --only <pruned> exits 1 with "pruned --only name(s), not installed in project mode", not 'unknown', and README:152-153 limits the claim to project mode. The global-mode gap is reported separately in D3.
C5	confirm	info	.github/workflows/runtime-worker-skills-freshness.yml:3-53	Triggers are schedule and workflow_dispatch only; there is no pull_request or pull_request_target, so it cannot run from forks. permissions: contents: read is set at workflow level with no job override. GH_TOKEN is scoped to the report step, and checkout uses persist-credentials: false. All four actions are pinned by 40-hex SHA, and the same SHAs appear in 10-20 other workflows. No github.event or other untrusted expression is interpolated into run blocks. zizmor 1.30.1 --offline reports no findings. The file is in the workflow-security coverage set, and that test passes with zizmor on PATH.
C6	confirm	info	blueprints/runtime-workers/skills/README.md:77-92	The cited file is openhands-sdk/openhands/sdk/skills/skill.py. It is the same blob, 134e6405, at fcc102a6 and da28c773. The brief's context/skills/skill.py path returns 404, but the PR itself cites the correct path. The cited lines match: 183-186 describe the listing; 278-284 and 528-538 read disable-model-invocation; 936-973 are the user dirs plus installed skills; 1123-1130 are the project dirs. Nothing in skill.py reads Claude Code settings or skillOverrides, so claude_listing has no effect there. The enumeration is incomplete; see D2.
C7	confirm	info	blueprints/runtime-workers/skills/validation.json:1-572	All 9 input_sha256 values match the tree. Every recorded repair-r2 command reproduces with TMPDIR under /var/tmp: 67 tests OK, test_blind_checkout 40, test_pre_push_gate 16, test_adoption_docs_consistency 37 (1 skipped), test_ecosystem_manifest 56, test_install_claude_profile 58 (3 skipped), test_token_e2e_preregistration 33, and --print-codex-config 16. scripts/validate.py passed (7734 hashed files). The manifests/evidence.json changes only register this PR's files. freshness-20260927.json covers the same 138 names, refs and trees. The worktree stayed clean.
D1	refute	medium	tools/adoption/install_skills.py:490-500 (preflight) vs 310-316 (add)	The project preflight fetches and caches every pinned tree but never compares a skill's tree_sha before the add. Tree identity is checked only after add, in classify_skill (lines 236-238). My probe used a fake gh returning a tree different from the manifest's tree_sha: the wrapper still ran add ... -y -a universal claude-code, then relied on rollback. Codex is a universal-directory agent (skills@7407f389 agents.ts:224-231; skillsDir '.agents/skills'; detected via $CODEX_HOME, ~/.codex or /etc/codex). Upstream remove.ts:293-310 keeps the canonical folder and lock entry whenever another detected agent's install path exists, and for universal agents that path is the canonical folder (installer.ts:151-158). On any Codex host, which is this stack's normal host and HOME defaults to the real home, a stale tree_sha after a repin therefore leaves unverified SKILL.md bytes in /.agents/skills. OpenHands (skill.py:1123-1130) and Codex both load from there, and every rerun re-adds and fails the same way. This does not trigger with today's pins, which all verify. Fix: refuse before any add when pinned_source_trees(source, ref).get(path) != tree_sha.
D2	amend	low	blueprints/runtime-workers/skills/README.md:77-92,134,240; blueprints/runtime-workers/skills/research.md:130-133	The README says the OpenHands loader reads skill directories, SKILL.md frontmatter and its installed-skill state; research.md says the same. The loader also reads other files. skill.py:1091-1121 loads third-party instruction files (.cursorrules, AGENTS.md, agent.md, CLAUDE.md, GEMINI.md; map at 346-352) as always-on skills. It searches both the work dir and the Git repo root, which _find_git_repo_root finds by walking parents (981-992), and it loads the project skill dirs at that root too. Project and user loading are opt-in (agent_context.py:93-135, default False), but README:240 tells operators to enable 'the normal native project loader'. The README's example OH_PROJECT="$STACK_ROOT/.runtime/openhands" (line 134) sits inside this repository's Git root. An OpenHands worker there would ingest this repo's AGENTS.md and CLAUDE.md, which also confounds the README's first-prompt measurement. The narrow 'no Claude settings' statement stays true.
D3	amend	low	tools/adoption/install_skills.py:445-446	Project scope is a convention, not enforced. Without --project-dir, the runtime manifest takes the global path. A dry run with a version-only fake CLI planned 138/138 add ... -g -y -a claude-code codex installs into ~/.agents/skills and ~/.claude/skills. That set includes 110 skills with no Codex or Claude gate. In global mode, pruned entries are also processed: a real fake-CLI run of --only <pruned> installed the skill globally and exited 0. README:109-110 documents that global behaviour is kept, every README command goes through worker_skills with --project-dir, and all 138 entries are currently 'trial', so there is no current effect. A manifest-level scope marker that the installer enforces would close this.
D4	amend	low	blueprints/runtime-workers/skills/README.md:166-179	The README says a post-add mismatch 'removes that skill with the native CLI' and that the in-use guard 'can retain' it. Per D1's upstream citations, retention is certain for universal and claude-code rollbacks whenever Codex or another universal-directory agent is detected under the wrapper's HOME. The caller's CODEX_HOME passes through even with a test --home. The README should say so, and should say the retained content stays loadable by workers. The retention itself is a deliberate, recorded choice (docs/harness-defaults.md row 'Widening rollback after a remover retains shared data'); only the wording overstates cleanup.
D5	amend	low	blueprints/runtime-workers/skills/validation.json:118	independent_review.source cites 'skills-review-r1.json' with a sha256, but that file is not in the tree (git ls-files finds 0 matches), so the cited review cannot be checked. Publish a sanitized receipt or relabel it as unpublished.
D6	amend	info	tools/adoption/install_skills.py:81,104-137	reuse_ref is an index-based JSON pointer, and reused entries repeat main's pins. Any insertion, reorder or repin in adoption/skills/manifest.json makes load_manifest refuse the whole runtime manifest, even for an unrelated --only run. It also fails the contract test on main's PR. This fails closed, but it couples the two lanes; document it in docs/lanes.md or key reuse by name.
D7	amend	info	tools/adoption/runtime_skill_freshness.py:96-97; .github/workflows/runtime-worker-skills-freshness.yml:12	ok hard-codes CLI 1.7.0, so a deliberate CLI repin fails the weekly job until the code changes. zizmor --persona=pedantic reports only concurrency-limits and an anonymous job definition.

Claude overall: changes-needed: The PR's seven headline claims hold on primary evidence. I re-fetched all 138 pins, and all 67 PR tests plus validate.py and the recorded commands reproduce. Name identity with the pinned skills CLI is clean. Reuse-gate refusal, the coverage contract, the 16 Codex disables, the pruned --only handling in project mode, global-copy-safe rollback and the least-privilege, SHA-pinned workflow all check out. Four things should change before merge. First, D1: compare each pinned tree against the manifest's tree_sha during preflight, before any add, because on Codex hosts the native rollback keeps whatever was added. Second, D4: state in README:166-179 that rollback retention is certain on hosts with Codex or another universal-directory agent. Third, D2: finish the OpenHands read-set in README:77-92 and research.md:130-133 (third-party AGENTS.md/CLAUDE.md files and the Git-root search), and move or caveat the example worker paths inside $STACK_ROOT. Fourth, D5: publish or relabel the uncommitted skills-review-r1.json. D3, D6 and D7 are low or informational follow-ups. None of the findings is blocker or high severity.

Neither review found a blocker or high. All findings go to one repair round (medium: GPT6-C1 project containment, CL-D1 tree check before add). Its dispositions will be recorded in blueprints/runtime-workers/skills/validation.json.

🤖 Generated with Claude Code