From 351436d71b6404ecc3731b6433dab4f1f75d17c6 Mon Sep 17 00:00:00 2001 From: seathatflowsinourveins <234074349+seathatflowsinourveins@users.noreply.github.com> Date: Wed, 23 Sep 2026 19:14:30 -0400 Subject: [PATCH 1/6] Run every Claude child at max effort under an Ultracode main loop Set effort: max in the five shipped agent definitions (adoption/agents/claude/*.md), keeping their task-matched models; the one-line change keeps them byte-identical to agent-lab's .claude/agents copies, which receive the same edit. Commit a project .claude/settings.json (byte-identical to the portable ultracode example and agent-lab's committed project settings) so single-repository cloud sessions on this repository start with Ultracode. It carries neither effortLevel (a max there is silently dropped) nor CLAUDE_CODE_EFFORT_LEVEL (any value but xhigh disables Ultracode, and max there overrides every child's effort). Update the Ultracode recipe: every child role at max with its task-matched model, the coordinator at Ultracode (xhigh plus orchestration), the measured constraints (max disables orchestration, cannot be persisted, and the environment variable overrides all child efforts), the per-stage rule (agent() calls pass effort: 'max'; a stage without it inherits xhigh) and GitHub/cloud-session guidance (claude-code-action has no effort input: use claude_args '--effort ultracode' or settings {"ultracode":true}, never '--effort max' beside Ultracode). Note the child-effort change in the native profile recipe. Record the decision with its probes, alternatives and overturn conditions in docs/decisions/2026-09-23-max-effort-default.md, linked from the Claude user profile decision, and rehash the two recipes in manifests/evidence.json. Co-Authored-By: Claude Opus 5.5 (1M context) --- .claude/settings.json | 7 + adoption/agents/claude/blind-judge.md | 2 +- adoption/agents/claude/evidence-reviewer.md | 2 +- adoption/agents/claude/isolated-builder.md | 2 +- .../claude/semantic-evidence-reviewer.md | 2 +- adoption/agents/claude/source-scout.md | 2 +- .../2026-09-23-claude-user-profile.md | 8 + .../2026-09-23-max-effort-default.md | 173 ++++++++++++++++++ manifests/evidence.json | 4 +- recipes/claude-native-profile.md | 21 ++- recipes/claude-native-ultracode.md | 116 ++++++++++-- 11 files changed, 320 insertions(+), 19 deletions(-) create mode 100644 docs/decisions/2026-09-23-max-effort-default.md diff --git a/.claude/settings.json b/.claude/settings.json index 91382fade..4b7a5ea4d 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -1,5 +1,12 @@ { "$schema": "https://json.schemastore.org/claude-code-settings.json", + "enableWorkflows": true, + "ultracode": true, + "workflowSizeGuideline": "unrestricted", + "env": { + "CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS": "8", + "CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH": "1" + }, "permissions": { "deny": [ "Read(~/.config/native-agent-stack/**)", diff --git a/adoption/agents/claude/blind-judge.md b/adoption/agents/claude/blind-judge.md index d4d5ff8bc..eb0e7ee02 100644 --- a/adoption/agents/claude/blind-judge.md +++ b/adoption/agents/claude/blind-judge.md @@ -3,7 +3,7 @@ name: blind-judge description: Judge or refute one stripped comparison packet on its preregistered metrics only; never sees arm identities, repository names or paths. tools: Read model: opus -effort: high +effort: max maxTurns: 30 omitClaudeMd: true --- diff --git a/adoption/agents/claude/evidence-reviewer.md b/adoption/agents/claude/evidence-reviewer.md index 01b6f8e16..15421194e 100644 --- a/adoption/agents/claude/evidence-reviewer.md +++ b/adoption/agents/claude/evidence-reviewer.md @@ -3,7 +3,7 @@ name: evidence-reviewer description: Independently review a supplied artifact against original source (a patch and its verification evidence, a drafted protocol, or a proposal to refute) and return source-cited defects and gaps. It is read-only (no Bash, Edit or Write), runs no acceptance commands and needs any command result supplied by the coordinator; use source-scout to run commands. tools: Read, Glob, Grep, ToolSearch, mcp__serena__find_symbol, mcp__serena__find_referencing_symbols, mcp__serena__find_declaration, mcp__serena__find_implementations, mcp__serena__get_symbols_overview, mcp__serena__get_diagnostics_for_file, mcp__socraticode__codebase_search, mcp__socraticode__codebase_symbol, mcp__socraticode__codebase_impact, mcp__socraticode__codebase_flow, mcp__jcodemunch__route, mcp__jcodemunch__order, mcp__plugin_context-mode_context-mode__ctx_execute, mcp__plugin_context-mode_context-mode__ctx_execute_file, mcp__plugin_context-mode_context-mode__ctx_batch_execute, mcp__plugin_context-mode_context-mode__ctx_search, mcp__ai-memory__memory_query, mcp__ai-memory__memory_read_page, mcp__ai-memory__memory_read_session_observations model: opus -effort: high +effort: max --- Read the supplied patch or changed files, relevant callers, acceptance criteria, and validation results. Report every concrete defect that causes incorrect behavior, a failed required check, or a misleading result, with exact file references and evidence; record missing evidence as a verification gap, not a defect; omit only pure style or naming preferences. Do not edit files. If you find none, say so and list remaining verification gaps. If you need a command result, ask the coordinator to supply it; do not derive counts or exit codes by eye. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. You have no Bash, Edit or Write tool; Context Mode execution can run commands in the working tree, so use it only to read (diffs, status listings, large outputs) and never to run acceptance commands or to create, modify or delete files or git state. Focused Read/Grep or Serena for known symbols and references, SocratiCode for conceptual code, jCodeMunch `order` (read-only actions) for indexed symbol retrieval, ai-memory query/read for prior decisions, Context Mode (ctx_execute) to derive answers from large outputs without loading them; choose one lane per artifact and verify original source before judging. diff --git a/adoption/agents/claude/isolated-builder.md b/adoption/agents/claude/isolated-builder.md index c8fac5356..7c0432213 100644 --- a/adoption/agents/claude/isolated-builder.md +++ b/adoption/agents/claude/isolated-builder.md @@ -3,7 +3,7 @@ name: isolated-builder description: Implement a bounded task in its own worktree and return a verified handoff. tools: Read, Edit, Write, Glob, Grep, Bash, ToolSearch, mcp__serena__find_symbol, mcp__serena__find_referencing_symbols, mcp__serena__find_declaration, mcp__serena__get_symbols_overview, mcp__serena__get_diagnostics_for_file, mcp__serena__replace_symbol_body, mcp__serena__insert_after_symbol, mcp__serena__insert_before_symbol, mcp__serena__rename_symbol, mcp__socraticode__codebase_search, mcp__socraticode__codebase_symbol, mcp__socraticode__codebase_impact, mcp__jcodemunch__route, mcp__jcodemunch__order, mcp__plugin_context-mode_context-mode__ctx_execute, mcp__plugin_context-mode_context-mode__ctx_execute_file, mcp__plugin_context-mode_context-mode__ctx_batch_execute, mcp__plugin_context-mode_context-mode__ctx_search, mcp__ai-memory__memory_query, mcp__ai-memory__memory_read_page model: sonnet -effort: medium +effort: max isolation: worktree --- diff --git a/adoption/agents/claude/semantic-evidence-reviewer.md b/adoption/agents/claude/semantic-evidence-reviewer.md index 0c0f10eeb..8db932024 100644 --- a/adoption/agents/claude/semantic-evidence-reviewer.md +++ b/adoption/agents/claude/semantic-evidence-reviewer.md @@ -3,7 +3,7 @@ name: semantic-evidence-reviewer description: Review supplied source claims and advisory TypeSafe semantic judgments against original source within a bounded evidence task. It is read-only (Read, Glob, Grep, with the typesafe-ai skill preloaded), makes no service calls and needs any TypeSafe inference result supplied by the coordinator; use evidence-reviewer for patches and general source review. tools: Read, Glob, Grep model: opus -effort: high +effort: max skills: - typesafe-ai --- diff --git a/adoption/agents/claude/source-scout.md b/adoption/agents/claude/source-scout.md index c39ff35bd..d38a497c7 100644 --- a/adoption/agents/claude/source-scout.md +++ b/adoption/agents/claude/source-scout.md @@ -3,7 +3,7 @@ name: source-scout description: Exact extraction and inventory from named files and commands, plus running the acceptance commands a task names; makes no edits of its own and returns source-cited facts, never judgments. tools: Read, Grep, Glob, Bash model: sonnet -effort: medium +effort: max maxTurns: 40 omitClaudeMd: true --- diff --git a/docs/decisions/2026-09-23-claude-user-profile.md b/docs/decisions/2026-09-23-claude-user-profile.md index eea74f707..958ad0583 100644 --- a/docs/decisions/2026-09-23-claude-user-profile.md +++ b/docs/decisions/2026-09-23-claude-user-profile.md @@ -115,3 +115,11 @@ apply/install tooling (`tools/adoption/apply_claude_settings.py`, the trust boundary `bootstrap-linux.sh`'s fixed system-package list already relies on), and poppler is not even installed on this host to describe. Left as a documented gap rather than a speculative mechanism. + +## Later decisions + +- [`max` effort for every Claude child, Ultracode for the main loop](2026-09-23-max-effort-default.md) + (2026-09-23): the shipped `adoption/agents/claude/` definitions now declare + `effort: max` beside their task-matched models, and this repository commits a + project `.claude/settings.json` that turns Ultracode on. Decision 3's coordinator + (Opus 5.5 at `xhigh`) is unchanged. diff --git a/docs/decisions/2026-09-23-max-effort-default.md b/docs/decisions/2026-09-23-max-effort-default.md new file mode 100644 index 000000000..275ba1fb4 --- /dev/null +++ b/docs/decisions/2026-09-23-max-effort-default.md @@ -0,0 +1,173 @@ +# Decision: `max` effort for every Claude child, Ultracode for the main loop (2026-09-23) + +**Decided by:** unit `max-effort-default`, its own dedicated worktree, branch +`claude/max-effort-default-20260923` (base `origin/main@40828dfe`), from the user's +requirement of 2026-09-23: "effort max and ultracode default for all future workflow +and github, max quality". The user expected one session to run `max` and Ultracode at +the same time; the probes below show that the native client does not allow that, so +the requirement is met per layer instead. + +**Scope:** the five shipped agent definitions (`adoption/agents/claude/*.md`, kept +byte-identical to agent-lab's `.claude/agents/` copies, which receive the same +one-line change), a new committed project settings file for this repository +(`.claude/settings.json`), the model/effort guidance in +[`recipes/claude-native-ultracode.md`](../../recipes/claude-native-ultracode.md) and +[`recipes/claude-native-profile.md`](../../recipes/claude-native-profile.md). The +user-scope settings template (`adoption/templates/claude.settings.template.json`) +already carries `ultracode: true`, the `xhigh` effort defaults and +`autoContinueAtUsageLimit: true`, and no `CLAUDE_CODE_EFFORT_LEVEL`; it needs no change. +This record extends the [Claude user profile decision](2026-09-23-claude-user-profile.md); +its coordinator choice (Opus 5.5 at `xhigh`) is unchanged. + +## Evidence + +**Probe method.** Research workflow `wf_3f4689fe-c86` ran headless probes P0 to P9 +with Claude Code 2.1.281, each in a fresh scratch directory, on the host whose user +settings carry `ultracode: true` and `modelSettings.claude-opus-5-5.effortLevel: +"xhigh"`. The user settings file's hash was the same before and after the probes. +Each probe's resolved effort, for the session and for each child, was read from its +native transcript. Ultracode was counted as active when the transcript carried the +Ultracode system reminder, whose text begins "Ultracode is on: optimize"; the +unrelated `/workflow-authoring` reference sentence "Ultracode is on for the +session" was excluded. The transcripts stay on the probing +host (they hold local session identifiers and paths) and are not published here. + +| Probe | Configuration | Result | +| --- | --- | --- | +| P0 | Baseline: user `ultracode: true` plus the per-model `xhigh` | Session at `xhigh`; reminder present (1 line) | +| P1 | Environment `CLAUDE_CODE_EFFORT_LEVEL=max` | Session at `max`; reminder absent (0 lines) | +| P2 | `--effort max` | Session at `max`; reminder absent (0 lines) | +| P3 | `--settings` with `effortLevel: "max"` | Session at `xhigh`, no warning; reminder present | +| P4 | `--settings` with `modelSettings.claude-opus-5-5.effortLevel: "max"` | Session at `xhigh`, no warning; reminder present | +| P5 | Environment `max`; one default Agent child | Child `claude-opus-5-5` at `max` | +| P6 | Environment `max`; one `source-scout` child, frontmatter then `sonnet`/`medium` | Child `claude-sonnet-5` at `max`: the variable overrides frontmatter effort | +| P7 | Project `.claude/settings.json` with `effortLevel: "max"` | Session at `xhigh`, no warning; reminder present | +| P8 | Project `effortLevel: "max"` plus `ultracode: true` | Session at `xhigh`, no warning; reminder present | +| P9 | Environment `max`; one Workflow stage passing `effort: 'high'` | Stage at `max`: the variable overrides stage effort; reminder absent | + +A further probe, H1, ran with the default Ultracode parent at `xhigh` (reminder +present): a Workflow stage that passed `effort: 'max'` ran at `max` (agent +`a659fbb94398ea280`, label `stage-max`); a stage with no `effort` inherited `xhigh` +(agent `abd2b64f159add7c2`, label `stage-inherit`); an Agent whose frontmatter says +`effort: max` ran at `max`. + +**Installed client.** The 2.1.281 binary's settings schema declares `effortLevel` and +`modelSettings..effortLevel` as +`z(["low","medium","high","xhigh"]).optional().catch(void 0)`, so any other value, +`"max"` included, is dropped without a warning (P3, P4, P7, P8). The binary has no +default-effort key for subagents or workflow stages; `CLAUDE_CODE_SUBAGENT_MODEL` +sets only a child's model. + +**Official documentation** ([model configuration](https://code.claude.com/docs/en/model-config), +fetched by the research workflow and again for this record on 2026-09-23): + +- "Ultracode is a Claude Code setting rather than a model effort level: it sends + `xhigh` to the model and additionally has Claude orchestrate dynamic workflows for + substantive tasks." When `CLAUDE_CODE_EFFORT_LEVEL` "is set to a level other than + `xhigh`, requests run at that level and ultracode's workflow orchestration stays + inactive." +- On the settings keys: "`max` isn't accepted as a level in either key". On + sessions: "Unless you set it through the `CLAUDE_CODE_EFFORT_LEVEL` environment + variable, Claude Code applies `max` to the current session only." +- `max` "may show diminishing returns and is prone to overthinking. Test before + adopting broadly". +- The effort table lists `max` for Opus 5.5, Fable 5.1 and Sonnet 5; a level the + model lacks falls back to the highest supported level at or below it; models the + table does not list, Haiku 4.5 among them, take no effort level. + +The platform effort documentation, as fetched by the research workflow, recommends +an effort sweep on your own evaluations for Opus 5.5; that page was not re-read for +this record. + +**GitHub and cloud sessions.** No GitHub Actions workflow in agent-lab or in this +catalog invokes Claude (searched 2026-09-23). The `action.yml` of +`anthropics/claude-code-action` at `8cf3482550831fb35a4fc3fbf7ca139cf8028b4c` +(committed 2026-09-23) declares 39 inputs, among them `settings` ("Claude Code +settings as JSON string or path to settings JSON file") and `claude_args` +("Additional arguments to pass directly to Claude CLI"), and no `effort`, +`ultracode` or `model` input. The +[cloud-session settings documentation](https://code.claude.com/docs/en/settings#settings-in-cloud-sessions) +says a session with one repository reads that repository's committed +`.claude/settings.json`, while a session with several repositories reads only its +`enabledPlugins` and `extraKnownMarketplaces` keys. Neither the action nor a cloud +session was run for this record. + +## Decision + +1. **The main loop stays at Ultracode everywhere** (`xhigh` plus dynamic workflow + orchestration): the user-scope settings template as it is, and a new committed + [`.claude/settings.json`](../../.claude/settings.json) in this repository, byte + for byte the [portable example](../../examples/claude-native/ultracode.settings.json) + and agent-lab's committed project settings (`enableWorkflows`, `ultracode`, the + `unrestricted` size guideline, the concurrency cap of eight and spawn depth one), + so single-repository cloud sessions on this repository start with it. It adds + neither `effortLevel` (a `max` there is dropped, P7 and P8) nor + `CLAUDE_CODE_EFFORT_LEVEL` (any value other than `xhigh` disables Ultracode, and + `max` there overrode every child's effort: P1, P5, P6 and P9). +2. **Every project agent and every saved workflow stage runs at `max`, with + task-matched models kept:** Sonnet for `source-scout` and `isolated-builder`; + Opus for `evidence-reviewer`, `semantic-evidence-reviewer` and `blind-judge`. + The five shipped definitions change only their frontmatter line to + `effort: max` (H1: frontmatter `max` runs at `max` under an Ultracode parent). + Ad-hoc workflow scripts pass `effort: 'max'` on every `agent()` call, because a + stage without it inherits `xhigh` (H1). +3. **One non-orchestrated session at `max` stays available on request:** + `claude --effort max` or `/effort max`. Either turns Ultracode off for that + session (P2 measured the flag; `/effort max` picks `max` in place of the + `ultracode` entry of the same menu), and neither persists. +4. **GitHub:** a future Actions workflow built on `claude-code-action` requests + Ultracode with `claude_args: '--effort ultracode'` or the `settings` input + `'{"ultracode": true}'`, and never puts `--effort max` in `claude_args` beside + Ultracode, because an explicit `max` wins and turns orchestration off (P2). + +**Cost.** Children are expected to spend more output tokens than at `medium` or +`high`; neither that cost nor a quality gain has been measured yet. +`autoContinueAtUsageLimit` stays on in the settings template. Record each run's +per-child model, effort and usage with `child-usage.mjs --latest` so the overturn +comparison below has data. + +## Alternatives considered + +- **`CLAUDE_CODE_EFFORT_LEVEL=max` everywhere.** Rejected: it disables Ultracode + orchestration for the main loop (P1) and forces every child to `max` regardless of + its definition (P5, P6, P9), so no stage could ever be set lower on evidence. +- **Keep the task-matched efforts (Sonnet/medium, Opus/high).** Rejected: contrary + to the user's requirement of `max` for all workflows. +- **`max` for reviewers and judges only.** Rejected: the user asked for all + workflows, scouts and builders included. +- **Persist `max` through `effortLevel` or `modelSettings`.** Not available: the + client drops the value without a warning (P3, P4, P7, P8). + +## Overturn conditions + +Revisit this record when any of these happens: + +- a measured effort sweep on the comparison harness (agent-lab `tools/compare`) + shows that `max` gives no quality gain over `xhigh` or `high` for a role at higher + cost; that role returns to the cheaper effort; +- usage limits block work; +- a Claude Code release accepts `max` together with Ultracode orchestration; the main + loop then moves to `max`; +- `max` overthinking is observed to regress a gated result. + +## Limitations + +- **Keep-but-compare.** The per-role qualifications in the Ultracode recipe were + recorded at the earlier efforts; no run has compared `max` with them on the same + tasks. +- **One host, one release.** The probes ran once each, on one account, with Claude + Code 2.1.281; a later release can change the schema or the interaction with + Ultracode. Repeat the probe method above after an upgrade before relying on it. +- **Not executed:** `claude-code-action` and cloud sessions (documentation and + source only). +- **Dated examples unchanged.** `examples/claude-native/agents/` and the saved + workflow examples keep the efforts they were qualified at, and their contract + suite pins those values; the Ultracode recipe says how to raise them when + adopting. +- **Released tags predate this change.** `v2026.09.23` has no + `adoption/agents/claude/`, and `v2026.09.23.1` (this record's base commit) ships the + five definitions at the earlier `medium`/`high` efforts. A host installing from + either tag gets the `max` definitions only after the next release and re-pin, or by + running `tools/adoption/install_claude_profile.py` from a checkout that has this + change. The installer refreshes any differing catalog-owned agent file, so a host + that already installed the earlier copies gets the `max` versions on a re-run. diff --git a/manifests/evidence.json b/manifests/evidence.json index d95c62145..42e0bb821 100644 --- a/manifests/evidence.json +++ b/manifests/evidence.json @@ -28641,8 +28641,8 @@ }, { "path": "recipes/claude-native-ultracode.md", - "sha256": "1333f3519a190e73b5bfadd37808164ccefebabe88617f2107f1b47c73a912e5", - "bytes": 14403 + "sha256": "fa8ff1b5d35b8af7dfc8fd5e4c4bfd4617c21bbd665d67dd283fe2e7edd28024", + "bytes": 21050 }, { "path": "recipes/native-upgrades-20260921.md", diff --git a/recipes/claude-native-profile.md b/recipes/claude-native-profile.md index 8b6b3f685..9a11194ef 100644 --- a/recipes/claude-native-profile.md +++ b/recipes/claude-native-profile.md @@ -184,9 +184,28 @@ their previous selection; Claude supports `/effort` for the current session. Do not interrupt active work to reload a default. [Codex worker settings](https://learn.chatgpt.com/docs/agent-configuration/subagents) and [Claude subagent frontmatter](https://code.claude.com/docs/en/sub-agents) can override effort/model inheritance, so do not claim all workers run at the -coordinator's maximum. The short global instruction example makes task-based +coordinator's effort. The short global instruction example makes task-based acceptance, original-source verification and independent review persistent. +**2026-09-23: Claude children run at `max`; the coordinator stays at +Ultracode.** Re-measured on Claude Code 2.1.281: `max` in `effortLevel` or +`modelSettings..effortLevel` is still silently dropped (the session +stays at `xhigh`), and a session started with `--effort max` or +`CLAUDE_CODE_EFFORT_LEVEL=max` ran at `max` with Ultracode orchestration off; +`/effort max` likewise replaces the `ultracode` entry of the same menu. The +environment variable also overrides every child's frontmatter and workflow-stage +effort, so keep it unset. The shipped [agent definitions](../adoption/agents/claude/) +therefore declare `effort: max` beside their task-matched models (Sonnet for +`source-scout` and `isolated-builder`; Opus for `evidence-reviewer`, +`semantic-evidence-reviewer` and `blind-judge`), and workflow stages pass +`effort: 'max'` explicitly, since a stage without it inherits the +coordinator's `xhigh`. Verify each child's resolved effort in its transcript +rather than inferring it from a definition. The +[Ultracode recipe](claude-native-ultracode.md#child-effort-max-under-an-ultracode-coordinator) +has the per-stage rule and the +[decision record](../docs/decisions/2026-09-23-max-effort-default.md) has the +probes, alternatives and overturn conditions. + The [settings receipt](../evidence/receipts/native-quality-defaults-20260920.json) records supported values, effective configuration and preservation checks. More reasoning can increase time and tokens; no quality improvement or savings is diff --git a/recipes/claude-native-ultracode.md b/recipes/claude-native-ultracode.md index 735f4d811..792e98388 100644 --- a/recipes/claude-native-ultracode.md +++ b/recipes/claude-native-ultracode.md @@ -85,10 +85,15 @@ on another account/provider. The [subsequent native observations](../evidence/ar retain fresh-session settings/role discovery and returned usage separately from the interrupted cross-client attempt; they are not another PC's acceptance. -Use this profile for substantial parallel work. Routine work stays at ordinary -effort. Native Ultracode can increase tokens and elapsed time; it is not a general -token-saving switch. In a `-p` run, load this opted-in settings file or pass -`--effort ultracode`; the word in a prompt does not enable the feature. +Since 2026-09-23 this profile is the default main loop for every session, not +only for substantial parallel work +([decision record](../docs/decisions/2026-09-23-max-effort-default.md)). +Ultracode orchestrates workflows only for substantive tasks, and the sizing rule +above keeps conversational and mechanical turns solo; a lower effort remains a +per-session `/effort` choice. Native Ultracode can increase tokens and elapsed +time; it is not a general token-saving switch. In a `-p` run, load this +opted-in settings file or pass `--effort ultracode`; the word in a prompt does +not enable the feature. [Persistent settings](https://code.claude.com/docs/en/settings-reference#ultracode), [native model configuration](https://code.claude.com/docs/en/model-config), [workflow behavior](https://code.claude.com/docs/en/workflows). @@ -103,14 +108,28 @@ Fable 5.1 is the escalation path when a task specifically needs the prior coordinator's demonstrated multi-agent-graph behavior (below) rather than the new default. +**2026-09-23, later: every child role runs at `max` effort.** The coordinator +stays Opus 5.5 at Ultracode (`xhigh` plus dynamic workflow orchestration). +Every child keeps its task-matched model and requests `max`: Sonnet for scouts +and builders, Opus for reviewers and judges. Each child row's Qualification +below predates this change and was recorded at the earlier task-matched +efforts (Sonnet/medium, Opus/high). No run has compared `max` with those +efforts on the same tasks yet, so the effort choice stays keep-but-compare +until the sweep named in the +[decision record](../docs/decisions/2026-09-23-max-effort-default.md) runs. +The [section below](#child-effort-max-under-an-ultracode-coordinator) has +the measured constraints and the per-stage rule. + | Role | Starting choice | Qualification | | --- | --- | --- | -| Requirements, decomposition, integration and hard judgments | Opus 5.5, Ultracode for substantial graphs; escalate to Fable 5.1 for a task needing its previously demonstrated graph-coordination behavior | Coordinator observed as Opus 5.5/xhigh on this host as of 2026-09-23; Fable 5.1/xhigh's own multi-agent-graph coordination (Sonnet 5 and Opus 5 workers) remains the escalation's own qualification below | -| Exact extraction, inventories, running acceptance commands | `source-scout` (Sonnet, medium; four built-in tools, no project instructions) | First prompt 8,048 tokens versus 42,396 for the default child on one identical task; ran the inventory stage of eight native reviews and the readers of two readiness audits (one deployed, one in the scratch adoption); the recheck stage exists since the eighth review and ran there and in the three scratch-adoption reviews | -| Implementation from a clear contract | `isolated-builder` (Sonnet, medium, own worktree, named MCP tools behind ToolSearch) | One real task: a manifest probe implemented, checked and committed from its own worktree (first prompt 17,864) | -| Independent review from source and recorded evidence | `evidence-reviewer` (Opus, high; read-only named MCP tools behind ToolSearch, no Bash/Edit/Write) | Eight native review runs; first prompt 12,164 for the deferred shape versus 42,220 with bare server grants | -| Cheap exact extraction | Haiku | Not routed: on one byte-identical packet the Opus verifier scored Sonnet 14/14 lane rows and Haiku 9/14 with a quote attributed to a file that does not contain it; overturn only after a repeat trial with no unanchored citation on two distinct packets | -| Independent cross-family review | Existing official Codex companion | Reuse its separately recorded native acceptance; this trial did not run Codex inside a Workflow graph | +| Requirements, decomposition, integration and hard judgments | Opus 5.5 at Ultracode (`xhigh` plus dynamic workflow orchestration), the default for every session; escalate to Fable 5.1 for a task needing its previously demonstrated graph-coordination behavior | Coordinator observed as Opus 5.5/xhigh on this host as of 2026-09-23; Fable 5.1/xhigh's own multi-agent-graph coordination (Sonnet 5 and Opus 5 workers) remains the escalation's own qualification below | +| Exact extraction, inventories, running acceptance commands | `source-scout` (Sonnet, max; four built-in tools, no project instructions) | First prompt 8,048 tokens versus 42,396 for the default child on one identical task; ran the inventory stage of eight native reviews and the readers of two readiness audits (one deployed, one in the scratch adoption); the recheck stage exists since the eighth review and ran there and in the three scratch-adoption reviews | +| Implementation from a clear contract | `isolated-builder` (Sonnet, max, own worktree, named MCP tools behind ToolSearch) | One real task: a manifest probe implemented, checked and committed from its own worktree (first prompt 17,864) | +| Independent review from source and recorded evidence | `evidence-reviewer` (Opus, max; read-only named MCP tools behind ToolSearch, no Bash/Edit/Write) | Eight native review runs; first prompt 12,164 for the deferred shape versus 42,220 with bare server grants | +| Review of supplied semantic (TypeSafe) judgments against original source | `semantic-evidence-reviewer` (Opus, max; Read, Glob and Grep, `typesafe-ai` skill preloaded) | One probe, `wf_20a5e69a-84d`, measured the skill preload (first prompt 15,059 tokens; [convergence record](../docs/harness-rules-convergence-20260922.md)); it proposes in the vendored [layer-verdict lane](../examples/claude-native/workflows/layer-verdict-lane.js); no quality comparison with another reviewer is recorded | +| Judging or refuting one sealed comparison packet | `blind-judge` (Opus, max; Read only, no project instructions) | Its frontmatter was checked against the agent contract in the [convergence record](../docs/harness-rules-convergence-20260922.md); its body was not reviewed there, and this catalog records no dated run of the role | +| Cheap exact extraction | Haiku | Not routed: on one byte-identical packet the Opus verifier scored Sonnet 14/14 lane rows and Haiku 9/14 with a quote attributed to a file that does not contain it; overturn only after a repeat trial with no unanchored citation on two distinct packets. Haiku 4.5 takes no effort level, so `max` does not apply to it | +| Independent cross-family review | Existing official Codex companion | Reuse its separately recorded native acceptance; this trial did not run Codex inside a Workflow graph. Its reasoning effort follows the Codex configuration, not this table | These are starting choices, not a universal quality ranking. Set worker model and effort explicitly. Inspect native child metadata and returned model identity; @@ -141,6 +160,14 @@ static contract suite, a mutation harness and receipt bindings driven by a sibli `contract.config.json`). `review-changes` re-runs every acceptance command with a second worker and compares both runs in code. Record each run's children with `node .claude/workflows/child-usage.mjs --latest`. +These examples are dated evidence: their agents, stages and contract suite +keep the efforts they were qualified at (Sonnet/medium, Opus/high). When +adopting them after 2026-09-23, raise each stage and agent effort to `max` +under the per-stage rule below, and update the suite to match: the `ROUTING` +table in `test-envelope.mjs` (which already accepts `max`) and the effort text +that cases in `test-contract-mutations.mjs` search for; the shipped +[`adoption/agents/claude/`](../adoption/agents/claude/) definitions already +declare it. Adopt selected files into a project's existing `.claude/` directories without overwriting its instructions, accounts or permissions. A saved script is a supported native extension, not an upstream-authored acceptance policy. @@ -162,10 +189,77 @@ Save a useful script from that view into the selected project's `.claude/workflo or the user's workflow directory, then `/reload-skills` when needed. Do not publish live host paths or save a one-off audit as a universal project workflow. +### Child effort: `max` under an Ultracode coordinator + +Measured on 2026-09-23 with Claude Code 2.1.281 in headless probes, each started +in a fresh directory; the probes left the user settings file unchanged. The +[decision record](../docs/decisions/2026-09-23-max-effort-default.md) lists each +probe and its result. + +- **`max` disables Ultracode orchestration.** Ultracode sends `xhigh` to the + model and additionally has Claude orchestrate dynamic workflows; any other + resolved effort, `max` included, leaves orchestration inactive. Sessions + started with `--effort max` or `CLAUDE_CODE_EFFORT_LEVEL=max` ran at `max` + and had no Ultracode system reminder, so one session cannot have both. +- **`max` cannot be persisted.** `effortLevel: "max"` and + `modelSettings..effortLevel: "max"` were silently dropped from + `--settings` and from a project `.claude/settings.json`, with or without + `ultracode: true` beside them: the session stayed at `xhigh`, with no + warning. The installed schema accepts only `low`, `medium`, `high` and + `xhigh` in both keys, and the official docs say `max` "isn't accepted as a + level in either key" and otherwise applies to the current session only. +- **`CLAUDE_CODE_EFFORT_LEVEL` overrides every child's effort.** With it set + to `max`, a default Agent child, a `source-scout` whose frontmatter then said + `sonnet`/`medium` and a workflow stage that passed `effort: 'high'` all ran + at `max`. Do not set it: besides disabling Ultracode, it erases every + per-agent and per-stage effort. + +Under the default Ultracode coordinator (`xhigh`, orchestration active), a +workflow stage that passed `effort: 'max'` ran at `max`, an Agent whose +frontmatter says `effort: max` ran at `max`, and a stage without `effort` +inherited `xhigh`. The installed binary has no default-effort key for +subagents or workflow stages (`CLAUDE_CODE_SUBAGENT_MODEL` sets only the +model), so name the effort wherever a child is defined: + +- every project agent declares `effort: max` in its frontmatter beside its + task-matched `model` (the shipped [agent definitions](../adoption/agents/claude/) + do); +- every saved workflow stage and every ad-hoc `agent()` call in a workflow + script passes `effort: 'max'` together with an explicit `model`, because a + stage without `effort` inherits the coordinator's `xhigh`; +- a single non-orchestrated session at `max` stays available on request with + `claude --effort max` or `/effort max`; either turns Ultracode off for that + session. + +The docs list `max` for Opus 5.5, Fable 5.1 and Sonnet 5 (the probes observed +it on Opus 5.5 and Sonnet 5); a model without it falls back to its highest +supported level, and Haiku 4.5 takes no effort level. The +official docs warn that `max` "may show diminishing returns and is prone to +overthinking" and advise testing before adopting it broadly. Children are +expected to spend more output tokens, and no sweep has measured that cost or a +quality gain yet (`autoContinueAtUsageLimit` stays on in the settings template). + +## GitHub and cloud sessions + +No GitHub Actions workflow in this catalog invokes Claude. For a future one +built on `anthropics/claude-code-action` (its `action.yml` read at +`8cf3482550831fb35a4fc3fbf7ca139cf8028b4c`, 2026-09-23): the action has no +`effort`, `ultracode` or `model` input. Request Ultracode through +`claude_args: '--effort ultracode'` or the `settings` input +`'{"ultracode": true}'`, and never put `--effort max` in `claude_args` beside +Ultracode: an explicit `max` wins and turns orchestration off. This +repository's committed [`.claude/settings.json`](../.claude/settings.json) is +the settings example above, byte for byte. Per the official +[cloud-session settings](https://code.claude.com/docs/en/settings#settings-in-cloud-sessions) +docs, a cloud session on this one repository reads it, while a session with +several repositories reads only its `enabledPlugins` and +`extraKnownMarketplaces` keys. Neither the action nor a cloud session was run; +this section rests on the documentation and the action's source. + ## Native sessions, result return and messaging ```sh -claude --bg --name scoped-worker --model sonnet --effort medium "Bounded task" +claude --bg --name scoped-worker --model sonnet --effort max "Bounded task" claude agents --json --all --cwd "$PWD" claude logs WORKER_ID claude attach WORKER_ID From 4140e88c0b03f846acd6f55fe00a67eecf7e023c Mon Sep 17 00:00:00 2001 From: seathatflowsinourveins <234074349+seathatflowsinourveins@users.noreply.github.com> Date: Wed, 23 Sep 2026 20:17:16 -0400 Subject: [PATCH 2/6] Carry max effort through the examples, vendored lane and docs; fix the review findings Rebased onto the current main (v2026.09.23.1 pinned by #148). The five adoption/agents/claude definitions differ from that release, so adoption/bootstrap.md now says they "changed after `v2026.09.23.1`", and tests/test_adoption_docs_consistency.py treats what install_claude_profile.py copies (the agent and hook directories, the MCP template) as install inputs, matching a directory by its path, never its last component. The stale v2026.09.23 notes are dropped from the bootstrap, platform, README and next-host pages: the pinned release contains everything they described. Examples: the four example agents and every agent() stage of review-changes.js, readiness-audit.js and layer-verdict-lane.js bind effort max with models unchanged; the three scripts are byte-identical to agent-lab 915e73e. The vendored lane now names blind-lane-reviewer, vendored beside it and pinned under required_agents in vendored-lanes.json; SHA256SUMS, vendored-lanes.json and two appended lane-provenance.json entries rebind the new bytes. The contract suite pins max in ROUTING (which also clears the routing failure the old vendored lane already had on main) and now rejects a stage or agent at any other effort, a second effort line, MODEL drift, a routing table that restates an agent differently, CLAUDE_CODE_EFFORT_LEVEL or an effort cap in the settings, and instructions without the `effort: 'max'` literal; of 24 new mutations, 22 fail on their named assertion and two harmless edits do not. examples/claude-native/CLAUDE.md takes the host's current worker sentence, and AGENTS.md tells the coordinator to pass effort: 'max' on ad-hoc agent() calls and to stay at xhigh. Recipes and decision record: GitHub runs are non-interactive Agent SDK runs whose Workflow launches go through normal permission evaluation; autoContinueAtUsageLimit is no cost control and headless, SDK, GitHub and background runs fail at a usage limit; the claude-code-action pin is "read 2026-09-23" without an input count; the action's one CI smoke is cited and no Ultracode or effort run is claimed; /effort max is marked as not probed; Q1/Q2 replace the confounded P3/P4/P7/P8 as the evidence that a persisted max is dropped, and Q3 adds that a stage's effort wins over the frontmatter; CLAUDE_CODE_EFFORT_LEVEL is never set. catalogs/foundation/decisions.json's lean-workflow-child-routing activation names the max efforts and cites the decision. A new test requires one "effort: max" frontmatter line in each shipped agent. Checks (raw): validate.py, validate_catalogs.py, evidence_manifest.py --check, new_host_grand_list.py --check, build_ecosystem.py --check and release_due.py --strict-if-repinned origin/main (exit 0, not strict: no re-pin, nothing due) pass; the example suites pass as CI runs them (sha256sum --check) and as their README runs them (envelope 207/207, mutations 47/47, child-usage 16/16, usage-receipts 18/18, codex-envelope, check-syntax); the named unittest modules and the related ones pass. Co-Authored-By: Claude Opus 5.5 (1M context) --- AGENTS.md | 9 + adoption/README.md | 14 +- adoption/bootstrap.md | 33 +-- adoption/platforms/linux-wsl2.md | 10 +- adoption/platforms/macos-arm64.md | 55 +--- catalogs/foundation/decisions.json | 8 +- .../2026-09-23-max-effort-default.md | 276 +++++++++++------- examples/claude-native/CLAUDE.md | 2 +- .../claude-native/agents/evidence-reviewer.md | 2 +- .../claude-native/agents/isolated-builder.md | 2 +- .../agents/semantic-evidence-reviewer.md | 4 +- examples/claude-native/agents/source-scout.md | 2 +- examples/claude-native/workflows/README.md | 65 +++-- .../workflows/readiness-audit.js | 8 +- .../claude-native/workflows/review-changes.js | 12 +- .../workflows/test-contract-mutations.mjs | 42 ++- .../claude-native/workflows/test-envelope.mjs | 49 +++- manifests/evidence.json | 44 +-- recipes/claude-native-profile.md | 26 +- recipes/claude-native-ultracode.md | 128 +++++--- tests/test_adoption_docs_consistency.py | 56 +++- tests/test_install_claude_profile.py | 33 +++ tests/test_verdict_lane_vendoring.py | 18 ++ 23 files changed, 585 insertions(+), 313 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f5f3f36c6..940c8f170 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -65,6 +65,15 @@ Use upstream executables and supported integration formats. Keep client accounts One coordinator integrates. Writing workers need separate worktrees and bounded file ownership. Evidence belongs in compact sanitized receipts; no raw conversations, tokens, personal paths or machine-specific active client configuration. +This repository commits `.claude/settings.json` with Ultracode on. The Claude +coordinator stays at `xhigh` under Ultracode, because a `max` session turns its +workflow orchestration off, and never sets `CLAUDE_CODE_EFFORT_LEVEL` (any value +overrides every child's effort). Pass `effort: 'max'` with an explicit +task-matched `model` on every ad-hoc workflow `agent()` call: a stage without +its own `effort` inherits the coordinator's `xhigh` unless its agent's +frontmatter sets one. Probes and overturn conditions: +`docs/decisions/2026-09-23-max-effort-default.md`. + For general engineering and ecosystem changes, start with `docs/convergence-architecture.md`. New convergence claims use `scripts/validate_convergence.py` with a scoped experiment record. Preserve failed diff --git a/adoption/README.md b/adoption/README.md index 8fa22dc32..05165d0d3 100644 --- a/adoption/README.md +++ b/adoption/README.md @@ -28,20 +28,20 @@ The pin columns count how many of a profile's components have a SHA-256 pin in [`pins-macos-arm64.json`](pins-macos-arm64.json), which is all the bootstrap scripts install; the repository test `test_adoption_docs_consistency.py` recomputes them. A count is main's; "(X at `vT`)" after it is the coverage release `vT`'s own -pin files give, shown while the pinned release differs (and kept as history -after a re-pin). The script refuses (exit 3) a profile with an unpinned -component until those ids are named in `--allow-unpinned`; it then installs -the pinned ones and skips the named ones, which you install through their -recipes ([bootstrap step 2](bootstrap.md)). +pin files give, shown while the pinned release differs (a re-pin may leave it +as history until a later change drops it). The script refuses (exit 3) a +profile with an unpinned component until those ids are named in +`--allow-unpinned`; it then installs the pinned ones and skips the named ones, +which you install through their recipes ([bootstrap step 2](bootstrap.md)). | Adoption profile | Selects | Linux pins | macOS pins | Next native acceptance | | --- | --- | --- | --- | --- | | `foundation-cpu` | Codex, Claude Code, Context Mode, RTK, QMD BM25, explicitly scoped ai-memory, MCPorter | all 7 | 5 of 7 | Native client setup; one useful context/document call and scoped memory retrieval. On macOS use `macos-arm64-foundation` | -| `macos-arm64-foundation` | macOS only, drafted: Codex, Claude Code, Context Mode, ai-memory, MCPorter, llama.cpp Metal embedding, Qdrant, SocratiCode | 5 of 8 | all 8 (7 of 8 at `v2026.09.23`) | The macOS acceptance lane on [the macOS page](platforms/macos-arm64.md); not accepted | +| `macos-arm64-foundation` | macOS only, drafted: Codex, Claude Code, Context Mode, ai-memory, MCPorter, llama.cpp Metal embedding, Qdrant, SocratiCode | 5 of 8 | all 8 | The macOS acceptance lane on [the macOS page](platforms/macos-arm64.md); not accepted | | `research-runtime` | Historical hash-locked SDK/DuckDB, Dagu and LEAN comparison lane | 2 of 11 | 2 of 11 | Reproduce the retained comparison; this profile does not override the Nautilus destination | | `trading-nautilus` | Selected pinned Nautilus engine and separate Alpaca boundary | none of 2 | none of 2 | Reproduce the bounded engine check; qualify each broker independently | | `observability` | Collector, Prometheus, Loki, Grafana, Alertmanager, ntfy | none of 6 | none of 6 | Native config validation, actual task/event delivery, matching usage categories | -| `semantic-rag` | HF, vLLM, Qdrant, SocratiCode | none of 4 | 2 of 4 (1 of 4 at `v2026.09.23`) | Hardware-compatible model serving, explicit project index and real retrieval/watcher behavior | +| `semantic-rag` | HF, vLLM, Qdrant, SocratiCode | none of 4 | 2 of 4 | Hardware-compatible model serving, explicit project index and real retrieval/watcher behavior | | `recovery` | Restic plus selected ai-memory/Qdrant application state | 1 of 3 | 2 of 3 | Isolated restore, logical comparison, independent key/destination, then explicit consumer cutover | The [reference manifest](manifest.json) maps **every selected component ID** to its native guide, including optional components outside these starting profiles. The offline HTML setup guide (`docs/ecosystem/index.html`) generates current counts and embeds these recipes alongside layer/profile selection, scoped acceptance and measured baseline choices; it is generated, not committed -- build it with `python3 scripts/build_ecosystem.py --write`, or download it from a `publish-catalog.yml` workflow artifact (7-day retention, `workflow_dispatch`/`v*`-tag runs only). The [lifecycle guide](lifecycle.md) covers ownership, restart, recovery and rollback. The [portability comparison](research.md) explains why native uv is the required dependency tool and other environment managers remain optional. diff --git a/adoption/bootstrap.md b/adoption/bootstrap.md index 417846c40..9ce263334 100644 --- a/adoption/bootstrap.md +++ b/adoption/bootstrap.md @@ -14,7 +14,7 @@ it (a commit cannot contain its own hash), and the pinned tag may predate ```sh git clone https://github.com/seathatflowsinourveins/native-agent-stack.git cd native-agent-stack -python3 scripts/release_due.py # on the default branch (added after v2026.09.23): steps main documents that the pinned release lacks +python3 scripts/release_due.py # on the default branch: steps main documents that the pinned release lacks tag="$(python3 -c "import json;print(json.load(open('adoption/manifest.json'))['source']['release_tag'])")" commit="$(python3 -c "import json;print(json.load(open('adoption/manifest.json'))['source']['release_commit'])")" git checkout "$tag" @@ -25,8 +25,8 @@ and confirms it resolves to `source.release_commit` (read from the manifest you cloned, never from a commit pasted into prose, which a later re-pin would leave stale), published with SLSA build provenance by `.github/workflows/publish-catalog.yml`. `scripts/release_due.py` -was added after `v2026.09.23`, so it runs here, on the default branch, before -the checkout. After checkout, follow the documents in your checkout. +runs here, on the default branch, before the checkout. After checkout, follow +the documents in your checkout. The pages here name the release a step was written against. "Added after `vT`" means release `vT` lacks the file that step uses (a path in @@ -120,16 +120,6 @@ GitHub-hosted macOS runner; see "none of N" pinned installs none of its own components through the script (only the `node`, `uv` and `gh` every run installs); use the recipes. - The macOS script and pins changed after `v2026.09.23`. At `v2026.09.23`, - `adoption/bootstrap-macos.sh` installs only a missing `jq` through Homebrew - (install the other formulae on - [the macOS page](platforms/macos-arm64.md#prerequisites) yourself first), - `adoption/pins-macos-arm64.json` has no `socraticode` pin (the script skips - it by default, so `macos-arm64-foundation` installs 7 of 8), no pinned - darwin binary for Codex or Claude Code (npm resolves those unverified) and - no embedding-model pin. Linux/WSL2's script and pins are the same at - `v2026.09.23` as on main. - 3. **Native sign-in.** Neither client's credentials transfer between machines (`adoption/manifest.json` `policy.authentication_transfer: native_login_on_target_only`). Use each client's own device flow: @@ -238,10 +228,13 @@ GitHub-hosted macOS runner; see template does not mention is kept), writes atomically and preserves the original file's mode bits. Never touches `~/.claude.json` or any credential store. - The profile installer, the apply tool and their assets - (`adoption/hooks/claude/`, `adoption/agents/claude/`, - `adoption/mcp/claude-user.json`) were added after `v2026.09.23`; at that - tag, merge the rendered settings by hand. + The agent definitions in `adoption/agents/claude/` changed after `v2026.09.23.1`: + at that tag they declare `effort: medium` (`source-scout`, + `isolated-builder`) or `effort: high` (`evidence-reviewer`, + `semantic-evidence-reviewer`, `blind-judge`) instead of `effort: max` + ([decision](../docs/decisions/2026-09-23-max-effort-default.md)). The + installer replaces a differing agent file, so rerunning its agents step + from a checkout that has the change installs the `max` definitions. **Plugin revision check** (added after `v2026.09.23.1`; it reads only this host's plugin registry, so it runs the same from any checkout). A Claude @@ -289,8 +282,7 @@ GitHub-hosted macOS runner; see `systemctl --user` on Linux/WSL2 (owned units only; never stop the shared MCPorter daemon to "clean up" another component), `launchctl` on macOS (table in [the macOS page](platforms/macos-arm64.md#launchd-services); run - only on a hosted runner, not yet on a Mac workstation; its launchd - templates were added after `v2026.09.23`). For the portable guarded runner wrappers used by + only on a hosted runner, not yet on a Mac workstation). For the portable guarded runner wrappers used by these services, see `adoption/tools/README.md`. 6. **Prerequisite report.** `uv run --no-project --python 3.13 python scripts/adoption_status.py --profile --json` @@ -310,8 +302,7 @@ GitHub-hosted macOS runner; see has neither. `scripts/host_receipts.py` (this same step's own recording tool, plus `scripts/component_matrix.py`, `scripts/new_host_grand_list.py`, `tools/sota-convergence/build_verdicts.py`, - `scripts/validate_convergence.py` and `scripts/release_due.py`, which was - added after `v2026.09.23`) needs + `scripts/validate_convergence.py` and `scripts/release_due.py`) needs **Python 3.9 or newer**: every one of those scripts parses under the Python 3.9 grammar and uses `from __future__ import annotations`, and this is exercised directly, not merely declared -- a macOS CI job runs the diff --git a/adoption/platforms/linux-wsl2.md b/adoption/platforms/linux-wsl2.md index 4980ba4e9..b983d9f6d 100644 --- a/adoption/platforms/linux-wsl2.md +++ b/adoption/platforms/linux-wsl2.md @@ -10,7 +10,7 @@ archive): ```sh git clone https://github.com/seathatflowsinourveins/native-agent-stack.git cd native-agent-stack -python3 scripts/release_due.py # on the default branch (added after v2026.09.23): steps main documents that the pinned release lacks +python3 scripts/release_due.py # on the default branch: steps main documents that the pinned release lacks tag="$(python3 -c "import json;print(json.load(open('adoption/manifest.json'))['source']['release_tag'])")" commit="$(python3 -c "import json;print(json.load(open('adoption/manifest.json'))['source']['release_commit'])")" git checkout "$tag" @@ -21,9 +21,7 @@ That checkout target is `adoption/manifest.json` `source.release_tag` (or a later tag), confirmed at `source.release_commit` and published with SLSA build provenance by `.github/workflows/publish-catalog.yml`. Read both values before the checkout, as above: the release's own manifest names the release -before it. `scripts/release_due.py` itself was added after `v2026.09.23`, so -it runs on the default branch; nothing else on this page needs a newer -release. Do +before it, so `scripts/release_due.py` runs on the default branch. Do **not** check out `source.baseline_commit`: that field predates `adoption/` and `tools/adoption/` entirely and is never a checkout target (Codex cross-family review finding, `codex-review-72`; `codex-review-64` is the @@ -78,10 +76,6 @@ itself evidence the WSL UVA gap closed. 1. Follow [`adoption/bootstrap.md`](../bootstrap.md) steps 1–3 (prerequisites, `bootstrap-linux.sh --profile `, native sign-in). - `adoption/bootstrap-linux.sh` and its pins changed after `v2026.09.23` - (native `claude-code` pin, uv-tool `markitdown`/`tavily-cli`, - `--configure-claude-user-profile`); at `v2026.09.23` the script installs - the npm `claude-code` 2.1.278 pin and has no profile step. 2. Recreate the SDK only for the `research-runtime` profile using [`adoption/sdk/README.md`](../sdk/README.md)'s transitive lock; retain the same exact-match and uncached-reinstall checks as diff --git a/adoption/platforms/macos-arm64.md b/adoption/platforms/macos-arm64.md index 5d0e457b4..60b7aeaaf 100644 --- a/adoption/platforms/macos-arm64.md +++ b/adoption/platforms/macos-arm64.md @@ -26,7 +26,7 @@ these one-liners and `scripts/release_due.py`. ```sh git clone https://github.com/seathatflowsinourveins/native-agent-stack.git cd native-agent-stack -python3 scripts/release_due.py # on the default branch (added after v2026.09.23): steps main documents that the pinned release lacks +python3 scripts/release_due.py # on the default branch: steps main documents that the pinned release lacks tag="$(python3 -c "import json;print(json.load(open('adoption/manifest.json'))['source']['release_tag'])")" commit="$(python3 -c "import json;print(json.load(open('adoption/manifest.json'))['source']['release_commit'])")" git checkout "$tag" @@ -34,11 +34,10 @@ if test "$(git rev-parse HEAD)" = "$commit"; then echo "at $tag ($commit)"; else ``` Read both values before the checkout, as above: the release's own manifest -names the release before it. `scripts/release_due.py` was added after -`v2026.09.23`, so it runs on the default branch. +names the release before it, so `scripts/release_due.py` runs on the default +branch. -This page describes main, and much of the macOS install path is newer than -`v2026.09.23`. A section whose file release `vT` lacks says "added after +This page describes main. A section whose file release `vT` lacks says "added after `vT`"; one whose file exists at `vT` but behaves differently there says "changed after `vT`" and states the difference. If your checkout is `vT`, follow the note: run an "added after" step from a separate clone of the @@ -55,11 +54,7 @@ release, the note is history and the step is in your checkout (`test -e 2. `bash adoption/bootstrap-macos.sh --profile macos-arm64-foundation` (usage and exit codes below). Use `macos-arm64-foundation`, not `foundation-cpu`: `qmd` and `rtk` have no macOS pin, so `foundation-cpu` - exits 3 here. The script and `adoption/pins-macos-arm64.json` changed after - `v2026.09.23`: at that tag the script installs only a missing `jq` through - Homebrew, so run the `brew install` line under "Prerequisites" yourself - first, and it skips `socraticode` (no pin there), installing 7 of the 8 - components; install `socraticode` through its recipe. + exits 3 here. 3. Native sign-in and config rendering: [`adoption/bootstrap.md`](../bootstrap.md) steps 3–4 (Codex, Claude and GitHub device flows; `tools/adoption/render_config.py` with this host's own `adoption/hosts/.json`). @@ -99,13 +94,11 @@ rules. None of these sizes has a real qualification run yet. ## Prerequisites Homebrew, for six formulae this profile still leaves floating: jq, -python@3.13, ripgrep, coreutils, restic and shellcheck. On main, +python@3.13, ripgrep, coreutils, restic and shellcheck. [`adoption/bootstrap-macos.sh`](../bootstrap-macos.sh) installs these itself (the equivalent of running the command below), one `brew list --versions ` check per formula, installing only the ones actually missing — -manually running it first is no longer required, only still possible. That -changed after `v2026.09.23`: the script at that tag installs only a missing -`jq`, so at that tag run this command yourself before the bootstrap: +manually running it first is not required, only still possible: ```sh brew install jq python@3.13 ripgrep coreutils restic shellcheck @@ -168,12 +161,9 @@ acceptance — remains unrun. | 3 | A selected component has no pin at all in [`adoption/pins-macos-arm64.json`](../pins-macos-arm64.json) and was not named in `--allow-unpinned`. Checked before anything is installed, and in `--plan` too; `--allow-unpinned ` skips the named ids instead and echoes them to the run log. | | 4 | A prerequisite (`curl`, `git`, `tar`, `shasum`, `unzip`, `jq`, `mktemp`) is still missing after the Homebrew step. With `--skip-system-packages` no `brew install` is attempted and the check lists what is missing. | -On main every selected `macos-arm64-foundation` component, including +Every selected `macos-arm64-foundation` component, including `socraticode` (below), has a pin, so the shipped profile needs no -`--allow-unpinned`. The script and pins changed after `v2026.09.23`: at that -tag `socraticode` has no pin and the script skips it by default -(`documented_unpinned_ids=(socraticode)`), so the profile exits 0 with 7 of 8 -components installed and `socraticode` left to its recipe. +`--allow-unpinned`. Exit codes 3 and 4 and the `--allow-unpinned` flag mirror [`adoption/bootstrap-linux.sh`](../bootstrap-linux.sh). One difference is disclosed rather than hidden: the script keeps a `documented_unpinned_ids=()` @@ -212,12 +202,7 @@ machine-readable copy with each `checksum_source` and `checksum_ref` is `ignore_scripts: true` field, read by the script's `install_npm`), the same convention [`recipes/README.md`](../../recipes/README.md#paths-pins-and-installation-conventions) documents for the Linux recipe; it has no `adoption/pins-linux-x86_64.json` -entry of its own there, only that documented manual recipe. The `socraticode` -row, both `platform_dependency` pins below and the `models[0]` embedding-model -pin were added to `adoption/pins-macos-arm64.json`, which changed after -`v2026.09.23`: at that tag npm resolves the Codex and Claude Code darwin -binaries unverified and the embedding model is not downloaded by the -bootstrap. Not in this table: +entry of its own there, only that documented manual recipe. Not in this table: `gitleaks`, `syft` and `dagu`, which are not in the `macos-arm64-foundation` component list. @@ -263,9 +248,7 @@ install 2.1.280`, exactly mirroring `adoption/pins-linux-x86_64.json`'s own existing claude-code step. There is no more nested platform package, no `install.cjs` postinstall to defer, and no `postinstall_binary_check`; the native binary manages its own version directory and launcher and keeps -auto-updating on the latest channel afterward. Both claude-code pins -(this page's and `adoption/pins-linux-x86_64.json`'s) changed after `v2026.09.23`; -at that tag they are npm pins. +auto-updating on the latest channel afterward. `llama-server` is a profile `required_command`, so llama.cpp is pinned rather than left to `brew install llama.cpp`. The macOS asset holds every executable @@ -284,8 +267,6 @@ The `platform_profiles` row for `macos-arm64` names a hosted smoke job `75a6e0d` (PR #94, `workflow_dispatch`, runner label `macos-15`, macOS 15.7.9 build 24G830), every job green, recorded in [`evidence/receipts/adoption-macos-hosted-smoke-20260923.json`](../../evidence/receipts/adoption-macos-hosted-smoke-20260923.json). -That run and its receipt were added after `v2026.09.23`: they exercise the -#94 script and pins, not the ones in that tag. Earlier runs (`35753384567`/`585032a`, `35753801691`/`9d9ce2b`, both recorded in the retired 2026-09-22 receipt) predate the round-3g through 3i redesign covered here and are superseded by this one; they are not @@ -397,11 +378,6 @@ observation: it establishes that these scripts run under a real macOS Python 3.9 interpreter as installed by Apple on this runner image, not that every real Mac's system Python matches it forever. -`scripts/release_due.py` was added after `v2026.09.23`; that does not block a -host, because recording runs on a branch of current `main` -([`docs/contributing-evidence.md`](../../docs/contributing-evidence.md) step 1) -and step 0 runs the check on the default branch before the checkout. - ## Embedding backend decision **These are two separate embedding spaces by design, not two ports serving @@ -468,8 +444,7 @@ deterministic before ever comparing across hosts). This is a Linux-side capture, never a macOS observation, and it does not by itself establish what a real Mac's Metal backend would return. The one command a real Mac (or the hosted CI step below) runs against a live `llama-server` on port -8232 is (`tools/adoption/embed_acceptance.py` and this reference vector were -added after `v2026.09.23`; at that tag run it from a default-branch clone): +8232 is: ```sh python3 tools/adoption/embed_acceptance.py http://127.0.0.1:8232 \ @@ -543,12 +518,6 @@ hosted run proves" above) `launchd-agents.sh` bootstrapped and booted out the `qdrant` and `llama-embed` agents; `ai-memory` has not run, and none of the three has run on a Mac workstation. -**Added after `v2026.09.23`.** The three templates, `launchd-agents.sh`, -`tools/adoption/render_launchd.py`, `adoption/hosts/macos-example.json` and -`tests/test_adoption_launchd.py` are not in that tag (`python3 -scripts/release_due.py` on the default branch lists them while it is pinned); -a host at that tag runs this section from a default-branch clone. - **2026-09-23 decision: brew-services semantics, no backup or reconcile.** - **Chosen:** stateless, path-verified ownership with no backup, no diff --git a/catalogs/foundation/decisions.json b/catalogs/foundation/decisions.json index 27484b442..bb06d9379 100644 --- a/catalogs/foundation/decisions.json +++ b/catalogs/foundation/decisions.json @@ -2659,7 +2659,7 @@ ], "selection": "conditional", "review_status": "accepted_within_scope", - "activation": "Name `agentType` where the role fits: `source-scout` (Sonnet/medium) for extraction, inventories and acceptance commands, `evidence-reviewer` (Opus/high, read-only) for independent review, `isolated-builder` (Sonnet/medium, own worktree) for bounded implementation; keep the default child for a stage that must re-run commands and judge them; never omit a stage's model or effort; run `workflowSizeGuideline: unrestricted` with three concurrent agents and size by task; record every run with `child-usage.mjs --latest`.", + "activation": "Name `agentType` where the role fits: `source-scout` (Sonnet/max) for extraction, inventories and acceptance commands, `evidence-reviewer` (Opus/max, read-only) for independent review, `isolated-builder` (Sonnet/max, own worktree) for bounded implementation; keep the default child for a stage that must re-run commands and judge them; never omit a stage's model or effort, and since 2026-09-23 bind effort `max` on every stage and agent with the models unchanged while the coordinator stays at xhigh under Ultracode (docs/decisions/2026-09-23-max-effort-default.md); run `workflowSizeGuideline: unrestricted` with three concurrent agents and size by task; record every run with `child-usage.mjs --latest`.", "component_ids": [ "claude-code", "codex" @@ -2676,13 +2676,15 @@ "The review workflow's strict accepted status was never returned during the deployment; the gate was fix-and-commit after independent review.", "Local suites are stubbed or synthetic; only the native runs and the Prometheus deltas are provider evidence. Per-session scoping of the exported metric is unavailable by design (the collector keeps no session id on metrics), so observational deltas are host-wide time windows with concurrent activity declared.", "Codex agent examples inherit model and effort and have no end-to-end run of their own.", - "The project-only settings profile is not qualified: three provider errors and, with the coordinator model pinned, a permission-layer refusal of the Workflow tool; headless runs need a loaded settings source whose permission mode or allow rule covers the Workflow tool." + "The project-only settings profile is not qualified: three provider errors and, with the coordinator model pinned, a permission-layer refusal of the Workflow tool; headless runs need a loaded settings source whose permission mode or allow rule covers the Workflow tool.", + "The routed efforts moved from Sonnet/medium and Opus/high to max on 2026-09-23 (docs/decisions/2026-09-23-max-effort-default.md) without a new native run; the evidence above was recorded at the earlier efforts, so the effort choice is keep-but-compare." ], "source_paths": [ "docs/ultracode-token-routing-20260921.md", "recipes/claude-native-ultracode.md", "recipes/claude-codex-cooperation-lanes.md", - "examples/claude-native/workflows/README.md" + "examples/claude-native/workflows/README.md", + "docs/decisions/2026-09-23-max-effort-default.md" ], "next_gap": "Overturn only on evidence: Haiku after a repeat trial with no unanchored citation on two distinct extraction packets; the unrestricted guideline if recorded runs exceed their independent units or pause on fan-out rate limits; a one-hour child cache class only for a child that idles more than five minutes between requests; the Codex agent examples after a task-level run of their own.", "lifecycle": { diff --git a/docs/decisions/2026-09-23-max-effort-default.md b/docs/decisions/2026-09-23-max-effort-default.md index 275ba1fb4..56cee972f 100644 --- a/docs/decisions/2026-09-23-max-effort-default.md +++ b/docs/decisions/2026-09-23-max-effort-default.md @@ -1,96 +1,149 @@ # Decision: `max` effort for every Claude child, Ultracode for the main loop (2026-09-23) **Decided by:** unit `max-effort-default`, its own dedicated worktree, branch -`claude/max-effort-default-20260923` (base `origin/main@40828dfe`), from the user's -requirement of 2026-09-23: "effort max and ultracode default for all future workflow -and github, max quality". The user expected one session to run `max` and Ultracode at -the same time; the probes below show that the native client does not allow that, so -the requirement is met per layer instead. - -**Scope:** the five shipped agent definitions (`adoption/agents/claude/*.md`, kept -byte-identical to agent-lab's `.claude/agents/` copies, which receive the same -one-line change), a new committed project settings file for this repository -(`.claude/settings.json`), the model/effort guidance in -[`recipes/claude-native-ultracode.md`](../../recipes/claude-native-ultracode.md) and -[`recipes/claude-native-profile.md`](../../recipes/claude-native-profile.md). The -user-scope settings template (`adoption/templates/claude.settings.template.json`) -already carries `ultracode: true`, the `xhigh` effort defaults and -`autoContinueAtUsageLimit: true`, and no `CLAUDE_CODE_EFFORT_LEVEL`; it needs no change. -This record extends the [Claude user profile decision](2026-09-23-claude-user-profile.md); -its coordinator choice (Opus 5.5 at `xhigh`) is unchanged. +`claude/max-effort-default-20260923` (first based on `origin/main@40828dfe`, later +rebased onto the current `main`), from the user's requirement of 2026-09-23: "effort max +and ultracode default for all future workflow and github, max quality". The user +expected one session to run `max` and Ultracode at the same time; the probes below +show that the native client does not allow that, so the requirement is met per layer +instead. + +**Scope:** + +- the five shipped agent definitions (`adoption/agents/claude/*.md`, kept byte-identical + to agent-lab's `.claude/agents/` copies at agent-lab commit `915e73e`, which made the + same one-line change there); +- a new committed project settings file for this repository (`.claude/settings.json`) and + the coordinator rule in [`AGENTS.md`](../../AGENTS.md); +- the portable examples: [`examples/claude-native/CLAUDE.md`](../../examples/claude-native/CLAUDE.md), + the agents in `examples/claude-native/agents/`, the saved workflows (`review-changes.js`, + `readiness-audit.js` and the vendored `layer-verdict-lane.js`, all byte-identical to + agent-lab `915e73e`), the `blind-lane-reviewer` agent that lane now names, and their + contract suites; +- the model/effort guidance in + [`recipes/claude-native-ultracode.md`](../../recipes/claude-native-ultracode.md) and + [`recipes/claude-native-profile.md`](../../recipes/claude-native-profile.md), and the + `lean-workflow-child-routing` activation in `catalogs/foundation/decisions.json`. + +The user-scope settings template (`adoption/templates/claude.settings.template.json`) +already carries `ultracode: true`, the `xhigh` effort defaults and no +`CLAUDE_CODE_EFFORT_LEVEL`; it needs no change. This record extends the +[Claude user profile decision](2026-09-23-claude-user-profile.md); its coordinator +choice (Opus 5.5 at `xhigh`) is unchanged. ## Evidence -**Probe method.** Research workflow `wf_3f4689fe-c86` ran headless probes P0 to P9 -with Claude Code 2.1.281, each in a fresh scratch directory, on the host whose user -settings carry `ultracode: true` and `modelSettings.claude-opus-5-5.effortLevel: -"xhigh"`. The user settings file's hash was the same before and after the probes. -Each probe's resolved effort, for the session and for each child, was read from its -native transcript. Ultracode was counted as active when the transcript carried the -Ultracode system reminder, whose text begins "Ultracode is on: optimize"; the -unrelated `/workflow-authoring` reference sentence "Ultracode is on for the -session" was excluded. The transcripts stay on the probing -host (they hold local session identifiers and paths) and are not published here. +**Probe method.** Research workflow `wf_3f4689fe-c86` ran headless probes P0 to P9, a +further probe H1 followed, and three more headless sessions ran Q1 to Q3 later the same +day, all with Claude Code 2.1.281, each in a fresh scratch directory, on the host whose user settings +carry `ultracode: true` and `modelSettings.claude-opus-5-5.effortLevel: "xhigh"`. The +user settings file's hash was the same before and after the probes. Each probe's +resolved effort, for the session and for each child, was read from its native +transcript. Ultracode was counted as active when the transcript carried the Ultracode +system reminder, whose text begins "Ultracode is on: optimize"; the unrelated +`/workflow-authoring` reference sentence "Ultracode is on for the session" was +excluded. The transcripts stay on the probing host (they hold local session +identifiers and paths) and are not published here. | Probe | Configuration | Result | | --- | --- | --- | | P0 | Baseline: user `ultracode: true` plus the per-model `xhigh` | Session at `xhigh`; reminder present (1 line) | | P1 | Environment `CLAUDE_CODE_EFFORT_LEVEL=max` | Session at `max`; reminder absent (0 lines) | | P2 | `--effort max` | Session at `max`; reminder absent (0 lines) | -| P3 | `--settings` with `effortLevel: "max"` | Session at `xhigh`, no warning; reminder present | -| P4 | `--settings` with `modelSettings.claude-opus-5-5.effortLevel: "max"` | Session at `xhigh`, no warning; reminder present | +| P3 | `--settings` with `effortLevel: "max"` | Session at `xhigh`, no warning; reminder present. Confounded: see below | +| P4 | `--settings` with `modelSettings.claude-opus-5-5.effortLevel: "max"` | Session at `xhigh`, no warning; reminder present. Confounded: see below | | P5 | Environment `max`; one default Agent child | Child `claude-opus-5-5` at `max` | | P6 | Environment `max`; one `source-scout` child, frontmatter then `sonnet`/`medium` | Child `claude-sonnet-5` at `max`: the variable overrides frontmatter effort | -| P7 | Project `.claude/settings.json` with `effortLevel: "max"` | Session at `xhigh`, no warning; reminder present | -| P8 | Project `effortLevel: "max"` plus `ultracode: true` | Session at `xhigh`, no warning; reminder present | +| P7 | Project `.claude/settings.json` with `effortLevel: "max"` | Session at `xhigh`, no warning; reminder present. Confounded: see below | +| P8 | Project `effortLevel: "max"` plus `ultracode: true` | Session at `xhigh`, no warning; reminder present. Confounded: see below | | P9 | Environment `max`; one Workflow stage passing `effort: 'high'` | Stage at `max`: the variable overrides stage effort; reminder absent | - -A further probe, H1, ran with the default Ultracode parent at `xhigh` (reminder -present): a Workflow stage that passed `effort: 'max'` ran at `max` (agent -`a659fbb94398ea280`, label `stage-max`); a stage with no `effort` inherited `xhigh` -(agent `abd2b64f159add7c2`, label `stage-inherit`); an Agent whose frontmatter says -`effort: max` ran at `max`. +| Q1 | `--settings '{"ultracode":false,"effortLevel":"max"}'` | Session at `xhigh`, the user's per-model level, no warning; reminder absent: the `max` was dropped | +| Q2 | Control: `--settings '{"ultracode":false,"effortLevel":"high"}'` | Session at `high`; reminder absent: the key applies a valid value | +| Q3 | Default Ultracode parent at `xhigh` (reminder present); Workflow run `wf_45d3a3bd-6e6` with four stages | `{model: 'sonnet', effort: 'max'}`: `claude-sonnet-5` at `max`. `{agentType: 'source-scout', effort: 'max'}`, frontmatter `max`: Sonnet 5 at `max`. `{agentType: 'source-scout'}` with no stage effort, frontmatter `max`: Sonnet 5 at `max`. `{agentType: 'scout-medium', effort: 'max'}`, a copy whose frontmatter says `effort: medium`: Sonnet 5 at `max`, so the stage effort wins over the frontmatter | + +H1 ran with the default Ultracode parent at `xhigh` (reminder present): a Workflow +stage that passed `effort: 'max'` ran at `max` (agent `a659fbb94398ea280`, label +`stage-max`); a stage with no `effort` and no agent type inherited `xhigh` (agent +`abd2b64f159add7c2`, label `stage-inherit`); an Agent whose frontmatter says +`effort: max` ran at `max`. In Q3 the four children were agents `adb4fe075ae1bd61d`, +`a3655d8bbcc4c8426`, `a21fd778630b97247` and `aee5ec9e1ff3c829e`, in the order of the +table row; the Q3 directory's project copy of `source-scout` (`effort: max`) won by name +over the user-level copy, which still said `medium`. + +**What shows that a persisted `max` is dropped.** P3, P4, P7 and P8 ran under the +user's `ultracode: true` (P8 also set it in the project file), and Ultracode "takes +precedence over `effortLevel` and `modelSettings` entries" +([settings reference](https://code.claude.com/docs/en/settings-reference#ultracode), +fetched 2026-09-23). Those sessions would have run at `xhigh` whatever level was +persisted, so they are +consistent with a dropped `max` but cannot show it on their own. Q1 and Q2 are the +unconfounded evidence: with `ultracode: false` in the same `--settings` source, +`effortLevel: "high"` ran at `high` (Q2), and `effortLevel: "max"` ran at `xhigh`, the +level of the user's `modelSettings.claude-opus-5-5.effortLevel` (Q1), so `max` was +dropped and the next source decided. Q1 and Q2 cover the top-level key in the +`--settings` source; for the per-model key and a project settings file the evidence is +the installed schema and the docs below. **Installed client.** The 2.1.281 binary's settings schema declares `effortLevel` and `modelSettings..effortLevel` as `z(["low","medium","high","xhigh"]).optional().catch(void 0)`, so any other value, -`"max"` included, is dropped without a warning (P3, P4, P7, P8). The binary has no -default-effort key for subagents or workflow stages; `CLAUDE_CODE_SUBAGENT_MODEL` -sets only a child's model. - -**Official documentation** ([model configuration](https://code.claude.com/docs/en/model-config), -fetched by the research workflow and again for this record on 2026-09-23): - -- "Ultracode is a Claude Code setting rather than a model effort level: it sends - `xhigh` to the model and additionally has Claude orchestrate dynamic workflows for - substantive tasks." When `CLAUDE_CODE_EFFORT_LEVEL` "is set to a level other than - `xhigh`, requests run at that level and ultracode's workflow orchestration stays - inactive." -- On the settings keys: "`max` isn't accepted as a level in either key". On - sessions: "Unless you set it through the `CLAUDE_CODE_EFFORT_LEVEL` environment - variable, Claude Code applies `max` to the current session only." -- `max` "may show diminishing returns and is prone to overthinking. Test before - adopting broadly". -- The effort table lists `max` for Opus 5.5, Fable 5.1 and Sonnet 5; a level the - model lacks falls back to the highest supported level at or below it; models the - table does not list, Haiku 4.5 among them, take no effort level. - -The platform effort documentation, as fetched by the research workflow, recommends -an effort sweep on your own evaluations for Opus 5.5; that page was not re-read for -this record. +`"max"` included, is dropped without a warning (measured for the top-level key by Q1, +with Q2 as its control). The binary has no default-effort key for subagents or +workflow stages; `CLAUDE_CODE_SUBAGENT_MODEL` sets only a child's model. + +**Official documentation** (fetched by the research workflow and again for this record +on 2026-09-23): + +- [Model configuration](https://code.claude.com/docs/en/model-config): "Ultracode is a + Claude Code setting rather than a model effort level: it sends `xhigh` to the model + and additionally has Claude orchestrate dynamic workflows for substantive tasks." + When `CLAUDE_CODE_EFFORT_LEVEL` "is set to a level other than `xhigh`, requests run at + that level and ultracode's workflow orchestration stays inactive." On the settings + keys: "`max` isn't accepted as a level in either key". On sessions: "Unless you set + it through the `CLAUDE_CODE_EFFORT_LEVEL` environment variable, Claude Code applies + `max` to the current session only." Frontmatter effort "applies when that skill or + subagent is active, overriding the session level but not the environment variable." + The `/effort` menu "also offers `ultracode`", beside the effort levels. +- The same page: `max` "may show diminishing returns and is prone to overthinking. + Test before adopting broadly". The effort table lists `max` for Opus 5.5, Fable 5.1 + and Sonnet 5; a level the model lacks falls back to the highest supported level at + or below it; models the table does not list, Haiku 4.5 among them, take no effort + level. +- [Workflows](https://code.claude.com/docs/en/workflows): "In `claude -p` and the Agent + SDK, Claude Code never shows this prompt. It runs the Workflow tool call through the + same permission evaluation as the rest of the session"; a launch there needs a + `Workflow` or `Workflow()` allow rule, auto or bypass permission mode, a + `PreToolUse` hook that allows it, or the host's own approval. A run pauses at a + claude.ai usage limit only when "The session is interactive and signed in with a + claude.ai subscription"; "A run doesn't pause in non-interactive mode with `claude -p` + or the Agent SDK, in a background session, or in a Remote Control or agent team + teammate session." +- [Interactive mode](https://code.claude.com/docs/en/interactive-mode#wait-for-a-usage-limit-to-reset): + "Automatic continue is on by default in interactive sessions signed in with a + claude.ai subscription"; the session "re-arms the wait on its own at most twice in a + row" and does not start a wait for a reset more than 24 hours away. + +The platform effort documentation, as fetched by the research workflow, recommends an +effort sweep on your own evaluations for Opus 5.5; that page was not re-read for this +record. **GitHub and cloud sessions.** No GitHub Actions workflow in agent-lab or in this catalog invokes Claude (searched 2026-09-23). The `action.yml` of -`anthropics/claude-code-action` at `8cf3482550831fb35a4fc3fbf7ca139cf8028b4c` -(committed 2026-09-23) declares 39 inputs, among them `settings` ("Claude Code -settings as JSON string or path to settings JSON file") and `claude_args` -("Additional arguments to pass directly to Claude CLI"), and no `effort`, -`ultracode` or `model` input. The -[cloud-session settings documentation](https://code.claude.com/docs/en/settings#settings-in-cloud-sessions) +`anthropics/claude-code-action`, read on 2026-09-23 at commit +`8cf3482550831fb35a4fc3fbf7ca139cf8028b4c`, has `settings` ("Claude Code settings as +JSON string or path to settings JSON file") and `claude_args` ("Additional arguments to +pass directly to Claude CLI") inputs and no `effort`, `ultracode` or `model` input; its +`base-action` runs Claude through the Agent SDK's `query()` +(`base-action/src/run-claude.ts` calls `runClaudeWithSdk`), so an action job is a +non-interactive Agent SDK run. The action itself was executed for this catalog only as +a 2026-09-23 CI smoke in a scratch repository (a smoke reply, an allow control, and a +settings-level deny rule passed through its `settings` input that produced one +permission denial; [gated items](../../catalogs/sota-convergence/sdk-runtime-coverage-20260922.md#gated-items-executed-2026-09-23)). +The [cloud-session settings documentation](https://code.claude.com/docs/en/settings#settings-in-cloud-sessions) says a session with one repository reads that repository's committed `.claude/settings.json`, while a session with several repositories reads only its -`enabledPlugins` and `extraKnownMarketplaces` keys. Neither the action nor a cloud -session was run for this record. +`enabledPlugins` and `extraKnownMarketplaces` keys. No Ultracode or effort run of the +action or of a cloud session was executed for this record. ## Decision @@ -101,30 +154,42 @@ session was run for this record. and agent-lab's committed project settings (`enableWorkflows`, `ultracode`, the `unrestricted` size guideline, the concurrency cap of eight and spawn depth one), so single-repository cloud sessions on this repository start with it. It adds - neither `effortLevel` (a `max` there is dropped, P7 and P8) nor - `CLAUDE_CODE_EFFORT_LEVEL` (any value other than `xhigh` disables Ultracode, and - `max` there overrode every child's effort: P1, P5, P6 and P9). + neither `effortLevel` (a `max` there is dropped: Q1, and the schema and docs for a + project file) nor `CLAUDE_CODE_EFFORT_LEVEL`, which is never set at any value: any + value overrides every child's own effort (the docs for frontmatter effort at any + value; at `max`, P6 for frontmatter and P9 for a stage's effort), and any value + other than `xhigh` also disables Ultracode (P1, docs). [`AGENTS.md`](../../AGENTS.md) + tells the coordinator so. 2. **Every project agent and every saved workflow stage runs at `max`, with task-matched models kept:** Sonnet for `source-scout` and `isolated-builder`; - Opus for `evidence-reviewer`, `semantic-evidence-reviewer` and `blind-judge`. - The five shipped definitions change only their frontmatter line to - `effort: max` (H1: frontmatter `max` runs at `max` under an Ultracode parent). - Ad-hoc workflow scripts pass `effort: 'max'` on every `agent()` call, because a - stage without it inherits `xhigh` (H1). -3. **One non-orchestrated session at `max` stays available on request:** - `claude --effort max` or `/effort max`. Either turns Ultracode off for that - session (P2 measured the flag; `/effort max` picks `max` in place of the - `ultracode` entry of the same menu), and neither persists. + Opus for `evidence-reviewer`, `semantic-evidence-reviewer`, `blind-judge` and the + vendored lane's `blind-lane-reviewer`. The shipped definitions change only their + frontmatter line to `effort: max` (H1 and Q3: frontmatter `max` runs at `max` under + an Ultracode parent). Workflow scripts, saved and ad-hoc, pass `effort: 'max'` on + every `agent()` call: a stage without its own effort inherits `xhigh` unless its + agent's frontmatter sets one (H1, Q3), and a stage's effort overrides the + frontmatter (Q3), so the stage literal decides. +3. **One non-orchestrated session at `max` stays available on request.** + `claude --effort max` ran at `max` with orchestration off (P2). `/effort max` was + not probed: the docs say Claude Code applies `max` to the current session only and + list `ultracode` as a separate entry of the same menu. Neither persists. 4. **GitHub:** a future Actions workflow built on `claude-code-action` requests Ultracode with `claude_args: '--effort ultracode'` or the `settings` input `'{"ultracode": true}'`, and never puts `--effort max` in `claude_args` beside - Ultracode, because an explicit `max` wins and turns orchestration off (P2). + Ultracode, because an explicit `max` wins and turns orchestration off (P2). A job + that should start workflows also needs a `Workflow` or `Workflow()` allow + rule, auto or bypass permission mode, or a hook that allows the call, because its + Workflow launches go through normal permission evaluation (workflows docs). **Cost.** Children are expected to spend more output tokens than at `medium` or -`high`; neither that cost nor a quality gain has been measured yet. -`autoContinueAtUsageLimit` stays on in the settings template. Record each run's -per-child model, effort and usage with `child-usage.mjs --latest` so the overturn -comparison below has data. +`high`; neither that cost nor a quality gain has been measured yet. Nothing here caps +that cost: `autoContinueAtUsageLimit`, on in the settings template, only lets an +interactive session signed in with a claude.ai subscription, and a workflow run inside +it, wait for a usage limit to reset instead of failing (a reset within 24 hours, at +most twice in a row); `claude -p`, Agent SDK and so GitHub Actions runs and background +sessions fail at the limit (interactive-mode and workflows docs above). The setting +does not reduce usage. Record each run's per-child model, effort and usage with +`child-usage.mjs --latest` so the overturn comparison below has data. ## Alternatives considered @@ -136,7 +201,8 @@ comparison below has data. - **`max` for reviewers and judges only.** Rejected: the user asked for all workflows, scouts and builders included. - **Persist `max` through `effortLevel` or `modelSettings`.** Not available: the - client drops the value without a warning (P3, P4, P7, P8). + client drops the value without a warning (Q1, with Q2 as its control; the schema and + docs for the per-model key). ## Overturn conditions @@ -152,22 +218,26 @@ Revisit this record when any of these happens: ## Limitations -- **Keep-but-compare.** The per-role qualifications in the Ultracode recipe were - recorded at the earlier efforts; no run has compared `max` with them on the same - tasks. +- **Keep-but-compare.** The per-role qualifications in the Ultracode recipe and the + examples were recorded at the earlier efforts; no run has compared `max` with them + on the same tasks, and no saved workflow has run with the `max` literals yet. - **One host, one release.** The probes ran once each, on one account, with Claude Code 2.1.281; a later release can change the schema or the interaction with Ultracode. Repeat the probe method above after an upgrade before relying on it. -- **Not executed:** `claude-code-action` and cloud sessions (documentation and - source only). -- **Dated examples unchanged.** `examples/claude-native/agents/` and the saved - workflow examples keep the efforts they were qualified at, and their contract - suite pins those values; the Ultracode recipe says how to raise them when - adopting. -- **Released tags predate this change.** `v2026.09.23` has no - `adoption/agents/claude/`, and `v2026.09.23.1` (this record's base commit) ships the - five definitions at the earlier `medium`/`high` efforts. A host installing from - either tag gets the `max` definitions only after the next release and re-pin, or by - running `tools/adoption/install_claude_profile.py` from a checkout that has this - change. The installer refreshes any differing catalog-owned agent file, so a host - that already installed the earlier copies gets the `max` versions on a re-run. +- **Probe coverage.** Q1 and Q2 cover the top-level `effortLevel` key in the + `--settings` source only; no unconfounded probe covered the per-model key or a + project settings file. `/effort max` was not probed. +- **Not executed:** an Ultracode or effort run of `claude-code-action`, and cloud + sessions (documentation and source only; the action's CI smoke above tested + replies and a settings-level permission rule, not Ultracode or effort). +- **Released tags predate this change.** `v2026.09.23.1`, pinned since #148 and this + record's first base, ships the five agent definitions at `medium`/`high`, so + `adoption/bootstrap.md` marks them "changed after `v2026.09.23.1`". A host installing + from that tag gets the `max` definitions after the next release and re-pin, or by + running `tools/adoption/install_claude_profile.py --only agents` from a default-branch + clone; the installer replaces any differing catalog-owned agent file. +- **Vendored pin on an unmerged commit.** agent-lab `915e73e`, the source commit of + the vendored lane, the two review scripts and `blind-lane-reviewer`, is on agent-lab + branch `claude/max-effort-default-20260923` and not yet merged; after its squash + merge, `vendored-lanes.json` should name the merged commit (the bytes, and so the + `SHA256SUMS` entries, stay the same). diff --git a/examples/claude-native/CLAUDE.md b/examples/claude-native/CLAUDE.md index 15625d644..6465680a7 100644 --- a/examples/claude-native/CLAUDE.md +++ b/examples/claude-native/CLAUDE.md @@ -12,7 +12,7 @@ Research maintained upstream documentation, skills and implementations before bu For requested harness features, inspect native CLI help, current official docs and relevant upstream implementations before asking the user to identify a discoverable capability. Compare community alternatives against demonstrated gaps, retain source pins and reasons, then run the selected native path through returned results. Dated exclusions are not current availability evidence. Research only the relevant layers. -For substantial parallel Claude work, use native Ultracode with explicit task-matched worker models/effort under the `unrestricted` size guideline (each workflow sized to its task, concurrency capped per host by `CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS`, starting at 8 under the client's default cap of min(16, available CPUs − 2) per workflow (the bundled `/workflow-authoring` reference; the setting overrides it and accepts 1–256 from 2.1.269); solo for conversational or mechanical turns, a scout plus review for a bounded change, one agent per independent unit for multi-unit work, tens of agents only for an enumerated work-list, within the documented 4,096-item and 1,000-agent run limits); avoid expensive model inheritance, unbounded fan-out and repeated word-count calls. Keep routine tasks at ordinary effort. Delegate a step when only its conclusion is needed, so the reads, searches and dead ends stay in the child; children do not fan out a second layer (`CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH=1`), and agent teams stay off because each teammate is a separate instance whose results are not summarized back. Consult the installed native Ultracode recipe only for dispatch, messaging or dashboard setup. Worker output is evidence to verify, not acceptance by agreement. +For substantial parallel Claude work, use native Ultracode with explicit task-matched worker models, every worker at effort max (`effort: 'max'` on each `agent()` call; project agents declare `effort: max`); the coordinator stays at xhigh because a max session turns ultracode orchestration off, and `CLAUDE_CODE_EFFORT_LEVEL` stays unset because it overrides every worker's effort. Run it under the `unrestricted` size guideline (each workflow sized to its task, concurrency capped per host by `CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS`, starting at 8 under the client's default cap of min(16, available CPUs − 2) per workflow (the bundled `/workflow-authoring` reference; the setting overrides it and accepts 1–256 from 2.1.269); solo for conversational or mechanical turns, a scout plus review for a bounded change, one agent per independent unit for multi-unit work, tens of agents only for an enumerated work-list, within the documented 4,096-item and 1,000-agent run limits); avoid expensive model inheritance, unbounded fan-out and repeated word-count calls. Keep routine tasks solo. Delegate a step when only its conclusion is needed, so the reads, searches and dead ends stay in the child; children do not fan out a second layer (`CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH=1`), and agent teams stay off because each teammate is a separate instance whose results are not summarized back. Consult the installed native Ultracode recipe only for dispatch, messaging or dashboard setup. Worker output is evidence to verify, not acceptance by agreement. Keep context small: load only the skill and source needed for the current task. Use a focused source read for known identifiers, scoped document search for prose and scoped semantic retrieval for unfamiliar code. Process large output outside the model; retain failures and a full-output recovery path. Select one sufficient retrieval or compression lane per artifact. Verify original source before editing or judging compressed or retrieved code. Preserve native prompt caching, deferred tool discovery and compaction. diff --git a/examples/claude-native/agents/evidence-reviewer.md b/examples/claude-native/agents/evidence-reviewer.md index ce520c423..5f05be7f1 100644 --- a/examples/claude-native/agents/evidence-reviewer.md +++ b/examples/claude-native/agents/evidence-reviewer.md @@ -3,7 +3,7 @@ name: evidence-reviewer description: Independently inspect an assigned patch and its verification evidence. tools: Read, Glob, Grep, ToolSearch, mcp__serena__find_symbol, mcp__serena__find_referencing_symbols, mcp__serena__find_declaration, mcp__serena__find_implementations, mcp__serena__get_symbols_overview, mcp__serena__get_diagnostics_for_file, mcp__socraticode__codebase_search, mcp__socraticode__codebase_symbol, mcp__socraticode__codebase_impact, mcp__socraticode__codebase_flow, mcp__jcodemunch__route, mcp__jcodemunch__order, mcp__plugin_context-mode_context-mode__ctx_execute, mcp__plugin_context-mode_context-mode__ctx_execute_file, mcp__plugin_context-mode_context-mode__ctx_batch_execute, mcp__plugin_context-mode_context-mode__ctx_search, mcp__ai-memory__memory_query, mcp__ai-memory__memory_read_page, mcp__ai-memory__memory_read_session_observations model: opus -effort: high +effort: max --- Read the supplied patch or changed files, relevant callers, acceptance criteria, and validation results. Find concrete correctness defects or missing checks. Do not edit files. Report actionable findings with exact file references and evidence; otherwise state that no actionable findings were found and list remaining verification gaps. If you need a command result, ask the coordinator to supply it; do not derive counts or exit codes by eye. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. You have no Bash, Edit or Write tool; Context Mode execution can run commands in the working tree, so use it only to read (diffs, status listings, large outputs) and never to run acceptance commands or to create, modify or delete files or git state. Focused Read/Grep or Serena for known symbols and references, SocratiCode for conceptual code, jCodeMunch `order` (read-only actions) for indexed symbol retrieval, ai-memory query/read for prior decisions, Context Mode (ctx_execute) to derive answers from large outputs without loading them; choose one lane per artifact and verify original source before judging. diff --git a/examples/claude-native/agents/isolated-builder.md b/examples/claude-native/agents/isolated-builder.md index c8fac5356..7c0432213 100644 --- a/examples/claude-native/agents/isolated-builder.md +++ b/examples/claude-native/agents/isolated-builder.md @@ -3,7 +3,7 @@ name: isolated-builder description: Implement a bounded task in its own worktree and return a verified handoff. tools: Read, Edit, Write, Glob, Grep, Bash, ToolSearch, mcp__serena__find_symbol, mcp__serena__find_referencing_symbols, mcp__serena__find_declaration, mcp__serena__get_symbols_overview, mcp__serena__get_diagnostics_for_file, mcp__serena__replace_symbol_body, mcp__serena__insert_after_symbol, mcp__serena__insert_before_symbol, mcp__serena__rename_symbol, mcp__socraticode__codebase_search, mcp__socraticode__codebase_symbol, mcp__socraticode__codebase_impact, mcp__jcodemunch__route, mcp__jcodemunch__order, mcp__plugin_context-mode_context-mode__ctx_execute, mcp__plugin_context-mode_context-mode__ctx_execute_file, mcp__plugin_context-mode_context-mode__ctx_batch_execute, mcp__plugin_context-mode_context-mode__ctx_search, mcp__ai-memory__memory_query, mcp__ai-memory__memory_read_page model: sonnet -effort: medium +effort: max isolation: worktree --- diff --git a/examples/claude-native/agents/semantic-evidence-reviewer.md b/examples/claude-native/agents/semantic-evidence-reviewer.md index 5558679ab..5b0e83162 100644 --- a/examples/claude-native/agents/semantic-evidence-reviewer.md +++ b/examples/claude-native/agents/semantic-evidence-reviewer.md @@ -2,7 +2,7 @@ name: semantic-evidence-reviewer description: Review supplied source claims and advisory semantic judgments within a bounded evidence task. model: opus -effort: high +effort: max tools: Read, Glob, Grep skills: - typesafe-ai @@ -25,5 +25,5 @@ availability decision; semantic confidence cannot override it. Return case IDs, your final dispositions, original source references, corrections and remaining limits. Explicitly distinguish retained provider judgments from your own source review. Report the skill path and actual model when the client exposes -it; otherwise report unavailable. Use the declared Opus/high review role; retain +it; otherwise report unavailable. Use the declared Opus/max review role; retain the returned model and effort separately from these requested settings. diff --git a/examples/claude-native/agents/source-scout.md b/examples/claude-native/agents/source-scout.md index c39ff35bd..d38a497c7 100644 --- a/examples/claude-native/agents/source-scout.md +++ b/examples/claude-native/agents/source-scout.md @@ -3,7 +3,7 @@ name: source-scout description: Exact extraction and inventory from named files and commands, plus running the acceptance commands a task names; makes no edits of its own and returns source-cited facts, never judgments. tools: Read, Grep, Glob, Bash model: sonnet -effort: medium +effort: max maxTurns: 40 omitClaudeMd: true --- diff --git a/examples/claude-native/workflows/README.md b/examples/claude-native/workflows/README.md index 8506a0c1b..6d8facb3b 100644 --- a/examples/claude-native/workflows/README.md +++ b/examples/claude-native/workflows/README.md @@ -3,10 +3,17 @@ These are project-authored saved scripts and agent definitions for the upstream Claude Workflow runtime. The saved workflow scripts and the three roles in the table below preserve the deployed agent-lab files from the retained portable -qualification. These are local integration assets, not upstream tests. The -separate `semantic-evidence-reviewer` example received a later explicit Opus/high -declaration and reporting instruction to satisfy the combined portable contract; -that change has local checks and no new native provider qualification. +qualification, with one later change: since 2026-09-23 every stage and agent +binds effort `max`, models unchanged +([decision](../../../docs/decisions/2026-09-23-max-effort-default.md)). +`review-changes.js`, `readiness-audit.js` and `layer-verdict-lane.js` are +byte-identical to agent-lab commit `915e73e`, which made that change there. The +qualification ran at the earlier efforts (Sonnet/medium, Opus/high); no native +run at `max` is recorded yet. These are local integration assets, not upstream +tests. The separate `semantic-evidence-reviewer` example received a later +explicit Opus declaration and reporting instruction to satisfy the combined +portable contract; that change has local checks and no new native provider +qualification. ## Byte-identity check `SHA256SUMS` in this directory (`sha256sum -- *.mjs *.js *.json`, excluding @@ -56,15 +63,18 @@ reads the inventoried files in full and marks diff-dependent claims unverifiable | Agent | Model, effort | Tools | Use | | --- | --- | --- | --- | -| `source-scout` | Sonnet, medium | Read, Grep, Glob, Bash; no project instructions loaded | exact extraction, inventories, running the acceptance commands a task names (raw through `rtk proxy` where `rtk` is installed) | -| `evidence-reviewer` | Opus, high | Read, Glob, Grep, ToolSearch and named read-only MCP tools; no Bash, Edit or Write | independent review from source and recorded evidence | -| `isolated-builder` | Sonnet, medium, own worktree | Read, Edit, Write, Glob, Grep, Bash, ToolSearch and named MCP tools | a bounded implementation from a clear contract | +| `source-scout` | Sonnet, max | Read, Grep, Glob, Bash; no project instructions loaded | exact extraction, inventories, running the acceptance commands a task names (raw through `rtk proxy` where `rtk` is installed) | +| `evidence-reviewer` | Opus, max | Read, Glob, Grep, ToolSearch and named read-only MCP tools; no Bash, Edit or Write | independent review from source and recorded evidence | +| `isolated-builder` | Sonnet, max, own worktree | Read, Edit, Write, Glob, Grep, Bash, ToolSearch and named MCP tools | a bounded implementation from a clear contract | Context Mode `ctx_execute*` can run commands, so the reviewer's read-only rule -there is an instruction, not a sandbox; the same holds for the scout's Bash. A -fourth definition, `semantic-evidence-reviewer` (Opus, high; Read, Glob, Grep; the -`typesafe-ai` skill), is project-local to agent-lab and published separately; the -contract suite covers every file in `agents/`. +there is an instruction, not a sandbox; the same holds for the scout's Bash. Two +more definitions ship here. `semantic-evidence-reviewer` (Opus, max; Read, Glob, +Grep; the `typesafe-ai` skill) is project-local to agent-lab and published +separately. `blind-lane-reviewer` (Opus, max; Read, Glob and Grep; no preloaded +skill and no project instructions) is the agent every `layer-verdict-lane` stage +names, vendored byte-identical from agent-lab commit `915e73e` and pinned in +`vendored-lanes.json`. The contract suite covers every file in `agents/`. ## Workflows @@ -118,7 +128,7 @@ lifecycle for a read-only cross-family review (lane C in the cooperation recipe) From this directory: ```sh -node check-syntax.mjs review-changes.js readiness-audit.js +node check-syntax.mjs review-changes.js readiness-audit.js layer-verdict-lane.js node test-envelope.mjs # envelope semantics plus the static contract below node test-contract-mutations.mjs # each listed defect must fail the suite on its named assertion node test-child-usage.mjs # child-usage.mjs against synthetic transcript rows @@ -132,10 +142,15 @@ binds the shipped receipts (run ids, figures, the builder commit) to this documentation; an adopting project keeps those receipts beside its own or drops that one file, it is evidence rather than a generic tool. The static contract fails a saved workflow whose worker packet drifts from the shared text, whose stage -omits `model` or `effort`, whose routing differs from the reviewed table, or whose -`agent(` is written where the scanner cannot see it, and an agent definition that -omits model or effort, grants a bare `mcp__server` prefix, grants MCP tools -without `ToolSearch`, or can edit files without `isolation: worktree`. Measured +omits `model` or `effort` or binds an effort other than `max`, whose `MODEL` +record differs from the model and effort its stages bind, whose routing differs +from the reviewed table, or whose `agent(` is written where the scanner cannot +see it; an agent definition that omits model or effort, carries an effort other +than one `effort: max` line, grants a bare `mcp__server` prefix, grants MCP tools +without `ToolSearch`, or can edit files without `isolation: worktree`; a routing +table that restates an agent's model or effort differently from its file; +project settings that set `CLAUDE_CODE_EFFORT_LEVEL` or cap effort below `max`; +and project instructions that do not state the `effort: 'max'` literal. Measured native results and remaining boundaries are in [the dated guide](../../../docs/ultracode-token-routing-20260921.md). @@ -147,7 +162,7 @@ Applies to the coordinator and every Workflow/Agent child (project agents and `. - **One context lane per artifact class.** Known source identifiers: focused `rg`/Serena read. Indexed Markdown: scoped QMD BM25. Unfamiliar or conceptual code: SocratiCode. Prior decisions: ai-memory. Large command output: Context Mode. Choose one lane per artifact; do not chain compressors, and verify original source before editing or judging retrieved text. - **Worker packet.** Bounded objective, explicit source paths, allowed effects (no writes except what an acceptance command named in the task itself produces, unless a worktree is owned), and a small return schema with source-cited fields. No word-count instructions; no whole-repository or transcript pastes. - **Stable policy prefix.** Keep the shared contract text byte-identical across saved workflows (asserted by `test-envelope.mjs`) and vary only the task packet. What the provider was observed to reuse across children is the agent type's system prompt and tool definitions, per model; identical packet text alone has no measured cache effect. Reuse one agent type and model for sibling workers, and preserve deferred tool discovery and compaction. Sibling stages share one prompt-cache prefix only when model, effort, agent type, tools, output schema and working directory all match (official workflows doc, fetched 2026-09-22); keep them identical and leave `CLAUDE_CODE_WORKFLOW_PREFIX_STAGGER_MS` at its 5,000 ms default. -- **Task-matched models.** Set model and effort explicitly per stage (Sonnet/medium bounded work, Opus/high review, Fable coordinator at xhigh under `ultracode`); record requested and resolved child model, and treat nulls, schema retries, stub payloads and substitutions as incomplete results. The routing table below is the default; `test-envelope.mjs` fails a saved workflow or project agent that omits model or effort. +- **Task-matched models at effort max.** Set model and effort explicitly per stage: Sonnet for bounded extraction, inventories, acceptance commands and implementation, Opus for review and judgment, and every saved stage and project agent at effort `max` (since 2026-09-23), with the coordinator at xhigh under `ultracode`, because a `max` session turns ultracode's orchestration off. A stage with no `effort` of its own inherits the coordinator's xhigh unless its agent's frontmatter sets one, and a stage's own effort overrides the frontmatter, so every `agent()` call, ad-hoc ones included, passes `effort: 'max'`. Never set `CLAUDE_CODE_EFFORT_LEVEL`: any value overrides every child's frontmatter and stage effort, and any value other than `xhigh` also turns ultracode's orchestration off. These effort rules were probed on Claude Code 2.1.281 on 2026-09-23 (`docs/decisions/2026-09-23-max-effort-default.md` in this catalog). Record requested and resolved child model and effort, and treat nulls, schema retries, stub payloads and substitutions as incomplete results. The routing table below is the default; `test-envelope.mjs` fails a saved workflow or project agent that omits model or effort or binds an effort other than `max`. - **Usage accounting.** Count each client separately: native `/usage`, `claude agents`/`/workflows` journals, `ccusage` offline reports and `ecosystem-token-report refresh`. Never sum RTK, Context Mode, jCodeMunch, Headroom and provider counters, and never state a savings percentage from a fixture. After a Workflow run, `node .claude/workflows/child-usage.mjs ` (or `--latest`) returns each child's requested and resolved model, effort, provider-returned usage and first-prompt size, and exits 1 when a child is null, substituted or inherited the coordinator model. - **Cross-family review.** Explicit `/codex:review` or `/codex:adversarial-review --background` at integration points (see `recipes/claude-codex-cooperation-lanes.md`); findings are verified against source, not accepted by agreement. - **Opt-in and limits.** The `ultracode` keyword starts a workflow only from a prompt typed in the session; it is inert from `-p`, an unstamped SDK prompt, a scheduled task or a relayed comment, so a headless run invokes a saved workflow by name under a settings source whose permission mode or allow rule (`Workflow` or `Workflow()`) covers the tool. Scripts take no mid-run user input, no `import()` and no `Date.now()`, `Math.random()` or argless `new Date()` (pass timestamps through `args`; run a stage that needs sign-off as its own workflow); one `parallel()`/`pipeline()` call takes at most 4,096 items and a run at most 1,000 agents. @@ -169,14 +184,18 @@ Spawning a child has a fixed prompt cost before any work. Run ids, the extractio In real runs the adopted definitions, with their full grants and bodies plus the task packet, started at 17,535 (`evidence-reviewer`, packet including the inventory) and 17,864 (`isolated-builder`). Sibling children launched together each wrote their own cache (first-request cache read 0); a repeat spawn of the same agent type about two minutes later read 34,591 of 42,091 tokens from cache. Children use the 5-minute cache class, so `subagentPromptCacheTtl` stays at its five-minute default: the one-hour class only pays for a child that idles more than five minutes between requests. +Effort column since 2026-09-23: every child role runs at `max` with its task-matched model, and the coordinator stays at xhigh under `ultracode`. The prompt-size figures above predate this change. + | Task class | Agent / stage | Model, effort | Lanes | | --- | --- | --- | --- | -| Requirements, decomposition, integration, hard judgments | coordinator | Fable, xhigh under `ultracode` | all, one per artifact | -| Exact extraction, inventory, running acceptance commands | `source-scout` | Sonnet, medium | `rg`/focused Read, `qmd search`, `jq` pipelines, RTK-filtered Bash with `rtk proxy` recovery; acceptance commands run raw through `rtk proxy` where `rtk` is installed. Bash is granted, so its read-only rule is an instruction, not a sandbox | -| Implementation from a clear contract | `isolated-builder` (own worktree) | Sonnet, medium | named Serena read and symbol-edit tools, SocratiCode, jCodeMunch, Context Mode, ai-memory, all deferred | -| Independent review from source and recorded evidence | `evidence-reviewer` | Opus, high | named Serena, SocratiCode, jCodeMunch and ai-memory read tools plus Context Mode `ctx_execute*`, all deferred. No Bash, Edit, Write or symbol-edit tool; `ctx_execute*` can still run commands in the working tree, so file safety there is an instruction, not a sandbox | -| Verification that must re-run commands | default workflow subagent | Opus, high | full tools; pays the 42k prompt deliberately | -| Web or documentation research | default workflow subagent | Sonnet, medium | Context Mode `ctx_fetch_and_index` then `ctx_search` (observed: 1 fetch, 5 searches, 8 requests) | +| Requirements, decomposition, integration, hard judgments | coordinator | Opus 5.5, xhigh under `ultracode` (a `max` session turns its orchestration off); Fable 5.1 as an explicit escalation | all, one per artifact | +| Exact extraction, inventory, running acceptance commands | `source-scout` | Sonnet, max | `rg`/focused Read, `qmd search`, `jq` pipelines, RTK-filtered Bash with `rtk proxy` recovery; acceptance commands run raw through `rtk proxy` where `rtk` is installed. Bash is granted, so its read-only rule is an instruction, not a sandbox | +| Implementation from a clear contract | `isolated-builder` (own worktree) | Sonnet, max | named Serena read and symbol-edit tools, SocratiCode, jCodeMunch, Context Mode, ai-memory, all deferred | +| Independent review from source and recorded evidence | `evidence-reviewer` | Opus, max | named Serena, SocratiCode, jCodeMunch and ai-memory read tools plus Context Mode `ctx_execute*`, all deferred. No Bash, Edit, Write or symbol-edit tool; `ctx_execute*` can still run commands in the working tree, so file safety there is an instruction, not a sandbox | +| Review of supplied semantic (TypeSafe) judgments against original source | `semantic-evidence-reviewer` | Opus, max | Read, Glob, Grep and the `typesafe-ai` skill preloaded (first prompt 15,059 in the 2026-09-22 probe); no MCP grants, Bash or writes | +| Layer-verdict lane stages: propose, refute and re-check one stripped packet from its repository root | `blind-lane-reviewer` | Opus, max | Read, Glob and Grep; no preloaded skill and no project instructions (`omitClaudeMd`); used by `layer-verdict-lane` | +| Verification that must re-run commands | default workflow subagent | Opus, max | full tools; pays the 42k prompt deliberately | +| Web or documentation research | default workflow subagent | Sonnet, max | Context Mode `ctx_fetch_and_index` then `ctx_search` (observed: 1 fetch, 5 searches, 8 requests) | | Cross-family review | `/codex:review` lanes | Codex | see `recipes/claude-codex-cooperation-lanes.md`; usage is not in the Claude journal | Haiku is not routed. In the same-packet trial the Opus verifier scored the Sonnet/medium inventory 14/14 lane rows and the Haiku inventory 9/14, including a quote attributed to a file that does not contain it; Haiku also ignored the requested effort and used more requests (22 vs 17). Overturn this with a repeat trial in which Haiku returns no unanchored citation on two distinct extraction packets. RTK and ai-memory hooks were observed firing inside workflow children (`rtk hook claude` rewrote child Bash calls); Context Mode has no child hook, so its use depends on the packet and agent text. Repomix, guarded Headroom and TOON stay coordinator-side: workers read focused ranges instead of packed sources, Context Mode already owns large worker output (no chained compressors), and a Workflow script has no filesystem or CLI access to run a TOON conversion on the packets it passes. diff --git a/examples/claude-native/workflows/readiness-audit.js b/examples/claude-native/workflows/readiness-audit.js index 659b28255..889769205 100644 --- a/examples/claude-native/workflows/readiness-audit.js +++ b/examples/claude-native/workflows/readiness-audit.js @@ -3,8 +3,8 @@ export const meta = { description: 'Extract source-cited claims from named documents and commands with Sonnet readers, then adversarially verify them with an Opus verifier', whenToUse: 'Status or readiness questions over project records: args = {docs: ["path", ...], commands: ["read-only command", ...], question: "what to judge"}', phases: [ - { title: 'Read', detail: 'Sonnet/medium readers, one per document group', model: 'sonnet' }, - { title: 'Verify', detail: 'Opus/high verifier refutes or confirms every claim', model: 'opus' }, + { title: 'Read', detail: 'Sonnet/max readers, one per document group', model: 'sonnet' }, + { title: 'Verify', detail: 'Opus/max verifier refutes or confirms every claim', model: 'opus' }, ], } @@ -81,7 +81,7 @@ const readers = await parallel(groups.map((g, i) => () => agent( ? 'Read these documents fully and extract every claim relevant to the question: ' + g.items.join(', ') : 'Run these read-only commands exactly and report their returned results as claims: ' + g.items.join(' ; ')) + '\nReturn exactly one sources entry for each listed document or command, copying its identifier exactly into source. Mark observed only after reading the document or receiving the command result, with nonblank returned evidence; observed command failures are evidence and must retain their integer exit_code. Use null exit_code for documents or commands not executed. Record unavailable documents as not_found and commands you did not run as not_executed. Never silently omit a requested source.', - { label: 'read:' + g.kind + '-' + (i + 1), phase: 'Read', schema: claimsForPacket(g.items), agentType: 'source-scout', model: 'sonnet', effort: 'medium' }, + { label: 'read:' + g.kind + '-' + (i + 1), phase: 'Read', schema: claimsForPacket(g.items), agentType: 'source-scout', model: 'sonnet', effort: 'max' }, ))) // Preserve every packet identity, including failed/null readers, so an unread // source cannot vanish from the verdict. @@ -94,7 +94,7 @@ log('claims extracted: ' + claims.length + ' from ' + (packets.length - unread.l phase('Verify') const verify = await agent( PACKET + '\nYou are an adversarial verifier. Open every cited source yourself and try to REFUTE each claim; default to unverifiable when you cannot. Return exactly one verdict for every reader claim, copying its claim string exactly, with nonblank evidence. Include a nonblank correction for corrected claims. Also return one source_verdicts entry for each requested packet item, copying packet_id and source exactly: independently confirm the returned source evidence and command outcome, including nonzero exits; use unverifiable if you cannot establish it. Emit only the exact (packet.id, item) pairs listed in each packet.items; never copy extra result.sources entries or combine one packet id with another packet\'s item. Packets whose result is null were NOT read: list their sources under missing and do not treat the audit as complete. The missing field is only for exact requested source identifiers that are unavailable or unverified. Put independently established additional facts the readers missed, with their evidence, in readiness_verdict; these are not missing sources. Then answer the question in readiness_verdict with the single most blocking gate and its evidence. A complete audit may conclude that the project is not ready.\nQuestion: ' + question + '\nReader packets (result null = unread): ' + JSON.stringify(packets), - { label: 'verify', phase: 'Verify', schema: VERIFY, model: 'opus', effort: 'high' }, + { label: 'verify', phase: 'Verify', schema: VERIFY, model: 'opus', effort: 'max' }, ) if (!verify) log('verifier returned null; claims are unverified') const evidenceIssues = [] diff --git a/examples/claude-native/workflows/review-changes.js b/examples/claude-native/workflows/review-changes.js index f169f8266..3df3abc67 100644 --- a/examples/claude-native/workflows/review-changes.js +++ b/examples/claude-native/workflows/review-changes.js @@ -3,9 +3,9 @@ export const meta = { description: 'Inventory a diff with a Sonnet worker, have an Opus reviewer independently verify it against original source, and reproduce the acceptance commands with a second worker', whenToUse: 'Before integrating a bounded change: args = {base: "", paths: ["optional", "scope"], checks: ["optional acceptance commands"]}', phases: [ - { title: 'Inventory', detail: 'Sonnet/medium: changed files, behavior, checks run', model: 'sonnet' }, - { title: 'Review', detail: 'Opus/high: refute or confirm each claimed behavior from source', model: 'opus' }, - { title: 'Recheck', detail: 'Sonnet/medium: a second worker re-runs the acceptance commands; the script compares exit codes', model: 'sonnet' }, + { title: 'Inventory', detail: 'Sonnet/max: changed files, behavior, checks run', model: 'sonnet' }, + { title: 'Review', detail: 'Opus/max: refute or confirm each claimed behavior from source', model: 'opus' }, + { title: 'Recheck', detail: 'Sonnet/max: a second worker re-runs the acceptance commands; the script compares exit codes', model: 'sonnet' }, ], } @@ -56,7 +56,7 @@ const inventory = await agent( 'Run: git status --short --untracked-files=all -- ' + (paths.length ? paths.join(' ') : '.') + '; git diff --stat ' + base + ' -- ' + (paths.length ? paths.join(' ') : '.') + '; git diff ' + base + ' -- ' + (paths.length ? paths.join(' ') : '.') + ' (read what changed, and Read every untracked in-scope file the status lists, since git diff does not show them; give each one a files entry and source-cited claims; every files[].path is repository-relative, never absolute).\n' + (checks.length ? 'Acceptance commands to run, ' + checks.length + ' in total, one per line below. Run each exactly as written, launched through `rtk proxy` where `rtk` is installed so its output is raw, without pipelines that mask exit status, and record exactly one checks_run entry per command with the command string copied verbatim (never the `rtk proxy` prefix), its exit code and concise exact output quoted in result. Copy deterministic passed/failed summaries verbatim; never eyeball or recount test totals; record explicitly when a command produced no output.\n' + checks.map((c) => '- ' + c).join('\n') + '\n' : 'No acceptance commands were supplied; record checks_run as empty.\n') + 'Describe the in-scope changed behavior with source-cited claims. If there is no change to review, return no claims rather than inventing one; the coordinator will mark the no-op incomplete. Return the schema only.', - { label: 'inventory', phase: 'Inventory', schema: INVENTORY, agentType: 'source-scout', model: 'sonnet', effort: 'medium' }, + { label: 'inventory', phase: 'Inventory', schema: INVENTORY, agentType: 'source-scout', model: 'sonnet', effort: 'max' }, ) if (!inventory) { log('inventory worker returned null; stopping without a review') @@ -72,10 +72,10 @@ const [review, recheck] = await parallel([() => agent( PACKET + '\nYou are an independent reviewer. Try to REFUTE each behavior claim below by reading the original source and the diff against ' + base + ' yourself (your role has no Bash: open changed files with Read, and obtain the diff with Context Mode ctx_execute, language shell, with cwd set to the repository root, running git diff ' + base + ' -- ' + (paths.length ? paths.join(' ') : '.') + ' and printing only the hunks you need; that diff omits untracked files, so also run git status --short --untracked-files=all -- ' + (paths.length ? paths.join(' ') : '.') + ' the same way (it lists each file inside a new directory) and Read every untracked in-scope file in full). When Context Mode is not among your tools, Read every file the inventory lists in full and judge each claim from that source; a claim that can only be settled by the diff is then unverifiable. Return exactly one verdict for every changed_behavior claim, copying its claim string exactly, with nonblank evidence. Default to unverifiable when you cannot open the cited source. Report concrete defects with file and line. List verification gaps. A corrected or refuted behavior claim requires coordinator resolution before acceptance.\n' + 'Requested scope: ' + scope + '. Requested acceptance commands: ' + JSON.stringify(checks) + '. Inspect the recorded evidence for these commands against the source (for example that a quoted total is consistent with the test file it came from) and report any inconsistency as a defect. Do not execute acceptance commands: they are caller-supplied and may write, your role is read-only, and a separate recheck stage re-runs them and the script compares both runs, so not having re-run them is not a gap. Use ctx_execute only for the git diff and git status reads named above. Record material unresolved gaps affecting the in-scope behavior or required checks, including missing evidence. Excluded files and disclosed general limitations are not themselves gaps; explain a concrete effect on this requested scope before treating them as one. Preserve every supported in-scope finding.\n' + 'Inventory: ' + JSON.stringify(inventory), - { label: 'review', phase: 'Review', schema: REVIEW, agentType: 'evidence-reviewer', model: 'opus', effort: 'high' }, + { label: 'review', phase: 'Review', schema: REVIEW, agentType: 'evidence-reviewer', model: 'opus', effort: 'max' }, ), () => (checks.length ? agent( PACKET + '\nIndependent reproduction of acceptance commands for the change in scope ' + scope + '. You have not seen any earlier result and must not look for one. From the repository root run each command below exactly as given, launched through `rtk proxy` where `rtk` is installed so its output is raw, without pipelines that mask exit status, and record exactly one checks_run entry per command with the command string copied verbatim (never the `rtk proxy` prefix), its exit code and the concise exact summary output quoted in result. Copy deterministic passed/failed summaries verbatim; never recount totals. Record explicitly when a command produced no output. Commands, ' + checks.length + ' in total, one per line:\n' + checks.map((c) => '- ' + c).join('\n'), - { label: 'recheck', phase: 'Recheck', schema: RECHECK, agentType: 'source-scout', model: 'sonnet', effort: 'medium' }, + { label: 'recheck', phase: 'Recheck', schema: RECHECK, agentType: 'source-scout', model: 'sonnet', effort: 'max' }, ) : Promise.resolve(null))]) if (!review) log('review worker returned null; inventory is unreviewed') // Acceptance requires complete claim coverage as well as checks and review. diff --git a/examples/claude-native/workflows/test-contract-mutations.mjs b/examples/claude-native/workflows/test-contract-mutations.mjs index f7ca74273..0cbb7ac4b 100644 --- a/examples/claude-native/workflows/test-contract-mutations.mjs +++ b/examples/claude-native/workflows/test-contract-mutations.mjs @@ -20,20 +20,54 @@ const relativeBindings = BINDINGS.filter(([, v]) => typeof v === 'string' && !v. const DEPTH = Math.max(0, ...relativeBindings.map(([, v]) => v.split('/').filter((seg) => seg === '..').length - 1)) + 2 let passed = 0, failed = 0 const expect = (n, c) => { console.log((c ? 'PASS ' : 'FAIL ') + n); if (c) passed++; else failed++ } +// Bound files as the copied tree places them (relative to its root), so the mutations below hold in every +// layout contract.config.json may describe; settings mutations start from the real env so each drifts one value. +const SETTINGS_FILE = join('workflows', CONFIG.settings) +const INSTRUCTIONS_FILE = join('workflows', CONFIG.instructions) +const ROUTING_DOC_FILE = join('workflows', CONFIG.routing_doc) +const ENV = JSON.parse(readFileSync(resolve(HERE, CONFIG.settings), 'utf8')).env || {} const MUTATIONS = [ ['a later stage names a nonexistent agent', 'workflows/review-changes.js', "agentType: 'evidence-reviewer'", "agentType: 'evidence-reviwer'", 'names only valid project agents in every stage'], - ['a later stage drops its model', 'workflows/review-changes.js', "agentType: 'evidence-reviewer', model: 'opus', effort: 'high'", "agentType: 'evidence-reviewer', effort: 'high'", 'binds model and effort inside every options literal'], + ['a later stage drops its model', 'workflows/review-changes.js', "agentType: 'evidence-reviewer', model: 'opus', effort: 'max'", "agentType: 'evidence-reviewer', effort: 'max'", 'binds model and effort inside every options literal'], ['the packet drifts by one space', 'workflows/readiness-audit.js', 'only the listed sources, no writes', 'only the listed sources, no writes', 'PACKET text is byte-identical'], ['the reviewer regains a bare server grant', 'agents/evidence-reviewer.md', 'tools: Read, Glob, Grep, ToolSearch, ', 'tools: Read, Glob, Grep, ToolSearch, mcp__serena, ', 'never a bare server prefix'], ['the reviewer gains a file-creating tool', 'agents/evidence-reviewer.md', 'tools: Read, Glob, Grep, ToolSearch, ', 'tools: Read, Glob, Grep, ToolSearch, mcp__serena__create_text_file, ', 'tool surface is exactly the reviewed list'], - ['an agent drops its effort', 'agents/source-scout.md', 'effort: medium\n', '', 'declares an explicit model and effort'], - ['an agent( hides inside a template literal', 'workflows/readiness-audit.js', "phase('Verify')", "phase('Verify')\nconst never = () => `${agent('x', { label: 'h', model: 'opus', effort: 'high' })}`", 'has no agent( inside a template literal'], + ['an agent drops its effort', 'agents/source-scout.md', 'effort: max\n', '', 'declares an explicit model and effort'], + ['an agent( hides inside a template literal', 'workflows/readiness-audit.js', "phase('Verify')", "phase('Verify')\nconst never = () => `${agent('x', { label: 'h', model: 'opus', effort: 'max' })}`", 'has no agent( inside a template literal'], ['a stage drops the shared packet but keeps its label', 'workflows/review-changes.js', "PACKET + '\\nYou are an independent reviewer.", "'Worker packet contract:' + '\\nYou are an independent reviewer.", 'starts every worker prompt with the full shared packet'], ['the review stage is routed to the builder', 'workflows/review-changes.js', "agentType: 'evidence-reviewer'", "agentType: 'isolated-builder'", 'routes each stage to its reviewed agent, model and effort'], ['a stage loses its agentType', 'workflows/readiness-audit.js', "agentType: 'source-scout', ", '', 'routes each stage to its reviewed agent, model and effort'], ['a call is written as agent (', 'workflows/review-changes.js', "phase('Inventory')", "phase('Inventory')\nif (a.extraStage) await agent ('extra', { label: 'extra' })", 'binds model and effort inside every options literal'], - ['a later duplicate key overrides the model', 'workflows/review-changes.js', "agentType: 'evidence-reviewer', model: 'opus', effort: 'high' }", "agentType: 'evidence-reviewer', model: 'opus', effort: 'high', model: undefined }", 'declares model and effort exactly once'], + ['a later duplicate key overrides the model', 'workflows/review-changes.js', "agentType: 'evidence-reviewer', model: 'opus', effort: 'max' }", "agentType: 'evidence-reviewer', model: 'opus', effort: 'max', model: undefined }", 'declares model and effort exactly once'], ['the builder loses worktree isolation', 'agents/isolated-builder.md', 'isolation: worktree\n', '', 'runs in its own worktree'], + // Effort max (docs/decisions/2026-09-23-max-effort-default.md): a stage or agent that drops its effort, or binds + // any level other than max, must fail; a stage without effort would inherit the coordinator's xhigh. + ['a later stage drops its effort', 'workflows/review-changes.js', "agentType: 'evidence-reviewer', model: 'opus', effort: 'max'", "agentType: 'evidence-reviewer', model: 'opus'", 'binds model and effort inside every options literal'], + ['a default-child stage drops its effort', 'workflows/readiness-audit.js', "schema: VERIFY, model: 'opus', effort: 'max'", "schema: VERIFY, model: 'opus'", 'binds model and effort inside every options literal'], + ['a review stage runs at effort high', 'workflows/review-changes.js', "agentType: 'evidence-reviewer', model: 'opus', effort: 'max'", "agentType: 'evidence-reviewer', model: 'opus', effort: 'high'", 'binds effort max in every options literal'], + ['a scout stage runs at effort medium', 'workflows/review-changes.js', "schema: RECHECK, agentType: 'source-scout', model: 'sonnet', effort: 'max'", "schema: RECHECK, agentType: 'source-scout', model: 'sonnet', effort: 'medium'", 'binds effort max in every options literal'], + ['a default-child stage runs at effort xhigh', 'workflows/readiness-audit.js', "schema: VERIFY, model: 'opus', effort: 'max'", "schema: VERIFY, model: 'opus', effort: 'xhigh'", 'binds effort max in every options literal'], + ['a reader stage names ultracode as its effort', 'workflows/readiness-audit.js', "agentType: 'source-scout', model: 'sonnet', effort: 'max'", "agentType: 'source-scout', model: 'sonnet', effort: 'ultracode'", 'binds effort max in every options literal'], + ['a lane stage runs at effort low while MODEL records max', 'workflows/layer-verdict-lane.js', "phase: 'Propose', agentType: 'blind-lane-reviewer', model: 'opus', effort: 'max'", "phase: 'Propose', agentType: 'blind-lane-reviewer', model: 'opus', effort: 'low'", 'records in MODEL, when declared, the same model and effort it binds'], + ['the lane MODEL literal records high while every call binds max', 'workflows/layer-verdict-lane.js', "const MODEL = { name: 'opus', effort: 'max' }", "const MODEL = { name: 'opus', effort: 'high' }", 'records in MODEL, when declared, the same model and effort it binds'], + ['the lane MODEL literal records a full model id while the calls bind aliases', 'workflows/layer-verdict-lane.js', "const MODEL = { name: 'opus', effort: 'max' }", "const MODEL = { name: 'claude-opus-5-5', effort: 'max' }", 'records in MODEL, when declared, the same model and effort it binds'], + ['an agent runs at effort low', 'agents/source-scout.md', 'effort: max\n', 'effort: low\n', 'runs at effort max on a single effort line'], + ['an agent runs at effort medium', 'agents/isolated-builder.md', 'effort: max\n', 'effort: medium\n', 'runs at effort max on a single effort line'], + ['an agent runs at effort high', 'agents/evidence-reviewer.md', 'effort: max\n', 'effort: high\n', 'runs at effort max on a single effort line'], + ['an agent runs at effort xhigh', 'agents/semantic-evidence-reviewer.md', 'effort: max\n', 'effort: xhigh\n', 'runs at effort max on a single effort line'], + ['an agent gains a second, lower effort line', 'agents/blind-lane-reviewer.md', 'effort: max\n', 'effort: max\neffort: high\n', 'runs at effort max on a single effort line'], + ['the routing table restates source-scout at medium', ROUTING_DOC_FILE, 'running acceptance commands | `source-scout` | Sonnet, max |', 'running acceptance commands | `source-scout` | Sonnet, medium |', 'lists every project agent once with the model and effort its file declares'], + ['the routing table restates a default child at high', ROUTING_DOC_FILE, '| default workflow subagent | Opus, max |', '| default workflow subagent | Opus, high |', 'every default workflow subagent row binds a model at effort max'], + ['the routing table moves the coordinator off xhigh under ultracode', ROUTING_DOC_FILE, '| coordinator | Opus 5.5, xhigh under `ultracode`', '| coordinator | Opus 5.5, max', 'the coordinator row stays at xhigh under ultracode'], + ['the instructions stop stating the stage effort literal', INSTRUCTIONS_FILE, "`effort: 'max'`", "`effort: 'high'`", 'state the effort literal every stage binds'], + // CLAUDE_CODE_EFFORT_LEVEL overrides every stage's and agent's effort at any value (docs; probes P6 and P9 at max), and any + // value other than xhigh also turns ultracode's orchestration off (P1); an effort cap below max clamps the stages. + ['the settings env sets CLAUDE_CODE_EFFORT_LEVEL=max', SETTINGS_FILE, 'env', { ...ENV, CLAUDE_CODE_EFFORT_LEVEL: 'max' }, 'CLAUDE_CODE_EFFORT_LEVEL stays unset and no maxEffortLevel caps the stage effort'], + ['the settings env sets CLAUDE_CODE_EFFORT_LEVEL=xhigh', SETTINGS_FILE, 'env', { ...ENV, CLAUDE_CODE_EFFORT_LEVEL: 'xhigh' }, 'CLAUDE_CODE_EFFORT_LEVEL stays unset and no maxEffortLevel caps the stage effort'], + ['the settings cap every model at xhigh', SETTINGS_FILE, 'maxEffortLevel', 'xhigh', 'CLAUDE_CODE_EFFORT_LEVEL stays unset and no maxEffortLevel caps the stage effort'], + ['a per-model settings entry caps Sonnet at high', SETTINGS_FILE, 'modelSettings', { 'claude-sonnet-5': { maxEffortLevel: 'high' } }, 'CLAUDE_CODE_EFFORT_LEVEL stays unset and no maxEffortLevel caps the stage effort'], + ['a harmless max cap in the settings', SETTINGS_FILE, 'maxEffortLevel', 'max', null], + ['a harmless comment mentions a lower effort', 'workflows/review-changes.js', "phase('Review')", "phase('Review') // before 2026-09-23 this stage bound effort: 'high'", null], // Config mutations are applied as JSON edits (key, new value) so they hold in every // layout the config may describe, not only the one whose strings appear here. ['the configured contract heading is absent from the contract doc', 'workflows/contract.config.json', 'contract_heading', '## Workflow contrac', 'contract_heading names the documented contract section'], diff --git a/examples/claude-native/workflows/test-envelope.mjs b/examples/claude-native/workflows/test-envelope.mjs index 48651c8ec..eeb184cd9 100644 --- a/examples/claude-native/workflows/test-envelope.mjs +++ b/examples/claude-native/workflows/test-envelope.mjs @@ -18,6 +18,12 @@ const configured = (key) => resolve(HERE, typeof CONFIG[key] === 'string' ? CONF // afterwards, so the suite always reaches its SUMMARY line. const readOr = (path) => { try { return readFileSync(path, 'utf8') } catch { return '' } } const WF = { review: 'workflows/review-changes.js', readiness: 'workflows/readiness-audit.js' } +// The one effort every saved workflow stage and project agent binds (since 2026-09-23; decision +// docs/decisions/2026-09-23-max-effort-default.md in the catalog). The coordinator stays at xhigh under +// ultracode: a stage with no effort of its own and no agent frontmatter effort inherits that xhigh, and +// CLAUDE_CODE_EFFORT_LEVEL overrides every child's effort, so max is bound per stage and per agent and the +// settings leave that variable unset. +const STAGE_EFFORT = 'max' function load(file, stubs) { const body = readFileSync(join(ROOT, file), 'utf8').replace(/^export const meta/m, 'const meta') const fn = new Function('args', 'agent', 'parallel', 'pipeline', 'phase', 'log', 'budget', 'workflow', 'return (async () => {\n' + body + '\n})()') @@ -358,10 +364,11 @@ function agentOptionLiterals(src) { const packetText = new Function(packets[files.indexOf(f)] + '; return PACKET')() expect('contract: ' + f + ' starts every worker prompt with the full shared packet', packetText.length > 200 && calls.every((c) => c.prompt.startsWith(packetText + '\n'))) // Reviewed routing, by stage label prefix: [agentType, model, effort]. A new saved workflow - // or a rerouted stage fails here until the routing is reviewed and listed. + // or a rerouted stage fails here until the routing is reviewed and listed. Models stay task-matched + // (Sonnet scouts, Opus reviewers and judges); every stage runs at effort max since 2026-09-23. const ROUTING = { - [WF.review]: { inventory: ['source-scout', 'sonnet', 'medium'], review: ['evidence-reviewer', 'opus', 'high'], recheck: ['source-scout', 'sonnet', 'medium'] }, - [WF.readiness]: { 'read:': ['source-scout', 'sonnet', 'medium'], verify: [undefined, 'opus', 'high'] }, + [WF.review]: { inventory: ['source-scout', 'sonnet', 'max'], review: ['evidence-reviewer', 'opus', 'max'], recheck: ['source-scout', 'sonnet', 'max'] }, + [WF.readiness]: { 'read:': ['source-scout', 'sonnet', 'max'], verify: [undefined, 'opus', 'max'] }, 'workflows/layer-verdict-lane.js': { 'propose:': ['blind-lane-reviewer', 'opus', 'max'], 'refute:': ['blind-lane-reviewer', 'opus', 'max'] }, } const routes = ROUTING[f] || {} @@ -380,6 +387,13 @@ function agentOptionLiterals(src) { expect('contract: ' + f + ' has no agent( inside a template literal', (structure(src, true).match(/\bagent\s*\(/g) || []).length === sites) expect('contract: ' + f + ' passes an inline options literal to every agent() call', literals.every((o) => o !== null)) expect('contract: ' + f + ' binds model and effort inside every options literal', literals.every((o) => ['haiku', 'sonnet', 'opus'].includes(optionValue(o, 'model')) && ['low', 'medium', 'high', 'xhigh', 'max'].includes(optionValue(o, 'effort')))) + // Policy, separate from validity: every call site, reached by the stub or not, binds the one stage effort. + expect('contract: ' + f + ' binds effort ' + STAGE_EFFORT + ' in every options literal', sites > 0 && literals.every((o) => optionValue(o, 'effort') === STAGE_EFFORT)) + // A workflow that records its model in a MODEL literal (the layer-verdict lane returns it) must record, in one + // strict single-line form, the same model and effort that every options literal binds. + const modelDecls = src.match(/^\s*(?:export\s+)?(?:const|let|var)\s+MODEL\s*=/mg) || [] + const declared = src.match(/^\s*(?:export\s+)?const\s+MODEL\s*=\s*\{\s*name:\s*'([^']+)',\s*effort:\s*'([^']+)'\s*\}\s*;?\s*$/m) + expect('contract: ' + f + ' records in MODEL, when declared, the same model and effort it binds in every options literal', modelDecls.length === 0 || (modelDecls.length === 1 && declared !== null && literals.every((o) => optionValue(o, 'model') === declared[1] && optionValue(o, 'effort') === declared[2]))) const times = (lit, key) => (lit ? lit.code.match(new RegExp('\\b' + key + ':', 'g')) || [] : []).length expect('contract: ' + f + ' declares model and effort exactly once per options literal and spreads nothing into it', literals.every((o) => o !== null && times(o, 'model') === 1 && times(o, 'effort') === 1 && times(o, 'agentType') <= 1 && !o.code.includes('...'))) const namedAgents = literals.map((o) => optionValue(o, 'agentType')).filter(Boolean) @@ -398,11 +412,40 @@ function agentOptionLiterals(src) { const tools = ((fm.match(/^tools: (.*)$/m) || [null, ''])[1]).split(',').map((t) => t.trim()).filter(Boolean) expect('agents: ' + n + ' name matches its file', nameOf(fm) === n.replace(/\.md$/, '')) expect('agents: ' + n + ' declares an explicit model and effort', /^model: (haiku|sonnet|opus)$/m.test(fm) && /^effort: (low|medium|high|xhigh|max)$/m.test(fm)) + // One effort line, and it is the stage effort: a second, lower line cannot hide behind the first. + expect('agents: ' + n + ' runs at effort ' + STAGE_EFFORT + ' on a single effort line', (fm.match(/^effort:/mg) || []).length === 1 && new RegExp('^effort: ' + STAGE_EFFORT + '$', 'm').test(fm)) expect('agents: ' + n + ' declares a tools allowlist', tools.length > 0) expect('agents: ' + n + ' runs in its own worktree when it can edit files', !tools.some((t) => ['Edit', 'Write', 'NotebookEdit'].includes(t)) || /^isolation: worktree$/m.test(fm)) expect('agents: ' + n + ' grants MCP tools by full name, never a bare server prefix', tools.filter((t) => t.startsWith('mcp__')).every((t) => /^mcp__.+__[A-Za-z0-9_]+$/.test(t) && !t.endsWith('__*'))) expect('agents: ' + n + ' keeps granted MCP tools deferred behind ToolSearch', !tools.some((t) => t.startsWith('mcp__')) || tools.includes('ToolSearch')) } + // The role routing table in the routing doc restates what the agent files and saved stages bind: one + // row per project agent carrying the model and effort its file declares, default-child rows at the stage + // effort, and the coordinator row at xhigh under ultracode. Drift on either side fails here. + const routingLines = readOr(configured('routing_doc')).split('\n') + const head = routingLines.findIndex((l) => l.startsWith('| Task class | Agent / stage | Model, effort |')) + const roleRows = [] + for (let i = head + 2; head >= 0 && i < routingLines.length && routingLines[i].startsWith('|'); i++) roleRows.push(routingLines[i].split('|').slice(1, -1).map((c) => c.trim())) + const title = { haiku: 'Haiku', sonnet: 'Sonnet', opus: 'Opus' } + expect('routing doc: the role routing table lists every project agent once with the model and effort its file declares', roleRows.length > 0 && names.every((n) => { + const fm = (readFileSync(join(dir, n), 'utf8').match(/^---\n([\s\S]*?)\n---/) || [null, ''])[1] + const rows = roleRows.filter((r) => (r[1] || '').startsWith('`' + n.replace(/\.md$/, '') + '`')) + return rows.length === 1 && rows[0][2] === title[(fm.match(/^model: (\w+)$/m) || [])[1]] + ', ' + (fm.match(/^effort: (\w+)$/m) || [])[1] + })) + const defaultRows = roleRows.filter((r) => r[1] === 'default workflow subagent') + expect('routing doc: every default workflow subagent row binds a model at effort ' + STAGE_EFFORT, defaultRows.length > 0 && defaultRows.every((r) => new RegExp('^(Sonnet|Opus), ' + STAGE_EFFORT + '$').test(r[2]))) + const coordinatorRows = roleRows.filter((r) => r[1] === 'coordinator') + expect('routing doc: the coordinator row stays at xhigh under ultracode', coordinatorRows.length === 1 && coordinatorRows[0][2].includes('xhigh under `ultracode`')) + // Effort profile: the coordinator keeps ultracode at xhigh, so the settings must not set + // CLAUDE_CODE_EFFORT_LEVEL (any value overrides every stage's and agent's own effort, and a value other than + // xhigh also leaves ultracode's orchestration inactive) and must not cap effort below the stage effort, either + // with a top-level maxEffortLevel or with one inside a modelSettings entry. + let settingsJson = {} + try { settingsJson = JSON.parse(readOr(configured('settings'))) || {} } catch { settingsJson = {} } + const settingsEnv = settingsJson.env && typeof settingsJson.env === 'object' ? settingsJson.env : {} + const effortCaps = [settingsJson.maxEffortLevel, ...Object.values(settingsJson.modelSettings && typeof settingsJson.modelSettings === 'object' ? settingsJson.modelSettings : {}).map((m) => (m && typeof m === 'object' ? m.maxEffortLevel : undefined))].filter((v) => v !== undefined) + expect('settings: CLAUDE_CODE_EFFORT_LEVEL stays unset and no maxEffortLevel caps the stage effort', !Object.prototype.hasOwnProperty.call(settingsEnv, 'CLAUDE_CODE_EFFORT_LEVEL') && effortCaps.every((v) => v === STAGE_EFFORT)) + expect('instructions: the project instructions state the effort literal every stage binds', readOr(configured('instructions')).includes("`effort: '" + STAGE_EFFORT + "'`")) const reviewer = (readFileSync(join(dir, 'evidence-reviewer.md'), 'utf8').match(/^tools: (.*)$/m) || [null, ''])[1] // The reviewer's surface is pinned exactly, so any added tool (for example a Serena // create_/replace_ tool) fails here until it is reviewed and listed. Two grants stay diff --git a/manifests/evidence.json b/manifests/evidence.json index 42e0bb821..f36252275 100644 --- a/manifests/evidence.json +++ b/manifests/evidence.json @@ -3185,8 +3185,8 @@ }, { "path": "adoption/README.md", - "sha256": "87101a069cd70cefcde02fa05d1fbc1ef32851dd816e75508fd16a24e25e6431", - "bytes": 15085 + "sha256": "6cc9c4a175073b3b87215214ee2f478fdd7a1f25db773dbeb42351352ae1e0bc", + "bytes": 15061 }, { "path": "adoption/bootstrap-linux.sh", @@ -9256,8 +9256,8 @@ }, { "path": "catalogs/foundation/decisions.json", - "sha256": "0bab7062a1acaa2f2292ce467966e3c6134f5be013b7021d61fbc92934a3bb39", - "bytes": 139189 + "sha256": "267bfb42c795b26b35907d48c283e55ed3f7b3ea108ed8b10ae9551569e0c2d2", + "bytes": 139699 }, { "path": "catalogs/foundation/manifest.json", @@ -28161,28 +28161,28 @@ }, { "path": "examples/claude-native/CLAUDE.md", - "sha256": "b4f083809afa233db0cd7bcc7e94f30d921de85cbc9a7d08a9b9d061a8b5bf9f", - "bytes": 5055 + "sha256": "fdb21dc1aa910ff68f2b558a02cdf98c9d836307b85dc8ecdb57aedfa3de4291", + "bytes": 5324 }, { "path": "examples/claude-native/agents/evidence-reviewer.md", - "sha256": "75067deabb854902f680e2edbd24d975d055ef64552aa2a9dd514c20cd7f58b1", - "bytes": 2069 + "sha256": "8ad444fdb16049eb7481e6c703250a0de887fbb537b259da6e72ebff7f3016cc", + "bytes": 2068 }, { "path": "examples/claude-native/agents/isolated-builder.md", - "sha256": "48938a39d10f33a2eef490d885da379cd63a03b7e42e53e4cb3ee3b8c80e1599", - "bytes": 2016 + "sha256": "ca683553bcc9fb824f8b642e77c55be3c5d77115004f6aa57144f0d326b0ca2d", + "bytes": 2013 }, { "path": "examples/claude-native/agents/semantic-evidence-reviewer.md", - "sha256": "a8896d029beed2334215e3cc0990c937b385c43d64683580627f1e0f611ad216", - "bytes": 1563 + "sha256": "52cdd8b58e9900ad43ea2195e4de0a11d95aa3b8c9a2ff6e4eabd149b68e96d9", + "bytes": 1561 }, { "path": "examples/claude-native/agents/source-scout.md", - "sha256": "2d5f5d9465a535d376a0c570a5d24105a939ad6771e504c8dde358469acbbad4", - "bytes": 1829 + "sha256": "035251c0168cc17d66861605f402cfc98ed76bddc5177a4fe28ef3acd35d2534", + "bytes": 1826 }, { "path": "examples/claude-native/ultracode.settings.json", @@ -28221,13 +28221,13 @@ }, { "path": "examples/claude-native/workflows/readiness-audit.js", - "sha256": "d0d5afb3f4f8d635d0a75e48b4c4cda4a074e842130d9c80eb6633a5bfe1701a", - "bytes": 13143 + "sha256": "1d3d816cc219a252760a471a3b68a979057e2c417c429d6c094df4c474e612bb", + "bytes": 13135 }, { "path": "examples/claude-native/workflows/review-changes.js", - "sha256": "581300deaa95c6a2e097ec824cc01aca21ff17f87cb497327e4ddbc1cc37d65f", - "bytes": 18073 + "sha256": "bb298f9ab63a6d43173d2c4f87ab31d00dd79184e961dc84d69cbef3966bce0b", + "bytes": 18059 }, { "path": "examples/claude-native/workflows/test-child-usage.mjs", @@ -28241,8 +28241,8 @@ }, { "path": "examples/claude-native/workflows/test-contract-mutations.mjs", - "sha256": "f418a9da354cb766cbab4f779be754359797b110ac4f3c28cdcda5e319d3c0a1", - "bytes": 9440 + "sha256": "2c2626d76e2113e0b86b372864001e45766d118b12fc0bf9441d937fa555069a", + "bytes": 15529 }, { "path": "examples/claude-native/workflows/test-envelope.mjs", @@ -28641,8 +28641,8 @@ }, { "path": "recipes/claude-native-ultracode.md", - "sha256": "fa8ff1b5d35b8af7dfc8fd5e4c4bfd4617c21bbd665d67dd283fe2e7edd28024", - "bytes": 21050 + "sha256": "338d149fc422372bf8f9784b3f02cc829e60b5cb303eee003a82ef9f7f7c1458", + "bytes": 24666 }, { "path": "recipes/native-upgrades-20260921.md", diff --git a/recipes/claude-native-profile.md b/recipes/claude-native-profile.md index 9a11194ef..e805280c0 100644 --- a/recipes/claude-native-profile.md +++ b/recipes/claude-native-profile.md @@ -188,19 +188,25 @@ coordinator's effort. The short global instruction example makes task-based acceptance, original-source verification and independent review persistent. **2026-09-23: Claude children run at `max`; the coordinator stays at -Ultracode.** Re-measured on Claude Code 2.1.281: `max` in `effortLevel` or -`modelSettings..effortLevel` is still silently dropped (the session -stays at `xhigh`), and a session started with `--effort max` or -`CLAUDE_CODE_EFFORT_LEVEL=max` ran at `max` with Ultracode orchestration off; -`/effort max` likewise replaces the `ultracode` entry of the same menu. The -environment variable also overrides every child's frontmatter and workflow-stage -effort, so keep it unset. The shipped [agent definitions](../adoption/agents/claude/) +Ultracode.** Re-measured on Claude Code 2.1.281: a persisted `max` is still +silently dropped. With `--settings '{"ultracode":false,"effortLevel":"max"}'` +the session ran at `xhigh`, while the same key at `high` ran at `high` (probes +Q1 and Q2); for `modelSettings..effortLevel` the installed schema and +the docs reject `max` as well. A session started with `--effort max` or +`CLAUDE_CODE_EFFORT_LEVEL=max` ran at `max` with Ultracode orchestration off. +`/effort max` was not probed; the docs say Claude Code applies `max` to the +current session only +([model configuration](https://code.claude.com/docs/en/model-config), fetched +2026-09-23). Never set `CLAUDE_CODE_EFFORT_LEVEL`: any value overrides every +child's frontmatter and workflow-stage effort, and any value other than `xhigh` +also turns Ultracode off. The shipped [agent definitions](../adoption/agents/claude/) therefore declare `effort: max` beside their task-matched models (Sonnet for `source-scout` and `isolated-builder`; Opus for `evidence-reviewer`, `semantic-evidence-reviewer` and `blind-judge`), and workflow stages pass -`effort: 'max'` explicitly, since a stage without it inherits the -coordinator's `xhigh`. Verify each child's resolved effort in its transcript -rather than inferring it from a definition. The +`effort: 'max'` explicitly: a stage without its own effort inherits the +coordinator's `xhigh` unless its agent's frontmatter sets one, and a stage's +effort overrides the frontmatter (probe Q3). Verify each child's resolved +effort in its transcript rather than inferring it from a definition. The [Ultracode recipe](claude-native-ultracode.md#child-effort-max-under-an-ultracode-coordinator) has the per-stage rule and the [decision record](../docs/decisions/2026-09-23-max-effort-default.md) has the diff --git a/recipes/claude-native-ultracode.md b/recipes/claude-native-ultracode.md index 792e98388..716d62cf5 100644 --- a/recipes/claude-native-ultracode.md +++ b/recipes/claude-native-ultracode.md @@ -54,9 +54,12 @@ these values and the planned-workstation profile are in `ultracode: true` requests effective `xhigh` effort, subject to applicable caps, and takes precedence over the stored `effortLevel` setting. Preserving that -stored value therefore does not mean effective effort is unchanged. An existing -`CLAUDE_CODE_EFFORT_LEVEL` value other than `xhigh` overrides Ultracode and leaves -its orchestration inactive. Ultracode is not a provider model name. The size +stored value therefore does not mean effective effort is unchanged. Never set +`CLAUDE_CODE_EFFORT_LEVEL`: any value overrides every child's frontmatter and +workflow-stage effort, and any value other than `xhigh` also overrides Ultracode +and leaves its orchestration inactive +([below](#child-effort-max-under-an-ultracode-coordinator)). Ultracode is not a +provider model name. The size guideline is advisory; the environment variable provides a per-workflow concurrency setting, not a total account or cross-session budget. Size by the task: solo for conversational or mechanical @@ -126,7 +129,8 @@ the measured constraints and the per-stage rule. | Exact extraction, inventories, running acceptance commands | `source-scout` (Sonnet, max; four built-in tools, no project instructions) | First prompt 8,048 tokens versus 42,396 for the default child on one identical task; ran the inventory stage of eight native reviews and the readers of two readiness audits (one deployed, one in the scratch adoption); the recheck stage exists since the eighth review and ran there and in the three scratch-adoption reviews | | Implementation from a clear contract | `isolated-builder` (Sonnet, max, own worktree, named MCP tools behind ToolSearch) | One real task: a manifest probe implemented, checked and committed from its own worktree (first prompt 17,864) | | Independent review from source and recorded evidence | `evidence-reviewer` (Opus, max; read-only named MCP tools behind ToolSearch, no Bash/Edit/Write) | Eight native review runs; first prompt 12,164 for the deferred shape versus 42,220 with bare server grants | -| Review of supplied semantic (TypeSafe) judgments against original source | `semantic-evidence-reviewer` (Opus, max; Read, Glob and Grep, `typesafe-ai` skill preloaded) | One probe, `wf_20a5e69a-84d`, measured the skill preload (first prompt 15,059 tokens; [convergence record](../docs/harness-rules-convergence-20260922.md)); it proposes in the vendored [layer-verdict lane](../examples/claude-native/workflows/layer-verdict-lane.js); no quality comparison with another reviewer is recorded | +| Review of supplied semantic (TypeSafe) judgments against original source | `semantic-evidence-reviewer` (Opus, max; Read, Glob and Grep, `typesafe-ai` skill preloaded) | One probe, `wf_20a5e69a-84d`, measured the skill preload (first prompt 15,059 tokens; [convergence record](../docs/harness-rules-convergence-20260922.md)); no quality comparison with another reviewer is recorded. The vendored layer-verdict lane no longer uses it (next row) | +| Proposing, refuting and re-checking one stripped layer-verdict packet | `blind-lane-reviewer` (Opus, max; Read, Glob and Grep, no preloaded skill, no project instructions) | Every stage of the vendored [layer-verdict lane](../examples/claude-native/workflows/layer-verdict-lane.js) names it (agent-lab `915e73e`, vendored with the lane). A skill it preloaded could be one of the candidates a packet judges, and project instructions can name incumbent selections, so it carries neither; this catalog records no dated run of the lane with it | | Judging or refuting one sealed comparison packet | `blind-judge` (Opus, max; Read only, no project instructions) | Its frontmatter was checked against the agent contract in the [convergence record](../docs/harness-rules-convergence-20260922.md); its body was not reviewed there, and this catalog records no dated run of the role | | Cheap exact extraction | Haiku | Not routed: on one byte-identical packet the Opus verifier scored Sonnet 14/14 lane rows and Haiku 9/14 with a quote attributed to a file that does not contain it; overturn only after a repeat trial with no unanchored citation on two distinct packets. Haiku 4.5 takes no effort level, so `max` does not apply to it | | Independent cross-family review | Existing official Codex companion | Reuse its separately recorded native acceptance; this trial did not run Codex inside a Workflow graph. Its reasoning effort follows the Codex configuration, not this table | @@ -154,20 +158,21 @@ without output pipelines that mask their exit status. The portable [saved workflow examples](../examples/claude-native/workflows/README.md) are byte-identical copies of the deployed review and readiness scripts, the -three [native agent definitions](../examples/claude-native/agents/), the per-child +[native agent definitions](../examples/claude-native/agents/), the per-child usage extractor `child-usage.mjs`, the Codex bridge and their local checks (a static contract suite, a mutation harness and receipt bindings driven by a sibling `contract.config.json`). `review-changes` re-runs every acceptance command with a second worker and compares both runs in code. Record each run's children with `node .claude/workflows/child-usage.mjs --latest`. -These examples are dated evidence: their agents, stages and contract suite -keep the efforts they were qualified at (Sonnet/medium, Opus/high). When -adopting them after 2026-09-23, raise each stage and agent effort to `max` -under the per-stage rule below, and update the suite to match: the `ROUTING` -table in `test-envelope.mjs` (which already accepts `max`) and the effort text -that cases in `test-contract-mutations.mjs` search for; the shipped -[`adoption/agents/claude/`](../adoption/agents/claude/) definitions already -declare it. +Since 2026-09-23 the examples' agents and stages bind effort `max` with their +task-matched models unchanged (the review, readiness and layer-verdict scripts +are byte-identical to agent-lab commit `915e73e`), as do the shipped +[`adoption/agents/claude/`](../adoption/agents/claude/) definitions. Their +contract suite now fails a stage or agent at any other effort, a `MODEL` record +that differs from the stages it describes, a routing table that restates an +agent's effort differently, and a settings file that sets +`CLAUDE_CODE_EFFORT_LEVEL` or caps effort below `max`. The qualifications above +were recorded at the earlier efforts (Sonnet/medium, Opus/high). Adopt selected files into a project's existing `.claude/` directories without overwriting its instructions, accounts or permissions. A saved script is a supported native extension, not an upstream-authored acceptance policy. @@ -200,36 +205,55 @@ probe and its result. model and additionally has Claude orchestrate dynamic workflows; any other resolved effort, `max` included, leaves orchestration inactive. Sessions started with `--effort max` or `CLAUDE_CODE_EFFORT_LEVEL=max` ran at `max` - and had no Ultracode system reminder, so one session cannot have both. -- **`max` cannot be persisted.** `effortLevel: "max"` and - `modelSettings..effortLevel: "max"` were silently dropped from - `--settings` and from a project `.claude/settings.json`, with or without - `ultracode: true` beside them: the session stayed at `xhigh`, with no - warning. The installed schema accepts only `low`, `medium`, `high` and - `xhigh` in both keys, and the official docs say `max` "isn't accepted as a + and had no Ultracode system reminder (P1, P2), so one session cannot have + both. +- **`max` cannot be persisted.** With `--settings + '{"ultracode":false,"effortLevel":"max"}'` the session ran at `xhigh`, the + level of the user's per-model `modelSettings` entry, with no warning: the + `max` was dropped and the next source decided (Q1). The control with + `"effortLevel":"high"` ran at `high`, so the key applies a valid value (Q2). + The earlier probes that put `max` in `effortLevel` or + `modelSettings..effortLevel` (`--settings` or a project + `.claude/settings.json`: P3, P4, P7, P8) also stayed at `xhigh`, but they ran + with `ultracode: true`, which takes precedence over both keys, so on their + own they cannot show the drop. For the per-model key and a project file the + evidence is the installed schema, which accepts only `low`, `medium`, `high` + and `xhigh` in both keys, and the official docs: `max` "isn't accepted as a level in either key" and otherwise applies to the current session only. - **`CLAUDE_CODE_EFFORT_LEVEL` overrides every child's effort.** With it set to `max`, a default Agent child, a `source-scout` whose frontmatter then said `sonnet`/`medium` and a workflow stage that passed `effort: 'high'` all ran - at `max`. Do not set it: besides disabling Ultracode, it erases every - per-agent and per-stage effort. + at `max` (P5, P6, P9), and the docs say frontmatter effort overrides the + session level "but not the environment variable". Never set it: any value + erases every per-agent and per-stage effort, and any value other than + `xhigh` also turns Ultracode off. Under the default Ultracode coordinator (`xhigh`, orchestration active), a -workflow stage that passed `effort: 'max'` ran at `max`, an Agent whose -frontmatter says `effort: max` ran at `max`, and a stage without `effort` -inherited `xhigh`. The installed binary has no default-effort key for -subagents or workflow stages (`CLAUDE_CODE_SUBAGENT_MODEL` sets only the -model), so name the effort wherever a child is defined: +workflow stage that passed `effort: 'max'` ran at `max` on Opus 5.5 (H1) and on +Sonnet 5 (Q3); a stage naming an agent whose frontmatter says `effort: max` +ran at `max` with or without its own `effort: 'max'` (Q3; H1 saw the same for +an Agent-tool child); a stage that +passed `effort: 'max'` to an agent whose frontmatter says `effort: medium` ran +at `max`, so the stage's effort wins over the frontmatter (Q3); and a stage +with neither inherited `xhigh` (H1). The installed binary has no +default-effort key for subagents or workflow stages +(`CLAUDE_CODE_SUBAGENT_MODEL` sets only the model), so name the effort +wherever a child is defined: - every project agent declares `effort: max` in its frontmatter beside its task-matched `model` (the shipped [agent definitions](../adoption/agents/claude/) - do); + and the [examples](../examples/claude-native/agents/) do); - every saved workflow stage and every ad-hoc `agent()` call in a workflow - script passes `effort: 'max'` together with an explicit `model`, because a - stage without `effort` inherits the coordinator's `xhigh`; -- a single non-orchestrated session at `max` stays available on request with - `claude --effort max` or `/effort max`; either turns Ultracode off for that - session. + script passes `effort: 'max'` together with an explicit `model`: a stage + without its own `effort` inherits the coordinator's `xhigh` unless its agent's + frontmatter sets one, and the stage's effort also overrides a lower + frontmatter; +- a single non-orchestrated session at `max` stays available on request: + `claude --effort max` ran at `max` with orchestration off (P2). `/effort max` + was not probed; the docs say Claude Code applies `max` "to the current + session only" and list `ultracode` as a separate entry of the `/effort` menu + ([model configuration](https://code.claude.com/docs/en/model-config), fetched + 2026-09-23). The docs list `max` for Opus 5.5, Fable 5.1 and Sonnet 5 (the probes observed it on Opus 5.5 and Sonnet 5); a model without it falls back to its highest @@ -237,24 +261,44 @@ supported level, and Haiku 4.5 takes no effort level. The official docs warn that `max` "may show diminishing returns and is prone to overthinking" and advise testing before adopting it broadly. Children are expected to spend more output tokens, and no sweep has measured that cost or a -quality gain yet (`autoContinueAtUsageLimit` stays on in the settings template). +quality gain yet. `autoContinueAtUsageLimit` (on in the settings template) is +not a cost control and does not reduce usage: it lets an interactive session +signed in with a claude.ai subscription, and a workflow run inside it, wait for +a usage limit to reset (a reset within 24 hours, at most twice in a row) instead +of failing. `claude -p`, Agent SDK and GitHub Actions runs and background +sessions get no wait and fail at the limit +([interactive mode](https://code.claude.com/docs/en/interactive-mode#wait-for-a-usage-limit-to-reset), +[workflows](https://code.claude.com/docs/en/workflows#when-a-run-hits-your-usage-limit), +both fetched 2026-09-23). ## GitHub and cloud sessions No GitHub Actions workflow in this catalog invokes Claude. For a future one -built on `anthropics/claude-code-action` (its `action.yml` read at -`8cf3482550831fb35a4fc3fbf7ca139cf8028b4c`, 2026-09-23): the action has no -`effort`, `ultracode` or `model` input. Request Ultracode through -`claude_args: '--effort ultracode'` or the `settings` input -`'{"ultracode": true}'`, and never put `--effort max` in `claude_args` beside -Ultracode: an explicit `max` wins and turns orchestration off. This +built on `anthropics/claude-code-action` (its `action.yml` and `base-action` +source read on 2026-09-23 at commit `8cf3482550831fb35a4fc3fbf7ca139cf8028b4c`): +the action has no `effort`, `ultracode` or `model` input, and `base-action` +runs Claude through the Agent SDK's `query()`, so a job is a non-interactive +Agent SDK run. Request Ultracode through `claude_args: '--effort ultracode'` or +the `settings` input `'{"ultracode": true}'`, and never put `--effort max` in +`claude_args` beside Ultracode: an explicit `max` wins and turns orchestration +off. In `claude -p` and Agent SDK runs Claude Code shows no workflow approval +prompt; a Workflow launch goes through the session's normal permission +evaluation, so it starts only under a `Workflow` or `Workflow()` allow +rule, auto or bypass permission mode, or a hook that allows the call +([workflows](https://code.claude.com/docs/en/workflows#approve-the-plan-before-it-runs), +fetched 2026-09-23). Such a job fails at a usage limit instead of waiting. This repository's committed [`.claude/settings.json`](../.claude/settings.json) is the settings example above, byte for byte. Per the official [cloud-session settings](https://code.claude.com/docs/en/settings#settings-in-cloud-sessions) docs, a cloud session on this one repository reads it, while a session with several repositories reads only its `enabledPlugins` and -`extraKnownMarketplaces` keys. Neither the action nor a cloud session was run; -this section rests on the documentation and the action's source. +`extraKnownMarketplaces` keys. The action itself has been executed for this +catalog only as a 2026-09-23 CI smoke in a scratch repository: a smoke reply, +an allow control, and a settings-level deny rule passed through its `settings` +input that produced one permission denial +([gated items](../catalogs/sota-convergence/sdk-runtime-coverage-20260922.md#gated-items-executed-2026-09-23)). +No Ultracode or effort run of the action or of a cloud session was executed; +the rest of this section rests on the documentation and the action's source. ## Native sessions, result return and messaging diff --git a/tests/test_adoption_docs_consistency.py b/tests/test_adoption_docs_consistency.py index 61a47203c..66e5433d4 100644 --- a/tests/test_adoption_docs_consistency.py +++ b/tests/test_adoption_docs_consistency.py @@ -20,10 +20,11 @@ - every script path those documents mention exists and is tracked at HEAD; - a new-host page unit (a heading section or a top-level numbered step) that mentions a path the pinned release lacks (scripts/release_due.py's ``due`` list) says "added after ``"; -- a new-host page that mentions an install input (bootstrap script or pin file) whose content - differs between the pinned release and HEAD says "changed after ``" in a unit - that mentions it. release_due.py reports every changed new-machine file in its ``changed`` - list; this check requires the per-page note for the install inputs; +- a new-host page that mentions an install input (bootstrap script, pin file, or an asset the + Claude profile installer copies onto the host: the agent and hook directories and the MCP + template) whose content differs between the pinned release and HEAD says "changed after + ``" in a unit that mentions it. release_due.py reports every changed new-machine + file in its ``changed`` list; this check requires the per-page note for these install inputs; - bootstrap.md's plugin revision check quotes the same commits as the recipes/README.md rows it names, and its install commands are the recipe's own commands (nothing else binds those copies, and a marketplace source cannot enforce a commit); @@ -68,7 +69,10 @@ COVERAGE_RE = re.compile(rf"^(all \d+|none of \d+|\d+ of \d+)(?: \((all \d+|none of \d+|\d+ of \d+) at `?({TAG})`?\))?$") HISTORICAL_TAG_RE = re.compile(rf"(?:\b(?:added|changed) after|\d+\)? at) `?{TAG}`?", re.I) INSTALL_INPUTS = ("adoption/bootstrap-linux.sh", "adoption/bootstrap-macos.sh", - "adoption/pins-linux-x86_64.json", "adoption/pins-macos-arm64.json") + "adoption/pins-linux-x86_64.json", "adoption/pins-macos-arm64.json", + # What tools/adoption/install_claude_profile.py copies onto the host. A directory + # (trailing slash) changed when its tracked files or any one file's text differ. + "adoption/agents/claude/", "adoption/hooks/claude/", "adoption/mcp/claude-user.json") # Independent store_true flags where one silently wins, so argparse cannot reject the pair. EXCLUSIVE_IN_EFFECT = {"component_matrix.py": [frozenset({"--write", "--check"})]} # write_mode = write and not check @@ -107,6 +111,25 @@ def show(commit: str, path: str) -> str | None: return result.stdout if result.returncode == 0 else None +def changed_since(commit: str, path: str) -> bool: + """Whether an install input differs between ``commit`` and this checkout: a file's text, or for a + directory (trailing slash) the set of tracked files under it or any one file's text.""" + if not path.endswith("/"): + return show(commit, path) != (ROOT / path).read_text(encoding="utf-8") + at_release = rd.git("ls-tree", "-r", "--name-only", commit, "--", path) + tracked = rd.git("ls-files", "--", path) + names = set(tracked.stdout.splitlines()) + if at_release.returncode != 0 or tracked.returncode != 0 or set(at_release.stdout.splitlines()) != names: + return True + return any(show(commit, name) != (ROOT / name).read_text(encoding="utf-8") for name in names) + + +def mention_names(path: str) -> tuple[str, ...]: + """A file input is mentioned by its path or file name; a directory only by its path, never by its + last component alone (``claude`` would match every page).""" + return (path, path.rstrip("/")) if path.endswith("/") else (path, Path(path).name) + + def resolve_commit(ref: str) -> str | None: result = rd.git("rev-parse", "--verify", "--quiet", f"{ref}^{{commit}}") return result.stdout.strip() if result.returncode == 0 else None @@ -444,7 +467,7 @@ def unmarked(self, text: str, due: set[str], label: str, tag: str) -> list[str]: def unmarked_changes(self, text: str, changed: list[str], label: str, tag: str) -> list[str]: errors = [] for path in changed: - names = (path, Path(path).name) + names = mention_names(path) mentioning = [unit for unit in units(text) if any(name in unit for name in names)] if mentioning and not any(self.marked(unit, "changed", tag) for unit in mentioning): errors.append(f"{label} mentions {path}, which changed after {tag}, but no unit that " @@ -464,8 +487,7 @@ def test_units_using_unreleased_paths_say_so(self): def test_pages_mentioning_a_changed_install_input_say_so(self): self.require_release() - changed = [path for path in INSTALL_INPUTS - if show(SOURCE["release_commit"], path) != (ROOT / path).read_text(encoding="utf-8")] + changed = [path for path in INSTALL_INPUTS if changed_since(SOURCE["release_commit"], path)] errors = [error for path in self.PAGES for error in self.unmarked_changes(path.read_text(encoding="utf-8"), changed, rel(path), SOURCE["release_tag"])] @@ -495,6 +517,24 @@ def test_the_check_rejects_an_undisclosed_install_change(self): self.assertEqual(self.unmarked_changes(disclosed, changed, "ok", "v2000.01.01"), []) self.assertEqual(self.unmarked_changes("## Other\n\nNo mention.\n", changed, "ok", "v2000.01.01"), []) + def test_a_changed_directory_input_is_matched_by_its_path_not_its_last_component(self): + changed = ["adoption/agents/claude/"] + text = ("## Profile\n\nCopies the [`adoption/agents/claude/*.md`](agents/claude/) files.\n\n" + "## Other\n\nClaude Code signs in natively.\n") + self.assertEqual(len(self.unmarked_changes(text, changed, "mutant", "v2000.01.01")), 1) + disclosed = text.replace(" files.", " files (changed after `v2000.01.01`).") + self.assertEqual(self.unmarked_changes(disclosed, changed, "ok", "v2000.01.01"), []) + self.assertEqual(self.unmarked_changes("## Other\n\nClaude Code signs in natively.\n", changed, "ok", + "v2000.01.01"), []) + + def test_the_directory_change_check_compares_tracked_files(self): + baseline = SOURCE["baseline_commit"] # predates adoption/ entirely + if rd.git("cat-file", "-e", f"{baseline}^{{commit}}").returncode != 0: + self.skipTest(f"baseline commit {baseline} is not in this clone") + self.assertTrue(changed_since(baseline, "adoption/agents/claude/")) + if rd.git("diff", "--quiet", "HEAD", "--", "adoption/hooks/claude/").returncode == 0: + self.assertFalse(changed_since("HEAD", "adoption/hooks/claude/")) + class PluginRevisionCheckTests(unittest.TestCase): """bootstrap.md's plugin revision check compares installed_plugins.json with a copy of the diff --git a/tests/test_install_claude_profile.py b/tests/test_install_claude_profile.py index 243efefc4..3f0aee6b4 100644 --- a/tests/test_install_claude_profile.py +++ b/tests/test_install_claude_profile.py @@ -150,6 +150,39 @@ def test_second_run_skips_unchanged_files(self): self.assertTrue(all(r == "skipped" for r in results)) +class ShippedAgentEffortTests(unittest.TestCase): + """Every shipped agent runs at effort max beside its task-matched model + (docs/decisions/2026-09-23-max-effort-default.md): an agent's frontmatter effort + is what a stage without its own effort runs at, and a second, lower effort line + must not hide behind the first.""" + + @staticmethod + def frontmatter(path: Path) -> list[str]: + lines = path.read_text(encoding="utf-8").splitlines() + if not lines or lines[0] != "---" or "---" not in lines[1:]: + return [] + return lines[1:lines.index("---", 1)] + + def test_each_shipped_agent_has_exactly_one_effort_max_line(self): + agents = sorted(icp.AGENTS_SRC_DIR.glob("*.md")) + self.assertTrue(agents) + for path in agents: + with self.subTest(agent=path.name): + effort_lines = [line for line in self.frontmatter(path) if line.startswith("effort:")] + self.assertEqual(effort_lines, ["effort: max"]) + + def test_the_check_rejects_a_lower_or_repeated_effort(self): + with tempfile.TemporaryDirectory() as tmp: + for body in ("---\nname: a\nmodel: opus\neffort: high\n---\nx\n", + "---\nname: a\nmodel: opus\neffort: max\neffort: high\n---\nx\n", + "---\nname: a\nmodel: opus\n---\neffort: max\n"): + path = Path(tmp) / "a.md" + path.write_text(body, encoding="utf-8") + with self.subTest(body=body): + effort_lines = [line for line in self.frontmatter(path) if line.startswith("effort:")] + self.assertNotEqual(effort_lines, ["effort: max"]) + + class McpMatchTests(unittest.TestCase): def test_http_server_matches_on_url_and_type(self): existing = "ai-memory:\n Type: http\n URL: http://127.0.0.1:49374/mcp\n" diff --git a/tests/test_verdict_lane_vendoring.py b/tests/test_verdict_lane_vendoring.py index a2e2441af..f48469990 100644 --- a/tests/test_verdict_lane_vendoring.py +++ b/tests/test_verdict_lane_vendoring.py @@ -9,6 +9,7 @@ import hashlib import importlib.util import json +import re import unittest from pathlib import Path @@ -41,6 +42,23 @@ def test_the_lane_prompt_it_fills_is_the_catalog_copy(self): # lane-prompt.md was already in the catalog; codex_lane.py and the packets read it from there. self.assertTrue((ROOT / "tools" / "sota-convergence" / "lane-prompt.md").is_file()) + def test_every_agent_the_vendored_lane_names_is_vendored_and_pinned_beside_it(self): + # Since agent-lab 915e73e every lane stage names agentType 'blind-lane-reviewer'; a host holding + # only this catalog needs that definition too, byte for byte the agent-lab file the pin names. + pin = json.loads((WORKFLOWS / "vendored-lanes.json").read_text(encoding="utf-8")) + entry = next(item for item in pin["files"] if item["path"] == "layer-verdict-lane.js") + lane = (WORKFLOWS / "layer-verdict-lane.js").read_text(encoding="utf-8") + named = set(re.findall(r"agentType: '([^']+)'", lane)) + required = {Path(item["path"]).stem: item for item in entry.get("required_agents", [])} + self.assertTrue(named) + self.assertEqual(set(required), named) + for name, item in required.items(): + path = (WORKFLOWS / item["path"]).resolve() + self.assertEqual(path, (ROOT / "examples" / "claude-native" / "agents" / f"{name}.md").resolve()) + self.assertEqual(hashlib.sha256(path.read_bytes()).hexdigest(), item["sha256"]) + self.assertEqual(item["source_path"], f".claude/agents/{name}.md") + self.assertRegex(item["agentlab_commit"], r"^[a-f0-9]{40}$") + TOOLS = ROOT / "tools" / "sota-convergence" From 961c78260642bc05cb8f9c275a715c2942b95e47 Mon Sep 17 00:00:00 2001 From: seathatflowsinourveins <234074349+seathatflowsinourveins@users.noreply.github.com> Date: Wed, 23 Sep 2026 20:44:07 -0400 Subject: [PATCH 3/6] Run the example contract suites in CI; fix the settings-example, adopt-step and concurrency wording Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/validate.yml | 5 +++++ catalogs/foundation/decisions.json | 2 +- examples/claude-native/workflows/README.md | 3 ++- recipes/claude-native-ultracode.md | 3 ++- 4 files changed, 10 insertions(+), 3 deletions(-) diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 1f3ca9072..ad8c801c4 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -40,6 +40,11 @@ jobs: run: | cd examples/claude-native/workflows sha256sum --check --strict SHA256SUMS + - name: Run the example workflow contract suites + run: | + cd examples/claude-native/workflows + node test-envelope.mjs + node test-contract-mutations.mjs - name: Install checksum-locked CI analyzer run: >- python3 -m pip install --disable-pip-version-check --no-input diff --git a/catalogs/foundation/decisions.json b/catalogs/foundation/decisions.json index bb06d9379..4e1a7acad 100644 --- a/catalogs/foundation/decisions.json +++ b/catalogs/foundation/decisions.json @@ -2659,7 +2659,7 @@ ], "selection": "conditional", "review_status": "accepted_within_scope", - "activation": "Name `agentType` where the role fits: `source-scout` (Sonnet/max) for extraction, inventories and acceptance commands, `evidence-reviewer` (Opus/max, read-only) for independent review, `isolated-builder` (Sonnet/max, own worktree) for bounded implementation; keep the default child for a stage that must re-run commands and judge them; never omit a stage's model or effort, and since 2026-09-23 bind effort `max` on every stage and agent with the models unchanged while the coordinator stays at xhigh under Ultracode (docs/decisions/2026-09-23-max-effort-default.md); run `workflowSizeGuideline: unrestricted` with three concurrent agents and size by task; record every run with `child-usage.mjs --latest`.", + "activation": "Name `agentType` where the role fits: `source-scout` (Sonnet/max) for extraction, inventories and acceptance commands, `evidence-reviewer` (Opus/max, read-only) for independent review, `isolated-builder` (Sonnet/max, own worktree) for bounded implementation; keep the default child for a stage that must re-run commands and judge them; never omit a stage's model or effort, and since 2026-09-23 bind effort `max` on every stage and agent with the models unchanged while the coordinator stays at xhigh under Ultracode (docs/decisions/2026-09-23-max-effort-default.md); run `workflowSizeGuideline: unrestricted` with `CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS=8` (the committed settings example) and size by task; record every run with `child-usage.mjs --latest`.", "component_ids": [ "claude-code", "codex" diff --git a/examples/claude-native/workflows/README.md b/examples/claude-native/workflows/README.md index 6d8facb3b..b5f215bd6 100644 --- a/examples/claude-native/workflows/README.md +++ b/examples/claude-native/workflows/README.md @@ -38,7 +38,8 @@ the Codex side. 2. Copy `contract.config.json` beside the workflows and point its paths at the project: `settings` (the project's `.claude/settings.json`, merged from [`ultracode.settings.json`](../ultracode.settings.json)), `instructions` (the - file that carries the sizing sentence, for example `AGENTS.md`), + file that carries the sizing sentence and the `effort: 'max'` stage rule, + for example `AGENTS.md`), `contract_doc` (a document holding this README's `## Workflow contract` section) and, when the project keeps usage receipts, `usage_receipts_dir`, `routing_doc` and `task_record`. A configured path that does not exist fails diff --git a/recipes/claude-native-ultracode.md b/recipes/claude-native-ultracode.md index 716d62cf5..96fffdda7 100644 --- a/recipes/claude-native-ultracode.md +++ b/recipes/claude-native-ultracode.md @@ -288,7 +288,8 @@ rule, auto or bypass permission mode, or a hook that allows the call ([workflows](https://code.claude.com/docs/en/workflows#approve-the-plan-before-it-runs), fetched 2026-09-23). Such a job fails at a usage limit instead of waiting. This repository's committed [`.claude/settings.json`](../.claude/settings.json) is -the settings example above, byte for byte. Per the official +byte-identical to the linked settings example file (the inline JSON above +omits its `$schema` line). Per the official [cloud-session settings](https://code.claude.com/docs/en/settings#settings-in-cloud-sessions) docs, a cloud session on this one repository reads it, while a session with several repositories reads only its `enabledPlugins` and From a56c62da3a75dac43afa2b81f8e99f99634023cf Mon Sep 17 00:00:00 2001 From: seathatflowsinourveins <234074349+seathatflowsinourveins@users.noreply.github.com> Date: Wed, 23 Sep 2026 22:13:11 -0400 Subject: [PATCH 4/6] Sync source-scout (maxTurns 100, foreground acceptance) and isolated-builder from agent-lab #46 (a7a67a3e) Co-Authored-By: Claude Opus 5.5 (1M context) --- adoption/agents/claude/isolated-builder.md | 2 +- adoption/agents/claude/source-scout.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/adoption/agents/claude/isolated-builder.md b/adoption/agents/claude/isolated-builder.md index 7c0432213..94cd8e427 100644 --- a/adoption/agents/claude/isolated-builder.md +++ b/adoption/agents/claude/isolated-builder.md @@ -7,4 +7,4 @@ effort: max isolation: worktree --- -Read AGENTS.md and the assigned task brief. Verify your actual working directory, branch, and starting commit. Work only on the assigned outcome and paths. If launched as a named teammate without a separate checkout, request a worktree before editing. Run the appropriate existing checks and report their outcomes. Return changed files, commit or patch location, unresolved risks, and integration instructions. Do not merge into another worker's branch. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. Focused Read/Grep or Serena for symbols, references and symbol-level edits, SocratiCode for conceptual code, jCodeMunch `order` for indexed retrieval, QMD (`qmd search`) for scoped Markdown, Context Mode (ctx_execute) for large command output, ai-memory query for prior decisions; one lane per artifact, verify original source before editing, keep RTK-filtered output and preserve raw-output recovery paths. A project skill is not preloaded: when the brief names one, Read its SKILL.md path. +Read the assigned task brief, and the target repository's AGENTS.md when it is not the loaded project. Verify your actual working directory, branch, and starting commit. Work only on the assigned outcome and paths. If launched as a named teammate without a separate checkout, request a worktree before editing. Run the appropriate existing checks and report their outcomes. Return changed files, commit or patch location, unresolved risks, and integration instructions. Do not merge into another worker's branch. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. Focused Read/Grep or Serena for symbols, references and symbol-level edits, SocratiCode for conceptual code, jCodeMunch `order` for indexed retrieval, QMD (`qmd search`) for scoped Markdown, Context Mode (ctx_execute) for large command output, ai-memory query for prior decisions; one lane per artifact, verify original source before editing, keep RTK-filtered output and preserve raw-output recovery paths. A project skill is not preloaded: when the brief names one, Read its SKILL.md path. diff --git a/adoption/agents/claude/source-scout.md b/adoption/agents/claude/source-scout.md index d38a497c7..8b826c16a 100644 --- a/adoption/agents/claude/source-scout.md +++ b/adoption/agents/claude/source-scout.md @@ -4,8 +4,8 @@ description: Exact extraction and inventory from named files and commands, plus tools: Read, Grep, Glob, Bash model: sonnet effort: max -maxTurns: 40 +maxTurns: 100 omitClaudeMd: true --- -You extract exact facts from the sources named in your task and nothing else. Project instructions are deliberately not loaded for this role; these rules replace them. Do not edit, create or delete files, change git state, install anything or use the network. Use `rg -n` and focused `Read` ranges for known identifiers, `qmd search` for scoped Markdown and `jq`/`rg` pipelines so that only the derived answer is printed; never print a whole large file or log. Bash output is condensed by the RTK hook: treat it as complete, and re-run as `rtk proxy ` only when a result is empty, garbled or contradicts its exit code. Acceptance commands named in your task are the one exception to the no-write rule and to condensed output: run each exactly as given, launched through `rtk proxy` so its output stays raw when `rtk` is installed (report the command as given, not the prefix), even when it writes build, test or temporary artifacts, without pipelines that hide the exit status, and copy the exit code and summary verbatim; never add a writing command of your own, and report a command you did not run as not run. Cite every fact with file path and line or the exact command. Copy numbers exactly. Never print credential or environment values. Report a missing, unreadable or empty source as such; do not fill gaps from memory, and do not restate a failure as a pass. Judgment, design and review belong to other roles: when the task needs one, return what you observed and say what remains undecided. +You extract exact facts from the sources named in your task and nothing else. Project instructions are deliberately not loaded for this role; these rules replace them. Do not edit, create or delete files, change git state, install anything or use the network. Use `rg -n` and focused `Read` ranges for known identifiers, `qmd search` for scoped Markdown and `jq`/`rg` pipelines so that only the derived answer is printed; never print a whole large file or log. Bash output is condensed by the RTK hook: treat it as complete, and re-run as `rtk proxy ` only when a result is empty, garbled or contradicts its exit code. Acceptance commands named in your task are the one exception to the no-write rule and to condensed output: run each exactly as given, launched through `rtk proxy` so its output stays raw when `rtk` is installed (report the command as given, not the prefix), even when it writes build, test or temporary artifacts, without pipelines that hide the exit status, and copy the exit code and summary verbatim; never add a writing command of your own, and report a command you did not run as not run. Run acceptance commands in the foreground with a timeout; do not use run_in_background or poll a background job. The tool limit is 10 minutes: report a command that needs longer as not run, for the coordinator to run. Cite every fact with file path and line or the exact command. Copy numbers exactly. Never print credential or environment values. Report a missing, unreadable or empty source as such; do not fill gaps from memory, and do not restate a failure as a pass. Judgment, design and review belong to other roles: when the task needs one, return what you observed and say what remains undecided. From cad8d255e5d54b7b1e8cada7d2916bd6ebadc364 Mon Sep 17 00:00:00 2001 From: seathatflowsinourveins <234074349+seathatflowsinourveins@users.noreply.github.com> Date: Thu, 24 Sep 2026 12:32:22 -0400 Subject: [PATCH 5/6] Rebase onto #145 and v2026.09.24.1: merge the secret guard into project settings, blind-adjudicator routing row, current agent-lab pins - .claude/settings.json keeps main's secret-read deny rules and guard hook and adds the ultracode keys; the recipe and decision no longer call it byte-identical. - bootstrap.md marks adoption/agents/claude and adoption/hooks/claude as changed after v2026.09.24.1; the lane test and lane files are #145's. - child-usage mirror synced from agent-lab; examples routing table lists blind-adjudicator. Co-Authored-By: Claude Opus 5.5 (1M context) --- adoption/bootstrap.md | 17 +++-- .../2026-09-23-max-effort-default.md | 29 +++---- examples/claude-native/workflows/README.md | 6 +- examples/claude-native/workflows/SHA256SUMS | 12 +-- .../claude-native/workflows/child-usage.mjs | 75 +++++++++++++++++-- .../workflows/test-child-usage.mjs | 43 ++++++++++- manifests/evidence.json | 64 ++++++++-------- recipes/claude-native-ultracode.md | 7 +- tests/test_verdict_lane_vendoring.py | 18 ----- 9 files changed, 180 insertions(+), 91 deletions(-) diff --git a/adoption/bootstrap.md b/adoption/bootstrap.md index 9ce263334..ef661dd00 100644 --- a/adoption/bootstrap.md +++ b/adoption/bootstrap.md @@ -228,13 +228,16 @@ GitHub-hosted macOS runner; see template does not mention is kept), writes atomically and preserves the original file's mode bits. Never touches `~/.claude.json` or any credential store. - The agent definitions in `adoption/agents/claude/` changed after `v2026.09.23.1`: - at that tag they declare `effort: medium` (`source-scout`, - `isolated-builder`) or `effort: high` (`evidence-reviewer`, - `semantic-evidence-reviewer`, `blind-judge`) instead of `effort: max` - ([decision](../docs/decisions/2026-09-23-max-effort-default.md)). The - installer replaces a differing agent file, so rerunning its agents step - from a checkout that has the change installs the `max` definitions. + The agent definitions in `adoption/agents/claude/` changed after `v2026.09.24.1`: + at that tag `source-scout` and `isolated-builder` declare `effort: medium` + (`source-scout` also `maxTurns: 40`), `evidence-reviewer`, + `semantic-evidence-reviewer` and `blind-judge` declare `effort: high`, and the + two blind lane roles are absent; here all seven declare `effort: max` + ([decision](../docs/decisions/2026-09-23-max-effort-default.md)). The guard + hooks' `adoption/hooks/claude/` also changed after `v2026.09.24.1` (its + `SHA256SUMS` gained the secret-path guard entry). The installer replaces a + differing agent file, so rerunning its agents step from a checkout that has + the change installs the `max` definitions. **Plugin revision check** (added after `v2026.09.23.1`; it reads only this host's plugin registry, so it runs the same from any checkout). A Claude diff --git a/docs/decisions/2026-09-23-max-effort-default.md b/docs/decisions/2026-09-23-max-effort-default.md index 56cee972f..7d99ff810 100644 --- a/docs/decisions/2026-09-23-max-effort-default.md +++ b/docs/decisions/2026-09-23-max-effort-default.md @@ -10,16 +10,16 @@ instead. **Scope:** -- the five shipped agent definitions (`adoption/agents/claude/*.md`, kept byte-identical - to agent-lab's `.claude/agents/` copies at agent-lab commit `915e73e`, which made the - same one-line change there); +- the shipped agent definitions (`adoption/agents/claude/*.md`; the five installer agents + change here, and the two blind lane roles arrived at `max` with #145), all byte-identical + to agent-lab's `.claude/agents/` copies at agent-lab `b31f640` (the `max` change is agent-lab + #43; `source-scout`'s `maxTurns: 100` is agent-lab #46); - a new committed project settings file for this repository (`.claude/settings.json`) and the coordinator rule in [`AGENTS.md`](../../AGENTS.md); - the portable examples: [`examples/claude-native/CLAUDE.md`](../../examples/claude-native/CLAUDE.md), the agents in `examples/claude-native/agents/`, the saved workflows (`review-changes.js`, - `readiness-audit.js` and the vendored `layer-verdict-lane.js`, all byte-identical to - agent-lab `915e73e`), the `blind-lane-reviewer` agent that lane now names, and their - contract suites; + `readiness-audit.js`; the vendored `layer-verdict-lane.js` and its two blind roles are + vendored at `max` by #145, not by this change), and their contract suites; - the model/effort guidance in [`recipes/claude-native-ultracode.md`](../../recipes/claude-native-ultracode.md) and [`recipes/claude-native-profile.md`](../../recipes/claude-native-profile.md), and the @@ -149,10 +149,12 @@ action or of a cloud session was executed for this record. 1. **The main loop stays at Ultracode everywhere** (`xhigh` plus dynamic workflow orchestration): the user-scope settings template as it is, and a new committed - [`.claude/settings.json`](../../.claude/settings.json) in this repository, byte - for byte the [portable example](../../examples/claude-native/ultracode.settings.json) - and agent-lab's committed project settings (`enableWorkflows`, `ultracode`, the + [`.claude/settings.json`](../../.claude/settings.json) in this repository: the + [portable example](../../examples/claude-native/ultracode.settings.json)'s keys, as in + agent-lab's committed project settings (`enableWorkflows`, `ultracode`, the `unrestricted` size guideline, the concurrency cap of eight and spawn depth one), + merged with this repository's existing secret-read `permissions.deny` rules and + secret-path guard hook, so single-repository cloud sessions on this repository start with it. It adds neither `effortLevel` (a `max` there is dropped: Q1, and the schema and docs for a project file) nor `CLAUDE_CODE_EFFORT_LEVEL`, which is never set at any value: any @@ -236,8 +238,7 @@ Revisit this record when any of these happens: from that tag gets the `max` definitions after the next release and re-pin, or by running `tools/adoption/install_claude_profile.py --only agents` from a default-branch clone; the installer replaces any differing catalog-owned agent file. -- **Vendored pin on an unmerged commit.** agent-lab `915e73e`, the source commit of - the vendored lane, the two review scripts and `blind-lane-reviewer`, is on agent-lab - branch `claude/max-effort-default-20260923` and not yet merged; after its squash - merge, `vendored-lanes.json` should name the merged commit (the bytes, and so the - `SHA256SUMS` entries, stay the same). +- **Vendored lane.** The lane and its two blind roles are vendored at `max` by catalog #145 + (pinned at agent-lab `e070125`), not by this change; the pre-merge `915e73e` pin on an + earlier revision of this branch never reached main. The review and readiness scripts and + the `child-usage.mjs` mirror here are byte-identical to agent-lab `b31f640`. diff --git a/examples/claude-native/workflows/README.md b/examples/claude-native/workflows/README.md index b5f215bd6..98f42b308 100644 --- a/examples/claude-native/workflows/README.md +++ b/examples/claude-native/workflows/README.md @@ -7,7 +7,8 @@ qualification, with one later change: since 2026-09-23 every stage and agent binds effort `max`, models unchanged ([decision](../../../docs/decisions/2026-09-23-max-effort-default.md)). `review-changes.js`, `readiness-audit.js` and `layer-verdict-lane.js` are -byte-identical to agent-lab commit `915e73e`, which made that change there. The +byte-identical to agent-lab `b31f640` (the `max` change is agent-lab #43; the lane is +pinned at agent-lab `e070125` in `vendored-lanes.json`). The qualification ran at the earlier efforts (Sonnet/medium, Opus/high); no native run at `max` is recorded yet. These are local integration assets, not upstream tests. The separate `semantic-evidence-reviewer` example received a later @@ -74,7 +75,7 @@ more definitions ship here. `semantic-evidence-reviewer` (Opus, max; Read, Glob, Grep; the `typesafe-ai` skill) is project-local to agent-lab and published separately. `blind-lane-reviewer` (Opus, max; Read, Glob and Grep; no preloaded skill and no project instructions) is the agent every `layer-verdict-lane` stage -names, vendored byte-identical from agent-lab commit `915e73e` and pinned in +names, vendored byte-identical from agent-lab `e070125` and pinned in `vendored-lanes.json`. The contract suite covers every file in `agents/`. ## Workflows @@ -195,6 +196,7 @@ Effort column since 2026-09-23: every child role runs at `max` with its task-mat | Independent review from source and recorded evidence | `evidence-reviewer` | Opus, max | named Serena, SocratiCode, jCodeMunch and ai-memory read tools plus Context Mode `ctx_execute*`, all deferred. No Bash, Edit, Write or symbol-edit tool; `ctx_execute*` can still run commands in the working tree, so file safety there is an instruction, not a sandbox | | Review of supplied semantic (TypeSafe) judgments against original source | `semantic-evidence-reviewer` | Opus, max | Read, Glob, Grep and the `typesafe-ai` skill preloaded (first prompt 15,059 in the 2026-09-22 probe); no MCP grants, Bash or writes | | Layer-verdict lane stages: propose, refute and re-check one stripped packet from its repository root | `blind-lane-reviewer` | Opus, max | Read, Glob and Grep; no preloaded skill and no project instructions (`omitClaudeMd`); used by `layer-verdict-lane` | +| Layer-verdict adjudication: judge, or refute a judgment on, one anonymous two-return disagreement input | `blind-adjudicator` | Opus, max | Read, Glob and Grep; no preloaded skill and no project instructions (`omitClaudeMd`); used by the catalog's adjudication lane | | Verification that must re-run commands | default workflow subagent | Opus, max | full tools; pays the 42k prompt deliberately | | Web or documentation research | default workflow subagent | Sonnet, max | Context Mode `ctx_fetch_and_index` then `ctx_search` (observed: 1 fetch, 5 searches, 8 requests) | | Cross-family review | `/codex:review` lanes | Codex | see `recipes/claude-codex-cooperation-lanes.md`; usage is not in the Claude journal | diff --git a/examples/claude-native/workflows/SHA256SUMS b/examples/claude-native/workflows/SHA256SUMS index 056c77287..9ab7403d4 100644 --- a/examples/claude-native/workflows/SHA256SUMS +++ b/examples/claude-native/workflows/SHA256SUMS @@ -1,13 +1,13 @@ ca62177687420fe22e03d0510a18e4f32b5d150e0156f6651c48367c3da506e1 check-syntax.mjs -75a6d74b07b62d8bb1be0ed37bcffece18781c9c1bdff4a94c23cd5ce9dbe60b child-usage.mjs +cb3bbe11127bc7856c751e555739aae2becc1464b3eb71e49c46ebfc6ffb1da0 child-usage.mjs cd303779df6521c804070a319aa04c86ed2d6fe1c099c537836f037dec5f4e8b codex-cross-review.mjs 2e73ed523e115e6a1f55a4cc992bf052e157d9cdd85aeb09cf4ef93d3f996771 contract.config.json fd77b74945e9d2bd0b26b7822aabcde00e7e650c327da16d852fdbfd5984c0d6 layer-verdict-lane.js -d0d5afb3f4f8d635d0a75e48b4c4cda4a074e842130d9c80eb6633a5bfe1701a readiness-audit.js -581300deaa95c6a2e097ec824cc01aca21ff17f87cb497327e4ddbc1cc37d65f review-changes.js -fd23f5c7d1118e4b67bd00fc6212f0db4b0f2cbac1f2aaa0c76ee5f07e780c56 test-child-usage.mjs +1d3d816cc219a252760a471a3b68a979057e2c417c429d6c094df4c474e612bb readiness-audit.js +bb298f9ab63a6d43173d2c4f87ab31d00dd79184e961dc84d69cbef3966bce0b review-changes.js +33c8bfb56a6765eeac8a62c40b5e89011447a74d597b29d5ad9330daf11661a0 test-child-usage.mjs 2617c081f606268ed20a6e00f6472495fad950db433ca3afbb44ea970aaac446 test-codex-envelope.mjs -f418a9da354cb766cbab4f779be754359797b110ac4f3c28cdcda5e319d3c0a1 test-contract-mutations.mjs -bb9314dc3c24de168149ba795351821957ec03e7afc05574708edb492adc7c29 test-envelope.mjs +2c2626d76e2113e0b86b372864001e45766d118b12fc0bf9441d937fa555069a test-contract-mutations.mjs +c2b4b9ec376c156e37186c5de1b586e5a310a94b8dde62282cb164867abdc124 test-envelope.mjs 2ed3323db76d0104327a0c0bb35f00cf6a05018bdd9f49982b0c279cf2540871 test-usage-receipts.mjs 452712bf2eeee0ad158a061cc6a9291d305aaf9e01d9564aca7b428e1d4ee83e vendored-lanes.json diff --git a/examples/claude-native/workflows/child-usage.mjs b/examples/claude-native/workflows/child-usage.mjs index b7a86b13f..f38182e02 100644 --- a/examples/claude-native/workflows/child-usage.mjs +++ b/examples/claude-native/workflows/child-usage.mjs @@ -4,11 +4,18 @@ // agent-.meta.json, agent-.jsonl) and changes nothing. // node .claude/workflows/child-usage.mjs (the "Transcript dir" the Workflow tool prints) // node .claude/workflows/child-usage.mjs --latest (newest run recorded for this working directory) +// add --require-effort max to also fail (exit 1) when any child ran at another effort +// (docs/tasks/2026-09-23-max-effort-default.md: a stage without effort inherits the +// coordinator's xhigh, and CLAUDE_CODE_EFFORT_LEVEL overrides every stage). // Streamed assistant lines repeat a message id, so usage is counted once per id // (largest output_tokens wins). Counters are provider-returned and kept per type; // they are not comparable to RTK/Context Mode/jCodeMunch/Headroom estimates. -// Exit 1 when any child is incomplete (null result, no transcript, no usage, or a -// resolved model outside the requested family). +// Exit 1 when any child is incomplete (null result, no transcript, no usage, a resolved +// model outside the requested family, a resolved model that changes within the child, +// or a model older than the requested alias's documented resolution for the client +// version that wrote the entry). The last two catch content-classifier fallback, which +// re-runs a flagged request on an older model and continues the child there; the +// family substring check alone accepts claude-opus-4-8 for a requested opus. import { readFileSync, existsSync, readdirSync, statSync } from 'node:fs' import { join, resolve } from 'node:path' import { homedir } from 'node:os' @@ -16,6 +23,34 @@ import { fileURLToPath } from 'node:url' const COUNTERS = ['input_tokens', 'output_tokens', 'cache_read_input_tokens', 'cache_creation_input_tokens'] const lines = (file) => readFileSync(file, 'utf8').split('\n').filter((l) => l.trim()).map((l) => { try { return JSON.parse(l) } catch { return null } }).filter(Boolean) +// Client-written rows (an API error such as a usage-limit notice) carry this model; the +// family check still reports them, and the two fallback checks below ignore them. +const SYNTHETIC = '' +// Documented resolution of an alias on the Anthropic API by the client version that wrote the +// transcript entry, as [first client version, model] rows in ascending order (model-config doc, +// fetched 2026-09-24: opus is Opus 5.5 from v2.1.280, Opus 5 from v2.1.219, Opus 4.8 from v2.1.154). +// An entry older than the first row, and an alias without rows (sonnet, haiku, fable), has no +// expectation. An ANTHROPIC_DEFAULT_OPUS_MODEL pin to an older model would be flagged; none is set here. +export const ALIAS_RESOLUTION = { opus: [['2.1.154', 'claude-opus-4-8'], ['2.1.219', 'claude-opus-5'], ['2.1.280', 'claude-opus-5-5']] } +const semver = (v) => { const m = /^(\d+)\.(\d+)\.(\d+)/.exec(typeof v === 'string' ? v : ''); return m ? m.slice(1).map(Number) : null } +const compareParts = (a, b) => { for (let i = 0; i < Math.max(a.length, b.length); i++) { const d = (a[i] || 0) - (b[i] || 0); if (d) return d < 0 ? -1 : 1 } return 0 } +// claude-opus-5-5 -> { family: 'opus', version: [5, 5] }; a date suffix and a [1m] suffix are ignored; other shapes -> null. +export function modelGeneration(model) { + const m = /^claude-([a-z]+)-(\d{1,2})(?:-(\d{1,2}))?(?:-\d{8})?(?:\[[^\]]*\])?$/.exec(String(model)) + return m ? { family: m[1], version: [Number(m[2]), ...(m[3] ? [Number(m[3])] : [])] } : null +} +export function expectedModel(alias, clientVersion) { + const rows = ALIAS_RESOLUTION[String(alias).toLowerCase()], v = semver(clientVersion) + if (!rows || !v) return null + let hit = null + for (const [from, model] of rows) if (compareParts(v, semver(from)) >= 0) hit = model + return hit +} +// True when `model` cannot be shown to be at least as new as `expected` (a different family or an unparseable name fails closed). +export function olderThan(model, expected) { + const got = modelGeneration(model), want = modelGeneration(expected) + return !got || !want || got.family !== want.family || compareParts(got.version, want.version) < 0 +} export function summarizeChild(started, result, meta, transcript) { const byId = new Map() @@ -48,6 +83,20 @@ export function summarizeChild(started, result, meta, transcript) { if (unresolved) issues.push(unresolved + ' assistant message(s) without a resolved model') if (!requested) issues.push('model not requested explicitly (inherits the coordinator model)') else if (!resolved.length || resolved.some((m) => !m.toLowerCase().includes(requested.toLowerCase()))) issues.push('resolved model outside requested family: ' + (resolved.join(',') || '(none resolved)')) + // Classifier fallback (model-config doc, "Automatic model fallback"): after a flagged request the + // child continues on the fallback model, so a change of resolved model within the child, or a model + // older than the alias's documented resolution for the entry's client version, is a substitution. + const real = messages.filter((r) => r.message.model && r.message.model !== SYNTHETIC) + const runs = real.map((r) => r.message.model).filter((m, i, all) => i === 0 || m !== all[i - 1]) + if (runs.length > 1) issues.push('resolved model changed within the child: ' + runs.join(' -> ')) + if (requested) { + const older = new Set() + for (const r of real) { + const want = expectedModel(requested, r.version) + if (want && r.message.model.toLowerCase().includes(requested.toLowerCase()) && olderThan(r.message.model, want)) older.add(r.message.model + ' on ' + r.version + ' (documented ' + requested + ': ' + want + ')') + } + if (older.size) issues.push('resolved model older than the documented alias resolution: ' + [...older].join(', ')) + } return { agent_id: started.agentId, label: started.label ?? null, phase: started.phase ?? null, agent_type: meta ? meta.agentType ?? null : null, @@ -84,12 +133,17 @@ export function summarizeRun(dir) { const incomplete = children.filter((c) => !c.complete) return { status: !children.length ? 'incomplete' : incomplete.length ? 'incomplete' : 'complete', - reason: !children.length ? 'journal has no started children' : incomplete.length ? incomplete.length + ' child(ren) incomplete' : 'every child has an explicit requested model, a matching resolved model, returned usage and a non-null result', + reason: !children.length ? 'journal has no started children' : incomplete.length ? incomplete.length + ' child(ren) incomplete' : 'every child has an explicit requested model, one matching resolved model no older than its documented alias resolution, returned usage and a non-null result', multi_model_children: children.filter((c) => c.resolved_models.length > 1).map((c) => c.label || c.agent_id), children, by_resolved_model: byModel, } } +// Children whose resolved efforts are not exactly [required] (none recorded counts as a mismatch). +export function effortMismatches(run, required) { + return run.children.filter((c) => c.efforts.length !== 1 || c.efforts[0] !== required).map((c) => ({ child: c.label || c.agent_id, efforts: c.efforts })) +} + // Newest native Workflow transcript directory for a working directory. The projects // layout (/projects///subagents/workflows/wf_*) // is observed native behavior, not a documented interface: return null rather than guess. @@ -111,10 +165,17 @@ export function latestRunDir(cwd, configDir) { } if (process.argv[1] && fileURLToPath(import.meta.url) === resolve(process.argv[1])) { - const dir = process.argv[2] === '--latest' ? latestRunDir(process.cwd(), process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude')) : process.argv[2] - if (process.argv[2] === '--latest' && !dir) { console.error('no native Workflow run found for ' + process.cwd()); process.exit(2) } - if (!dir) { console.error('usage: child-usage.mjs '); process.exit(2) } + const argv = process.argv.slice(2) + const ri = argv.indexOf('--require-effort') + const required = ri >= 0 ? argv[ri + 1] : null + if (ri >= 0 && !['low', 'medium', 'high', 'xhigh', 'max'].includes(required)) { console.error('--require-effort needs one of low, medium, high, xhigh, max'); process.exit(2) } + if (argv.filter((a) => a === '--require-effort').length > 1) { console.error('--require-effort may be given once'); process.exit(2) } + const target = argv.filter((_, i) => i !== ri && i !== ri + 1 || ri < 0)[0] + const dir = target === '--latest' ? latestRunDir(process.cwd(), process.env.CLAUDE_CONFIG_DIR || join(homedir(), '.claude')) : target + if (target === '--latest' && !dir) { console.error('no native Workflow run found for ' + process.cwd()); process.exit(2) } + if (!dir) { console.error('usage: child-usage.mjs | --latest [--require-effort ]'); process.exit(2) } const out = { transcript_dir: dir, ...summarizeRun(dir) } + if (required) out.effort_mismatches = effortMismatches(out, required) console.log(JSON.stringify(out, null, 2)) - process.exit(out.status === 'complete' ? 0 : 1) + process.exit(out.status === 'complete' && !(required && out.effort_mismatches.length) ? 0 : 1) } diff --git a/examples/claude-native/workflows/test-child-usage.mjs b/examples/claude-native/workflows/test-child-usage.mjs index cb2045b59..9009b12b3 100644 --- a/examples/claude-native/workflows/test-child-usage.mjs +++ b/examples/claude-native/workflows/test-child-usage.mjs @@ -2,8 +2,10 @@ // OFFLINE check of child-usage.mjs against SYNTHETIC transcript rows (no provider // call, not native evidence). Real runs are checked by passing their directory; // stored receipts from real runs are bound to the documentation by test-usage-receipts.mjs. -import { summarizeChild, summarizeRun, latestRunDir } from './child-usage.mjs' +import { summarizeChild, summarizeRun, latestRunDir, effortMismatches, modelGeneration, expectedModel } from './child-usage.mjs' import { mkdtempSync, writeFileSync, rmSync, mkdirSync, utimesSync } from 'node:fs' +import { spawnSync } from 'node:child_process' +import { fileURLToPath } from 'node:url' import { tmpdir } from 'node:os' import { join } from 'node:path' let passed = 0, failed = 0 @@ -29,6 +31,27 @@ const noModel = summarizeChild(started, done, { model: 'sonnet' }, [{ ...msg('m1 expect('usage without a resolved model is incomplete, not vacuously matching', !noModel.complete && noModel.issues.some((i) => i.includes('without a resolved model'))) expect('no assistant usage is incomplete', !summarizeChild(started, done, { model: 'opus' }, [{ type: 'user' }]).complete) +// Classifier fallback (model-config doc, "Automatic model fallback"): a flagged request re-runs on an +// older model and the child continues there. Rows carry the client version that wrote them. +const vmsg = (id, model, version) => ({ ...msg(id, model, 5, 0, 0, 'max'), version }) +const synthetic = (id, version) => ({ type: 'assistant', version, isApiErrorMessage: true, effort: null, message: { id, model: '', usage: { input_tokens: 0, output_tokens: 0, cache_read_input_tokens: 0, cache_creation_input_tokens: 0 } } }) +const has = (child, text) => child.issues.some((i) => i.includes(text)) +const switched = summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-opus-5-5', '2.1.281'), vmsg('m2', 'claude-opus-4-8', '2.1.281'), vmsg('m3', 'claude-opus-4-8', '2.1.281')]) +expect('fallback: a resolved model that changes within the child is incomplete and names the change once', !switched.complete && switched.issues.some((i) => i.endsWith('changed within the child: claude-opus-5-5 -> claude-opus-4-8'))) +expect('fallback: the substring family check alone would have accepted claude-opus-4-8 for opus', !has(switched, 'outside requested family')) +const wholeChild = summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-opus-5', '2.1.281'), vmsg('m2', 'claude-opus-5', '2.1.281')]) +expect('fallback: a child that ran entirely on an older model than its version documents is incomplete', !wholeChild.complete && !has(wholeChild, 'changed within') && has(wholeChild, 'claude-opus-5 on 2.1.281 (documented opus: claude-opus-5-5)')) +expect('fallback: opus on claude-opus-5 under 2.1.278 is the documented resolution, not a fallback', summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-opus-5', '2.1.278'), vmsg('m2', 'claude-opus-5', '2.1.278')]).complete) +expect('fallback: opus on claude-opus-5-5 under 2.1.280 and 2.1.281 is complete', summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-opus-5-5', '2.1.280'), vmsg('m2', 'claude-opus-5-5', '2.1.281')]).complete) +expect('fallback: a cybersecurity fallback to claude-opus-4-8 under 2.1.278 is older than claude-opus-5', has(summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-opus-4-8', '2.1.278')]), 'older than the documented alias resolution')) +const withError = summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-opus-5-5', '2.1.281'), synthetic('s1', '2.1.281'), vmsg('m2', 'claude-opus-5-5', '2.1.281')]) +expect('fallback: a API-error row is not a model change or an older model; the family check still reports it', !has(withError, 'changed within') && !has(withError, 'older than') && has(withError, 'outside requested family: claude-opus-5-5,')) +expect('fallback: an entry older than the first documented row has no expectation', summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-opus-4-8', '2.1.100')]).complete) +expect('fallback: an alias without documented rows (sonnet) is checked for changes only', summarizeChild(started, done, { model: 'sonnet' }, [vmsg('m1', 'claude-sonnet-5', '2.1.281')]).complete && !summarizeChild(started, done, { model: 'sonnet' }, [vmsg('m1', 'claude-sonnet-5', '2.1.281'), vmsg('m2', 'claude-haiku-4-5-20251001', '2.1.281')]).complete) +expect('fallback: an opus model name that cannot be compared fails closed', has(summarizeChild(started, done, { model: 'opus' }, [vmsg('m1', 'claude-3-opus-20240229', '2.1.281')]), 'older than the documented alias resolution')) +expect('fallback: model names parse to family and version, ignoring date and [1m] suffixes', JSON.stringify([modelGeneration('claude-opus-5-5'), modelGeneration('claude-opus-5'), modelGeneration('claude-haiku-4-5-20251001'), modelGeneration('claude-opus-5-5[1m]'), modelGeneration('claude-3-opus-20240229')]) === JSON.stringify([{ family: 'opus', version: [5, 5] }, { family: 'opus', version: [5] }, { family: 'haiku', version: [4, 5] }, { family: 'opus', version: [5, 5] }, null])) +expect('fallback: the opus version table matches the documented boundaries', JSON.stringify(['2.1.153', '2.1.154', '2.1.218', '2.1.219', '2.1.279', '2.1.280', '2.1.281'].map((v) => expectedModel('opus', v))) === JSON.stringify([null, 'claude-opus-4-8', 'claude-opus-4-8', 'claude-opus-5', 'claude-opus-5', 'claude-opus-5-5', 'claude-opus-5-5']) && expectedModel('sonnet', '2.1.281') === null && expectedModel('opus', undefined) === null) + const dir = mkdtempSync(join(tmpdir(), 'child-usage-')) try { expect('missing journal fails closed', summarizeRun(dir).status === 'incomplete') @@ -37,7 +60,25 @@ try { writeFileSync(join(dir, 'agent-a1.jsonl'), JSON.stringify(msg('m1', 'claude-sonnet-5', 9, 100)) + '\n') const run = summarizeRun(dir) expect('run keeps every started child and is incomplete when one has no result, meta or transcript', run.status === 'incomplete' && run.children.length === 2 && run.children[0].complete && !run.children[1].complete) + const efforts = { children: [{ label: 'max', efforts: ['max'] }, { label: 'inherit', efforts: ['xhigh'] }, { label: 'mixed', efforts: ['max', 'high'] }, { label: 'none', efforts: [] }] } + expect('require-effort flags every child not exactly at the level, including none recorded', JSON.stringify(effortMismatches(efforts, 'max').map((m) => m.child)) === '["inherit","mixed","none"]') + expect('require-effort passes a run whose children all ran at the level', effortMismatches({ children: [{ label: 'a', efforts: ['max'] }] }, 'max').length === 0) expect('per-model totals come from returned usage only', run.by_resolved_model['claude-sonnet-5'].output_tokens === 9 && run.by_resolved_model['claude-sonnet-5'].children === 1) + // CLI: --require-effort on a complete one-child run (either argument order), and bad levels exit 2 + const cli = (...a) => spawnSync(process.execPath, [fileURLToPath(new URL('./child-usage.mjs', import.meta.url)), ...a], { encoding: 'utf8' }).status + const one = join(dir, 'one'); mkdirSync(one) + writeFileSync(join(one, 'journal.jsonl'), [started, done].map((e) => JSON.stringify(e)).join('\n') + '\n') + writeFileSync(join(one, 'agent-a1.meta.json'), JSON.stringify({ model: 'sonnet', agentType: 'workflow' })) + writeFileSync(join(one, 'agent-a1.jsonl'), JSON.stringify(msg('m1', 'claude-sonnet-5', 9, 100, 0, 'max')) + '\n') + expect('cli: a max-only run passes --require-effort max in either argument order', cli(one, '--require-effort', 'max') === 0 && cli('--require-effort', 'max', one) === 0) + expect('cli: the same run fails --require-effort high', cli(one, '--require-effort', 'high') === 1) + expect('cli: a missing, flag-like, unknown or repeated level exits 2', cli(one, '--require-effort') === 2 && cli('--require-effort', '--latest', one) === 2 && cli(one, '--require-effort', 'MAX') === 2 && cli('--require-effort', 'max', '--require-effort', 'max', one) === 2) + // A run whose one opus child fell back mid-child exits 1 even when every row is at the required effort. + const fell = join(dir, 'fell'); mkdirSync(fell) + writeFileSync(join(fell, 'journal.jsonl'), [started, done].map((e) => JSON.stringify(e)).join('\n') + '\n') + writeFileSync(join(fell, 'agent-a1.meta.json'), JSON.stringify({ model: 'opus', agentType: 'evidence-reviewer' })) + writeFileSync(join(fell, 'agent-a1.jsonl'), [vmsg('m1', 'claude-opus-5-5', '2.1.281'), vmsg('m2', 'claude-opus-4-8', '2.1.281')].map((e) => JSON.stringify(e)).join('\n') + '\n') + expect('cli: a run with a fallback child exits 1 and reports the run incomplete', cli(fell, '--require-effort', 'max') === 1 && summarizeRun(fell).status === 'incomplete') // --latest: newest journal under /projects///subagents/workflows/ const cfg = join(dir, 'cfg'), cwd = '/work/agent-lab.x' expect('latest: unknown working directory returns null', latestRunDir(cwd, cfg) === null) diff --git a/manifests/evidence.json b/manifests/evidence.json index f36252275..40b77cc3a 100644 --- a/manifests/evidence.json +++ b/manifests/evidence.json @@ -918,7 +918,7 @@ "prometheus", "loki" ], - "claim": "After native sign-in restored readiness, the freshly reproduced SDK completed a real LEAN backtest→Dagu→DuckDB→GPT-6 Astra→Claude Opus 5 prepared-packet research workflow; both native reports passed structural citation/value/unit/scope checks and the accepted Astra report was bound into Claude's prompt by verified hashes.", + "claim": "After native sign-in restored readiness, the freshly reproduced SDK completed a real LEAN backtest\u2192Dagu\u2192DuckDB\u2192GPT-6 Astra\u2192Claude Opus 5 prepared-packet research workflow; both native reports passed structural citation/value/unit/scope checks and the accepted Astra report was bound into Claude's prompt by verified hashes.", "limitations": [ "This validates a bounded historical simulation research workflow on the authoring host, not a profitable strategy, current market source, broker execution or unattended production service.", "The two reported model tasks used no model tools. Their native hooks and telemetry remain separate evidence; this does not repeat every MCP, memory or GPU-RAG behavior.", @@ -1900,7 +1900,7 @@ "component_ids": [ "ai-memory" ], - "claim": "Mac arm64 synthetic2.3.1→2.3.2 migration, scoped exact reads/FTS, reserved and Unicode paths, restart and no-capture checks passed.", + "claim": "Mac arm64 synthetic2.3.1\u21922.3.2 migration, scoped exact reads/FTS, reserved and Unicode paths, restart and no-capture checks passed.", "limitations": [ "Separate synthetic store and explicit no-model policy only; no production migration claimed by this receipt.", "OS filesystem/network confinement was not established; failed restricted-sandbox attempt retained. Native execution succeeded without security overrides; Gatekeeper assessment rejected the ad-hoc distribution.", @@ -3040,8 +3040,8 @@ "files": [ { "path": ".claude/settings.json", - "sha256": "7b586a519802aa839d6405825d6965855fcb9f225d6d0e32c5ee874304bde665", - "bytes": 1134 + "sha256": "604543fb3a2a54f4b2d87789ca0f28adc748bdcca387f05bc2928bcd577bbe90", + "bytes": 1344 }, { "path": ".gitattributes", @@ -3140,8 +3140,8 @@ }, { "path": ".github/workflows/validate.yml", - "sha256": "2752d215e78278e4937107bf98fde5f4ad95c31a5447104417ec0cad5a176e7e", - "bytes": 17272 + "sha256": "d7c8c95c9725859263c2b2677484b33e00a97ffacaac4087869d2efc3be528da", + "bytes": 17464 }, { "path": ".gitignore", @@ -3160,8 +3160,8 @@ }, { "path": "AGENTS.md", - "sha256": "06a1eaf808a783cd3821e566ef1837503cc13a1f0ba5f516bc59538c79fdcaac", - "bytes": 10279 + "sha256": "9b56cee5d4b2a98a2bd6b2a84fdbd272b035f006ee5ef5b92b355e1a6d0f5305", + "bytes": 10844 }, { "path": "CLAUDE.md", @@ -3200,8 +3200,8 @@ }, { "path": "adoption/bootstrap.md", - "sha256": "6a8bc180930f45402f0e7502e42cba656250c7fa70b0d8b3d3e554faa8981467", - "bytes": 21263 + "sha256": "b6e772ba2f5f52601e78a25f04d90266e5faa2e99b3f8ccb9c44f58147716a46", + "bytes": 21005 }, { "path": "adoption/credential-inventory.json", @@ -3300,13 +3300,13 @@ }, { "path": "adoption/platforms/linux-wsl2.md", - "sha256": "cd517021c129ee2ef93ef8f44e2ffba693dccfbb0226ae7223fe15dab747aab8", - "bytes": 6401 + "sha256": "e6a509cf6453014aa9b296f55f8472437d02fc9259144206ea166396d0373a87", + "bytes": 6008 }, { "path": "adoption/platforms/macos-arm64.md", - "sha256": "146e5065d42199409327668fefd6f88c890ffe8060adbe9b7c21b9105bcedc16", - "bytes": 40489 + "sha256": "2d6cf4b8c002c427040efb69a15251737c3da896e42d4efcd0a1244cdcc2423f", + "bytes": 38219 }, { "path": "adoption/pre-login-observation.json", @@ -5425,7 +5425,7 @@ "bytes": 126 }, { - "path": "blueprints/convergence-practice/wsl-retrieval/seed/corpus/primary/notes/aéé.md", + "path": "blueprints/convergence-practice/wsl-retrieval/seed/corpus/primary/notes/a\u00e9\u00e9.md", "sha256": "cb36cd06858449475efbfaddaedfd78d20a2b7b4f30993bca02cd1cb360e0cb3", "bytes": 111 }, @@ -9256,8 +9256,8 @@ }, { "path": "catalogs/foundation/decisions.json", - "sha256": "267bfb42c795b26b35907d48c283e55ed3f7b3ea108ed8b10ae9551569e0c2d2", - "bytes": 139699 + "sha256": "0daad888d852dc86b479e8253dcd7bcde6bad937542607860eddc7420be08661", + "bytes": 139755 }, { "path": "catalogs/foundation/manifest.json", @@ -28191,12 +28191,12 @@ }, { "path": "examples/claude-native/workflows/README.md", - "sha256": "fd2bb74cb0f15f52ef96f241c056b5be010cb66b6a35e53cea7e894c9a8b3052", - "bytes": 17670 + "sha256": "c11dc279e70da3256f0eb20d38ff1a172e252e1547791cd63ab4280c82d8b9f2", + "bytes": 20733 }, { "path": "examples/claude-native/workflows/SHA256SUMS", - "sha256": "c659b36ea75690467e353d73f9b9f44caaa8c9b8832bbfc5b09d0d0e6eb18d7d", + "sha256": "d78e30e9a3f9b99d6053f1647f8715da8c0e3c39ecff12a3bebce6d616ccc880", "bytes": 1129 }, { @@ -28206,8 +28206,8 @@ }, { "path": "examples/claude-native/workflows/child-usage.mjs", - "sha256": "75a6d74b07b62d8bb1be0ed37bcffece18781c9c1bdff4a94c23cd5ce9dbe60b", - "bytes": 7780 + "sha256": "cb3bbe11127bc7856c751e555739aae2becc1464b3eb71e49c46ebfc6ffb1da0", + "bytes": 12642 }, { "path": "examples/claude-native/workflows/codex-cross-review.mjs", @@ -28231,8 +28231,8 @@ }, { "path": "examples/claude-native/workflows/test-child-usage.mjs", - "sha256": "fd23f5c7d1118e4b67bd00fc6212f0db4b0f2cbac1f2aaa0c76ee5f07e780c56", - "bytes": 5612 + "sha256": "33c8bfb56a6765eeac8a62c40b5e89011447a74d597b29d5ad9330daf11661a0", + "bytes": 12656 }, { "path": "examples/claude-native/workflows/test-codex-envelope.mjs", @@ -28246,8 +28246,8 @@ }, { "path": "examples/claude-native/workflows/test-envelope.mjs", - "sha256": "bb9314dc3c24de168149ba795351821957ec03e7afc05574708edb492adc7c29", - "bytes": 44501 + "sha256": "c2b4b9ec376c156e37186c5de1b586e5a310a94b8dde62282cb164867abdc124", + "bytes": 49552 }, { "path": "examples/claude-native/workflows/test-usage-receipts.mjs", @@ -28636,13 +28636,13 @@ }, { "path": "recipes/claude-native-profile.md", - "sha256": "8edc71e983a2d42ee50cc717530b5750bc5b80b2b621c9cd8feb1c31c59795f3", - "bytes": 16871 + "sha256": "54619b59d9ae542ef05830728417038cd0c610ed0f921cdb105ea2e677c25286", + "bytes": 18584 }, { "path": "recipes/claude-native-ultracode.md", - "sha256": "338d149fc422372bf8f9784b3f02cc829e60b5cb303eee003a82ef9f7f7c1458", - "bytes": 24666 + "sha256": "c4268f3abaabf8747af5c144285e7572640b9ee6e8007c33ac7c4c8a1d734e8b", + "bytes": 24853 }, { "path": "recipes/native-upgrades-20260921.md", @@ -29106,8 +29106,8 @@ }, { "path": "tests/test_install_claude_profile.py", - "sha256": "e57a159819925000baddff8f7ee00f315382287cb3402f8910415a5f2ebb01ad", - "bytes": 17923 + "sha256": "758c725d424541a466a84342b204a180f97f9c110520a352495ec4b4f9924746", + "bytes": 19610 }, { "path": "tests/test_job_recovery.py", diff --git a/recipes/claude-native-ultracode.md b/recipes/claude-native-ultracode.md index 96fffdda7..28f19face 100644 --- a/recipes/claude-native-ultracode.md +++ b/recipes/claude-native-ultracode.md @@ -130,7 +130,7 @@ the measured constraints and the per-stage rule. | Implementation from a clear contract | `isolated-builder` (Sonnet, max, own worktree, named MCP tools behind ToolSearch) | One real task: a manifest probe implemented, checked and committed from its own worktree (first prompt 17,864) | | Independent review from source and recorded evidence | `evidence-reviewer` (Opus, max; read-only named MCP tools behind ToolSearch, no Bash/Edit/Write) | Eight native review runs; first prompt 12,164 for the deferred shape versus 42,220 with bare server grants | | Review of supplied semantic (TypeSafe) judgments against original source | `semantic-evidence-reviewer` (Opus, max; Read, Glob and Grep, `typesafe-ai` skill preloaded) | One probe, `wf_20a5e69a-84d`, measured the skill preload (first prompt 15,059 tokens; [convergence record](../docs/harness-rules-convergence-20260922.md)); no quality comparison with another reviewer is recorded. The vendored layer-verdict lane no longer uses it (next row) | -| Proposing, refuting and re-checking one stripped layer-verdict packet | `blind-lane-reviewer` (Opus, max; Read, Glob and Grep, no preloaded skill, no project instructions) | Every stage of the vendored [layer-verdict lane](../examples/claude-native/workflows/layer-verdict-lane.js) names it (agent-lab `915e73e`, vendored with the lane). A skill it preloaded could be one of the candidates a packet judges, and project instructions can name incumbent selections, so it carries neither; this catalog records no dated run of the lane with it | +| Proposing, refuting and re-checking one stripped layer-verdict packet | `blind-lane-reviewer` (Opus, max; Read, Glob and Grep, no preloaded skill, no project instructions) | Every stage of the vendored [layer-verdict lane](../examples/claude-native/workflows/layer-verdict-lane.js) names it (agent-lab `e070125`, vendored with the lane). A skill it preloaded could be one of the candidates a packet judges, and project instructions can name incumbent selections, so it carries neither; this catalog records no dated run of the lane with it | | Judging or refuting one sealed comparison packet | `blind-judge` (Opus, max; Read only, no project instructions) | Its frontmatter was checked against the agent contract in the [convergence record](../docs/harness-rules-convergence-20260922.md); its body was not reviewed there, and this catalog records no dated run of the role | | Cheap exact extraction | Haiku | Not routed: on one byte-identical packet the Opus verifier scored Sonnet 14/14 lane rows and Haiku 9/14 with a quote attributed to a file that does not contain it; overturn only after a repeat trial with no unanchored citation on two distinct packets. Haiku 4.5 takes no effort level, so `max` does not apply to it | | Independent cross-family review | Existing official Codex companion | Reuse its separately recorded native acceptance; this trial did not run Codex inside a Workflow graph. Its reasoning effort follows the Codex configuration, not this table | @@ -166,7 +166,7 @@ second worker and compares both runs in code. Record each run's children with `node .claude/workflows/child-usage.mjs --latest`. Since 2026-09-23 the examples' agents and stages bind effort `max` with their task-matched models unchanged (the review, readiness and layer-verdict scripts -are byte-identical to agent-lab commit `915e73e`), as do the shipped +are byte-identical to agent-lab `b31f640`; the max change is agent-lab #43), as do the shipped [`adoption/agents/claude/`](../adoption/agents/claude/) definitions. Their contract suite now fails a stage or agent at any other effort, a `MODEL` record that differs from the stages it describes, a routing table that restates an @@ -288,8 +288,7 @@ rule, auto or bypass permission mode, or a hook that allows the call ([workflows](https://code.claude.com/docs/en/workflows#approve-the-plan-before-it-runs), fetched 2026-09-23). Such a job fails at a usage limit instead of waiting. This repository's committed [`.claude/settings.json`](../.claude/settings.json) is -byte-identical to the linked settings example file (the inline JSON above -omits its `$schema` line). Per the official +the settings example's keys (`enableWorkflows`, `ultracode`, `workflowSizeGuideline` and the two env values) merged with this repository's secret-read `permissions.deny` rules and secret-path guard hook. Per the official [cloud-session settings](https://code.claude.com/docs/en/settings#settings-in-cloud-sessions) docs, a cloud session on this one repository reads it, while a session with several repositories reads only its `enabledPlugins` and diff --git a/tests/test_verdict_lane_vendoring.py b/tests/test_verdict_lane_vendoring.py index f48469990..a2e2441af 100644 --- a/tests/test_verdict_lane_vendoring.py +++ b/tests/test_verdict_lane_vendoring.py @@ -9,7 +9,6 @@ import hashlib import importlib.util import json -import re import unittest from pathlib import Path @@ -42,23 +41,6 @@ def test_the_lane_prompt_it_fills_is_the_catalog_copy(self): # lane-prompt.md was already in the catalog; codex_lane.py and the packets read it from there. self.assertTrue((ROOT / "tools" / "sota-convergence" / "lane-prompt.md").is_file()) - def test_every_agent_the_vendored_lane_names_is_vendored_and_pinned_beside_it(self): - # Since agent-lab 915e73e every lane stage names agentType 'blind-lane-reviewer'; a host holding - # only this catalog needs that definition too, byte for byte the agent-lab file the pin names. - pin = json.loads((WORKFLOWS / "vendored-lanes.json").read_text(encoding="utf-8")) - entry = next(item for item in pin["files"] if item["path"] == "layer-verdict-lane.js") - lane = (WORKFLOWS / "layer-verdict-lane.js").read_text(encoding="utf-8") - named = set(re.findall(r"agentType: '([^']+)'", lane)) - required = {Path(item["path"]).stem: item for item in entry.get("required_agents", [])} - self.assertTrue(named) - self.assertEqual(set(required), named) - for name, item in required.items(): - path = (WORKFLOWS / item["path"]).resolve() - self.assertEqual(path, (ROOT / "examples" / "claude-native" / "agents" / f"{name}.md").resolve()) - self.assertEqual(hashlib.sha256(path.read_bytes()).hexdigest(), item["sha256"]) - self.assertEqual(item["source_path"], f".claude/agents/{name}.md") - self.assertRegex(item["agentlab_commit"], r"^[a-f0-9]{40}$") - TOOLS = ROOT / "tools" / "sota-convergence" From 152cfa42dea9c4ee2b10da15c9ed9bbf20de68bf Mon Sep 17 00:00:00 2001 From: seathatflowsinourveins <234074349+seathatflowsinourveins@users.noreply.github.com> Date: Thu, 24 Sep 2026 12:37:36 -0400 Subject: [PATCH 6/6] Resolve the rebase review: current pin in the record, finish the macOS #94 cleanup, sync example agents - The decision record names v2026.09.24.1 and the settings file main added (#182). - The macOS step 4 re-pin note and next-host-stages' 'as the macOS page marks' go, since #94 is in the pinned tag. - The four example agents are byte-identical to agent-lab's again (source-scout maxTurns 100 and the foreground-acceptance sentence), as the recipe states. - The install-input docstring and comment name the secret-path guard's SHA256SUMS coverage. Co-Authored-By: Claude Opus 5.5 (1M context) --- adoption/platforms/macos-arm64.md | 10 +------- .../2026-09-23-max-effort-default.md | 10 ++++---- docs/next-host-stages.md | 11 ++++----- .../claude-native/agents/evidence-reviewer.md | 4 ++-- .../claude-native/agents/isolated-builder.md | 2 +- .../agents/semantic-evidence-reviewer.md | 7 +++--- examples/claude-native/agents/source-scout.md | 4 ++-- manifests/evidence.json | 24 +++++++++---------- tests/test_adoption_docs_consistency.py | 9 ++++--- 9 files changed, 37 insertions(+), 44 deletions(-) diff --git a/adoption/platforms/macos-arm64.md b/adoption/platforms/macos-arm64.md index 60b7aeaaf..1a5f055c2 100644 --- a/adoption/platforms/macos-arm64.md +++ b/adoption/platforms/macos-arm64.md @@ -59,15 +59,7 @@ release, the note is history and the step is in your checkout (`test -e steps 3–4 (Codex, Claude and GitHub device flows; `tools/adoption/render_config.py` with this host's own `adoption/hosts/.json`). 4. launchd services and the embedding acceptance ("launchd services" and - "Embedding backend decision" below). `adoption/launchd/launchd-agents.sh` - and `tools/adoption/embed_acceptance.py` were added after `v2026.09.23`, and - so were their inputs: at that tag `bootstrap-macos.sh` neither downloads the - embedding model (`models[0]` in main's darwin pins) into `state/models` nor - writes `config/qdrant.yaml`, which the qdrant and llama-embed agents need. - **Until main is re-pinned to a release that contains the macOS clean install - (#94), run this page from step 2 onward from a default-branch clone** and - record the receipts with that clone's `catalog_revision` (main-only - evidence, labelled as such). + "Embedding backend decision" below). 5. `uv run --no-project --python 3.13 python scripts/adoption_status.py --profile macos-arm64-foundation --json` (changed after `v2026.09.23.1`, which runs plain `python3`; the manifest supports Python 3.13 only, so run this form there too), diff --git a/docs/decisions/2026-09-23-max-effort-default.md b/docs/decisions/2026-09-23-max-effort-default.md index 7d99ff810..8d95ba418 100644 --- a/docs/decisions/2026-09-23-max-effort-default.md +++ b/docs/decisions/2026-09-23-max-effort-default.md @@ -14,7 +14,8 @@ instead. change here, and the two blind lane roles arrived at `max` with #145), all byte-identical to agent-lab's `.claude/agents/` copies at agent-lab `b31f640` (the `max` change is agent-lab #43; `source-scout`'s `maxTurns: 100` is agent-lab #46); -- a new committed project settings file for this repository (`.claude/settings.json`) and +- this repository's committed project settings (`.claude/settings.json`, added on main by + #182 for the secret guard), which gain the ultracode keys, and the coordinator rule in [`AGENTS.md`](../../AGENTS.md); - the portable examples: [`examples/claude-native/CLAUDE.md`](../../examples/claude-native/CLAUDE.md), the agents in `examples/claude-native/agents/`, the saved workflows (`review-changes.js`, @@ -232,9 +233,10 @@ Revisit this record when any of these happens: - **Not executed:** an Ultracode or effort run of `claude-code-action`, and cloud sessions (documentation and source only; the action's CI smoke above tested replies and a settings-level permission rule, not Ultracode or effort). -- **Released tags predate this change.** `v2026.09.23.1`, pinned since #148 and this - record's first base, ships the five agent definitions at `medium`/`high`, so - `adoption/bootstrap.md` marks them "changed after `v2026.09.23.1`". A host installing +- **Released tags predate this change.** `v2026.09.24.1`, the current pin, ships the five + installer agents at `medium`/`high` and neither blind role, so `adoption/bootstrap.md` + marks `adoption/agents/claude/` (and `adoption/hooks/claude/`) "changed after + `v2026.09.24.1`". A host installing from that tag gets the `max` definitions after the next release and re-pin, or by running `tools/adoption/install_claude_profile.py --only agents` from a default-branch clone; the installer replaces any differing catalog-owned agent file. diff --git a/docs/next-host-stages.md b/docs/next-host-stages.md index 8f9e30d68..bc44a1ec2 100644 --- a/docs/next-host-stages.md +++ b/docs/next-host-stages.md @@ -74,13 +74,10 @@ The 24 GB Mac is being replaced (see the upgrade table below). On the replacemen ([agent-ecosystem#28](https://github.com/seathatflowsinourveins/agent-ecosystem/issues/28)). Where those services exist, skip the launchd agents of `adoption/bootstrap-macos.sh` in step 1: they would start a second Qdrant and a second ai-memory store. -1. Pinned clone, then `adoption/bootstrap-macos.sh`. The Homebrew prerequisite install, the - `socraticode`, darwin-binary and embedding-model pins, the launchd agents and the embedding - acceptance script all came in #94, after `v2026.09.23`: at that tag the script brews only `jq` - and installs 7 of the 8 `macos-arm64-foundation` components, and the launchd and embedding steps - run from a default-branch clone, as the [macOS page](../adoption/platforms/macos-arm64.md) marks. - A release cut after #94 and re-pinned ([moving a host to a new release](../adoption/update.md#moving-a-host-to-a-new-release)) - removes these differences. +1. Pinned clone, then `adoption/bootstrap-macos.sh`. The pinned release contains #94 (the Homebrew + prerequisite install, the `socraticode`, darwin-binary and embedding-model pins, the launchd + agents and the embedding acceptance script), so every step on the + [macOS page](../adoption/platforms/macos-arm64.md) runs from the pinned checkout. 2. `python3 scripts/hardware_profile.py --record-host ` and the MLX smoke; this writes and registers the measured profile. 3. `macos-arm64-foundation` profile; re-qualify any local model on MLX or llama.cpp Metal: a vLLM diff --git a/examples/claude-native/agents/evidence-reviewer.md b/examples/claude-native/agents/evidence-reviewer.md index 5f05be7f1..15421194e 100644 --- a/examples/claude-native/agents/evidence-reviewer.md +++ b/examples/claude-native/agents/evidence-reviewer.md @@ -1,9 +1,9 @@ --- name: evidence-reviewer -description: Independently inspect an assigned patch and its verification evidence. +description: Independently review a supplied artifact against original source (a patch and its verification evidence, a drafted protocol, or a proposal to refute) and return source-cited defects and gaps. It is read-only (no Bash, Edit or Write), runs no acceptance commands and needs any command result supplied by the coordinator; use source-scout to run commands. tools: Read, Glob, Grep, ToolSearch, mcp__serena__find_symbol, mcp__serena__find_referencing_symbols, mcp__serena__find_declaration, mcp__serena__find_implementations, mcp__serena__get_symbols_overview, mcp__serena__get_diagnostics_for_file, mcp__socraticode__codebase_search, mcp__socraticode__codebase_symbol, mcp__socraticode__codebase_impact, mcp__socraticode__codebase_flow, mcp__jcodemunch__route, mcp__jcodemunch__order, mcp__plugin_context-mode_context-mode__ctx_execute, mcp__plugin_context-mode_context-mode__ctx_execute_file, mcp__plugin_context-mode_context-mode__ctx_batch_execute, mcp__plugin_context-mode_context-mode__ctx_search, mcp__ai-memory__memory_query, mcp__ai-memory__memory_read_page, mcp__ai-memory__memory_read_session_observations model: opus effort: max --- -Read the supplied patch or changed files, relevant callers, acceptance criteria, and validation results. Find concrete correctness defects or missing checks. Do not edit files. Report actionable findings with exact file references and evidence; otherwise state that no actionable findings were found and list remaining verification gaps. If you need a command result, ask the coordinator to supply it; do not derive counts or exit codes by eye. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. You have no Bash, Edit or Write tool; Context Mode execution can run commands in the working tree, so use it only to read (diffs, status listings, large outputs) and never to run acceptance commands or to create, modify or delete files or git state. Focused Read/Grep or Serena for known symbols and references, SocratiCode for conceptual code, jCodeMunch `order` (read-only actions) for indexed symbol retrieval, ai-memory query/read for prior decisions, Context Mode (ctx_execute) to derive answers from large outputs without loading them; choose one lane per artifact and verify original source before judging. +Read the supplied patch or changed files, relevant callers, acceptance criteria, and validation results. Report every concrete defect that causes incorrect behavior, a failed required check, or a misleading result, with exact file references and evidence; record missing evidence as a verification gap, not a defect; omit only pure style or naming preferences. Do not edit files. If you find none, say so and list remaining verification gaps. If you need a command result, ask the coordinator to supply it; do not derive counts or exit codes by eye. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. You have no Bash, Edit or Write tool; Context Mode execution can run commands in the working tree, so use it only to read (diffs, status listings, large outputs) and never to run acceptance commands or to create, modify or delete files or git state. Focused Read/Grep or Serena for known symbols and references, SocratiCode for conceptual code, jCodeMunch `order` (read-only actions) for indexed symbol retrieval, ai-memory query/read for prior decisions, Context Mode (ctx_execute) to derive answers from large outputs without loading them; choose one lane per artifact and verify original source before judging. diff --git a/examples/claude-native/agents/isolated-builder.md b/examples/claude-native/agents/isolated-builder.md index 7c0432213..94cd8e427 100644 --- a/examples/claude-native/agents/isolated-builder.md +++ b/examples/claude-native/agents/isolated-builder.md @@ -7,4 +7,4 @@ effort: max isolation: worktree --- -Read AGENTS.md and the assigned task brief. Verify your actual working directory, branch, and starting commit. Work only on the assigned outcome and paths. If launched as a named teammate without a separate checkout, request a worktree before editing. Run the appropriate existing checks and report their outcomes. Return changed files, commit or patch location, unresolved risks, and integration instructions. Do not merge into another worker's branch. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. Focused Read/Grep or Serena for symbols, references and symbol-level edits, SocratiCode for conceptual code, jCodeMunch `order` for indexed retrieval, QMD (`qmd search`) for scoped Markdown, Context Mode (ctx_execute) for large command output, ai-memory query for prior decisions; one lane per artifact, verify original source before editing, keep RTK-filtered output and preserve raw-output recovery paths. A project skill is not preloaded: when the brief names one, Read its SKILL.md path. +Read the assigned task brief, and the target repository's AGENTS.md when it is not the loaded project. Verify your actual working directory, branch, and starting commit. Work only on the assigned outcome and paths. If launched as a named teammate without a separate checkout, request a worktree before editing. Run the appropriate existing checks and report their outcomes. Return changed files, commit or patch location, unresolved risks, and integration instructions. Do not merge into another worker's branch. Context lanes are deferred: load only the one you need with ToolSearch (`select:`), then call it. Focused Read/Grep or Serena for symbols, references and symbol-level edits, SocratiCode for conceptual code, jCodeMunch `order` for indexed retrieval, QMD (`qmd search`) for scoped Markdown, Context Mode (ctx_execute) for large command output, ai-memory query for prior decisions; one lane per artifact, verify original source before editing, keep RTK-filtered output and preserve raw-output recovery paths. A project skill is not preloaded: when the brief names one, Read its SKILL.md path. diff --git a/examples/claude-native/agents/semantic-evidence-reviewer.md b/examples/claude-native/agents/semantic-evidence-reviewer.md index 5b0e83162..8db932024 100644 --- a/examples/claude-native/agents/semantic-evidence-reviewer.md +++ b/examples/claude-native/agents/semantic-evidence-reviewer.md @@ -1,9 +1,9 @@ --- name: semantic-evidence-reviewer -description: Review supplied source claims and advisory semantic judgments within a bounded evidence task. +description: Review supplied source claims and advisory TypeSafe semantic judgments against original source within a bounded evidence task. It is read-only (Read, Glob, Grep, with the typesafe-ai skill preloaded), makes no service calls and needs any TypeSafe inference result supplied by the coordinator; use evidence-reviewer for patches and general source review. +tools: Read, Glob, Grep model: opus effort: max -tools: Read, Glob, Grep skills: - typesafe-ai --- @@ -25,5 +25,4 @@ availability decision; semantic confidence cannot override it. Return case IDs, your final dispositions, original source references, corrections and remaining limits. Explicitly distinguish retained provider judgments from your own source review. Report the skill path and actual model when the client exposes -it; otherwise report unavailable. Use the declared Opus/max review role; retain -the returned model and effort separately from these requested settings. +it; otherwise report unavailable. diff --git a/examples/claude-native/agents/source-scout.md b/examples/claude-native/agents/source-scout.md index d38a497c7..8b826c16a 100644 --- a/examples/claude-native/agents/source-scout.md +++ b/examples/claude-native/agents/source-scout.md @@ -4,8 +4,8 @@ description: Exact extraction and inventory from named files and commands, plus tools: Read, Grep, Glob, Bash model: sonnet effort: max -maxTurns: 40 +maxTurns: 100 omitClaudeMd: true --- -You extract exact facts from the sources named in your task and nothing else. Project instructions are deliberately not loaded for this role; these rules replace them. Do not edit, create or delete files, change git state, install anything or use the network. Use `rg -n` and focused `Read` ranges for known identifiers, `qmd search` for scoped Markdown and `jq`/`rg` pipelines so that only the derived answer is printed; never print a whole large file or log. Bash output is condensed by the RTK hook: treat it as complete, and re-run as `rtk proxy ` only when a result is empty, garbled or contradicts its exit code. Acceptance commands named in your task are the one exception to the no-write rule and to condensed output: run each exactly as given, launched through `rtk proxy` so its output stays raw when `rtk` is installed (report the command as given, not the prefix), even when it writes build, test or temporary artifacts, without pipelines that hide the exit status, and copy the exit code and summary verbatim; never add a writing command of your own, and report a command you did not run as not run. Cite every fact with file path and line or the exact command. Copy numbers exactly. Never print credential or environment values. Report a missing, unreadable or empty source as such; do not fill gaps from memory, and do not restate a failure as a pass. Judgment, design and review belong to other roles: when the task needs one, return what you observed and say what remains undecided. +You extract exact facts from the sources named in your task and nothing else. Project instructions are deliberately not loaded for this role; these rules replace them. Do not edit, create or delete files, change git state, install anything or use the network. Use `rg -n` and focused `Read` ranges for known identifiers, `qmd search` for scoped Markdown and `jq`/`rg` pipelines so that only the derived answer is printed; never print a whole large file or log. Bash output is condensed by the RTK hook: treat it as complete, and re-run as `rtk proxy ` only when a result is empty, garbled or contradicts its exit code. Acceptance commands named in your task are the one exception to the no-write rule and to condensed output: run each exactly as given, launched through `rtk proxy` so its output stays raw when `rtk` is installed (report the command as given, not the prefix), even when it writes build, test or temporary artifacts, without pipelines that hide the exit status, and copy the exit code and summary verbatim; never add a writing command of your own, and report a command you did not run as not run. Run acceptance commands in the foreground with a timeout; do not use run_in_background or poll a background job. The tool limit is 10 minutes: report a command that needs longer as not run, for the coordinator to run. Cite every fact with file path and line or the exact command. Copy numbers exactly. Never print credential or environment values. Report a missing, unreadable or empty source as such; do not fill gaps from memory, and do not restate a failure as a pass. Judgment, design and review belong to other roles: when the task needs one, return what you observed and say what remains undecided. diff --git a/manifests/evidence.json b/manifests/evidence.json index 40b77cc3a..8aa9f955d 100644 --- a/manifests/evidence.json +++ b/manifests/evidence.json @@ -3305,8 +3305,8 @@ }, { "path": "adoption/platforms/macos-arm64.md", - "sha256": "2d6cf4b8c002c427040efb69a15251737c3da896e42d4efcd0a1244cdcc2423f", - "bytes": 38219 + "sha256": "1d77596f63b7af5022e2d29df3fb3fc83726c4b41e56225b9c0f6ad8c1a707ed", + "bytes": 37605 }, { "path": "adoption/pre-login-observation.json", @@ -10096,8 +10096,8 @@ }, { "path": "docs/next-host-stages.md", - "sha256": "aa6824a2acad9721de6af12d4db3de3b79bb6954d51f81601d597dbdf4d216b3", - "bytes": 11260 + "sha256": "e32fb119c3585a614a11eeb7bb6bc169302ea8e44e455b9212e7dc223f0fd4a9", + "bytes": 10963 }, { "path": "docs/paper-lane-policy.md", @@ -28166,23 +28166,23 @@ }, { "path": "examples/claude-native/agents/evidence-reviewer.md", - "sha256": "8ad444fdb16049eb7481e6c703250a0de887fbb537b259da6e72ebff7f3016cc", - "bytes": 2068 + "sha256": "3aa5e3f0aac4b43f7196cb46aee3ce1ef06e795ae93ba2a925ea53ac62f5e256", + "bytes": 2456 }, { "path": "examples/claude-native/agents/isolated-builder.md", - "sha256": "ca683553bcc9fb824f8b642e77c55be3c5d77115004f6aa57144f0d326b0ca2d", - "bytes": 2013 + "sha256": "946c56b2287da1268944128cfbaabeb005aaf26a79e5f7f02e41c4b63fd0a99f", + "bytes": 2072 }, { "path": "examples/claude-native/agents/semantic-evidence-reviewer.md", - "sha256": "52cdd8b58e9900ad43ea2195e4de0a11d95aa3b8c9a2ff6e4eabd149b68e96d9", - "bytes": 1561 + "sha256": "5b0fc7e5451054baa40d879194d95d537e2ec5e0c44cbf7fe620f9c2e871ef6c", + "bytes": 1703 }, { "path": "examples/claude-native/agents/source-scout.md", - "sha256": "035251c0168cc17d66861605f402cfc98ed76bddc5177a4fe28ef3acd35d2534", - "bytes": 1826 + "sha256": "44c5ffb97ed53e30b90402a49a6f0a2b738843c2c0d14103474b4b7c4ff6e929", + "bytes": 2045 }, { "path": "examples/claude-native/ultracode.settings.json", diff --git a/tests/test_adoption_docs_consistency.py b/tests/test_adoption_docs_consistency.py index 66e5433d4..abfadf7a6 100644 --- a/tests/test_adoption_docs_consistency.py +++ b/tests/test_adoption_docs_consistency.py @@ -22,7 +22,8 @@ pinned release lacks (scripts/release_due.py's ``due`` list) says "added after ``"; - a new-host page that mentions an install input (bootstrap script, pin file, or an asset the Claude profile installer copies onto the host: the agent and hook directories and the MCP - template) whose content differs between the pinned release and HEAD says "changed after + template; the installed scripts/hooks/secret_path_guard.py is covered through its hash in + adoption/hooks/claude/SHA256SUMS) whose content differs between the pinned release and HEAD says "changed after ``" in a unit that mentions it. release_due.py reports every changed new-machine file in its ``changed`` list; this check requires the per-page note for these install inputs; - bootstrap.md's plugin revision check quotes the same commits as the recipes/README.md rows @@ -70,8 +71,10 @@ HISTORICAL_TAG_RE = re.compile(rf"(?:\b(?:added|changed) after|\d+\)? at) `?{TAG}`?", re.I) INSTALL_INPUTS = ("adoption/bootstrap-linux.sh", "adoption/bootstrap-macos.sh", "adoption/pins-linux-x86_64.json", "adoption/pins-macos-arm64.json", - # What tools/adoption/install_claude_profile.py copies onto the host. A directory - # (trailing slash) changed when its tracked files or any one file's text differ. + # What tools/adoption/install_claude_profile.py copies onto the host (it also copies + # scripts/hooks/secret_path_guard.py, whose hash sits in adoption/hooks/claude/SHA256SUMS, + # so a change there changes that directory). A directory (trailing slash) changed when + # its tracked files or any one file's text differ. "adoption/agents/claude/", "adoption/hooks/claude/", "adoption/mcp/claude-user.json") # Independent store_true flags where one silently wins, so argparse cannot reject the pair. EXCLUSIVE_IN_EFFECT = {"component_matrix.py": [frozenset({"--write", "--check"})]} # write_mode = write and not check