Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion adoption/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ which you install through their recipes ([bootstrap step 2](bootstrap.md)).
| Adoption profile | Selects | Linux pins | macOS pins | Next native acceptance |
| --- | --- | --- | --- | --- |
| `foundation-cpu` | Codex, Claude Code, Context Mode, RTK, QMD BM25, explicitly scoped ai-memory, MCPorter | all 7 | all 7 (5 of 7 at `v2026.09.26`) | Native client setup; one useful context/document call and scoped memory retrieval. On macOS use `macos-arm64-foundation` |
| `new-wsl-clean-foundation` | [Clean replacement WSL source profile](new-wsl-profile.md): Codex 0.159.3 and Claude Code 2.1.284, with the existing bootstrap's pinned Node/uv/gh prerequisites and CPython 3.13.15 via uv | all 2 | all 2 | Source-review recommendations only; optional rows and comparison arms are excluded from the default install. Pin coverage counts do not qualify either host. New-host acceptance and the dual-image recipe remain pending |
| `new-wsl-clean-foundation` | [Clean replacement WSL source profile](new-wsl-profile.md), with the generated [new WSL handbook](../docs/new-wsl-handbook.md): Codex 0.159.3 and Claude Code 2.1.284, with the existing bootstrap's pinned Node/uv/gh prerequisites and CPython 3.13.15 via uv | all 2 | all 2 | Source-review recommendations only; optional rows and comparison arms are excluded from the default install. Pin coverage counts do not qualify either host. New-host acceptance and the dual-image recipe remain pending |
| `macos-arm64-foundation` | macOS only, drafted: Codex, Claude Code, Context Mode, ai-memory, MCPorter, llama.cpp Metal embedding, Qdrant, SocratiCode | 6 of 8 (5 of 8 at `v2026.09.25.2`) | all 8 | The macOS acceptance lane on [the macOS page](platforms/macos-arm64.md); not accepted |
| `token-efficiency` | Accepted 2026-09-30 as the selection ([routing record](../docs/decisions/2026-09-30-task-model-routing.md)), and changed after `v2026.09.25.1`, whose manifest lacks it: the selected token practice (RTK, Context Mode, explicit-file Repomix, guarded Headroom and TOON, ccusage, QMD, MarkItDown, Serena, SocratiCode, ai-memory, MCPorter) wired into both native clients | all 14 (8 of 14 at `v2026.09.25.2`) | all 14 (6 of 14 at `v2026.09.26`) | The [coverage check](../docs/token-efficiency-stack.md#coverage-check) for command presence and client wiring, then pinned-version evidence and one useful native call per tool in each client, recorded through a PR. Both bootstraps now pin all of it; the macOS pins changed after `v2026.09.26` |
| `research-runtime` | Historical hash-locked SDK/DuckDB, Dagu and LEAN comparison lane | 2 of 11 | 2 of 11 | Reproduce the retained comparison; this profile does not override the Nautilus destination |
Expand Down
4 changes: 4 additions & 0 deletions catalogs/landscape/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@
{
"path": "blueprints/catalog-clean-install/README.md",
"label": "Clean install checks"
},
{
"path": "docs/new-wsl-handbook.md",
"label": "New WSL handbook: source recommendations and acceptance stages"
}
]
}
8 changes: 8 additions & 0 deletions docs/harness-defaults.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,3 +215,11 @@ When a claim or action proves wrong, correct it where it was relayed and record
| 2026-10-01 | Citing a second-party reproduction that was a quotation | The coordinator told the user that semble's benchmark score had been reproduced by a second party (LightOn, 0.853 against 0.854). The decision round's critic read the source: LightOn's pull request says the semble and BM25 numbers "come from semble's README" and were not re-run, and the quoted row is an earlier model. | Before calling a number reproduced, read how the second party obtained it: a re-run, or a quotation. Two tables that agree to three digits are a sign of copying, not of replication. | `evidence/artifacts/new-wsl-definitive-defaults-20261001/foundation-definitive.compact.json` (code-search, corrections_by_critics) |
| 2026-10-01 | Reconciling a second model family that ran with fewer tools | The first blind GPT-6.1 Sol judge of the memory layer ran without shell network or page fetching (`gh auth status` exit 1, `curl` exit 6) while the Claude judges had both. It picked a repository the Claude judge had left out for facts it could not see (created a month earlier, no release); the rerun with network access dropped that pick. | Before reconciling two families, read the second family's event log for failed tool calls and its own evidence gaps, and rerun a degraded unit as a separately labeled unit, keeping the first. Identical packets with unequal tools produce disagreements that come from missing evidence. | `evidence/artifacts/new-wsl-definitive-defaults-20261001/gpt-memory-first-round-units.json`; `docs/decisions/2026-10-01-new-wsl-definitive-defaults.md` |
| 2026-10-01 | Copying a reviewer's own evidence-gap text into a blind packet | The decision round's memory packet gave the deciders the second first-round reviewer's evidence gaps verbatim. Two of those sentences carried routing wording (an "Astra/Max arbitration" attempt and an approval-policy note), from which a decider could tell which model family had written the notes. The cross-family read of the pull request found it; the packets' own check searched only for model names. | Strip a reviewer's process wording (model routing, tool approvals, budgets) from anything a blind judge reads, keep only the factual limitation, and test packets for routing terms, not only for names. A frozen packet that was already used is published as it was, with the leak recorded. | `evidence/artifacts/new-wsl-definitive-defaults-20261001/README.md`; `tests/test_new_wsl_definitive_defaults.py` |
| 2026-10-02 | Keeping owned worktrees and unpushed receipts in a directory that is emptied at boot | A WSL restart emptied the temporary directory that held the coordinator's scratch worktrees, two unpushed model-server gate folders and three running reviews. Both gate folders had to be rebuilt from retained raw runs, and the original confirmatory receipt is lost. | Put an owned worktree in a durable directory outside every checkout, push a worker's evidence to a branch when the worker returns, and keep only disposable files in a scratch directory. | this log; `docs/decisions/2026-10-01-new-wsl-definitive-defaults.md` records the rebuild and the loss |
| 2026-10-02 | Embedding double quotes in an argument that Windows PowerShell hands to a native program | The new-distribution recipe's F10 loop and an R1 probe carried a double quote inside the single-quoted argument of `wsl.exe`. Windows PowerShell 5.1 does not escape it for a native program, so the argument broke apart in the rehearsal (`unexpected EOF`, `stat: missing operand`). The commands had been reviewed as text and had not been run from PowerShell. | Write a command that crosses PowerShell into `wsl.exe` without an embedded double quote, and run each recipe command once from the shell the page names before the page merges. | `tests/test_wsl_new_distro_recipe.py` (`powershell_quote_errors`, with a control that restores the quoted form); `evidence/artifacts/new-wsl-rehearsal-20261002/runs-on-wsl-3.0.1.json` |
| 2026-10-02 | Counting log lines by a pattern that also matches the system's own configuration echo | The recipe's storage-error count matched `hv_storvsc` and left out the driver's registration line, but not the kernel's command-line echoes, which name `hv_storvsc.storvsc_max_hw_queues=4`. On kernel 6.18.40.1 every boot logs them, so a healthy boot counted as storage errors. | Before a count becomes a stop rule, read the lines it matches on a healthy run of the release in use and exclude the configuration and registration lines. | `tests/test_wsl_new_distro_recipe.py` (`test_all_three_storage_filters_exclude_the_command_line_echo`) |
| 2026-10-02 | Stating a recurrence that the observations do not cover | The recipe page and its decision record said that `systemd-binfmt.service` fails at every boot of the new distribution. The failure had been observed on one host, and WSL's source shows that it depends on the binfmt protection, a default-enabled and best-effort lock, so it is conditional. The cross-family read of pull request 593 found the overstatement. | State on which host and under which setting a behavior was observed, and keep "every" and "always" for a mechanism that upstream source shows to be unconditional or for a measured series. | this log; the qualified wording in `adoption/platforms/linux-wsl2-new-distro.md` and `docs/decisions/2026-10-01-new-wsl-distro-recipe.md` |
| 2026-10-02 | Editing a convergence record's frozen input in a pull request that does not own the record | A wording fix to the new-distribution recipe page was made in the terminal-profile pull request (604). The page is a frozen input of the recipe's convergence record, so the hosted validate job failed with `frozen_inputs.sources[0]: SHA-256 mismatch`. The local `scripts/validate.py` had passed, because it does not read the records' input hashes. | Change a frozen input only in the pull request that owns its record and refreezes it, and run `python3 scripts/validate_convergence.py --all-recorded --root . --json` beside `scripts/validate.py` before a push. | `.github/workflows/validate.yml` (the step "Validate declared convergence evidence and scope" failed on this mistake) |
| 2026-10-02 | Writing an acceptance check against an optional field without seeing the tool's output on a real install | The install plan's check of the engineering-process skills compared a `ref` field of the installer's lock file with the pinned tag. The check had not run before the first install on a real distribution. The lock that this run returned had no `ref`, so the check failed; the installer at the pinned release supports that field as an optional branch or tag, and why it was absent in that run is unresolved. | Take an acceptance check's fields from the tool's own output on a real install, and count the check as validated only after it has run there. The check now compares the lock's recorded `skillFolderHash` values with the expected tree hashes of the skill folders at the tag; it does not hash the installed directories. | this log; `evidence/artifacts/new-wsl-install-plan-20261002/VALIDATION.md` and `real-distribution-validation.json` record the failed check and the later result |
| 2026-10-02 | Running an upstream installer unattended without its non-interactive switch | The install plan piped Codex's installer to a shell. The installer asks whether to start Codex on the terminal device, not on standard input, so the run on a real distribution waited at the prompt until it was stopped; the earlier container run had no terminal and had not shown it. | Read the installer's source at the pinned tag for prompts and set its documented non-interactive variable (`CODEX_NON_INTERACTIVE=1` in `scripts/install/install.sh` at `rust-v0.160.0`); validate an unattended install where a terminal device exists. | this log; `evidence/artifacts/new-wsl-install-plan-20261002/install.sh` sets the variable and `VALIDATION.md` records the run |
| 2026-10-02 | Pinning with a reference syntax the client does not have | The install plan added the skills marketplace as `owner/repository@<commit>`. The client reads the text after `@` as a branch or tag, and the shorthand cloned over SSH, which failed on the host key in a fresh distribution. | Check a pin's syntax in the client's own help or source before writing it; where the client cannot pin a commit, add the source by HTTPS URL and compare the cloned HEAD with the reviewed commit in the acceptance check. | this log; `evidence/artifacts/new-wsl-install-plan-20261002/install.sh` and `accept.sh` |
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Native self-updating Claude/Codex installers remain preferred over npm; reviewed

- **Rows follow the manifest.** Trafilatura, ccusage, Phoenix, Promptfoo and chezmoi lost their install and acceptance functions, mise tools, config files (`config/phoenix-compose.yaml`) and source entries; so did the rows that never had a host install (Claude Code Action, the CodeQL slot, trufflehog, the LSP plugins, the structural-diff row). attest and Dependabot became `installed: false`, with their reviewed workflow pointers kept in `notes`. Loki, Grafana and Playwright CLI keep their functions behind `--only`.
- **Added.** ast-grep 0.45.3 through mise (`structural-search`); Alertmanager v0.34.1 as a checksum-verified release binary with `config/alertmanager.yaml` (one receiver that sends nothing), port 21093 on loopback and clustering off (`alerting`); mattpocock/skills per skill for both agents, through the `skills` installer pinned to 1.7.0, source tag v1.2.3 and telemetry off (`engineering-process-skills`). `setup-matt-pocock-skills` is included because the upstream README asks that it be one of the selected skills.
- **Three repairs from the real run** (VALIDATION.md, findings 1 to 3): the Codex installer line sets `CODEX_NON_INTERACTIVE=1`, which the installer documents as skipping prompts; the Claude marketplace line for Trail of Bits adds the repository by its HTTPS clone URL, unpinned (the client takes a branch or tag there, not a commit, and the repository has no tag), and the acceptance prints the clone's `HEAD` beside the reviewed commit and fails when they differ; and the scripts stop with a clear message when `repo_root` is not a git checkout. The mattpocock row needs no such comparison: the installer clones the pinned tag and the acceptance checks the tag recorded in its lock file.
- **Three repairs from the real run** (VALIDATION.md, findings 1 to 3): the Codex installer line sets `CODEX_NON_INTERACTIVE=1`, which the installer documents as skipping prompts; the Claude marketplace line for Trail of Bits adds the repository by its HTTPS clone URL, unpinned (the client takes a branch or tag there, not a commit, and the repository has no tag), and the acceptance prints the clone's `HEAD` beside the reviewed commit and fails when they differ; and the scripts stop with a clear message when `repo_root` is not a git checkout. The mattpocock row needs no such comparison: the installer clones the pinned tag. Its acceptance compares the `skillFolderHash` values that the installer recorded in its lock file with the expected git tree hashes of the six skill folders at the tag; it does not hash the installed directories. The clean run's record reports that the lock it returned had no `ref` field; the installer at v1.7.0 supports that optional field, and why it was absent in that run is unresolved.
- **`check_plan.py`** fails and names each problem when a selected row has no install function or no post-install acceptance, a function exists for a row that is neither selected nor measurement-only, `install.sh --list` does not print exactly the rows of `install-plan.json`, a `mise.toml` tool belongs to no selected or measurement-only row, a selected row's manifest counterpart is missing, `not_installed` or `split`, two rows claim one port, or a command has no source URL. It also checks that the rows are the manifest's foundation rows, that `owners.json` agrees, that the `run_command` lines and `check` calls in the scripts are the commands in the JSON, and that every `config/` file is copied.

The original [validation.json](validation.json) records the earlier source/static review and [container-validation.json](container-validation.json) the round 1 container run. The clean run of 2026-10-02 claims installation, post-install acceptance and the health of six services in a throwaway distribution, and nothing more: no provider call, model run, GPU use or sign-in, and no run on the destination distribution. After that run the header comments of both scripts and the plan's `status` text were corrected to say so; no command changed, and the record names the executed commits and file hashes.
10 changes: 6 additions & 4 deletions evidence/artifacts/new-wsl-install-plan-20261002/VALIDATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,12 @@ no provider call, no model run. The distribution was removed afterwards.
owner, is installed by the script's first step, which prints no result line.
- Post-install acceptance at `b48321ea`, first pass, counted in result lines, one per plan row (64): 34 exit 0, 29
skipped and one exit 1 (`engineering-process-skills`). The 29 are the 25 rows that are not installed, the three
measurement-only owners and `credential-guard`, which has no host-executable check. The install was right and
the check was wrong: the installer's lock file records no `ref`. Each of the six skills' `skillFolderHash`
equals the git tree hash of its folder at tag `v1.2.3` and differs from main's, so the pin had been honoured.
- Post-install acceptance at `1b290218`, final: the acceptance now compares those hashes. `accept.sh --only
measurement-only owners and `credential-guard`, which has no host-executable check. The check failed because
the lock that this run returned had no `ref` field; the installer at v1.7.0 supports that field as an optional
branch or tag, and why it was absent here is unresolved. Each of the six skills' recorded `skillFolderHash` equals
the git tree hash of its folder at tag `v1.2.3` and differs from main's.
- Post-install acceptance at `1b290218`, final: the acceptance now compares those recorded hashes with the expected
ones; it does not hash the installed directories, and it says nothing about a client loading a skill. `accept.sh --only
engineering-process-skills` returned exit 0, and one full `accept.sh` run returned 35 exit 0 and 29 skipped. Of
that full run only the counts were kept, not its per-owner lines, so the record's `per_owner` table is the first
pass and still shows this owner's failure; `per_owner_final` holds the one line that was kept.
Expand Down
Loading
Loading