Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/harness-defaults.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,3 +225,9 @@ When a claim or action proves wrong, correct it where it was relayed and record
| 2026-10-02 | Writing an acceptance check against an optional field without seeing the tool's output on a real install | The install plan's check of the engineering-process skills compared a `ref` field of the installer's lock file with the pinned tag. The check had not run before the first install on a real distribution. The lock that this run returned had no `ref`, so the check failed; the installer at the pinned release supports that field as an optional branch or tag, and why it was absent in that run is unresolved. | Take an acceptance check's fields from the tool's own output on a real install, and count the check as validated only after it has run there. The check now compares the lock's recorded `skillFolderHash` values with the expected tree hashes of the skill folders at the tag; it does not hash the installed directories. | this log; `evidence/artifacts/new-wsl-install-plan-20261002/VALIDATION.md` and `real-distribution-validation.json` record the failed check and the later result |
| 2026-10-02 | Running an upstream installer unattended without its non-interactive switch | The install plan piped Codex's installer to a shell. The installer asks whether to start Codex on the terminal device, not on standard input, so the run on a real distribution waited at the prompt until it was stopped; the earlier container run had no terminal and had not shown it. | Read the installer's source at the pinned tag for prompts and set its documented non-interactive variable (`CODEX_NON_INTERACTIVE=1` in `scripts/install/install.sh` at `rust-v0.160.0`); validate an unattended install where a terminal device exists. | this log; `evidence/artifacts/new-wsl-install-plan-20261002/install.sh` sets the variable and `VALIDATION.md` records the run |
| 2026-10-02 | Pinning with a reference syntax the client does not have | The install plan added the skills marketplace as `owner/repository@<commit>`. The client reads the text after `@` as a branch or tag, and the shorthand cloned over SSH, which failed on the host key in a fresh distribution. | Check a pin's syntax in the client's own help or source before writing it; where the client cannot pin a commit, add the source by HTTPS URL and compare the cloned HEAD with the reviewed commit in the acceptance check. | this log; `evidence/artifacts/new-wsl-install-plan-20261002/install.sh` and `accept.sh` |
| 2026-10-02 | Inherited host UI overrides in an unchanged upstream test run | GitHub CLI v2.102.0's native suite failed in pager, editor-color and interactive-prompt tests under inherited `GH_PAGER`, `NO_COLOR` and `TERM=dumb`. Source diagnosis and unchanged single-variable controls distinguished test-environment interference; the complete command passed after those overrides were cleared. The failed run and its native outputs are kept. | Read the pinned implementation's environment inputs, keep the original failed condition and test one source-backed environment change at a time before diagnosing an upstream defect: [Pager](https://github.com/cli/cli/blob/fc4b137cdef0a6bd28fd461b7cf9c84a5812a8cd/internal/ghcmd/cmd.go#L387); Survey v2.3.7 `core/template.go:27-29`; [Huh accessible mode](https://github.com/charmbracelet/huh/blob/3c0116c6fd1f578ef50a25d868d2db19bd985222/form.go#L129-L132). These controls apply to the pinned upstream test run; user sessions get no blanket TERM, color or pager reset. | this log; the native failed, full and focused qualification runs are recorded outside this repository, with no new global hook |
| 2026-10-02 | A measurement window opened while the lane's own model server still held a model from an earlier trial | The server ran with an unlimited keep-alive, the trial script ended without unloading, and the next window's script recorded the GPU memory in use (18,094 MiB) without checking it, so the first load started under a condition the preregistration excluded. The attempt was stopped before any result was read and is kept as void. | A window checks its preconditions before the first load (the measurement server lists no resident model, no other model server runs, the workstation's own model services are inactive, the memory in use is within what the desktop alone explains) and stops if one fails; trial scripts unload what they loaded. | deviation 1 of the lane's local-model preregistration (kept outside the repository and hashed before each trial); the lane's measurement scripts; no repository check |
| 2026-10-02 | A run label reused across two arms, and copies named without the parameter that varied | The second arm's ten client runs were refused because their label existed, and the workstation-side copies of the first arm's records were overwritten by the refusal output because their names did not carry the model. | Name every run and every copy by every parameter that varies, and refuse to overwrite an existing record. | deviation 3 of the same preregistration; the lane's runner scripts; no repository check |
| 2026-10-02 | Normalizing trailing slashes so that the root directory becomes empty | `shell_dir` in `tools/adoption/managed_block.py` ended with `normal.rstrip("/")`, so the explicit `--extra-dir /` that pull request 608 accepted became the empty string and the profile block wrote an empty PATH component, which names the current directory: a control on the original function turned `/usr/bin:/bin` into `/opt/ecosystem/bin::/usr/bin:/bin`. The lane finalizing pull request 610 reported it on pull request 608, and the Codex lane confirmed it. A search for the same strip found `host_values` in `tools/adoption/new_wsl_client_config.py`: a host value file that declares HOME as `/` stripped to the empty prefix that every absolute path has, so `--apply --home` moved the first entry of HOST_PATH (`/usr/local/sbin` in the example host's values) under the new home and applied the run. | A normalization of a directory keeps `/`: no empty PATH component is written, and no path is tested against the empty prefix that `/` strips to; where the root cannot serve, as a home whose paths are moved, the input is refused. A test pins the root case. | `tests/test_managed_block.py` (`test_an_explicit_root_directory_is_one_component_and_no_component_is_empty`) and `tests/test_new_wsl_client_config.py` (`test_a_host_value_file_whose_home_is_the_root_is_refused_and_nothing_is_moved_or_written`), each seen failing on the code before the fix; three tests in `tests/test_managed_block.py` that pin the unchanged bytes of the other inputs bound the fix and pass on that code too |
| 2026-10-02 | Printing a whole multi-megabyte registry instead of a bounded patch | A registry helper printed the old and the new `manifests/evidence.json` in full, and the tool output was truncated. The 13 file registrations had completed, but the receipt entry had not been written. A patch that inserted only that entry at the end of the receipt array completed it; `scripts/validate.py` then passed (69 components, 9,330 files, 4 profiles, 187 receipts), and so did the all-recorded convergence check. No truncated output was used as a patch or a verdict. | Derive a minimal patch and its counts outside the model, and never print a complete registry. | this log (reported by the SDK lane); no automated check |
| 2026-10-02 | Counting usage once per content block instead of once per message | A prepared role adapter for the Claude Agent SDK repeated the assistant usage on every content block and dropped the native `message_id`, so messages from parallel tool use could not be deduplicated. A review before execution (GPT-6 Astra) rejected those bytes before they ran. | Take per-step usage from the SDK's own message (`usage` and `message_id`), count each `message_id` once, and keep a message without an id uncountable. | [`message_parser.py` at `1ef6d8c7`](https://github.com/anthropics/claude-agent-sdk-python/blob/1ef6d8c71bb0e44a6b33fe61497864f21e17fdb7/src/claude_agent_sdk/_internal/message_parser.py#L210-L220); [per-step usage](https://code.claude.com/docs/en/agent-sdk/cost-tracking#track-per-step-usage); caught by review, no runtime test |
12 changes: 6 additions & 6 deletions manifests/evidence.json
Original file line number Diff line number Diff line change
Expand Up @@ -15936,8 +15936,8 @@
},
{
"path": "docs/harness-defaults.md",
"sha256": "3b9c833fdfb36c81e13e64eba29a19803d0f09c0691042381fb202eda7ecef1a",
"bytes": 136634
"sha256": "85a0f5d92161b16c67bdeb111bdb5b67c247f0fd241e0a9ef032d8cb9b51e8eb",
"bytes": 142363
},
{
"path": "docs/harness-rules-convergence-20260922.md",
Expand Down Expand Up @@ -49941,8 +49941,8 @@
},
{
"path": "tests/test_managed_block.py",
"sha256": "65191a29f502605e296fdd924c38f889ea6bda4c239674fd1b5f14352a2c9bdc",
"bytes": 28016
"sha256": "103c30facbb744404abcef68734b9d46b78b7fe04593484a899293b4407dd25e",
"bytes": 33095
},
{
"path": "tests/test_memory_lifecycle.py",
Expand Down Expand Up @@ -50441,8 +50441,8 @@
},
{
"path": "tools/adoption/managed_block.py",
"sha256": "aa8147a6662a1083cd841f29c9eab7ad50bfb078c0f416b5980bffce6cfd575a",
"bytes": 23196
"sha256": "4a5d7c2001be450209aa903af691e653b39989af654bf0e61d70b2199012bba8",
"bytes": 23567
},
{
"path": "tools/adoption/prove_codex_lane.py",
Expand Down
87 changes: 84 additions & 3 deletions tests/test_managed_block.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
the markers survive; an earlier hand copy of the example is replaced only when it equals the current example, and any
other copy, a damaged block, a symlink or a non-UTF-8 file is refused with nothing written. The PATH block is sourced
by a real POSIX sh: its first PATH entry is the ecosystem's bin directory whatever characters the root holds, and a
second sourcing adds nothing.
second sourcing adds nothing. An explicit root directory stays `/`, so no PATH component is empty, and every other
input writes the bytes it wrote before that fix.
"""

import contextlib
Expand Down Expand Up @@ -453,19 +454,99 @@ def change_after_read(path):


class ProfilePathBlockTests(ManagedBlockCase):
# Blocks pinned as text, not computed by the code under test. ORDINARY_BLOCK, OMITTED_PROFILE and ROOT_ECO_BLOCK
# are the bytes the code wrote before the root-directory fix; ROOT_EXTRA_BLOCK replaces the empty component that
# code wrote for `--extra-dir /` (PATH="${PATH:+:$PATH}": /usr/bin:/bin became /opt/ecosystem/bin::/usr/bin:/bin).
HEAD = ("# native-agent-stack:profile-path:begin (adoption/bootstrap-linux.sh --configure-full-profile; edit "
"outside these markers)\n"
"# The ecosystem's bin directory first on PATH, so `claude` in a login shell is its launcher.\n")
EXTRAS_NOTE = ("# These directories are added when they are not on PATH yet: the native installers' bin directory "
"and mise's shims.\n")
TAIL = "export PATH\n# native-agent-stack:profile-path:end\n"
ORDINARY_BLOCK = (HEAD + EXTRAS_NOTE +
'case ":${PATH-}:" in\n'
' *":/opt/tools:"*) ;;\n'
' *) PATH="/opt/tools${PATH:+:$PATH}" ;;\n'
"esac\n"
'case ":${PATH-}:" in\n'
' *":$HOME/.local/bin:"*) ;;\n'
' *) PATH="$HOME/.local/bin${PATH:+:$PATH}" ;;\n'
"esac\n"
'case ":${PATH-}:" in\n'
' ":$HOME/.local/share/codex-ecosystem/bin:"*) ;;\n'
' *) PATH="$HOME/.local/share/codex-ecosystem/bin${PATH:+:$PATH}" ;;\n'
"esac\n" + TAIL)
OMITTED_PROFILE = (HEAD +
'case ":${PATH-}:" in\n'
' ":$HOME/.local/share/codex-ecosystem/bin:"*) ;;\n'
' *) PATH="$HOME/.local/share/codex-ecosystem/bin${PATH:+:$PATH}" ;;\n'
"esac\n" + TAIL).encode("utf-8")
ROOT_ECO_BLOCK = (HEAD +
'case ":${PATH-}:" in\n'
' ":/bin:"*) ;;\n'
' *) PATH="/bin${PATH:+:$PATH}" ;;\n'
"esac\n" + TAIL)
ROOT_EXTRA_BLOCK = (HEAD + EXTRAS_NOTE +
'case ":${PATH-}:" in\n'
' *":/:"*) ;;\n'
' *) PATH="/${PATH:+:$PATH}" ;;\n'
"esac\n"
'case ":${PATH-}:" in\n'
' ":/opt/ecosystem/bin:"*) ;;\n'
' *) PATH="/opt/ecosystem/bin${PATH:+:$PATH}" ;;\n'
"esac\n" + TAIL)

def setUp(self):
super().setUp()
self.target = self.home / ".profile"

def apply(self, eco_root: str, *extra: str) -> tuple:
return run("--home", str(self.home), *extra, "profile-path", "--eco-root", eco_root)

def path_after_sourcing(self, times: int = 1) -> list:
def path_after_sourcing(self, times: int = 1, start: str = "/usr/bin:/bin") -> list:
script = ". \"$HOME/.profile\"; " * times + 'printf "%s" "$PATH"'
result = subprocess.run([SH, "-c", script], env={"HOME": str(self.home), "PATH": "/usr/bin:/bin"},
result = subprocess.run([SH, "-c", script], env={"HOME": str(self.home), "PATH": start},
capture_output=True, text=True, timeout=30, check=True)
return result.stdout.split(":")

@unittest.skipUnless(SH, "needs a POSIX sh")
def test_an_explicit_root_directory_is_one_component_and_no_component_is_empty(self):
for extra in ("/", "//", "/./"):
with self.subTest(extra=extra):
self.assertEqual(managed_block.profile_block("/opt/ecosystem", str(self.home), (extra,)),
self.ROOT_EXTRA_BLOCK)
if self.target.exists():
self.target.unlink()
code, _, err = run("--home", str(self.home), "profile-path", "--eco-root", "/opt/ecosystem",
"--extra-dir", extra)
self.assertEqual(code, 0, err)
text = self.target.read_text(encoding="utf-8")
self.assertIn(' *) PATH="/${PATH:+:$PATH}" ;;\n', text)
self.assertEqual(text, self.ROOT_EXTRA_BLOCK)
path = self.path_after_sourcing()
self.assertEqual(":".join(path), "/opt/ecosystem/bin:/:/usr/bin:/bin")
self.assertNotIn("", path)
self.assertEqual(self.path_after_sourcing(times=2), path)
self.assertEqual(":".join(self.path_after_sourcing(start="")), "/opt/ecosystem/bin:/")
# With the home at the root there is no $HOME form either, and the root stays a component of its own.
self.assertEqual(managed_block.profile_block("/opt/ecosystem", "/", ("/",)), self.ROOT_EXTRA_BLOCK)

def test_ordinary_extra_directories_give_the_bytes_written_before(self):
# One directory under home (written as $HOME/...) and one outside it.
block = managed_block.profile_block(str(self.home / ".local/share/codex-ecosystem"), str(self.home),
(str(self.home / ".local/bin"), "/opt/tools"))
self.assertEqual(block, self.ORDINARY_BLOCK)

def test_without_the_option_the_default_profile_gets_the_bytes_written_before(self):
code, _, err = self.apply(str(self.home / ".local/share/codex-ecosystem"))
self.assertEqual(code, 0, err)
self.assertEqual(self.target.read_bytes(), self.OMITTED_PROFILE)

def test_a_root_ecosystem_directory_still_puts_slash_bin_first(self):
for eco in ("/", "//"):
with self.subTest(eco=eco):
self.assertEqual(managed_block.profile_block(eco, str(self.home)), self.ROOT_ECO_BLOCK)

def test_an_eco_root_under_home_is_written_relative_to_home(self):
eco = self.home / ".local/share/codex-ecosystem"
self.assertEqual(self.apply(str(eco))[0], 0)
Expand Down
29 changes: 29 additions & 0 deletions tests/test_new_wsl_client_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -1457,6 +1457,35 @@ def test_an_environment_that_names_another_config_directory_is_refused_for_the_r
self.assertIn("CLAUDE_CONFIG_DIR or CODEX_HOME is set", err)
self.assertEqual(list(self.home.iterdir()), [])

def test_a_host_value_file_whose_home_is_the_root_is_refused_and_nothing_is_moved_or_written(self):
# The example host's values with its home at the root: HOME "/" (or "//"), its home paths written from the root
# and HOST_PATH as it is. Without its trailing slashes such a HOME is empty, a prefix of every absolute path:
# the code before the fix moved /usr/local/sbin, HOST_PATH's first entry, under the new home and applied that.
example = json.loads((render_config.HOSTS / f"{EXAMPLE_HOST}.json").read_text(encoding="utf-8"))
hosts = self.base / "hosts"
hosts.mkdir()
plan = cfg.analyse(ROOT)[2]
for declared in ("/", "//"):
root_home = {k: (declared if k == "HOME" else v.replace(example["HOME"], "")) for k, v in example.items()}
(hosts / "root-home.json").write_text(json.dumps(root_home, indent=2) + "\n", encoding="utf-8")
with self.subTest(declared=declared), mock.patch.object(render_config, "HOSTS", hosts):
code, out, err = run_main("--apply", "--host", "root-home", "--home", str(self.home), "--claude-bin",
str(self.claude), "--codex-bin", str(self.codex), "--codex-process-name",
NO_PROCESS)
self.assertEqual(code, 1, out[-600:])
self.assertTrue(err.startswith(f"apply failed: the host value file declares HOME as {declared!r}"), err)
self.assertEqual(list(self.home.iterdir()), [])
self.assertFalse(self.marker.exists(), "a client ran")
with self.assertRaises(cfg.ConfigError):
cfg.host_values("root-home", plan, self.home, [])
# With the new home at the root as well nothing moves: the values are those of a render without --home.
self.assertEqual(cfg.host_values("root-home", plan, Path("/"), []),
cfg.host_values("root-home", plan, None, []))
# A home below the root moves the paths under it and leaves the system's directories where they are.
values = cfg.host_values(EXAMPLE_HOST, plan, self.home, [])
self.assertEqual((values["HOME"], values["ECO_ROOT"], values["HOST_PATH"]),
(str(self.home), str(self.eco), example["HOST_PATH"]))


def leaves(node, prefix=()):
"""{key path: JSON text of the value} of every leaf of a parsed settings or config tree."""
Expand Down
11 changes: 8 additions & 3 deletions tools/adoption/managed_block.py
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,11 @@ def sh_double_quoted(text: str) -> str:


def shell_dir(path: str, home: str, what: str) -> str:
"""path as the inside of a POSIX sh double-quoted string: under home it is written as $HOME/<relative>."""
"""path as the inside of a POSIX sh double-quoted string: under home it is written as $HOME/<relative>.

A path of slashes only is the root directory and stays "/", as POSIX basename makes it (for "//" that is
implementation-defined, and Linux takes "//" as the root), so the result is never empty: an empty PATH component
names the current directory (XBD 8.3, PATH)."""
normal = os.path.normpath(path)
base = os.path.normpath(home)
if not os.path.isabs(normal):
Expand All @@ -232,11 +236,12 @@ def shell_dir(path: str, home: str, what: str) -> str:
relative = os.path.relpath(normal, base) if base != "/" else None
if relative is not None and relative != "." and not relative.startswith(".."):
return "$HOME/" + sh_double_quoted(relative)
return sh_double_quoted(normal.rstrip("/"))
return sh_double_quoted(normal.rstrip("/") or "/")


def profile_block(eco_root: str, home: str, extra_dirs: tuple[str, ...] = ()) -> str:
directory = shell_dir(eco_root, home, "ecosystem root") + "/bin"
root = shell_dir(eco_root, home, "ecosystem root")
directory = ("" if root == "/" else root) + "/bin" # the root's bin directory is /bin, not //bin
extras = ""
if extra_dirs:
# Each extra directory goes in front of the system path unless it is already on PATH, in reverse order so the
Expand Down
Loading
Loading