Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .agents/skills/harness-adapters/references/harness/codex.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,15 @@ A directory trust dialog appears on the first run for a repository root: "Do you
Accept it with Enter and verify the instructions begin processing.
The decision persists for the repository, so later worktrees of the same project skip it.

## Hook trust

A second dialog, "Hooks need review - N hooks are new or changed", appears whenever the machine's `~/.codex/hooks.json` or a project's own `.codex/hooks.json` carries a hook Codex has not persisted trust for.
It is unanswerable rather than merely inconvenient: its selection starts on "Review hooks", which is neither trusting nor declining, and Firstmate's key plane carries Enter, Escape and Ctrl-C with no arrow navigation.
Writing Codex's own trust store to pre-accept it would manufacture an operator consent that was never given.
So crewmate and scout launches disable Codex's hook layer outright (`bin/fm-spawn.sh`'s launch template owns the flag), which is the opposite of `--dangerously-bypass-hook-trust` - that flag RUNS the untrusted hooks.
A crewmate loses nothing: its turn-end signal is the `-c notify=` program on the same launch, and the Firstmate hooks in a project's `.codex/hooks.json` are primary-session infrastructure that stands down in a child worktree.
A secondmate is a primary in its own home and keeps its hooks, so an unanswerable modal there is still possible and is the operator's own hook review to settle.

## Skill popup

A `$<skill>` invocation opens a `$` autocomplete popup.
Expand Down
24 changes: 23 additions & 1 deletion bin/fm-spawn.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1685,11 +1685,33 @@ launch_template() {
fi
printf '%s' '__MODELFLAG____EFFORTFLAG__"$(__OPINPUT__ encode launch-brief < __BRIEF__)"'
;;
# --disable hooks (equivalent to -c features.hooks=false) turns codex's whole
# lifecycle-hook layer off for CREWMATE and SCOUT launches only.
# Without it a crewmate launch parks forever on codex's hook-trust modal
# ("N hooks are new or changed"), whose selection sits on "Review hooks" -
# neither trusting nor declining. Firstmate's key plane carries Enter, Escape
# and Ctrl-C with no arrow navigation, so the selection cannot be moved, and
# pre-accepting the prompt by writing codex's own trust store would manufacture
# an operator consent that was never given. The hooks it asks about are the
# OPERATOR's machine-level ~/.codex/hooks.json plus any project-local
# .codex/hooks.json, and a crewmate needs none of them: its turn-end signal is
# the -c notify= program on this same launch (verified still firing with hooks
# disabled, codex-cli 0.151.0), and firstmate's own .codex/hooks.json registers
# PRIMARY-session infrastructure that already stands down in a child worktree.
# This is the opposite of --dangerously-bypass-hook-trust, which RUNS untrusted
# hooks; disabling the feature runs none of them and leaves the operator's
# ~/.codex untouched. An unknown feature name is a hard codex error, so a future
# release that drops this flag fails the launch loudly instead of silently
# restoring the modal.
# A secondmate is a firstmate PRIMARY in its own home, and its turn-end guard,
# session-start digest, and cd/arm seatbelts are exactly those project hooks
# (docs/turnend-guard.md, docs/sessionstart-nudge.md, docs/cd-guard.md), so the
# secondmate launch deliberately keeps hooks on.
codex)
if [ "$kind" = secondmate ]; then
printf '%s' 'codex __MODELFLAG____EFFORTFLAG__--dangerously-bypass-approvals-and-sandbox "$(__OPINPUT__ encode launch-brief < __BRIEF__)"'
else
printf '%s' 'codex __MODELFLAG____EFFORTFLAG__--dangerously-bypass-approvals-and-sandbox -c "notify=[\"bash\",\"-c\",\"touch __TURNEND__\"]" "$(__OPINPUT__ encode launch-brief < __BRIEF__)"'
printf '%s' 'codex __MODELFLAG____EFFORTFLAG__--dangerously-bypass-approvals-and-sandbox --disable hooks -c "notify=[\"bash\",\"-c\",\"touch __TURNEND__\"]" "$(__OPINPUT__ encode launch-brief < __BRIEF__)"'
fi
;;
opencode) printf '%s' 'OPENCODE_CONFIG_CONTENT='\''{"permission":{"*":"allow"}}'\'' opencode __MODELFLAG__--prompt "$(__OPINPUT__ encode launch-brief < __BRIEF__)"' ;;
Expand Down
3 changes: 2 additions & 1 deletion bin/fm-test-run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -344,7 +344,8 @@ family_for_basename() {
fm-cmux-claude-composer-live-e2e.test.sh|\
fm-composer-matrix-live-e2e.test.sh|\
fm-composer-codex-idle-live-e2e.test.sh|\
fm-codex-continuity-live-e2e.test.sh|fm-grok-continuity-live-e2e.test.sh|\
fm-codex-continuity-live-e2e.test.sh|fm-codex-hook-layer-live-e2e.test.sh|\
fm-grok-continuity-live-e2e.test.sh|\
fm-cursor-primary-live-e2e.test.sh|\
fm-grok-stop-live-e2e.test.sh|fm-harness-adapter-instructions-live-e2e.test.sh|\
fm-harness-liveness-drift-live-e2e.test.sh|\
Expand Down
57 changes: 57 additions & 0 deletions docs/verification/runtime-backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -508,6 +508,63 @@ The lab home was deleted and the test entry was removed from the store and verif
That automated spawn case runs against a fake claude, so it asserts the store entry and the launch command and nothing more; the live arms above are what establish that the entry actually suppresses the dialog.
The composer-classification record below observes the same gate from the other side, where an untrusted worktree left Claude, Grok, and Muse unverified because the guard reads a first-launch trust dialog as an unreadable composer.

## Codex hook trust

Verified 2026-09-16 on codex-cli 0.151.0, macOS arm64, in a fresh linked worktree of this repository.

Codex gates hooks it has no persisted trust for behind an interactive modal.
A crewmate launch built by `bin/fm-spawn.sh` was driven under a real PTY and stopped there before the brief was ever submitted:

```text
Hooks need review
12 hooks are new or changed.
Hooks can run outside the sandbox after you trust them.
> 1. Review hooks
2. Trust all and continue
3. Continue without trusting (hooks won't run)
Press enter to confirm or esc to go back
```

The selection starts on "Review hooks", which is neither trusting nor declining, and Firstmate's key plane carries only Enter, Escape, and C-c with no arrow navigation, so the selection cannot be moved.
That count covers every hook Codex had no persisted trust for, drawn from both the machine's own `~/.codex/hooks.json` and this repository's tracked `.codex/hooks.json`.
Writing Codex's own trust store to pre-accept the modal would record an operator consent that was never given, so it is not an option either.

`codex --help` documents `--dangerously-bypass-hook-trust` as "Run enabled hooks without requiring persisted hook trust for this invocation", which RUNS the untrusted hooks.
That is the opposite of what an unattended worker needs, so the control used is the hook feature flag:

```sh
codex features list | grep '^hooks'
codex --disable hooks features list | grep '^hooks'
codex --disable no_such_feature features list
```

```text
hooks stable true
hooks stable false
Error: Unknown feature flag: no_such_feature
```

The last arm is what makes the control safe to depend on: an unknown feature name is a hard error, so a release that renames or drops the flag fails the launch loudly instead of silently restoring the modal.

The same launch with the hook layer disabled reached the composer with no modal, answered the prompt, and fired the turn-end program that rides the launch rather than any hook:

```sh
codex --dangerously-bypass-approvals-and-sandbox --disable hooks \
-c "notify=[\"bash\",\"-c\",\"touch $TURNEND\"]" "Say ACK and stop."
```

```text
> Say ACK and stop.
- ACK, captain.
$ ls "$TURNEND"
<turn-end file present>
```

`tests/fm-codex-hook-layer-live-e2e.test.sh` is the command that refreshes this record.
It captures the launch `bin/fm-spawn.sh` actually builds, replays those exact flags against the installed Codex, and fails naming the harness and version if the hook layer comes back on.
It spends no model tokens, so it runs by default wherever Codex is installed.
The portable half, `tests/fm-spawn-dispatch-profile.test.sh`, pins the split the launch template makes: a crewmate launches hook-free while a secondmate, which runs a primary session on this repository's own project hooks, keeps them.

## Composer classification matrix

The shared composer classifier (`bin/fm-composer-lib.sh`, `fm_composer_classify_screen`) owns every composer shape fleet-wide; each backend contributes only a capture and a capability descriptor.
Expand Down
97 changes: 97 additions & 0 deletions tests/fm-codex-hook-layer-live-e2e.test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
#!/usr/bin/env bash
# Live guard for the codex crewmate launch's hook posture.
#
# The verdict here comes from the installed codex, not from a stub: a stub can
# only confirm the assumption already written into it, and what this guard
# protects is exactly a vendor-owned surface. Codex blocks a fresh crewmate
# launch on an unanswerable "Hooks need review" modal whenever the machine's
# ~/.codex/hooks.json or a project's .codex/hooks.json carries a hook it has no
# persisted trust for, so the crewmate launch disables codex's hook layer
# outright (bin/fm-spawn.sh's launch template owns the flag).
#
# The guard replays the REAL launch flags fm-spawn builds - captured from a
# spawn driven through a fake pane - against the installed codex and asks codex
# itself whether hooks ended up disabled. If a codex release renames or drops
# the feature, the flag becomes a hard "Unknown feature flag" error and this
# guard fails naming the harness and version instead of letting the modal
# silently come back.
#
# It spends no model tokens (`codex features list` resolves configuration only),
# so it runs by default wherever codex is installed.
set -u

# shellcheck source=tests/fixtures.sh
. "$(dirname "${BASH_SOURCE[0]}")/fixtures.sh"

fm_live_gate default-on FM_CODEX_HOOK_LAYER_LIVE codex

CODEX_VERSION=$(codex --version 2>&1)
TMP_ROOT=$(fm_test_tmproot fm-codex-hook-layer-live)

# capture_codex_launch <name> <extra fm-spawn args...>: spawns a codex crewmate
# against a fake pane and echoes the literal launch command firstmate sent.
capture_codex_launch() {
local name=$1
shift
local case_dir home proj wt fakebin launchlog id
case_dir="$TMP_ROOT/$name"
home="$case_dir/home"
proj="$case_dir/project"
wt="$case_dir/wt"
launchlog="$case_dir/launch.log"
id="codex-hook-layer-$name"
fakebin=$(fm_test_make_spawn_fakebin "$case_dir/fake")
fm_test_spawn_home "$home" codex
fm_test_spawn_brief "$home" "$id"
fm_git_worktree "$proj" "$wt" "wt-$name"
: > "$launchlog"
FM_FAKE_LAUNCH_LOG="$launchlog" \
fm_test_run_spawn "$home" "$wt" "$fakebin" "$id" "$proj" "$@" >/dev/null 2>&1 ||
fail "codex $CODEX_VERSION: fm-spawn could not build a crewmate launch"
cat "$launchlog"
}

# codex_global_flags <launch command>: the flags between the codex executable
# and the positional brief, which is everything codex itself is configured by.
codex_global_flags() {
local launch=$1 flags
flags=${launch#*codex }
flags=${flags%%\"\$(*}
printf '%s' "$flags"
}

test_installed_codex_disables_hooks_for_the_captured_crewmate_launch() {
local launch flags state
launch=$(capture_codex_launch ship --mode no-mistakes --yolo off)
flags=$(codex_global_flags "$launch")

# The whole point: every flag firstmate will launch with, handed to the real
# codex, must leave the hook layer off. `features list` reports the effective
# state after those flags are applied and contacts no model.
state=$(eval "codex $flags features list" 2>&1) ||
fail "codex $CODEX_VERSION rejected firstmate's crewmate launch flags: $state"
case "$state" in
*"Unknown feature flag"*)
fail "codex $CODEX_VERSION no longer knows the hook feature firstmate disables: $state"
;;
esac
printf '%s\n' "$state" | awk '$1 == "hooks" { print $NF }' | grep -qx false ||
fail "codex $CODEX_VERSION left hooks enabled for firstmate's crewmate launch flags, so a fresh launch can park on the hook-trust modal"

printf 'ok - codex %s runs a firstmate crewmate launch with its hook layer disabled\n' "$CODEX_VERSION"
}

test_installed_codex_still_reports_the_hook_feature() {
local listing
listing=$(codex features list 2>&1) ||
fail "codex $CODEX_VERSION could not list its feature flags: $listing"
printf '%s\n' "$listing" | awk '{ print $1 }' | grep -qx hooks ||
fail "codex $CODEX_VERSION no longer publishes a hook feature flag; firstmate's crewmate launch needs a new control"

printf 'ok - codex %s still publishes the hook feature flag firstmate disables\n' "$CODEX_VERSION"
}

test_installed_codex_still_reports_the_hook_feature
test_installed_codex_disables_hooks_for_the_captured_crewmate_launch

echo "# all fm-codex-hook-layer-live-e2e tests passed"
46 changes: 46 additions & 0 deletions tests/fm-spawn-dispatch-profile.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,50 @@ test_codex_omits_max_effort_for_unsupported_model() {
pass "codex omits max for models without the catalog capability"
}

# Codex parks a crewmate launch forever on its unanswerable hook-trust modal
# unless the launch turns the hook layer off. These two cases pin the split:
# a crewmate runs hook-free, a secondmate keeps the project hooks that carry its
# own primary-session turn-end guard and session-start digest.
test_codex_crewmate_launch_disables_the_hook_layer() {
local rec id out status launch
id=profile-codex-hooks-z4c
rec=$(make_spawn_case profile-codex-hooks codex "$id")
read_case_record "$rec"

out=$(run_ship_spawn "$HOME_DIR" "$WT_DIR" "$FAKEBIN_DIR" "$LAUNCH_LOG" "$id" "$PROJ_DIR")
status=$?
expect_code 0 "$status" "codex crewmate spawn should succeed"$'\n'"$out"
launch=$(cat "$LAUNCH_LOG")
assert_contains "$launch" "--disable hooks" \
"codex crewmate launch did not disable the hook layer that blocks it on a trust modal"
# The opposite posture: this flag RUNS the untrusted hooks instead of
# disabling them, so a launch must never reach for it.
assert_not_contains "$launch" "--dangerously-bypass-hook-trust" \
"codex crewmate launch ran the operator's untrusted hooks instead of disabling them"
# Firstmate goes blind without the turn-end signal, which rides this same
# launch rather than any hook.
assert_contains "$launch" "notify=" \
"codex crewmate launch lost the turn-end notify program"
pass "a codex crewmate launches with no hook layer and keeps its turn-end signal"
}

test_codex_secondmate_launch_keeps_the_hook_layer() {
local rec id sm out status launch
id=profile-codex-secondmate-hooks-z4d
rec=$(make_spawn_case profile-codex-secondmate-hooks codex "$id")
read_case_record "$rec"
sm="$CASE_DIR/secondmate-home"
make_seeded_secondmate_home "$sm" "$id"

out=$(run_spawn "$HOME_DIR" "$WT_DIR" "$FAKEBIN_DIR" "$LAUNCH_LOG" "$id" "$sm" --secondmate)
status=$?
expect_code 0 "$status" "codex secondmate spawn should succeed"$'\n'"$out"
launch=$(cat "$LAUNCH_LOG")
assert_not_contains "$launch" "--disable hooks" \
"codex secondmate launch disabled the project hooks its own primary supervision depends on"
pass "a codex secondmate keeps the project hook layer its primary session runs on"
}

test_grok_threads_model_and_reasoning_effort() {
local rec id out status launch
id=profile-grok-z5
Expand Down Expand Up @@ -1386,6 +1430,8 @@ test_claude_threads_model_and_effort
test_codex_threads_model_and_effort
test_codex_threads_model_and_max_effort
test_codex_omits_max_effort_for_unsupported_model
test_codex_crewmate_launch_disables_the_hook_layer
test_codex_secondmate_launch_keeps_the_hook_layer
test_grok_threads_model_and_reasoning_effort
test_grok_omits_invalid_max_reasoning_effort
test_grok_omits_invalid_xhigh_reasoning_effort
Expand Down
Loading