Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 23 additions & 11 deletions bin/fm-spawn.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5432,30 +5432,39 @@ spawn_record_traceparent() {
return "$status"
}

# Export GOTMPDIR into the crewmate's pane shell so the agent and every child
# process (go build, go test, ...) inherit it. Sent before the launch command so
# the env is set when the agent starts; the brief sleep lets the export land.
spawn_send_text_line "$T" "export GOTMPDIR=$TASK_TMP/gotmp"
# Export the compact-adviser kill switch into the pane shell through the same
# pre-launch channel, so later commands in that shell inherit it too. The launch
# command independently establishes the value for the agent process itself.
spawn_send_text_line "$T" "export COMPACT_ADVISER_DISABLE=1"
# Herdr's pane run returns before zsh has consumed the submitted line. Several
# rapid exports can enter its paste buffer together and lose their separators,
# merging with the later source command. Source the setup with the staged launch
# instead, in the pane shell and outside any env-i wrapper. Other backends retain
# their existing delivery path.
SPAWN_PANE_SETUP=
spawn_export_line() { # <target> <export command>
if [ "$BACKEND" = herdr ]; then
SPAWN_PANE_SETUP+="$2"$'\n'
else
spawn_send_text_line "$1" "$2"
fi
}
# Export GOTMPDIR into the pane shell so the agent and every child inherit it.
spawn_export_line "$T" "export GOTMPDIR=$TASK_TMP/gotmp"
# The launch independently establishes the compact-adviser switch for the agent.
spawn_export_line "$T" "export COMPACT_ADVISER_DISABLE=1"
if [ "$LAVISH_AXI_HOST_CONFIG_PRESENT" = 1 ]; then
spawn_send_text_line "$T" "export LAVISH_AXI_HOST=$(shell_quote "$LAVISH_AXI_HOST")"
spawn_export_line "$T" "export LAVISH_AXI_HOST=$(shell_quote "$LAVISH_AXI_HOST")"
fi
# Mark the pane as a task worker so bin/fm-test-run.sh can refuse to run the
# suite in the repository's primary checkout. Ship and scout workers are the
# ones assigned an isolated worktree; a secondmate runs its own home instead.
# The id reached a validated bare-slug charset above, so it carries no shell
# syntax of its own.
if [ "$KIND" = ship ] || [ "$KIND" = scout ]; then
spawn_send_text_line "$T" "export FM_TASK_ID=$ID"
spawn_export_line "$T" "export FM_TASK_ID=$ID"
fi
# Send through the exact channel that already ships GOTMPDIR, so every backend
# and harness - ship, scout, and secondmate - gets it before launch. Skipped
# entirely when trace context is off.
if [ -n "$SPAWN_TRACEPARENT" ]; then
if spawn_send_text_line "$T" "export TRACEPARENT=$SPAWN_TRACEPARENT"; then
if spawn_export_line "$T" "export TRACEPARENT=$SPAWN_TRACEPARENT"; then
if ! spawn_record_traceparent; then
LAUNCH="unset TRACEPARENT; $LAUNCH"
fi
Expand Down Expand Up @@ -5501,6 +5510,9 @@ if [ "$LAUNCH_ENV_ENABLED" = 1 ]; then
fi
LAUNCH="$LAUNCH_ENV_PREFIX /bin/sh -c $(shell_quote "$LAUNCH")"
fi
# Apply deferred pane exports before the cleared-environment wrapper expands
# its forwarded variables, preserving them in the pane shell after agent exit.
LAUNCH="$SPAWN_PANE_SETUP$LAUNCH"
# Implement the launch-delivery contract in this script's header. The full
# home-identity hash isolates equal task ids across homes, and the spawn token in
# the final filename keeps a buffered source line bound to this incarnation.
Expand Down
2 changes: 1 addition & 1 deletion docs/trace-context.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ This feature adds only that carrier seam.
When enabled, for each spawn Firstmate resolves one W3C `traceparent` carrier for the task - minted as a fresh root on the task's first spawn and reused verbatim from the meta on relaunch - and:

- forms it as `00-<32 hex trace id>-<16 hex span id>-<2 hex flags>`, with random ids for a new root;
- injects it into the agent's pane shell as the `TRACEPARENT` environment variable immediately before launch, through the same `spawn_send_text_line` channel that already ships `GOTMPDIR`; and
- injects it into the agent's pane shell as the `TRACEPARENT` environment variable immediately before launch, through the same pre-launch pane-export channel that already ships `GOTMPDIR` (on `herdr` those exports are prepended to the staged launch file rather than typed as separate lines); and
- records the identical value as `traceparent=` in `state/<id>.meta`.

`TRACEPARENT` as an environment variable is a Firstmate convention carrying a W3C-formatted value: W3C Trace Context standardizes the `traceparent` HTTP header, not an env var, and OpenTelemetry SDKs do not read it from the environment automatically, so a downstream observer must explicitly read this env value or the `traceparent=` meta field.
Expand Down
2 changes: 1 addition & 1 deletion docs/verification/trace-context.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ A final assertion drives the file-decided path (`FM_TRACE_CONTEXT` unset) and pr
The suite touches no real harness or live fleet.
`tests/fm-session-start.test.sh` additionally proves only a lock-owning session start writes the effective state and a lock-refused read-only start leaves it unchanged.

The remote-route suite `tests/fm-remote-secondmate-trace-context.test.sh` (6 assertions) covers the Secondmate path that never reaches the local export site, driving the real chain - the parent's `bin/fm-spawn.sh`, `bin/fm-on.sh`, the real remote entrypoint, `bin/fm-remote-secondmate-control.sh`, and the remote host's own `bin/fm-spawn.sh` - over the deterministic SSH boundary with a stateful fake Herdr CLI, the backend a remote second mate always runs on, so the carrier the remote pane receives is read back from that pane's own log: disabled, the parent records no `traceparent=`, the remote pane receives no export, the remote home inherits no enablement flag, and the delivered snapshot is `FM_TRACE_CONTEXT=off` while `GOTMPDIR` still ships; enabled, the parent's recorded carrier, the remote endpoint's own record, and the exported pane value are one identical valid carrier sent after `GOTMPDIR` and before the launch command, with `FM_TRACE_CONTEXT=on` and the inherited flag delivered; a relaunch keeps that carrier verbatim in both the parent record and the pane export; a second remote route resolved from an environment holding a fixed ambient `TRACEPARENT` roots a trace id distinct from both that ambient carrier and the first route; the remote receiver accepts `config/trace-context` as ordinary declared inherited material while refusing `config/secondmate-harness`, which the primary deliberately does not propagate; and the delivery argument that carries a parent's carrier to a remote host is refused on a ship spawn, on a shell-metacharacter value, on an all-zero trace id, and on an empty value, so nothing but a strict W3C carrier on a Secondmate launch can reach a pane export.
The remote-route suite `tests/fm-remote-secondmate-trace-context.test.sh` (6 assertions) covers the Secondmate path that never reaches the local export site, driving the real chain - the parent's `bin/fm-spawn.sh`, `bin/fm-on.sh`, the real remote entrypoint, `bin/fm-remote-secondmate-control.sh`, and the remote host's own `bin/fm-spawn.sh` - over the deterministic SSH boundary with a stateful fake Herdr CLI, the backend a remote second mate always runs on, so the carrier the remote pane receives is read back from the staged launch file delivered to that pane: disabled, the parent records no `traceparent=`, the remote pane receives no export, the remote home inherits no enablement flag, and the delivered snapshot is `FM_TRACE_CONTEXT=off` while `GOTMPDIR` still ships; enabled, the parent's recorded carrier, the remote endpoint's own record, and the exported pane value are one identical valid carrier exported after `GOTMPDIR` and before the launch command in that same staged file, with `FM_TRACE_CONTEXT=on` and the inherited flag delivered; a relaunch keeps that carrier verbatim in both the parent record and the pane export; a second remote route resolved from an environment holding a fixed ambient `TRACEPARENT` roots a trace id distinct from both that ambient carrier and the first route; the remote receiver accepts `config/trace-context` as ordinary declared inherited material while refusing `config/secondmate-harness`, which the primary deliberately does not propagate; and the delivery argument that carries a parent's carrier to a remote host is refused on a ship spawn, on a shell-metacharacter value, on an all-zero trace id, and on an empty value, so nothing but a strict W3C carrier on a Secondmate launch can reach a pane export.

```console
$ bash tests/fm-trace-context-lib.test.sh | tail -1
Expand Down
101 changes: 99 additions & 2 deletions tests/fm-backend-herdr-launcher-workspace-e2e.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,21 @@ WS_PRIMARY=$(workspace_of_pane "$UNIQA_PANE")
[ "$(focused_workspace)" = "$WS_OTHER" ] || fail "the spawn stole focus from the captain's workspace"
pass "real herdr E2E: with one 'firstmate' workspace and no herdr parent, a crewmate still lands in this home's own workspace without stealing focus"

# Placement and publication are not proof that the command actually executed.
# Rapid pre-launch exports used to merge with the staged source line in zsh,
# leaving a valid task record attached to a shell that never ran the worker.
LAUNCH_CAPTURE=
i=0
while [ "$i" -lt 20 ]; do
LAUNCH_CAPTURE=$(lab pane read "$UNIQA_PANE" --source recent --lines 80 2>/dev/null)
case "$LAUNCH_CAPTURE" in *launcher-ws-ok*) break ;; esac
sleep 0.25
i=$((i + 1))
done
assert_contains_local "$LAUNCH_CAPTURE" launcher-ws-ok \
"the spawned command never executed; pre-launch exports must not merge with the staged source line"
pass "real herdr E2E: the staged worker command actually executes after the pane environment is established"

# --- 2. unique label, WITH a launcher pane: same workspace, now by identity --

read -r _ _ LAUNCH_PRIMARY_PANE <<EOF
Expand Down Expand Up @@ -427,10 +442,92 @@ SM2_WS=$(workspace_of_pane "$SM2_PANE")
|| fail "a --secondmate launch should land in '2ndmate-$SM2_ID', got '$(label_of_workspace "$SM2_WS")'"
pass "real herdr E2E: a --secondmate launch still stands up that secondmate's own workspace instead of inheriting the launcher's"

# --- 7b. pane setup survives an empty launch-env allowlist and trace export -

# An empty allowlist still wraps the launch in `env -i`, which forwards only
# variables already set in the pane shell when it expands. The pane exports must
# therefore land before that wrapper, and must persist in the pane shell after
# the worker exits.
ENV_HOME="$TMP_ROOT/env-home"
mkdir -p "$ENV_HOME/state" "$ENV_HOME/config" "$ENV_HOME/data/envA"
printf 'off\n' > "$ENV_HOME/config/herdr-presentation-spaces"
: > "$ENV_HOME/config/launch-env-allowlist"
write_ship_brief "$ENV_HOME/data/envA/brief.md" envA
SM3_ID="lwsm3"
SM3_HOME="$TMP_ROOT/secondmate-home-3"
mkdir -p "$SM3_HOME/state" "$SM3_HOME/config" "$SM3_HOME/projects" "$SM3_HOME/bin" "$SM3_HOME/data" \
"$ENV_HOME/data/$SM3_ID"
printf 'off\n' > "$SM3_HOME/config/herdr-presentation-spaces"
printf '# scratch secondmate home AGENTS.md placeholder\n' > "$SM3_HOME/AGENTS.md"
printf '%s\n' "$SM3_ID" > "$SM3_HOME/.fm-secondmate-home"
printf 'trivial e2e secondmate charter: nothing to do.\n' > "$SM3_HOME/data/charter.md"
printf '%s\n' 'projects/' 'state/' 'data/' 'config/' '.no-mistakes/' > "$SM3_HOME/.gitignore"
git -C "$SM3_HOME" init -q -b main
printf 'trivial secondmate charter brief: nothing to do.\n' > "$ENV_HOME/data/$SM3_ID/brief.md"
ENV_TRACEPARENT=00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

# The probe reports into a fixture-owned file rather than the pane screen: a
# full trace carrier is wider than the visible pane, so a screen read can see a
# wrapped prefix. Each report is renamed into place, so it is complete once seen.
ENV_PROBE_SCRIPT="$TMP_ROOT/envprobe.sh"
cat > "$ENV_PROBE_SCRIPT" <<'SH'
printf 'envprobe task=%s gotmp=%s cad=%s tp=%s\n' "${FM_TASK_ID:-unset}" "${GOTMPDIR:+set}" \
"${COMPACT_ADVISER_DISABLE:-unset}" "${TRACEPARENT:-unset}" > "$1.tmp" && mv "$1.tmp" "$1"
SH
spawn_env_probe() { # <task-id> <project> [extra fm-spawn args...]
local id=$1 proj=$2
shift 2
env HERDR_ENV=1 HERDR_PANE_ID="$LAUNCH_PRIMARY_PANE" HERDR_SESSION="$HERDR_LAB_SESSION" \
HERDR_SOCKET_PATH="$LAB_SOCKET" \
FM_SPAWN_NO_GUARD=1 FM_HOME="$ENV_HOME" FM_ROOT_OVERRIDE="$ROOT" \
"$ROOT/bin/fm-spawn.sh" "$id" "$proj" "sh '$ENV_PROBE_SCRIPT' '$TMP_ROOT/$id.probe'" \
--backend herdr "$@" >"$TMP_ROOT/$id.out" 2>"$TMP_ROOT/$id.err"
}
wait_probe_file() { # <file> <pane> -> prints the report, or the pane on timeout
local i=0
while [ "$i" -lt 40 ]; do
[ -s "$1" ] && { cat "$1"; return 0; }
sleep 0.25
i=$((i + 1))
done
lab pane read "$2" --source recent --lines 80 2>/dev/null
return 1
}

spawn_env_probe envA "$PROJ" --mode no-mistakes --yolo off \
|| fail "the empty-allowlist ship spawn failed"$'\n'"$(cat "$TMP_ROOT/envA.err")"
ENVA_META="$ENV_HOME/state/envA.meta"
record_worktree "$ENVA_META"
ENVA_PANE=$(grep '^herdr_pane_id=' "$ENVA_META" | cut -d= -f2-)
[ -n "$ENVA_PANE" ] || fail "envA meta is missing herdr_pane_id"
ENVA_LINE=$(wait_probe_file "$TMP_ROOT/envA.probe" "$ENVA_PANE") \
|| fail "the empty-allowlist worker command never executed"$'\n'"$ENVA_LINE"
[ "$ENVA_LINE" = "envprobe task=envA gotmp=set cad=1 tp=unset" ] \
|| fail "an empty allowlist must still forward the pane's task marker, GOTMPDIR, and compact-adviser switch"$'\n'"--- got ---"$'\n'"$ENVA_LINE"
lab pane run "$ENVA_PANE" "printf 'task=%s\\n' \"\${FM_TASK_ID:-unset}\" > '$TMP_ROOT/envA.after.tmp' && mv '$TMP_ROOT/envA.after.tmp' '$TMP_ROOT/envA.after'" >/dev/null 2>&1 \
|| fail "could not probe envA's pane shell after the worker exited"
ENVA_AFTER=$(wait_probe_file "$TMP_ROOT/envA.after" "$ENVA_PANE") \
|| fail "envA's pane shell did not answer the post-exit probe"$'\n'"$ENVA_AFTER"
[ "$ENVA_AFTER" = "task=envA" ] \
|| fail "the task marker must remain set in the pane shell after the worker exits"$'\n'"--- got ---"$'\n'"$ENVA_AFTER"
pass "real herdr E2E: with an empty launch-env allowlist the worker sees its pane setup, and the task marker persists in the pane shell after exit"

spawn_env_probe "$SM3_ID" "$SM3_HOME" --secondmate --traceparent "$ENV_TRACEPARENT" \
|| fail "the empty-allowlist traced secondmate spawn failed"$'\n'"$(cat "$TMP_ROOT/$SM3_ID.err")"
SM3_PANE=$(grep '^herdr_pane_id=' "$ENV_HOME/state/$SM3_ID.meta" | cut -d= -f2-)
[ -n "$SM3_PANE" ] || fail "$SM3_ID meta is missing herdr_pane_id"
SM3_LINE=$(wait_probe_file "$TMP_ROOT/$SM3_ID.probe" "$SM3_PANE") \
|| fail "the traced secondmate command never executed"$'\n'"$SM3_LINE"
[ "$SM3_LINE" = "envprobe task=unset gotmp=set cad=1 tp=$ENV_TRACEPARENT" ] \
|| fail "a traced secondmate under an empty allowlist must receive its pane setup and the trace export"$'\n'"--- got ---"$'\n'"$SM3_LINE"
grep -qx "traceparent=$ENV_TRACEPARENT" "$ENV_HOME/state/$SM3_ID.meta" \
|| fail "the delivered trace carrier was not recorded in the secondmate's task record"
pass "real herdr E2E: a traced secondmate under an empty allowlist receives and records its trace carrier"

# --- 8. teardown closes only the worker's own pane --------------------------

FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$PRIMARY_HOME/state" FM_DATA_OVERRIDE="$PRIMARY_HOME/data" \
FM_CONFIG_OVERRIDE="$PRIMARY_HOME/config" \
FM_HOME="$PRIMARY_HOME" FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$PRIMARY_HOME/state" \
FM_DATA_OVERRIDE="$PRIMARY_HOME/data" FM_CONFIG_OVERRIDE="$PRIMARY_HOME/config" \
"$ROOT/bin/fm-teardown.sh" dupC >"$TMP_ROOT/teardown.out" 2>&1
status=$?
[ "$status" -eq 0 ] || fail "fm-teardown.sh failed for dupC"$'\n'"$(cat "$TMP_ROOT/teardown.out")"
Expand Down
Loading
Loading