Skip to content

feat(bin): add agy subscription routing and in-run model fallback - #19

Closed
adibirzu wants to merge 7 commits into
mainfrom
fm/fm-routing-model-fallback
Closed

adibirzu wants to merge 7 commits into
mainfrom
fm/fm-routing-model-fallback

Conversation

@adibirzu

@adibirzu adibirzu commented Aug 24, 2026 •

Copy link
Copy Markdown
Owner

Intent

Support agy subscription provider and automatic model fallback on depletion

What Changed

  • Added agy as a routable subscription provider: fm-dispatch-select.mjs now includes it in PROVIDERS/NATIVE_PROVIDER (and admits muse/agy as verified harnesses), fm-spawn.sh --provider accepts cursor and agy with the native harness/provider match enforced, and fm-control-lib.sh plus fm-runtime-handoff.sh gained copilot/agy (and cursor/muse/cline) adapter facts — interrupt keys, exit command vs. cline's C-c key exit — while still refusing those adapters for --secondmate.
  • Introduced the modelFallback config object (legacy alias _model_fallback) mapping a harness to its ordered model chain: fm-bootstrap.sh validates the object shape, verified harness keys, and non-empty model-id chains as CREW_DISPATCH diagnostics, and AGENTS.md/docs/configuration.md/docs/architecture.md/the dispatch skill document the post-dispatch relaunch-in-place path through fm-runtime-handoff.sh --model before moving to the next harness lane, plus an agy example rule and chain in docs/examples/crew-dispatch.json.
  • Tightened RATE_LIMIT_RE in fm-dispatch-select.mjs to subscription vocabulary (framed 429, explicit rate limit, named quota/credit/allowance exhaustion) so a bare limit/token/unframed 429 — e.g. "context token limit reached" — no longer parks a provider for a full cooldown, and consolidated the previously contradictory harness-support rosters in docs/configuration.md and AGENTS.md into a single owner.

Risk Assessment

⚠️ Medium: The agy provider support and the new validation/regex work are well-bounded and behavior-tested, but the automatic model-fallback flow this change mandates relies on a relaunch path that silently drops the recorded effort axis and leaves a stale routing provider behind, so the documented depletion response can downgrade reasoning class and cannot record the cooldown after a lane move.

Testing

I ran the five targeted suites covering every file the change touches (fm-dispatch-select, fm-runtime-handoff, fm-control, the crew-dispatch validation test in fm-bootstrap, and the pre-existing fm-agy-harness suite) and they all pass. Because green units alone do not show the feature, I also built and ran an end-to-end operator walkthrough that drives the real shipped scripts against fake tmux/quota-axi/harness CLIs and captures the whole depletion story as a CLI transcript: an operator declares an agy lane plus a modelFallback chain and real bootstrap accepts it while rejecting empty chains, unverified harness keys, blank model ids and the both-spellings conflict with actionable CREW_DISPATCH diagnostics; the selector prices agy on the declared Antigravity pool (fails closed on the spent claude_gpt_5h pool, selects on the healthy gemini_5h pool with native provider identity); a real 429/quota status line records a cooldown while three working-ceiling lines are correctly refused; fm-runtime-handoff.sh then relaunches the same task in the same worktree on the next chain model twice over (3.7 to 3.6 to 3.5) with HEAD, uncommitted work and PR metadata all preserved; and only after that does the cooled-down agy lane fail over to codex, with a clear restoring eligibility. The same three commands replayed against the base commit refuse agy outright and silently ignore every malformed modelFallback chain, which is the fail-before half of the contrast. No test failures, setup problems or flakiness; the two findings I filed are informational observations about fallback observability, not broken tests. The worktree is clean — all evidence lives in the evidence directory.

Evidence: End-to-end agy + model-fallback walkthrough transcript (5 acts, base-commit contrast included)

Source: End-to-end agy + model-fallback walkthrough transcript (5 acts, base-commit contrast included)


================================================================
ACT 1 — the operator declares an agy lane and its model-fallback chain
================================================================
config/crew-dispatch.json is the file an operator edits. Before this change the
`agy` harness was not a routable provider and `modelFallback` did not exist, so
bootstrap would have rejected both. Below: the real bin/fm-bootstrap.sh reading
the real config.

$ cat config/crew-dispatch.json | jq -c .
{"rules":[{"when":"Standard coding work suited for Gemini models on Antigravity.","use":[{"harness":"agy","model":"gemini-3.7-flash-high","effort":"high","quotaWindow":"gemini_5h"}],"why":"Antigravity bills its Gemini and Claude/GPT pools separately."}],"default":[{"harness":"agy","model":"gemini-3.7-flash-high","quotaWindow":"gemini_5h"},{"harness":"codex","provider":"codex","model":"gpt-5.5"}],"modelFallback":{"agy":["gemini-3.7-flash-high","gemini-3.6-flash-high","gemini-3.5-flash-high"],"claude":["claude-sonnet-5","sonnet","haiku"]}}
$ bin/fm-bootstrap.sh
(no CREW_DISPATCH diagnostic — config accepted)

--- and a chain the operator got wrong is refused with an actionable diagnostic, not ignored

$ cat config/crew-dispatch.json | jq -c .
{"default":{"harness":"agy"},"modelFallback":{"agy":[]}}
$ bin/fm-bootstrap.sh
CREW_DISPATCH: invalid config/crew-dispatch.json - modelFallback chain must be a non-empty array of non-empty model ids: agy

$ cat config/crew-dispatch.json | jq -c .
{"default":{"harness":"agy"},"modelFallback":{"spaceship":["a"]}}
$ bin/fm-bootstrap.sh
CREW_DISPATCH: invalid config/crew-dispatch.json - modelFallback has an unverified harness: spaceship

$ cat config/crew-dispatch.json | jq -c .
{"default":{"harness":"agy"},"modelFallback":{"agy":["gemini-3.7-flash-high",""]}}
$ bin/fm-bootstrap.sh
CREW_DISPATCH: invalid config/crew-dispatch.json - modelFallback chain must be a non-empty array of non-empty model ids: agy

$ cat config/crew-dispatch.json | jq -c .
{"modelFallback":{"agy":["a"]},"_model_fallback":{"agy":["b"]}}
$ bin/fm-bootstrap.sh
CREW_DISPATCH: invalid config/crew-dispatch.json - modelFallback and its legacy alias _model_fallback cannot both be declared

--- the documented legacy spelling still loads

$ cat config/crew-dispatch.json | jq -c .
{"default":{"harness":"agy"},"_model_fallback":{"agy":["gemini-3.6-flash-high"]}}
$ bin/fm-bootstrap.sh
(no CREW_DISPATCH diagnostic — config accepted)

================================================================
ACT 2 — agy is dispatchable, priced on the Antigravity pool it draws from
================================================================
quota-axi reports agy with four windows. The Gemini pool is healthy; the
Claude/GPT pool is spent. A profile without quotaWindow is conservatively priced
on the WORST window and must fail closed.

$ quota-axi --json   (fixture)
{"provider":"agy","windows":["gemini_5h=100%","gemini_weekly=93%","claude_gpt_5h=0%","claude_gpt_weekly=41%"]}

--- undeclared window -> priced on the empty claude_gpt_5h pool -> fail closed (exit 3)

$ bin/fm-dispatch-select.mjs select '[{"harness":"agy","model":"gemini-3.7-flash-high"}]'
fm-dispatch-select: candidate provider=agy unavailable: quota headroom 0% is at or below 20% reserve
fm-dispatch-select: no subscription candidate has current dispatch capacity evidence
(exit 3)

--- declared gemini_5h -> agy selected, native provider identity established

$ bin/fm-dispatch-select.mjs select '[{"harness":"agy","model":"gemini-3.7-flash-high","quotaWindow":"gemini_5h"}]'
fm-dispatch-select: candidate provider=agy window=gemini_5h eligible: fresh window gemini_5h headroom=100% reserve=20%
fm-dispatch-select: selection provider=agy basis=least-recent eligible subscription sequence=1
{"harness":"agy","provider":"agy","model":"gemini-3.7-flash-high"}
(exit 0)

--- declared claude_gpt_5h (the spent pool) -> refused by name

$ bin/fm-dispatch-select.mjs select '[{"harness":"agy","model":"claude-sonnet-4-6","quotaWindow":"claude_gpt_5h"}]'
fm-dispatch-select: candidate provider=agy window=claude_gpt_5h unavailable: window claude_gpt_5h headroom 0% is at or below 20% reserve
fm-dispatch-select: no subscription candidate has current dispatch capacity evidence
(exit 3)

--- fm-spawn.sh now accepts agy as a routing provider (and still rejects nonsense)

$ bin/fm-spawn.sh --harness agy --provider agy   (argument gate; stops later on a real requirement)
error: ship spawns require --mode <no-mistakes|direct-PR|local-only>; resolve it at intake from the captain's instruction and the project's registered posture in data/projects.md

$ bin/fm-spawn.sh --harness agy --provider spaceship
error: --provider must be one of claude, codex, grok, cursor, agy

================================================================
ACT 2b - the same three commands run against the BASE commit (before the change)
================================================================

$ (base 3c544d6a1958784845356e62bc09d3dd56ee8a67) fm-spawn.sh --harness agy --provider agy
error: --provider must be one of claude, codex, grok

$ (base) fm-dispatch-select.mjs select agy profile on gemini_5h
fm-dispatch-select: subscription dispatch requires a verified harness, not agy
(exit 2)

--- (base) a broken model-fallback chain is accepted and ignored; after the change it is refused

$ cat config/crew-dispatch.json | jq -c .
{"default":{"harness":"agy"},"modelFallback":{"agy":[]}}
$ (base) bin/fm-bootstrap.sh
(base emits NO diagnostic: the whole modelFallback object is an unknown key it silently ignores)

$ cat config/crew-dispatch.json | jq -c .
{"default":{"harness":"agy"},"modelFallback":{"spaceship":["a"]}}
$ (base) bin/fm-bootstrap.sh
(base emits NO diagnostic: the whole modelFallback object is an unknown key it silently ignores)

$ cat config/crew-dispatch.json | jq -c .
{"modelFallback":{"agy":["a"]},"_model_fallback":{"agy":["b"]}}
$ (base) bin/fm-bootstrap.sh
(base emits NO diagnostic: the whole modelFallback object is an unknown key it silently ignores)

================================================================
ACT 3 — the running agy worker depletes; firstmate records the depletion
================================================================

$ cat state/ship-42.status
failed: 429 Too Many Requests - you have reached your 5-hour Gemini usage limit

$ bin/fm-dispatch-select.mjs record-failure --provider agy --task ship-42
fm-dispatch-select: provider=agy cooldown recorded until epoch 2800
(exit 0)

--- a working ceiling is NOT spent quota — it must not park a healthy lane

$ echo working:\ context\ token\ limit\ reached\;\ compacting > state/benign.status && record-failure --provider agy --task benign
fm-dispatch-select: task status contains no rate-limit or quota-exhaustion evidence
(exit 2)

$ echo failed:\ exceeded\ the\ tool\ output\ limit > state/benign.status && record-failure --provider agy --task benign
fm-dispatch-select: task status contains no rate-limit or quota-exhaustion evidence
(exit 2)

$ echo working:\ applying\ the\ hunk\ at\ line\ 429\ of\ the\ diff > state/benign.status && record-failure --provider agy --task benign
fm-dispatch-select: task status contains no rate-limit or quota-exhaustion evidence
(exit 2)

================================================================
ACT 4 — automatic model fallback: relaunch in place on the next chain entry
================================================================
The chain configured in ACT 1 for harness agy is:
  ["gemini-3.7-flash-high","gemini-3.6-flash-high","gemini-3.5-flash-high"]
The worker was running gemini-3.7-flash-high (chain[0]). Firstmate relaunches the
SAME task, in the SAME worktree, on chain[1] via bin/fm-runtime-handoff.sh --model.

$ cat state/ship-42.meta   (before)
worktree=/var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/wt
harness=agy
model=gemini-3.7-flash-high
effort=high
pr=https://example.test/pr/1
HEAD=ea36c62eace0a0079df0e855e6a1cca1957e6d28  dirty.txt present=yes

$ bin/fm-runtime-handoff.sh ship-42 --harness agy --model gemini-3.6-flash-high --skip-exit --progress-note '...'
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
●  WATCHER DOWN - SUPERVISION IS OFF
●  1 task(s) in flight, but no watcher has a fresh beacon (last beat: never, grace 300s).
●  Trust the emitted supervision protocol for this harness; do not use shell & for watcher repair.
●  This is a supervision warning only; the guarded operation WILL still run.
●  watcher supervision needs Stop-owned automatic recovery; inspect the hook registration and startup status before ending the turn.
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
warning: /var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/home/data/ship-42/brief.md records no delivery contract line (scaffolded before ship briefs recorded one); launching on the explicit --mode no-mistakes - confirm its definition of done matches
spawned ship-42 harness=agy kind=ship mode=no-mistakes yolo=off window=firstmate:fm-ship-42 worktree=/var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/wt
handed-off ship-42 from harness=agy to harness=agy worktree=/var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/wt
(exit 0)

$ cat state/ship-42.meta   (after)
pr=https://example.test/pr/1
worktree=/var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/wt
harness=agy
model=gemini-3.6-flash-high
effort=default
HEAD=ea36c62eace0a0079df0e855e6a1cca1957e6d28  (unchanged: yes)
dirty.txt preserved: yes

$ cat state/ship-42.status   (what status reporting shows)
working: runtime handoff to agy; continue from original brief

$ cat state/ship-42.handoff-prompt   (what the replacement worker is told)
# Runtime handoff - continue this in-flight task

You are replacing a previous worker runtime on the SAME task identity, worktree, and branch.
Preserve every commit and every uncommitted change.
Do not create a new worktree. Do not start over from a clean tree.

## Progress so far

agy gemini-3.7-flash-high 5h pool depleted; falling back to chain[1]. 1 commit + uncommitted work remain.

## Active pipeline


--- walk chain[1] -> chain[2] the same way
WARNING: watcher still down (same stale episode; last beat: never, grace 300s) - full banner already printed this episode.
warning: /var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/home/data/ship-42/brief.md records no delivery contract line (scaffolded before ship briefs recorded one); launching on the explicit --mode no-mistakes - confirm its definition of done matches
spawned ship-42 harness=agy kind=ship mode=no-mistakes yolo=off window=firstmate:fm-ship-42 worktree=/var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/wt
handed-off ship-42 from harness=agy to harness=agy worktree=/var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T//fm-agy-fallback-evidence.mgLAD2/handoff/wt
(exit 0)
harness=agy
model=gemini-3.5-flash-high

================================================================
ACT 5 — chain exhausted: only now does work leave the agy lane
================================================================
agy is on cooldown from the ACT 3 depletion, so the selector routes the next
dispatch to the next harness lane instead of re-picking a spent subscription.

$ bin/fm-dispatch-select.mjs select '[{"harness":"agy","model":"gemini-3.5-flash-high","quotaWindow":"gemini_5h"},{"harness":"codex","model":"gpt-5.5"}]'
fm-dispatch-select: candidate provider=agy unavailable: cooldown until epoch 2800
fm-dispatch-select: candidate provider=codex eligible: fresh quota headroom=90% reserve=20%
fm-dispatch-select: selection provider=codex basis=least-recent eligible subscription sequence=1
{"harness":"codex","provider":"codex","model":"gpt-5.5"}
(exit 0)

--- and once the operator clears the cooldown, agy is a first-class candidate again

$ bin/fm-dispatch-select.mjs clear --provider agy
fm-dispatch-select: provider=agy cooldown cleared

$ bin/fm-dispatch-select.mjs select '[{"harness":"agy","model":"gemini-3.5-flash-high","quotaWindow":"gemini_5h"},{"harness":"codex","model":"gpt-5.5"}]'
fm-dispatch-select: candidate provider=agy window=gemini_5h eligible: fresh window gemini_5h headroom=100% reserve=20%
fm-dispatch-select: candidate provider=codex eligible: fresh quota headroom=90% reserve=20%
fm-dispatch-select: selection provider=agy basis=least-recent eligible subscription sequence=2
{"harness":"agy","provider":"agy","model":"gemini-3.5-flash-high"}
(exit 0)


== end of walkthrough ==
Evidence: Driver script that produced the transcript (runs the real shipped scripts against fake tmux/quota-axi fixtures)

Source: Driver script that produced the transcript (runs the real shipped scripts against fake tmux/quota-axi fixtures)

#!/usr/bin/env bash
# End-to-end operator walkthrough for the intent:
#   "Support agy subscription provider and automatic model fallback on depletion"
#
# Drives the REAL shipped scripts against fake tmux / quota-axi / harness CLIs and
# narrates exactly what an operator sees at each step of a depletion.
set -u
FM_REPO="/Users/adrianb/.no-mistakes/worktrees/80e4cf1781af/01M0SEQE0C1ZFKZDTZWW97YA0Y"
. "$FM_REPO/tests/lib.sh"

TMP_ROOT=$(fm_test_tmproot fm-agy-fallback-evidence)
fm_git_identity
NODE_BIN_DIR=$(dirname "$(command -v node)")
BASE_PATH="$NODE_BIN_DIR:/usr/bin:/bin:/usr/sbin:/sbin"
SELECTOR="$ROOT/bin/fm-dispatch-select.mjs"
STAMP=1970-01-01T00:16:40.000Z

hr()  { printf '\n================================================================\n%s\n================================================================\n' "$1"; }
sub() { printf '\n--- %s\n' "$1"; }
run() { printf '\n$ %s\n' "$*"; }

# ------------------------------------------------------------------ fixtures
mk_toolchain() {  # <dir> -> fakebin with everything bootstrap probes
  local dir=$1 fakebin real_jq
  fakebin=$(fm_fakebin "$dir")
  fm_fake_exit0 "$fakebin" tmux node chrome-devtools-axi gh
  fm_fake_version_tool "$fakebin" lavish-axi FM_FAKE_LAVISH_AXI_VERSION 0.1.46
  fm_fake_version_tool "$fakebin" gh-axi FM_FAKE_GH_AXI_VERSION 0.1.29
  fm_fake_version_tool "$fakebin" quota-axi FM_FAKE_QUOTA_AXI_VERSION 0.1.29
  fm_fake_version_tool "$fakebin" no-mistakes FM_FAKE_NO_MISTAKES_VERSION \
    'no-mistakes version v1.31.2 (fake) 2026-06-27T00:02:18Z'
  cat > "$fakebin/treehouse" <<'SH'
#!/usr/bin/env bash
if [ "${1:-}" = get ] && [ "${2:-}" = --help ]; then
  printf '%s\n' 'Usage: treehouse get [--lease] [--lease-holder <holder>]'
  exit 0
fi
exit 0
SH
  chmod +x "$fakebin/treehouse"
  cat > "$fakebin/tasks-axi" <<'SH'
#!/usr/bin/env bash
[ "${1:-}" != --version ] || { printf '0.2.4\n'; exit 0; }
if [ "${1:-}" = update ] && [ "${2:-}" = --help ]; then
  printf 'usage: tasks-axi update <id> [flags]\n  --body-file <path>\n  --archive-body\n'; exit 0
fi
if [ "${1:-}" = mv ] && [ "${2:-}" = --help ]; then
  printf 'usage: tasks-axi mv <id> [<id>...] --to <path-or-dir>\n'; exit 0
fi
exit 0
SH
  chmod +x "$fakebin/tasks-axi"
  real_jq=$(command -v jq)
  printf '#!/usr/bin/env bash\nexec %q "$@"\n' "$real_jq" > "$fakebin/jq"
  chmod +x "$fakebin/jq"
  printf '%s\n' "$fakebin"
}

validate_config() {  # <label> <json>
  local label=$1 body=$2 dir out
  dir="$TMP_ROOT/cfg-$(printf '%s' "$label" | tr -cd '[:alnum:]')"
  mkdir -p "$dir/home/config"
  printf 'manual\n' > "$dir/home/config/backlog-backend"
  printf '%s\n' "$body" > "$dir/home/config/crew-dispatch.json"
  local fakebin; fakebin=$(mk_toolchain "$dir")
  out=$(PATH="$fakebin:$BASE_PATH" FM_HOME="$dir/home" FM_ROOT_OVERRIDE="$dir/home" \
    FM_FAKE_TREEHOUSE_LEASE_HELP=1 "$ROOT/bin/fm-bootstrap.sh" 2>/dev/null)
  printf '\n$ cat config/crew-dispatch.json | jq -c .\n'
  printf '%s\n' "$body" | jq -c .
  printf '$ bin/fm-bootstrap.sh\n'
  if [ -z "$out" ]; then
    printf '(no CREW_DISPATCH diagnostic — config accepted)\n'
  else
    printf '%s\n' "$out"
  fi
}

select_home=$TMP_ROOT/select/home
mkdir -p "$select_home/state" "$select_home/config"
select_fakebin=$TMP_ROOT/select/fakebin
mkdir -p "$select_fakebin"
QUOTA="$select_home/quota.json"

write_quota() {  # <agy-gemini-5h> <agy-gemini-weekly> <agy-claude-5h> <codex>
  cat > "$QUOTA" <<JSON
{"schemaVersion":3,"generatedAt":"$STAMP","providers":[
  {"provider":"agy","state":{"status":"fresh","stale":false},
   "windows":[
     {"id":"gemini_5h","percentRemaining":$1},
     {"id":"gemini_weekly","percentRemaining":$2},
     {"id":"claude_gpt_5h","percentRemaining":$3},
     {"id":"claude_gpt_weekly","percentRemaining":41}
   ]},
  {"provider":"codex","state":{"status":"fresh","stale":false},
   "windows":[{"id":"all","percentRemaining":$4}]}
]}
JSON
}

sel() {  # <state-file> <profiles-json> <now>
  FM_HOME="$select_home" FM_STATE_OVERRIDE="$select_home/state" \
  FM_CONFIG_OVERRIDE="$select_home/config" FM_DISPATCH_STATE_FILE="$select_home/state/$1" \
  PATH="$select_fakebin:$BASE_PATH" "$SELECTOR" select --quota-json "$QUOTA" --now "$3" "$2"
}

selcmd() {  # <state-file> <profiles-json> <now>
  run "bin/fm-dispatch-select.mjs select '$2'"
  sel "$1" "$2" "$3" 2>&1
  printf '(exit %s)\n' "$?"
}

CHAIN='["gemini-3.7-flash-high","gemini-3.6-flash-high","gemini-3.5-flash-high"]'
CONFIG=$(cat <<'JSON'
{
  "rules": [
    {
      "when": "Standard coding work suited for Gemini models on Antigravity.",
      "use": [
        { "harness": "agy", "model": "gemini-3.7-flash-high", "effort": "high", "quotaWindow": "gemini_5h" }
      ],
      "why": "Antigravity bills its Gemini and Claude/GPT pools separately."
    }
  ],
  "default": [
    { "harness": "agy", "model": "gemini-3.7-flash-high", "quotaWindow": "gemini_5h" },
    { "harness": "codex", "provider": "codex", "model": "gpt-5.5" }
  ],
  "modelFallback": {
    "agy": ["gemini-3.7-flash-high", "gemini-3.6-flash-high", "gemini-3.5-flash-high"],
    "claude": ["claude-sonnet-5", "sonnet", "haiku"]
  }
}
JSON
)

################################################################################
hr "ACT 1 — the operator declares an agy lane and its model-fallback chain"
################################################################################
cat <<'TXT'
config/crew-dispatch.json is the file an operator edits. Before this change the
`agy` harness was not a routable provider and `modelFallback` did not exist, so
bootstrap would have rejected both. Below: the real bin/fm-bootstrap.sh reading
the real config.
TXT
validate_config "accepted" "$CONFIG"

sub "and a chain the operator got wrong is refused with an actionable diagnostic, not ignored"
validate_config "emptychain"   '{"default":{"harness":"agy"},"modelFallback":{"agy":[]}}'
validate_config "badharness"   '{"default":{"harness":"agy"},"modelFallback":{"spaceship":["a"]}}'
validate_config "blankmodel"   '{"default":{"harness":"agy"},"modelFallback":{"agy":["gemini-3.7-flash-high",""]}}'
validate_config "bothspelling" '{"modelFallback":{"agy":["a"]},"_model_fallback":{"agy":["b"]}}'
sub "the documented legacy spelling still loads"
validate_config "legacyalias"  '{"default":{"harness":"agy"},"_model_fallback":{"agy":["gemini-3.6-flash-high"]}}'

################################################################################
hr "ACT 2 — agy is dispatchable, priced on the Antigravity pool it draws from"
################################################################################
cat <<'TXT'
quota-axi reports agy with four windows. The Gemini pool is healthy; the
Claude/GPT pool is spent. A profile without quotaWindow is conservatively priced
on the WORST window and must fail closed.
TXT
write_quota 100 93 0 90
run "quota-axi --json   (fixture)"
jq -c '.providers[] | select(.provider=="agy") | {provider, windows: [.windows[] | "\(.id)=\(.percentRemaining)%"]}' "$QUOTA"

sub "undeclared window -> priced on the empty claude_gpt_5h pool -> fail closed (exit 3)"
selcmd a1.json '[{"harness":"agy","model":"gemini-3.7-flash-high"}]' 1000

sub "declared gemini_5h -> agy selected, native provider identity established"
selcmd a2.json '[{"harness":"agy","model":"gemini-3.7-flash-high","quotaWindow":"gemini_5h"}]' 1000

sub "declared claude_gpt_5h (the spent pool) -> refused by name"
selcmd a3.json '[{"harness":"agy","model":"claude-sonnet-4-6","quotaWindow":"claude_gpt_5h"}]' 1000

sub "fm-spawn.sh now accepts agy as a routing provider (and still rejects nonsense)"
run "bin/fm-spawn.sh --harness agy --provider agy   (argument gate; stops later on a real requirement)"
PATH="$select_fakebin:$BASE_PATH" "$ROOT/bin/fm-spawn.sh" --harness agy --provider agy 2>&1 | head -2
run "bin/fm-spawn.sh --harness agy --provider spaceship"
PATH="$select_fakebin:$BASE_PATH" "$ROOT/bin/fm-spawn.sh" --harness agy --provider spaceship 2>&1 | head -2

################################################################################
hr "ACT 2b - the same three commands run against the BASE commit (before the change)"
#############################################################################

... [837 bytes truncated] ...

model":"gemini-3.7-flash-high","quotaWindow":"gemini_5h"}]' 2>&1
printf '(exit %s)\n' "$?"

base_validate() {  # <label> <json>
  local label=$1 body=$2 dir out fakebin
  dir="$TMP_ROOT/basecfg-$(printf '%s' "$label" | tr -cd '[:alnum:]')"
  mkdir -p "$dir/home/config"
  printf 'manual\n' > "$dir/home/config/backlog-backend"
  printf '%s\n' "$body" > "$dir/home/config/crew-dispatch.json"
  fakebin=$(mk_toolchain "$dir")
  out=$(PATH="$fakebin:$BASE_PATH" FM_HOME="$dir/home" FM_ROOT_OVERRIDE="$dir/home" \
    FM_FAKE_TREEHOUSE_LEASE_HELP=1 bash "$TMP_ROOT/base/bin/fm-bootstrap.sh" 2>/dev/null)
  printf '\n$ cat config/crew-dispatch.json | jq -c .\n'
  printf '%s\n' "$body" | jq -c .
  printf '$ (base) bin/fm-bootstrap.sh\n'
  [ -n "$out" ] && printf '%s\n' "$out" \
    || printf '(base emits NO diagnostic: the whole modelFallback object is an unknown key it silently ignores)\n'
}

sub "(base) a broken model-fallback chain is accepted and ignored; after the change it is refused"
base_validate "emptychain" '{"default":{"harness":"agy"},"modelFallback":{"agy":[]}}'
base_validate "badharness" '{"default":{"harness":"agy"},"modelFallback":{"spaceship":["a"]}}'
base_validate "bothspelling" '{"modelFallback":{"agy":["a"]},"_model_fallback":{"agy":["b"]}}'

################################################################################
hr "ACT 3 — the running agy worker depletes; firstmate records the depletion"
################################################################################
printf 'harness=agy\n' > "$select_home/state/ship-42.meta"
printf 'failed: 429 Too Many Requests - you have reached your 5-hour Gemini usage limit\n' \
  > "$select_home/state/ship-42.status"
run "cat state/ship-42.status"
cat "$select_home/state/ship-42.status"
run "bin/fm-dispatch-select.mjs record-failure --provider agy --task ship-42"
FM_HOME="$select_home" FM_STATE_OVERRIDE="$select_home/state" FM_CONFIG_OVERRIDE="$select_home/config" \
  PATH="$select_fakebin:$BASE_PATH" "$SELECTOR" record-failure --provider agy --task ship-42 --now 1000 2>&1
printf '(exit %s)\n' "$?"

sub "a working ceiling is NOT spent quota — it must not park a healthy lane"
for line in \
  'working: context token limit reached; compacting' \
  'failed: exceeded the tool output limit' \
  'working: applying the hunk at line 429 of the diff'; do
  printf 'harness=agy\n' > "$select_home/state/benign.meta"
  printf '%s\n' "$line" > "$select_home/state/benign.status"
  printf '\n$ echo %q > state/benign.status && record-failure --provider agy --task benign\n' "$line"
  FM_HOME="$select_home" FM_STATE_OVERRIDE="$select_home/state" FM_CONFIG_OVERRIDE="$select_home/config" \
    PATH="$select_fakebin:$BASE_PATH" "$SELECTOR" record-failure --provider agy --task benign --now 1000 2>&1
  printf '(exit %s)\n' "$?"
done

################################################################################
hr "ACT 4 — automatic model fallback: relaunch in place on the next chain entry"
################################################################################
cat <<TXT
The chain configured in ACT 1 for harness agy is:
  $CHAIN
The worker was running gemini-3.7-flash-high (chain[0]). Firstmate relaunches the
SAME task, in the SAME worktree, on chain[1] via bin/fm-runtime-handoff.sh --model.
TXT

CASE_DIR="$TMP_ROOT/handoff"
CASE_HOME="$CASE_DIR/home"; CASE_PROJ="$CASE_DIR/project"; CASE_WT="$CASE_DIR/wt"
mkdir -p "$CASE_HOME/state" "$CASE_HOME/data/ship-42" "$CASE_HOME/config" "$CASE_HOME/projects"
fm_git_worktree "$CASE_PROJ" "$CASE_WT" "fm/ship-42"
printf 'landed work\n' > "$CASE_WT/feature.txt"
git -C "$CASE_WT" add feature.txt && git -C "$CASE_WT" commit -qm 'task work'
printf 'work in progress, never committed\n' > "$CASE_WT/dirty.txt"
printf '# brief for ship-42: wire the invoice export\n' > "$CASE_HOME/data/ship-42/brief.md"

hfakebin=$(fm_fakebin "$CASE_DIR")
cat > "$hfakebin/tmux" <<'SH'
#!/usr/bin/env bash
set -u
case "$*" in
  *"#{pane_current_path}"*) printf '%s\n' "${FM_FAKE_PANE_PATH:-}"; exit 0 ;;
  *"#{pane_current_command}"*) printf '%s\n' "${FM_FAKE_PANE_CMD:-bash}"; exit 0 ;;
  *"list-windows"*) exit 0 ;;
esac
case "${1:-}" in
  capture-pane)
    # agy past-trust idle footer, so the spawn-time project-trust gate clears.
    printf '%s\n' '? for shortcuts                                   Gemini 3.6 Flash - low'
    exit 0 ;;
esac
case "${1:-}" in
  display-message)
    case "$*" in
      *'#{pane_id}'*) printf '%%1\n' ;;
      *'#{pane_current_path}'*) printf '%s\n' "${FM_FAKE_PANE_PATH:-}" ;;
      *) printf '%s\n' "${FM_FAKE_SESSION:-firstmate}" ;;
    esac; exit 0 ;;
  new-window) printf '@9\n'; exit 0 ;;
esac
exit 0
SH
chmod +x "$hfakebin/tmux"
fm_fake_exit0 "$hfakebin" agy claude codex grok cursor cline copilot muse kimi pi pi-signed opencode

export FM_HOME="$CASE_HOME" FM_FAKE_PANE_PATH="$CASE_WT" FM_FAKE_SESSION=firstmate \
       FM_FAKE_PANE_CMD=bash FM_FAKE_WINDOW_PRESENT=0 FM_FAKE_TREEHOUSE_WT="$CASE_WT"
export PATH="$hfakebin:$BASE_PATH"

fm_write_meta "$CASE_HOME/state/ship-42.meta" \
  "window=firstmate:fm-ship-42" "endpoint_task_id=ship-42" "worktree=$CASE_WT" \
  "project=$CASE_PROJ" "harness=agy" "kind=ship" "mode=no-mistakes" "yolo=off" \
  "tasktmp=/tmp/fm-evidence-ship-42" "model=gemini-3.7-flash-high" "effort=high" \
  "pr=https://example.test/pr/1" "pr_head=abc123"

run "cat state/ship-42.meta   (before)"
grep -E '^(harness|model|effort|worktree|pr)=' "$CASE_HOME/state/ship-42.meta"
head_before=$(git -C "$CASE_WT" rev-parse HEAD)
printf 'HEAD=%s  dirty.txt present=%s\n' "$head_before" "$([ -f "$CASE_WT/dirty.txt" ] && echo yes || echo no)"

run "bin/fm-runtime-handoff.sh ship-42 --harness agy --model gemini-3.6-flash-high --skip-exit --progress-note '...'"
set +e
out=$(FM_SPAWN_SETTLE_POLLS=2 FM_AGY_TRUST_POLLS=3 FM_AGY_POLL_INTERVAL=0 "$ROOT/bin/fm-runtime-handoff.sh" ship-42 \
  --harness agy --model gemini-3.6-flash-high --skip-exit \
  --progress-note "agy gemini-3.7-flash-high 5h pool depleted; falling back to chain[1]. 1 commit + uncommitted work remain." 2>&1)
rc=$?
set -e
printf '%s\n(exit %s)\n' "$out" "$rc"

run "cat state/ship-42.meta   (after)"
grep -E '^(harness|model|effort|worktree|pr)=' "$CASE_HOME/state/ship-42.meta"
printf 'HEAD=%s  (unchanged: %s)\n' "$(git -C "$CASE_WT" rev-parse HEAD)" \
  "$([ "$(git -C "$CASE_WT" rev-parse HEAD)" = "$head_before" ] && echo yes || echo NO)"
printf 'dirty.txt preserved: %s\n' "$([ -f "$CASE_WT/dirty.txt" ] && echo yes || echo NO)"
run "cat state/ship-42.status   (what status reporting shows)"
cat "$CASE_HOME/state/ship-42.status" 2>/dev/null || echo '(none)'
run "cat state/ship-42.handoff-prompt   (what the replacement worker is told)"
sed -n '1,12p' "$CASE_HOME/state/ship-42.handoff-prompt" 2>/dev/null || echo '(none)'

sub "walk chain[1] -> chain[2] the same way"
set +e
out=$(FM_SPAWN_SETTLE_POLLS=2 FM_AGY_TRUST_POLLS=3 FM_AGY_POLL_INTERVAL=0 "$ROOT/bin/fm-runtime-handoff.sh" ship-42 \
  --harness agy --model gemini-3.5-flash-high --skip-exit \
  --progress-note "gemini-3.6-flash-high depleted too; chain[2]." 2>&1)
rc=$?
set -e
printf '%s\n(exit %s)\n' "$out" "$rc"
grep -E '^(harness|model)=' "$CASE_HOME/state/ship-42.meta"

################################################################################
hr "ACT 5 — chain exhausted: only now does work leave the agy lane"
################################################################################
cat <<'TXT'
agy is on cooldown from the ACT 3 depletion, so the selector routes the next
dispatch to the next harness lane instead of re-picking a spent subscription.
TXT
PROFILES='[{"harness":"agy","model":"gemini-3.5-flash-high","quotaWindow":"gemini_5h"},{"harness":"codex","model":"gpt-5.5"}]'
selcmd .dispatch-routing.json "$PROFILES" 1001

sub "and once the operator clears the cooldown, agy is a first-class candidate again"
run "bin/fm-dispatch-select.mjs clear --provider agy"
FM_HOME="$select_home" FM_STATE_OVERRIDE="$select_home/state" FM_CONFIG_OVERRIDE="$select_home/config" \
  PATH="$select_fakebin:$BASE_PATH" "$SELECTOR" clear --provider agy 2>&1
selcmd .dispatch-routing.json "$PROFILES" 1002

printf '\n\n== end of walkthrough ==\n'
Evidence: Key excerpt: agy selected on its own pool, then the in-place model fallback
$ bin/fm-dispatch-select.mjs select '[{"harness":"agy","model":"gemini-3.7-flash-high","quotaWindow":"gemini_5h"}]'
fm-dispatch-select: candidate provider=agy window=gemini_5h eligible: fresh window gemini_5h headroom=100% reserve=20%
fm-dispatch-select: selection provider=agy basis=least-recent eligible subscription sequence=1
{"harness":"agy","provider":"agy","model":"gemini-3.7-flash-high"}
(exit 0)

--- same command on base commit 3c544d6 ---
fm-dispatch-select: subscription dispatch requires a verified harness, not agy
(exit 2)

$ cat state/ship-42.status
failed: 429 Too Many Requests - you have reached your 5-hour Gemini usage limit
$ bin/fm-dispatch-select.mjs record-failure --provider agy --task ship-42
fm-dispatch-select: provider=agy cooldown recorded until epoch 2800

$ bin/fm-runtime-handoff.sh ship-42 --harness agy --model gemini-3.6-flash-high --progress-note '...'
spawned ship-42 harness=agy kind=ship mode=no-mistakes yolo=off window=firstmate:fm-ship-42 worktree=.../wt
handed-off ship-42 from harness=agy to harness=agy worktree=.../wt
(exit 0)
meta model: gemini-3.7-flash-high -> gemini-3.6-flash-high -> gemini-3.5-flash-high
HEAD unchanged: yes | dirty.txt preserved: yes | pr= preserved: yes

$ bin/fm-dispatch-select.mjs select '[agy on gemini_5h, codex]' (agy still cooled down)
fm-dispatch-select: candidate provider=agy unavailable: cooldown until epoch 2800
fm-dispatch-select: selection provider=codex basis=least-recent eligible subscription sequence=1
{"harness":"codex","provider":"codex","model":"gpt-5.5"}
- Outcome: ⚠️ 2 infos across 1 run (19m48s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 5 issues (3 warnings, 2 infos)
  • 🚨 AGENTS.md:215 - Intent requires "automatic model fallback on depletion", but that half is delivered only as prose pointing at a config/crew-dispatch.json _model_fallback key that exists nowhere else. AGENTS.md:215 ("Read model fallback chains from config/crew-dispatch.json _model_fallback without hardcoding a duplicate copy") and docs/configuration.md:362 are the only two references in the repo. The field is absent from the canonical schema block at docs/configuration.md:309-327 (a section that declares itself "the single owner of the canonical schema and its per-field semantics"), absent from docs/examples/crew-dispatch.json, and read by no script. It is also unvalidated: crew_dispatch_validate in bin/fm-bootstrap.sh:1046-1090 never enumerates top-level keys, so an operator-authored _model_fallback with a typo or malformed chain is silently ignored instead of reported, contradicting the same file's "malformed configuration must be reported and corrected rather than selected around". As shipped, the fallback behavior has no data source and cannot trigger.
  • ⚠️ bin/fm-dispatch-select.mjs:104 - RATE_LIMIT_RE was widened to include generic reach|exceed paired with limit|token|usage, plus a bare \b429\b. That gate has two consumers with real side effects: the record-failure evidence check (line 588) and the automatic telemetry-evidence cooldown at line 496. A benign status line such as working: context token limit reached; compacting or failed: exceeded the tool output limit now qualifies as verified quota evidence and parks a provider with full headroom for cooldownSeconds (default 1800s), pushing dispatch onto a weaker lane. Coverage is also missing for the change: the new agy test (tests/fm-dispatch-select.test.sh:464) uses 429 Too Many Requests, which the pre-change regex already matched, and there is no negative case asserting benign text is rejected. Recommend anchoring the new alternatives to quota vocabulary (e.g. require quota/credit/usage adjacency rather than bare limit) and adding one positive plus one negative case.
  • ⚠️ AGENTS.md:216 - The new always-loaded line says depletion detection uses pane/status-log errors "for runtimes without telemetry (such as ClinePass and Grok)". Grok does have telemetry: docs/configuration.md:359 in this same commit says "Providers exposed by quota-axi, including Claude, Codex, Grok, Cursor, and agy, require fresh telemetry", and bin/fm-dispatch-select.mjs:82 keeps grok in PROVIDERS so it is priced against the reserve and refused on staleness. An agent following AGENTS.md would drive grok depletion purely off pane text and bypass the record-failure/telemetry contract. cline (ClinePass) is the correct example of a telemetry-free runtime; grok appears to be a mistake.
  • ⚠️ tests/fm-dispatch-select.test.sh:509 - The closing assertion of test_agy_record_failure_and_cooldown accepts agy OR codex, which are the only two candidates in the profile array, so it can never fail. If clear --provider agy regressed to a no-op, agy would remain in cooldown, codex would be selected, and the test would still pass while claiming to prove "cleared agy cooldown did not restore candidate eligibility". Rotation is least-recently-used and agy has not been dispatched at that point, so asserting = agy is deterministic and actually exercises the clear path.
  • ℹ️ bin/fm-runtime-handoff.sh:194 - Beyond agy, this commit admits cursor, muse, cline, and copilot as runtime-handoff targets and wires copilot through the full control-plane fact table (bin/fm-control-lib.sh:66,92,107,119,130,148,160,176,191). The facts match .agents/skills/harness-adapters/SKILL.md, so nothing is invented, but four extra harnesses gain a new exit-then-relaunch lifecycle entry point in a change scoped to "agy subscription provider and automatic model fallback", and tests/fm-runtime-handoff.test.sh:344 only swaps them out of the refusal case rather than adding accept-path coverage. Confirm the widening belongs in this commit.
  • ℹ️ docs/examples/crew-dispatch.json:28 - The new agy example (and the header example at bin/fm-dispatch-select.mjs:53) uses gemini-3.7-flash-high. .agents/skills/harness-adapters/SKILL.md:664 enumerates gemini-3.6-flash-{low,medium,high}, gemini-3.5-flash-*, and gemini-3.1-pro-* for the verified Antigravity CLI 1.1.9, so a copied-verbatim 3.7 id would be rejected at launch and, per that adapter doc, surface only as an agy trust-gate timeout rather than a clean model error. The sibling cursor example carries a "confirm against quota-axi --json before copying this" caveat; the agy why has no equivalent for the model id.
  • ℹ️ AGENTS.md:214 - .agents/skills/quota-array-dispatch/SKILL.md:62 says to "stop and report that the strongest-class choice cannot proceed rather than downgrading it", while AGENTS.md:214 forbids blocking, parking, or escalating a routine depletion and AGENTS.md:218 says the strongest-class rule "still wins over conserving quota". For the common case where the depleted model IS the strongest-class one, the two always-loaded surfaces prescribe opposite actions (relaunch on a weaker chain entry vs stop and report) and nothing in the change resolves which governs.
  • ℹ️ bin/fm-control-lib.sh:97 - fm_control_harness_supports_kind now refuses secondmate for muse|cline|copilot|agy (line 107), but the explanatory comment above it still names only muse and cline. bin/fm-spawn.sh:729,1404 and the harness-adapters doc confirm copilot and agy are crewmate/scout-only; the comment should name them so the refusal is not later read as unexplained.

🔧 Fix: define and validate modelFallback, tighten quota-evidence regex
5 issues (3 warnings, 2 infos) still open:

  • ⚠️ bin/fm-runtime-handoff.sh:355 - The mandated automatic model-fallback invocation silently drops the recorded effort axis. fm-runtime-handoff.sh reads only mode and yolo from meta (lines 166-168) and appends --model/--effort to SPAWN_ARGS only when they were passed on its own command line (354-355). fm-spawn.sh's reuse path drops effort= from the preserved meta (drop_re, bin/fm-spawn.sh:3130) and rewrites effort=${EFFORT:-default} (3176), and effort_flag_for_harness emits nothing for default. So following AGENTS.md:214 / docs/configuration.md:365 verbatim (fm-runtime-handoff.sh &lt;id&gt; --harness &lt;name&gt; --model &lt;next-model&gt;) relaunches a task recorded at effort=max at the harness default — exactly the 'silently downgrading reasoning class' that AGENTS.md:217 forbids two lines later. For agy it is worse than a downgrade: a base chain entry such as gemini-3.6-flash REQUIRES --effort (.agents/skills/harness-adapters/SKILL.md:665), so the relaunch fails and, per that same doc, surfaces only as an agy trust-gate timeout. The competing owner is explicit: the sibling caller of the same spawn reuse path, bin/fm-control.sh:617-620,672,681, reads PRIOR_MODEL/PRIOR_EFFORT from meta and carries them forward unless overridden. Earliest shared boundary: have fm-runtime-handoff.sh default MODEL/EFFORT from the recorded meta the way it already does for mode/yolo (or preserve effort= in the spawn reuse path), rather than documenting the flag at every call site.
  • ⚠️ bin/fm-runtime-handoff.sh:348 - After the cross-harness lane move this change mandates ('move work to the next harness lane only when a harness's whole model chain is exhausted', AGENTS.md:215), the task's recorded routing provider goes stale and the cooldown contract can no longer be honoured. provider= is not in fm-spawn.sh's reuse drop_re (bin/fm-spawn.sh:3130), and handoff never passes --provider, so a task spawned as harness=claude provider=claude becomes harness=agy provider=claude after the lane move. taskMetaProvider then reads provider claude, computes NATIVE_PROVIDER.get(&#39;agy&#39;) === &#39;agy&#39;, and dies with 'task <id> has mismatched native harness and provider metadata' (bin/fm-dispatch-select.mjs:598-599) — exit 2, no cooldown. The exhausted agy lane therefore stays a full-headroom dispatch candidate, defeating the fail-closed capacity contract AGENTS.md:218 says remains enforced. This commit widens reachability by adding agy/cursor to NATIVE_PROVIDER and admitting them as handoff targets, and by making the lane move a routine automatic step. Same fix site as the effort gap: carry the recorded routing identity forward (or clear/re-derive provider= for the new native harness) in the handoff relaunch rather than leaving the previous lane's identity on the record.
  • ⚠️ tests/fm-runtime-handoff.test.sh:608 - The fix round's new accept-path test relaunches on --harness cursor, but the cursor spawn path resolves a real binary: fm_cursor_resolve_binary searches PATH for cursor-agent then agent, then ~/.local/bin/ (bin/fm-cursor-lib.sh:164-185), and fm-spawn.sh:1434 exits 1 when that fails. The fixture stubs a binary named cursor (tests/fm-runtime-handoff.test.sh:115), which no lookup name matches, and PATH keeps the host's entries (line 190). On a host without Cursor Agent CLI installed the test fails with 'handoff from a key-exit harness to cursor should succeed'; on a host with it, the suite shells out to the real Cursor binary in a fixture built entirely on fakes. fm_cursor_verify_executable accepts any executable whose basename is cursor-agent, so adding cursor-agent to the stub list at line 115 makes the test hermetic without changing what it proves.
  • ℹ️ bin/fm-dispatch-select.mjs:110 - The new 429 alternative accepts any of http|status|code|error|response within 16 characters of a bare 429. error is loose enough to keep a false positive alive: a status line such as failed: error at line 429 matches, and both consumers act on it — record-failure (line 602) parks the provider for the full cooldown (default 1800s) with untouched headroom, and providerReadiness (line 383) marks it as quota evidence. The new negative test only covers the unframed variant (working: applying the hunk at line 429 of the diff), so the framed line-number case is uncovered. Dropping bare error (or requiring tight adjacency, e.g. status[ _-]?code[ _-]?429) keeps every accepted case in the new positive table matching, since those rely on status code 429, too many requests, and quota vocabulary.
  • ℹ️ docs/examples/crew-dispatch.json:46 - The shipped example chains, which docs/configuration.md:363 tells operators to copy, contain two questionable entries. &#34;claude&#34;: [&#34;claude-sonnet-5&#34;, &#34;sonnet&#34;, &#34;haiku&#34;] — sonnet is claude's alias for the same model as claude-sonnet-5 (.agents/skills/harness-adapters/SKILL.md:150), so the first fallback hop relaunches the worker on the model that just depleted and burns a full exit-and-relaunch cycle before reaching a genuinely different model. &#34;agy&#34;: [&#34;gemini-3.7-flash-high&#34;, ...] heads with an id agy models does not list (SKILL.md:664 enumerates gemini-3.6-flash-, gemini-3.5-flash-, gemini-3.1-pro-*) while its other two entries are real ids, so the example mixes a launch-failing head with valid tail entries; per that adapter doc a bad agy id surfaces only as a trust-gate timeout. The rule profile above got a 'confirm before copying' caveat; the chain has none of its own.
⚠️ **Test** - 2 infos
  • ℹ️ bin/fm-runtime-handoff.sh:367 - The in-run model-fallback relaunch is not distinguishable in status reporting. bin/fm-runtime-handoff.sh:367 logs only working: runtime handoff to &lt;harness&gt;, so an agy gemini-3.7-flash-high -> gemini-3.6-flash-high fallback and a later gemini-3.6 -> gemini-3.5 fallback produce byte-identical status lines with no model named. AGENTS.md section 4 (added by this change) requires that "Every automatic model switch must be logged and visible in status reporting rather than silently downgrading reasoning class"; the new model id is only observable by reading state/<id>.meta. Confirmed in the captured walkthrough (ACT 4).
  • ℹ️ bin/fm-runtime-handoff.sh:355 - Following the documented fallback procedure literally drops the recorded effort axis. AGENTS.md section 4 instructs relaunching in place "via bin/fm-runtime-handoff.sh with --model"; because bin/fm-runtime-handoff.sh:355 only forwards --effort when explicitly supplied, a task recorded at effort=high comes back as effort=default after the model switch. Observed in the captured walkthrough (ACT 4): meta goes from model=gemini-3.7-flash-high, effort=high to model=gemini-3.6-flash-high, effort=default. This is pre-existing handoff behaviour, but it now sits on the automatic depletion path the change introduces.
  • bash tests/fm-dispatch-select.test.sh — 14 cases pass, including the three new ones (agy per-pool quotaWindow pricing, agy record-failure/cooldown/clear, depletion-evidence gate vs working ceilings)
  • bash tests/fm-runtime-handoff.test.sh — all pass, including the new key-exit-harness -> newly-admitted-target (cline -> cursor) handoff case
  • bash tests/fm-control.test.sh — all pass, including the copilot/agy adapter contract rows and cline/copilot/agy harness-family resolution
  • bash tests/fm-bootstrap.test.sh (test_crew_dispatch_validation) — all crew-dispatch rows pass, including the 9 new agy/modelFallback validation rows
  • bash tests/fm-agy-harness.test.sh — pre-existing agy adapter suite still green (busy regex, composer classification, project-trust gate)
  • Manual end-to-end walkthrough driving the real bin/fm-bootstrap.sh, bin/fm-dispatch-select.mjs, bin/fm-spawn.sh and bin/fm-runtime-handoff.sh against fake tmux/quota-axi/harness CLIs: bash /var/folders/j8/ztw_4_wx3691gww5x_1n9r4c0000gn/T/no-mistakes-evidence/01M0SEQE0C1ZFKZDTZWW97YA0Y/agy-model-fallback-e2e.sh
  • Fail-before/pass-after contrast inside that walkthrough: the same three commands replayed against base commit 3c544d6a1958784845356e62bc09d3dd56ee8a67 (extracted via git archive)
⚠️ **Document** - 1 info
  • ⚠️ docs/configuration.md:268 - docs/configuration.md "Harness support" states the same verified-harness fact twice and the two copies contradict each other. Line 262 says claude/codex/opencode/pi/pi-signed/grok/kimi/cursor are verified for crewmate and secondmate launches with cline and copilot crewmate-only; line 268 repeats the sentence with a different set (drops cursor, omits cline/copilot, names agy crewmate-only). AGENTS.md:190 carries a third variant that lists cline and copilot in the general set even though bin/fm-spawn.sh:1406-1411 refuses all of muse, agy, cline, and copilot for --secondmate. Both duplicates predate this change (line 268 was introduced by 93cc802), so this is a consolidation follow-up: collapse docs/configuration.md to one authoritative sentence covering all crewmate-only adapters and reduce AGENTS.md:190 to match it or point at it. Out of scope here because this change did not touch either line and the fix spans an always-loaded agent contract.

🔧 Fix: consolidate contradictory harness-support rosters into one owner
1 info still open:

  • ℹ️ tests/fm-dispatch-select.test.sh:516 - bin/fm-lint.sh exits 1 on this branch: SC2034 (warning) 'case appears unused' for the local declaration local home fakebin quota out rc case name status. The unused case variable is new on this branch (base commit 3c544d6 has no such declaration at any of its local home fakebin quota out rc ... sites), so the change introduced the lint failure. Not fixable here because this phase is restricted to documentation and doc comments and this is an executable test file; the outer executor's lint phase owns the fix (drop case from the declared locals).
🔧 **Lint** - 1 issue found → auto-fixed ✅
  • ⚠️ linter found issues (exit code 1)

🔧 Fix: drop unused local case in dispatch-select test
✅ Re-checked - no issues remain.

✅ **Push** - passed

✅ No issues found.

@adibirzu adibirzu changed the title feat: support agy routing provider and model fallback on depletion feat(bin): add agy subscription routing and in-run model fallback Aug 24, 2026
@adibirzu

Copy link
Copy Markdown
Owner Author

Superseded by #20, which completes this feature as genuine code per the captain's fix-round ruling (engine + watcher auto-trigger wiring + consumption-boundary validation + docs). All content landed to main via #20.

@adibirzu adibirzu closed this Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant