Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .agents/skills/harness-adapters/references/harness/omp.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ The optional claude-bridge extension runs a nested executable literally named `c

[`fm-spawn.sh --help`](../../../bin/fm-spawn.sh) owns session posture, worker-only memory scope, and the invariant that the captain's own configuration is never written.
OMP 18.6.1 natively rotates pooled credentials on `usage_limit_reached`; native rotation also applies to review processes launched with extensions disabled.
Workers enable usage-aware selection without a generic percentage reserve or reset wait, and a later per-task overlay supplies only the selected rule's exact model-and-effort fallback chain.
Inspect pooled headroom through `bin/fm-dispatch-capacity.sh`, not quota-axi's single Codex account; the schema and supported cross-harness stand-ins belong to `docs/configuration.md`.
Workers enable usage-aware selection without a generic percentage reserve or reset wait, and a later per-task overlay, `state/<id>.omp-fallback.yml`, supplies only the selected rule's exact model-and-effort fallback chain.
Inspect pooled headroom through `bin/fm-dispatch-capacity.sh`, not quota-axi's single Codex account; the schema, the tracked session overlay's Sol-to-DeepSeek-Flash chain, and supported cross-harness stand-ins belong to `docs/configuration.md`.

## Extension loading

Expand Down
31 changes: 18 additions & 13 deletions .omp/fm-session-overlay.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# unattended worker on an interactive prompt, change its pinned model under it,
# or make its composer unreadable to bin/fm-composer-lib.sh. Verified against
# omp 18.1.11's own settings schema (`omp config list`); the fallback keys against
# omp 18.8.7 by tests/fm-omp-fallback-chain-live-e2e.test.sh.
# omp 18.11.0 by tests/fm-omp-fallback-chain-live-e2e.test.sh.
composer:
# Pins the bare `❯` (U+276F) row for predictable worker rendering.
# omp live-reloads this overlay, so removing the pin also changes sessions
Expand All @@ -27,19 +27,24 @@ retry:
# `confirm` pops a dialog when a usage window runs low; `auto` lets omp fall
# through the fallback chain without a keystroke.
usageReservePolicy: auto
# A session on the strongest class never falls to a weaker model, and fleet
# agents never run on OpenRouter, so the Sol chain is empty: when the
# recorded model fails, the run stops with the provider's error instead of
# moving on. The global chain for this model once continued to a weaker one,
# which stranded a lane there after a provider error burst. omp merges
# overlays key by key and replaces an array whole, so this entry replaces that
# one chain and leaves every other chain as the operator set it. The
# session's live-model record (bin/fm-omp-live-model.ts) carries the stop to
# the supervisor, and a stand-in is chosen at dispatch or relaunch.
# A Sol session whose provider fails moves at once to direct DeepSeek Flash,
# the approved stand-in, and never to an OpenRouter route. With both Codex
# accounts out there is no credential to rotate to, and without a chain omp
# stops at its wait cap instead (a usage window of several days exceeds
# retry.maxDelayMs) and the lane sits dead. The switch happens on any eligible
# provider error, not only on a quota-out. omp merges overlays key by key and
# replaces an array whole, so this entry replaces the global Sol chain and
# leaves every other chain as the operator set it. A worker's per-task
# overlay (state/<id>.omp-fallback.yml, bin/fm-spawn.sh) loads after this one
# and replaces the entry for the worker's own model. The session's live-model
# record (bin/fm-omp-live-model.ts) follows the session to the stand-in.
fallbackChains:
openai-codex/gpt-6.1-sol: []
# omp's own way back to the recorded model, for every chain that remains: the
# next prompt after the primary's suppression window ends restores it.
openai-codex/gpt-6.1-sol:
- deepseek/deepseek-v4-flash:high
# omp's own way back to the recorded model, for every chain including the Sol
# one above: the next prompt after the primary's suppression window ends
# restores it, so a Sol session that moved to Flash returns to Sol once its
# window is over.
# `never` would leave a session on its fallback until someone changes the
# model by hand.
fallbackRevertPolicy: cooldown-expiry
2 changes: 1 addition & 1 deletion bin/fm-crew-state.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
# compared only when the launch recorded an explicit provider/id.
# run-error: <message>
# the session's last run ended in an error that no retry or fallback
# recovered, so a session that stopped rather than downgrade says so.
# recovered, so a session that stopped on an error says so.
# model-stale: <model> on <harness> is older than <newer> in the <harness> catalog
# any harness: the launch was told, at spawn, that a newer model of the
# same family was in its catalog (bin/fm-model-freshness.sh owns the rule).
Expand Down
10 changes: 6 additions & 4 deletions bin/fm-spawn.sh
Original file line number Diff line number Diff line change
Expand Up @@ -310,10 +310,12 @@
# .omp/fm-session-overlay.yml through --config. That overlay pins composer
# shape, plan mode off, prewalk off, and the non-interactive usage-reserve
# policy for the one session only (--auto-approve alone owns approval,
# forcing tools.approvalMode: yolo for the session). It also empties the Sol
# model's fallback chain, so a failing Sol run stops and reports through the
# live-model record instead of moving to a weaker model, and it pins omp's own
# return to the recorded model once the primary serves again.
# forcing tools.approvalMode: yolo for the session). It also chains the Sol
# model to direct DeepSeek Flash, so a Sol session whose provider fails, a
# usage window that is out included, moves to the stand-in at once (a worker's
# per-task state/<id>.omp-fallback.yml replaces that entry for its own model),
# and it pins omp's own return to the recorded model once the primary serves
# again.
# Each session publishes the model it is serving through
# bin/fm-omp-live-model.ts (a crewmate or scout from its per-task extension, a
# secondmate from the home's tracked extension), and bin/fm-crew-state.sh
Expand Down
10 changes: 9 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1687,8 +1687,16 @@ A recorded rule still contains the task's profile when typed resolution raised i
A recorded identifier that no longer exists, or no longer contains the task's profile, counts as absent.
A task that switched to a stand-in stays on it until it is launched again from the matrix.
Without an identifier, identical matching lists apply, and differing matching lists permit no fallback.
OMP workers receive a later per-task configuration overlay with exact model-and-effort chains, preventing ambient chains from weakening the task.
OMP workers receive a later per-task configuration overlay, `state/<id>.omp-fallback.yml`, with exact model-and-effort chains, preventing ambient chains from weakening the task.
Those chains contain only the order-preserving eligible native prefix; the first supported cross-harness destination ends the prefix and leaves the transition to recovery.
The tracked session overlay `.omp/fm-session-overlay.yml` chains `openai-codex/gpt-6.1-sol` to direct `deepseek/deepseek-v4-flash:high` for every omp launch, so a Sol session moves to the stand-in on any eligible provider error, a quota-out included, and `retry.fallbackRevertPolicy: cooldown-expiry` returns it to Sol once the suppression window ends.
A worker's per-task chain replaces that entry for its own model, and is empty when the rule names no eligible native stand-in.
Dispatch-time selection and the overlay chain are separate mechanisms.
Dispatch-time selection keeps the strongest-reasoning class, and a tight choice with no supported equal-class stand-in stops and is reported.
The overlay chain is a run-time provider-error fallback inside one session, not a dispatch decision, and it is a standing exception to that rule for omp Sol sessions.
Its destination is the direct `deepseek` provider only, never an API-billed router such as OpenRouter.
A lane that fell back runs on a lower reasoning class until omp returns it to Sol when the suppression window ends, or until the lane is launched again.
Narrowing the exception means editing the one chain key, `retry.fallbackChains."openai-codex/gpt-6.1-sol"`, in `.omp/fm-session-overlay.yml`.
Native OMP account rotation precedes model fallback, and successful native model switches publish the serving selector.
An idle terminal quota error after native rotation is handled by the existing bounded recovery scan through `fm-control.sh relaunch`, retaining its pause, captain-call, generation, and work-preservation guards.
Launch and successful replacement append the selected route to task status.
Expand Down
32 changes: 18 additions & 14 deletions docs/verification/runtime-backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -1730,22 +1730,25 @@ Additional isolated live validation on 2026-10-09, with omp 18.8.1 on Herdr 0.9.
After the bounded wait, the extension submitted the wake alone, omp consumed the queued follow-up, and the editor retained exactly the unsent draft.
The before-change extension also reproduced the stranded older-wiring wake: it remained above the draft until the live check timed out; the changed extension delivered it and left exactly the draft.

### 2026-10-09 omp fallback chain
### 2026-10-16 omp fallback chain

Verified on 2026-10-09 on macOS arm64 against omp 18.8.7, with the built-in `openai-codex` and `deepseek` providers pointed at scripted local servers through an isolated agent directory's `models.yml`, so no model tokens were spent.
The agent directory's `config.yml` carries the operator's Sol chain as it stands, `openai-codex/gpt-6.1-sol` to `deepseek/deepseek-v4-pro`, and one other chain, `deepseek/deepseek-v4-pro` to a lab-only route.
Verified on 2026-10-16 on macOS arm64 against omp 18.11.0, with the built-in `openai-codex` and `deepseek` providers pointed at scripted local servers through an isolated agent directory's `models.yml`, so no model tokens were spent.
The agent directory's `config.yml` carries the operator's Sol chain as it stood on 2026-10-09, `openai-codex/gpt-6.1-sol` to `deepseek/deepseek-v4-pro`, and one other chain, `deepseek/deepseek-v4-pro` to a lab-only route.
The guard drives `omp -p` with the tracked session overlay `.omp/fm-session-overlay.yml`, the overlay every Firstmate-launched omp session carries.

- **The incident reproduces without the overlay.**
With the primary refusing, the global chain moved the session on to the weak model, which answered.
That is how a lane recorded on Sol ended on `deepseek-v4-flash` on 2026-10-09: the primary hit its usage limit, then the equal refused for credits.
- **The overlay empties the Sol chain.**
With the overlay, the weak model received no request, the run exited non-zero with the primary's rate-limit error, and the live-model record named the recorded model and carried the error.
omp merges `--config` overlays key by key and replaces an array whole, so the overlay's empty Sol entry replaced the global Sol chain and left every other chain alone: with the overlay, a session on the other chain's model still moved to that chain's route when its model refused.
Fleet agents run on no OpenRouter route, so the equal is not a chain hop; a stand-in is chosen at dispatch or relaunch.
That is how a lane recorded on Sol ended on `deepseek-v4-flash` through OpenRouter on 2026-10-09: the primary hit its usage limit, then the equal refused for credits.
- **The overlay chains Sol to direct DeepSeek Flash.**
With a scripted `usage_limit_reached` answer and a retry-after of one hour, far above `retry.maxDelayMs`, the session moved to `deepseek/deepseek-v4-flash`, the run succeeded, and the live-model record followed the session to the stand-in.
The guard requires a request to reach Flash, so an empty Sol chain fails it.
The stop an empty chain caused in production on 2026-10-16, when both Codex accounts were out, is not exercised by the guard: omp stopped at its wait cap, `retry.maxDelayMs`, with the provider's usage-limit error.
omp merges `--config` overlays key by key and replaces an array whole, so the overlay's Sol entry replaced the global Sol chain: the global chain's weak model received no request, and a session on the other chain's model still moved to that chain's route when its model refused.
- **A 500 moves the session too.**
A 500 from a provider that still has quota also moved the session to the stand-in and the record followed it, so the chain applies on any eligible provider error, not only on a quota-out.
- **omp returns to the recorded model by itself.**
`retry.fallbackRevertPolicy: cooldown-expiry` restores the original selector at the next prompt once its suppression window ends; it never steps back to an intermediate route, and a primary that is out of quota for days stays suppressed for days.
The guard layers a lab-only chain after the tracked overlay (a later `--config` wins) so a fallback can happen at all.
The guard layers a lab-only chain after the tracked overlay (a later `--config` wins) so the fallback route answers slowly enough for the primary's window to end.
A refusal classed as a concurrency limit is suppressed for 5 seconds, so the guard's second prompt ran on the primary again and the record followed.
The overlay pins the policy, because a global `never` left the same session on its fallback in the control run.
- **A provider's own short `retry-after` never reaches the chain.**
Expand All @@ -1759,11 +1762,12 @@ FM_OMP_FALLBACK_LIVE=1 tests/fm-omp-fallback-chain-live-e2e.test.sh
```

```text
ok - omp (omp/18.8.7): control - the global Sol chain alone still falls to the weak model
ok - omp (omp/18.8.7): the overlay empties the Sol chain, the run stops with the provider error, and the record carries it
ok - omp (omp/18.8.7): the overlay leaves another model's global chain in place
ok - omp (omp/18.8.7): with the overlay a fallen-back session returns to its recorded model at the next prompt even under a global fallbackRevertPolicy: never
ok - omp (omp/18.8.7): control - without the overlay a global fallbackRevertPolicy: never keeps the session on the fallback
ok - omp (omp/18.11.0): control - the global Sol chain alone still falls to the weak model
ok - omp (omp/18.11.0): the overlay moves a Sol session whose usage window is out to DeepSeek Flash, and the record follows
ok - omp (omp/18.11.0): the overlay moves a Sol session whose provider answers 500 to DeepSeek Flash, and the record follows
ok - omp (omp/18.11.0): the overlay leaves another model's global chain in place
ok - omp (omp/18.11.0): with the overlay a fallen-back session returns to its recorded model at the next prompt even under a global fallbackRevertPolicy: never
ok - omp (omp/18.11.0): control - without the overlay a global fallbackRevertPolicy: never keeps the session on the fallback
```

### 2026-10-10 agy composer through Herdr
Expand Down
Loading
Loading