Skip to content

feat: centralize fleet model selection in a role-based index - #13

Merged
MrGTV-love merged 33 commits into
mainfrom
fm/fm-model-index
Oct 7, 2026
Merged

MrGTV-love merged 33 commits into
mainfrom
fm/fm-model-index

Conversation

@MrGTV-love

@MrGTV-love MrGTV-love commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Intent

Question: "If were able to standardize, would it be better to have a standard model index so we did not need to make multiople updates for every model change?"

Context given with the decision page: on 2026-09-29 switching from gpt-6-sol to gpt-6.1-sol needed hand edits in config/crew-dispatch.json (2 model lines and 4 rule texts naming the id), the 7 lane-home copies of that file, memory and rule prose, and Vernant's separate product registry (scripts/model_registry.json, handled in Vernant GRE-1872). The captain's same-day rulings: "use 6.1 sol instead of 6.0 sol. 6.1 released today and it is smarter and more efficient than its predecessor" and "for the purposes of model alignment, gpt-6.1-sol should be considered a suitable alternative to opus 5.5". Standing rule: a retired or prior-generation model id is a hallucination alarm.

Answer chosen on the decision page: "Model index: A-fleet-index". Option A as presented: one fleet model index maps roles to current ids (for example opus-grade: Claude opus[1m] or Codex gpt-6.1-sol; sonnet-grade: claude-sonnet-5-5[1m], stand-in GLM 5.3; easy: gpt-6-luna, stand-in GLM 5.3 Flash); dispatch profiles name the role, never the id, and the spawn step looks the id up at launch; rules and memory say the role; a change to the index is checked against each harness's live catalog and refused if an id is absent; the index propagates to lane homes like other inheritable config; Vernant keeps its own product registry, checked against the index's retired list. Next model release: one edit to the index.

What Changed

  • Add config/model-index.json and fm-model-index.sh for per-harness model roles, explicit stand-ins, retired-model rejection, and read-only registry checks against the retired list.
  • Resolve roles across dispatch, spawn, and secondmate restart/relaunch paths, with account-aware catalog checks that reject known absent or retired models and report unavailable catalog evidence without blocking launches.
  • Stage and validate the model index and dispatch configuration together for local and remote secondmate inheritance, preserve existing routing files when validation or destination guards refuse updates, and retain remote reread retries. Update operator documentation, role-based examples, and regression coverage.

Risk Assessment

⚠️ Medium: The change spans fleet model selection, account-sensitive catalog checks, and local/remote replacement and propagation lifecycles, but the reviewed safeguards are coherent and no concrete material defect or intent contradiction was substantiated.

Testing

Public CLI, inheritance, receiver, and private-tmux launch scenarios passed, as did focused regressions after correcting disposable socket/config/root setup; evidence captures emitted commands, diagnostics, persisted configuration, metadata, and mutation timing. Workers remained at trust dialogs, so no worker model turn is claimed; default-secondmate replacement was not live-exercisable within the workspace boundary. Remote receiver checks were direct, not SSH lifecycle checks. All fixtures were removed; no full suite, lint, formatter, or static analysis ran.

  • Live validation: ✅ go - 13 of 14 scenarios driven live against the product
Scenario Result Live Evidence
Edit only the index and resolve unchanged dispatch profiles to new models ✅ pass live model-selection-live.json: before/after model and profiles commands, with identical dispatch hashes.
Check a model against the installed Claude harness's native catalog ✅ pass live model-selection-live.json: real SDK initialization and native check commands.
Refuse absent catalog entries and retired selections without inventing unavailable evidence ✅ pass live model-selection-live.json: readable, missing, malformed, and concatenated catalog cases; retired literal and alias-target cases.
Check a separate product registry for retired identifiers without modifying it ✅ pass live model-selection-live.json: check-registry output and before/after registry hashes.
Reject malformed index and dispatch envelopes before selecting a profile ✅ pass live model-selection-live.json and typed-intake-guards.json.
Propagate an edited routing pair and resolve the destination's roles ✅ pass live propagation-live.json: shared-valid persisted files and destination profiles/model output.
Withhold incoherent or malformed routing pairs while continuing unrelated inheritance ✅ pass live propagation-live.json: removed-role, missing-index, empty-dispatch, and concatenated-dispatch shared API and public config-push cases.
Respect destination filesystem guards and preserve documented local link behavior ✅ pass live propagation-live.json: tracked/nonregular destination guards and local source/destination link cases.
Mirror absence of both routing sources without requiring jq ✅ pass live propagation-live.json: shared-both-absent-without-jq.
Apply routing payloads through the real receiver and refuse unsafe transfers ✅ pass live propagation-live.json: direct receiver put/check/absent commands and persisted state.
Launch real workers using roles and stand-ins, then launch again after one index edit ✅ pass live launch-live.json and launch-live.text: live-role, live-edit, and live-standin.
Mutate the source index after selection and retain the staged launch generation ✅ pass live launch-live.json: concurrent_source_mutation and synchronized staging/edit timestamps.
Refuse a catalog-absent spawn or replacement before creating or stopping an agent ✅ pass live launch-live.json and launch-live.text: prelaunch and pre-stop refusal records.
Replace a default-model local secondmate despite unsafe routing sources ⏸️ untested no Actual public launch attempts rejected both a home nested under the active lab and a sibling fixture nested under the source root. Every permitted fixture location is inside that root, while the produ…
Evidence: Model selection, native catalog, retirement, and envelope evidence

Source: Model selection, native catalog, retirement, and envelope evidence

{
  "assignment": "Manual live public model-selection CLI scenarios; no unit tests, suite, build, lint, formatter, source edits, operator Codex data, prompt, or session persistence.",
  "recorded_at": "2026-10-07T07:41:14.274495+00:00",
  "public_cli": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/bin/fm-model-index.sh",
  "public_cli_sha256": "f2b4d9b608330c02538352729eadc7a8cace175ea967bd60cd73945a83e1176e",
  "artifact": "~/.no-mistakes/evidence/01M4A2J6ENDRSX124XXEZ5XC1T/model-selection-live.json",
  "fixture_root": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
  "fixture_data": {
    "initial_index": {
      "version": 1,
      "roles": {
        "selection": {
          "claude": {
            "model": "current-primary-v1",
            "stand_in": "current-stand-in-v1"
          }
        }
      },
      "retired": [
        "retired-model"
      ]
    },
    "one_edited_index": {
      "version": 1,
      "roles": {
        "selection": {
          "claude": {
            "model": "current-primary-v2",
            "stand_in": "current-stand-in-v2"
          }
        }
      },
      "retired": [
        "retired-model"
      ]
    },
    "unchanged_dispatch": {
      "rules": [
        {
          "when": "primary",
          "use": {
            "harness": "claude",
            "role": "selection",
            "effort": "high"
          }
        },
        {
          "when": "alternate",
          "use": [
            {
              "harness": "claude",
              "role": "selection",
              "stand_in": true,
              "effort": "low"
            }
          ]
        }
      ],
      "default": {
        "harness": "claude",
        "role": "selection"
      }
    },
    "registry_fixtures": {
      "retired-values-and-keys": {
        "models": [
          "retired-model",
          "provider/retired-model[1m]",
          "current-primary-v2"
        ],
        "retired-model": {
          "description": "Contains retired-model in prose only"
        }
      },
      "prose-only": {
        "description": "This retired-model appears in prose and should not match.",
        "notes": [
          "Use retired-model only as historical prose",
          "Not a current retired-model selection"
        ]
      },
      "current-only": {
        "model": "current-primary-v2",
        "current-stand-in-v2": {
          "model": "provider/current-primary-v2[1m]"
        }
      }
    },
    "native_proof_index": {
      "version": 1,
      "roles": {
        "native-proof": {
          "claude": {
            "model": "default",
            "stand_in": "fm-live-proof-absent-01M4A2J6ENDRSX124XXEZ5XC1T"
          }
        }
      },
      "retired": []
    }
  },
  "dispatch_read_only_assertion": {
    "before_sha256": "9f4608a73122bb3589f328ad24713fb7ada2175b8272ef1b320d8b474a11e271",
    "after_sha256": "9f4608a73122bb3589f328ad24713fb7ada2175b8272ef1b320d8b474a11e271",
    "passed": true
  },
  "registry_read_only_assertions": [
    {
      "path": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture/registry-retired-values-and-keys.json",
      "before_sha256": "77b3efca37d3f9313b80230a756d0d814f840c241057eaa6fcd578eb3ff0b34a",
      "after_sha256": "77b3efca37d3f9313b80230a756d0d814f840c241057eaa6fcd578eb3ff0b34a",
      "passed": true
    },
    {
      "path": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture/registry-prose-only.json",
      "before_sha256": "a085f4346e7f5f9d5a22db55e1420bbfadb9e42d151fc774f0b47bc5d35e8e05",
      "after_sha256": "a085f4346e7f5f9d5a22db55e1420bbfadb9e42d151fc774f0b47bc5d35e8e05",
      "passed": true
    },
    {
      "path": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture/registry-current-only.json",
      "before_sha256": "ab45784990858873752957e19b90cca66f66f259122d2d4fd343dd00fc304ed2",
      "after_sha256": "ab45784990858873752957e19b90cca66f66f259122d2d4fd343dd00fc304ed2",
      "passed": true
    }
  ],
  "native_catalog_proof": {
    "installed_binary": "~/.local/bin/claude",
    "method": "Claude SDK initialize control request with no prompt, normal inherited login, --no-session-persistence, --setting-sources empty; live public checker performed its own native discovery with catalog export override unset.",
    "listed_selectors": [
      "default",
      "opus",
      "claude-fable-5-1[1m]",
      "sonnet",
      "haiku"
    ],
    "selected_id": "default",
    "synthetic_absent_id": "fm-live-proof-absent-01M4A2J6ENDRSX124XXEZ5XC1T",
    "absent_from_captured_native_ids": true
  },
  "scenarios": [
    {
      "scenario": "role-model-before-index-edit",
      "command": [
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/bin/fm-model-index.sh",
        "model",
        "claude",
        "role:selection"
      ],
      "cwd": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
      "environment": {
        "FM_HOME": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
        "FM_MODEL_CATALOG_DIR": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture/catalogs",
        "fleet_overrides": "all inherited FM_* variables removed"
      },
      "stdin": null,
      "duration_seconds": 0.036,
      "result": {
        "exit_code": 0,
        "stdout": "current-primary-v1\n",
        "stderr": ""
      },
      "assertions": [
        {
          "name": "expected exit code",
          "expected": 0,
          "actual": 0,
          "passed": true
        },
        {
          "name": "stdout contains expected text",
          "expected": "current-primary-v1\n",
          "passed": true
        },
        {
          "name": "scenario output assertion",
          "passed": true
        }
      ],
      "passed": true
    },
    {
      "scenario": "stand-in-model-before-index-edit",
      "command": [
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/bin/fm-model-index.sh",
        "model",
        "claude",
        "stand-in:selection"
      ],
      "cwd": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
      "environment": {
        "FM_HOME": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
        "FM_MODEL_CATALOG_DIR": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture/catalogs",
        "fleet_overrides": "all inherited FM_* variables removed"
      },
      "stdin": null,
      "duration_seconds": 0.039,
      "result": {
        "exit_code": 0,
        "stdout": "current-stand-in-v1\n",
        "stderr": ""
      },
      "assertions": [
        {
          "name": "expected exit code",
          "expected": 0,
          "actual": 0,
          "passed": true
        },
        {
          "name": "stdout contains expected text",
          "expected": "current-stand-in-v1\n",
          "passed": true
        },
        {
          "name": "scenario output assertion",
          "passed": true
        }
      ],
      "passed": true
    },
    {
      "scenario": "offline-profiles-before-index-edit",
      "command": [
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/bin/fm-model-index.sh",
        "profiles",
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture/crew-dispatch.json"
      ],
      "cwd": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixt

... [59825 bytes truncated] ...

 endpoints\\\"), or answer \\\"where is X defined / which files reference Y.\\\" Do NOT use it for code review, design-doc auditing, cross-file consistency checks, or open-ended analysis \u2014 it reads excerpts rather than whole files and will miss content past its read window. When calling, specify search breadth: \\\"quick\\\" for a single targeted lookup, \\\"medium\\\" for moderate exploration, or \\\"very thorough\\\" to search across multiple locations and naming conventions.\",\"model\":\"inherit\"},{\"name\":\"general-purpose\",\"description\":\"General-purpose agent for researching complex questions, searching for code, and executing multi-step tasks. When you are searching for a keyword or file and are not confident that you will find the right match in the first few tries use this agent to perform the search for you.\"},{\"name\":\"Plan\",\"description\":\"Software architect agent for designing implementation plans. Use this when you need to plan the implementation strategy for a task. Returns step-by-step plans, identifies critical files, and considers architectural trade-offs.\",\"model\":\"inherit\"},{\"name\":\"statusline-setup\",\"description\":\"Use this agent to configure the user's Claude Code status line setting.\",\"model\":\"sonnet\"}],\"output_style\":\"default\",\"available_output_styles\":[\"default\",\"Proactive\",\"Concise\",\"Explanatory\",\"Learning\"],\"user_output_styles_dir\":\"~/.claude/output-styles\",\"models\":[{\"value\":\"default\",\"resolvedModel\":\"claude-opus-5-5\",\"displayName\":\"Default (recommended)\",\"description\":\"Opus 5.5 \u00b7 Best for everyday, complex tasks\",\"supportsEffort\":true,\"supportedEffortLevels\":[\"low\",\"medium\",\"high\",\"xhigh\",\"max\"],\"supportsAdaptiveThinking\":true,\"supportsFastMode\":true,\"supportsAutoMode\":true},{\"value\":\"opus\",\"resolvedModel\":\"claude-opus-5-5\",\"displayName\":\"Opus\",\"description\":\"Opus 5.5 \u00b7 Best for everyday, complex tasks\",\"supportsEffort\":true,\"supportedEffortLevels\":[\"low\",\"medium\",\"high\",\"xhigh\",\"max\"],\"supportsAdaptiveThinking\":true,\"supportsFastMode\":true,\"supportsAutoMode\":true},{\"value\":\"claude-fable-5-1[1m]\",\"resolvedModel\":\"claude-fable-5-1\",\"displayName\":\"Fable\",\"description\":\"Fable 5.1 \u00b7 Most capable for your hardest and longest-running tasks\",\"supportsEffort\":true,\"supportedEffortLevels\":[\"low\",\"medium\",\"high\",\"xhigh\",\"max\"],\"supportsAdaptiveThinking\":true,\"supportsAutoMode\":true},{\"value\":\"sonnet\",\"resolvedModel\":\"claude-sonnet-5-5\",\"displayName\":\"Sonnet\",\"description\":\"Sonnet 5.5 \u00b7 Efficient for routine tasks\",\"supportsEffort\":true,\"supportedEffortLevels\":[\"low\",\"medium\",\"high\",\"xhigh\",\"max\"],\"supportsAdaptiveThinking\":true,\"supportsAutoMode\":true},{\"value\":\"haiku\",\"resolvedModel\":\"claude-haiku-4-5-20251001\",\"displayName\":\"Haiku\",\"description\":\"Haiku 4.5 \u00b7 Fastest for quick answers\"}],\"account\":{\"email\":\"charles@greentreetherapeutics.com\",\"organization\":\"charles@greentreetherapeutics.com's Organization\",\"subscriptionType\":\"Claude Max\",\"apiProvider\":\"firstParty\"},\"pid\":93144,\"current_permission_mode\":\"default\",\"feedback_mode\":{\"kind\":\"post\"},\"analytics_disabled\":false,\"remote_control_auto_enable\":false,\"remote_control_auto_connect_default\":false,\"remote_control_available\":true,\"remote_control_auto_on_by_default\":false,\"ide_rc_auto_enable_gate\":true,\"fast_mode_state\":\"off\",\"fast_mode_disabled_reason\":\"sdk_opt_in_required\",\"session_state\":\"idle\",\"capabilities\":[\"ui_surface_v1\"]},\"pending_permission_requests\":[],\"pending_user_dialog_requests\":[]}}\n",
        "stderr": ""
      },
      "assertions": [
        {
          "name": "expected exit code",
          "expected": 0,
          "actual": 0,
          "passed": true
        },
        {
          "name": "scenario output assertion",
          "passed": true
        }
      ],
      "passed": true
    },
    {
      "scenario": "native-catalog-accepts-harness-listed-selected-id",
      "command": [
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/bin/fm-model-index.sh",
        "check",
        "claude",
        "default"
      ],
      "cwd": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
      "environment": {
        "FM_HOME": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
        "FM_MODEL_CATALOG_DIR": null,
        "fleet_overrides": "all inherited FM_* variables removed"
      },
      "stdin": null,
      "duration_seconds": 1.017,
      "result": {
        "exit_code": 0,
        "stdout": "",
        "stderr": ""
      },
      "assertions": [
        {
          "name": "expected exit code",
          "expected": 0,
          "actual": 0,
          "passed": true
        },
        {
          "name": "scenario output assertion",
          "passed": true
        }
      ],
      "passed": true
    },
    {
      "scenario": "native-catalog-rejects-synthetic-absent-id",
      "command": [
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/bin/fm-model-index.sh",
        "check",
        "claude",
        "fm-live-proof-absent-01M4A2J6ENDRSX124XXEZ5XC1T"
      ],
      "cwd": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
      "environment": {
        "FM_HOME": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
        "FM_MODEL_CATALOG_DIR": null,
        "fleet_overrides": "all inherited FM_* variables removed"
      },
      "stdin": null,
      "duration_seconds": 0.961,
      "result": {
        "exit_code": 2,
        "stdout": "",
        "stderr": "model-index: id 'fm-live-proof-absent-01M4A2J6ENDRSX124XXEZ5XC1T' absent or retired in claude catalog (role 'native-proof')\n"
      },
      "assertions": [
        {
          "name": "expected exit code",
          "expected": 2,
          "actual": 2,
          "passed": true
        },
        {
          "name": "stderr contains expected text",
          "expected": "id 'fm-live-proof-absent-01M4A2J6ENDRSX124XXEZ5XC1T' absent or retired in claude catalog",
          "passed": true
        }
      ],
      "passed": true
    },
    {
      "scenario": "valid-index-only-no-dispatch-profiles-no-arguments-success",
      "command": [
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/bin/fm-model-index.sh",
        "profiles"
      ],
      "cwd": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
      "environment": {
        "FM_HOME": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture/index-only",
        "FM_MODEL_CATALOG_DIR": null,
        "fleet_overrides": "all inherited FM_* variables removed"
      },
      "stdin": null,
      "duration_seconds": 0.044,
      "result": {
        "exit_code": 0,
        "stdout": "",
        "stderr": ""
      },
      "assertions": [
        {
          "name": "expected exit code",
          "expected": 0,
          "actual": 0,
          "passed": true
        },
        {
          "name": "scenario output assertion",
          "passed": true
        }
      ],
      "passed": true
    }
  ],
  "summary": {
    "scenario_count": 34,
    "passed": 34,
    "failed": [],
    "dispatch_unchanged": true,
    "all_registries_unchanged": true,
    "blockers": []
  },
  "cleanup": {
    "disposable_fixture_removed": "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.model-selection-live-fixture",
    "removal_command": ["rm", "-rf", ".model-selection-live-fixture"],
    "removal_exit_code": 0,
    "no_scripts_created": true,
    "source_files_modified": false
  }
}
Evidence: Typed intake malformed-dispatch guards

Source: Typed intake malformed-dispatch guards

{
  "network_isolation": "All HTTPS proxies point to loopback discard port 9; no external services used. Actual product preflight runs before API.",
  "scenarios": [
    {
      "name": "empty present dispatch",
      "command": [
        "bin/fm-dispatch-resolve.sh",
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.gate-intake-n0e_u2nu/brief.md"
      ],
      "dispatch": "",
      "exit": 2,
      "stdout": "",
      "stderr": "error: malformed rules file: ~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.gate-intake-n0e_u2nu/config/crew-dispatch.json (not JSON)\n",
      "passed": true
    },
    {
      "name": "concatenated dispatch with appended approval",
      "command": [
        "bin/fm-dispatch-resolve.sh",
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.gate-intake-n0e_u2nu/brief.md"
      ],
      "dispatch": "{\"rules\": [], \"default\": {\"harness\": \"claude\", \"role\": \"fast\"}}\n{\"rules\": [], \"default\": {\"harness\": \"claude\", \"role\": \"fast\"}, \"approval\": \"required\"}",
      "exit": 2,
      "stdout": "",
      "stderr": "jq: error (at /var/folders/z8/7g01lsx95d77m5k2z1qzsngr0000gn/T/tmp.NiVlNbyCgV:1): dispatch must contain exactly one JSON object\nmodel-index: malformed dispatch: /var/folders/z8/7g01lsx95d77m5k2z1qzsngr0000gn/T/tmp.NiVlNbyCgV\nerror: model index/profile resolution failed\n",
      "passed": true
    },
    {
      "name": "nonobject dispatch",
      "command": [
        "bin/fm-dispatch-resolve.sh",
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.gate-intake-n0e_u2nu/brief.md"
      ],
      "dispatch": "[]",
      "exit": 2,
      "stdout": "",
      "stderr": "jq: error (at /var/folders/z8/7g01lsx95d77m5k2z1qzsngr0000gn/T/tmp.IxMZwhGbH6:0): dispatch must contain exactly one JSON object\nmodel-index: malformed dispatch: /var/folders/z8/7g01lsx95d77m5k2z1qzsngr0000gn/T/tmp.IxMZwhGbH6\nerror: model index/profile resolution failed\n",
      "passed": true
    },
    {
      "name": "valid one-object dispatch without rules",
      "command": [
        "bin/fm-dispatch-resolve.sh",
        "~/.no-mistakes/worktrees/32d18ed9638d/01M4A2J6ENDRSX124XXEZ5XC1T/.gate-intake-n0e_u2nu/brief.md"
      ],
      "dispatch": "{\"rules\": [], \"default\": {\"harness\": \"claude\", \"role\": \"fast\"}}",
      "exit": 0,
      "stdout": "dispatch-resolve:\n  status: escalate\n  reason: no rules to match\n",
      "stderr": "",
      "passed": true
    }
  ],
  "cleanup": "Disposable fixture removed in finally"
}

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed (5) ✅
  • ⚠️ tests/fm-bootstrap.test.sh:1298 - The new remote-notification fixture checks python3 on the original PATH at line 1269, then invokes it through a restricted PATH that excludes Homebrew, virtualenv, and /usr/local/bin installations. If Python is available only there, prerequisite detection succeeds but fake SSH cannot execute it, so the cases fail before exercising inheritance or notification. Both caller paths are affected: tests/fm-bootstrap.test.sh:1343 runs guarded fm-config-push/fm-bootstrap scenarios; :1380 runs ordinary convergence. Resolve Python once and expose that exact executable in each fixture fakebin, following the existing jq dependency pattern at :1291.
  • ⚠️ tests/fm-dispatch-resolve.test.sh:1518 - The new native-account and unknown-context cases inherit the operator's documented FM_MODEL_CATALOG_DIR override despite requiring native discovery or no export. For example, with an exported missing catalog directory, this pinned-Claude case never invokes its fake native catalog and leaves claude-catalog-roots unwritten; a readable unrelated export can instead reject synthetic IDs in the unknown-context cases. The checker intentionally prioritizes exports at bin/fm-model-index.sh:129 and permits them despite unknown worker context at :174. Clear the ambient override for these cases while preserving explicitly supplied fixture exports. Other affected sites: tests/fm-dispatch-resolve.test.sh:1547 — unknown Codex/Pi intake; tests/fm-control-relaunch.test.sh:1025 — unpinned indexed relaunch; tests/fm-agy-harness.test.sh:913,943 — indexed role and literal launches; tests/fm-worker-account.test.sh:410,432 — pinned Claude/Pi catalogs; :470 — unpinned Codex/Pi/Pi-signed contexts; :501 — ordinary Claude context; :554 — indexed Cursor/omp selections.

🔧 Fix applied.
✅ Re-checked - no issues remain.

  • ⚠️ bin/fm-secondmate-restart.sh:285 - The prior role-resolution fix (dafa7b4) leaves remote restart using two different index generations. Concrete sequence: a remote mate retains an index mapping its role to model-old; the parent changes that role to model-new, then runs the documented update/restart flow before config convergence. This code resolves model-new from the parent index but forwards only that literal, without propagating the index. Even when the remote mate has a valid pinned Claude/Pi account whose readable catalog excludes model-new, its pre-stop check finds no matching index entry and therefore performs no catalog check; replacement spawn likewise treats model-new as a non-entry literal. The working agent is stopped for a replacement that a converged index would reject beforehand. Affected changed siblings: bin/fm-control.sh:954-956,974-976 — resolution and pre-stop check against remote config; bin/fm-spawn.sh:2436-2442,2577-2582 — membership classification and conditional catalog gate. The unchanged transport path is bin/fm-remote-secondmate-relaunch.sh:51-56 → bin/fm-remote-secondmate-control.sh:277-281, which explicitly disables inheritance. Reuse the existing guarded routing-pair propagation before remote relaunch, ensuring resolution and remote pre-stop validation use the same selected pair; if propagation refuses, leave the running mate intact. No new durable state is needed.

🔧 Fix applied.
1 warning still open:

  • ⚠️ bin/fm-remote-secondmate-relaunch.sh:72 - The R9-1 fix round (c7fb4f4) introduces live inheritance without preserving reread intent when relaunch refuses. Concrete sequence: the parent changes a role from model-old to model-new; this wrapper publishes the new routing pair; the destination's pinned catalog excludes model-new, so pre-stop validation refuses and the old secondmate remains running. No reread or pending marker is recorded. On the next bootstrap, tracked files are already current and inherited files are unchanged, so bootstrap removes its newly created marker without notifying the surviving worker. Configuration changes can therefore remain unannounced indefinitely. The same invariant applies at bin/fm-remote-secondmate-relaunch.sh:75-79 (partial inheritance failure after unrelated writes), :86-90 (relaunch refusal), and :99-105 (missing route confirmation). Reuse the existing remote nudge marker before transfer, retain it across unsuccessful or unconfirmed replacement, and clear it only after confirmed replacement or successful reread delivery. Existing sibling notification patterns are bin/fm-config-push.sh:199-230 and bin/fm-bootstrap.sh:614-649. Extend the refusal scenarios at tests/fm-remote-secondmate-relaunch.test.sh:357-373 to cover subsequent notification of the surviving worker; no new persistence machinery is needed.

🔧 Fix applied.
1 warning still open:

  • ⚠️ bin/fm-spawn.sh:1071 - The round-6 frozen-pair fix and R9-1 remote-relaunch fix leave launch/control siblings selecting and validating different index generations. Concrete remote sequence: role:fast resolves to model-A; during the readiness/sync hops, the operator changes the coherent routing pair to fast=model-B with retired=[]; inheritance publishes that later pair, but launch forwards model-A. Destination spawn now classifies A as a non-entry literal and skips its selected-entry catalog check, allowing a launch inconsistent with the inherited role—even when the destination's authoritative catalog excludes A. Remaining sites: bin/fm-spawn.sh:1145,1170 — later inheritance and earlier-model forwarding; :2437,2438,2582 — independent resolution, membership, and catalog snapshots; :3185 — local secondmate inheritance after selection. The pre-stop sibling at bin/fm-control.sh:956,976,1168 has the same failure: changing the role while the pinned account's external authentication check runs makes the newer index omit the previously selected id, so validation succeeds and control stops the working agent for an unchecked replacement. bin/fm-model-index.sh:205 is the shared exact-model filter that skips these formerly indexed selections. Both role and stand-in selectors, supplied explicitly or through secondmate-harness, are affected. Freeze one routing pair at the launch/control selection boundary and reuse that generation for resolution, membership, catalog validation, inheritance, and replacement launch, using the existing staging mechanism without changing account pins. The staged remote-relaunch wrapper already preserves this invariant.

🔧 Fix applied.
3 issues (1 error, 2 warnings) still open:

  • ⚠️ bin/fm-spawn.sh:1071 - The round-6 frozen-pair fix and R9-1 remote-relaunch fix leave launch/control siblings selecting and validating different index generations. Concrete remote sequence: role:fast resolves to model-A; during the readiness/sync hops, the operator changes the coherent routing pair to fast=model-B with retired=[]; inheritance publishes that later pair, but launch forwards model-A. Destination spawn now classifies A as a non-entry literal and skips its selected-entry catalog check, allowing a launch inconsistent with the inherited role—even when the destination's authoritative catalog excludes A. Remaining sites: bin/fm-spawn.sh:1145,1170 — later inheritance and earlier-model forwarding; :2437,2438,2582 — independent resolution, membership, and catalog snapshots; :3185 — local secondmate inheritance after selection. The pre-stop sibling at bin/fm-control.sh:956,976,1168 has the same failure: changing the role while the pinned account's external authentication check runs makes the newer index omit the previously selected id, so validation succeeds and control stops the working agent for an unchecked replacement. bin/fm-model-index.sh:205 is the shared exact-model filter that skips these formerly indexed selections. Both role and stand-in selectors, supplied explicitly or through secondmate-harness, are affected. Freeze one routing pair at the launch/control selection boundary and reuse that generation for resolution, membership, catalog validation, inheritance, and replacement launch, using the existing staging mechanism without changing account pins. The staged remote-relaunch wrapper already preserves this invariant.
  • 🚨 bin/fm-spawn.sh:2440 - The R11-1 fix (c104dea) unnecessarily extends mandatory preselection staging to local secondmates with no selected model. Concrete sequence: a working Claude secondmate has a bare claude harness pin and a dangling routing-source symlink or nonregular routing source. Default-model control skips staging, stops the agent, then replacement spawn exits at line 2443; rollback leaves no agent running. Fresh launches also fail before unrelated inheritance can continue. This contradicts the existing warning-only local propagation contract at .agents/skills/secondmate-provisioning/SKILL.md:111-115. No model selection requires this extra branch. Revert this portion of the round to the narrower fix: freeze at model selection, while default-model local launches retain the existing guarded, warning-only inheritance boundary. Related sites: bin/fm-control.sh:939,956 — default selection skips pre-stop staging; :1163,1173-1180 — stop and replacement; :873-874 — stopped worker remains stopped; bin/fm-secondmate-restart.sh:174-175 — update/restart caller; bin/fm-spawn.sh:3204-3206 — intended warning-only propagation.
  • ⚠️ bin/fm-model-index.sh:255 - The round-6 routing-coherence fix still approves malformed present dispatch envelopes. Unlike the round-7 index/catalog validators, profiles accepts an empty file with success and transforms concatenated objects independently. With a valid index and a zero-byte crew-dispatch.json, config-push approves and publishes the empty dispatch over a working destination configuration; typed intake subsequently rejects that same file. Concatenated dispatch objects likewise propagate, while typed resolution consumes only the first object, potentially ignoring an appended approval gate. Require exactly one object at the shared present-dispatch boundary, preserving index-only validation by representing dispatch absence explicitly rather than treating a present empty file like the /dev/null sentinel. Related sites: bin/fm-model-index.sh:258-259 — separate streaming warning pass; bin/fm-config-inherit-lib.sh:605-620,636 — shared coherence validation and local publication gate; bin/fm-config-push.sh:139,160-162 — fleet-wide approval; bin/fm-remote-inherit-push.sh:53 — remote approval; bin/fm-spawn.sh:1018 and bin/fm-remote-secondmate-relaunch.sh:63 — launch/relaunch approval; bin/fm-bootstrap.sh:1046 — diagnostic consumer; bin/fm-dispatch-resolve.sh:163-168,422,470 — stream acceptance followed by first-document consumption.

🔧 Fix applied.
✅ Re-checked - no issues remain.

🔧 **Test** - 1 issue found ✅
  • ⚠️ live validation verdict: inconclusive (11 of 12 scenarios were driven live against the product); untested: Deliver the mid-session reread nudge to a running lane
  • Live validation: ⚠️ inconclusive - 11 of 12 scenarios driven live against the product
Scenario Result Live Evidence
Resolve role-based dispatch and explicit stand-ins; an index edit changes unchanged profiles ✅ pass live Public model-index and receiver transcripts
Reject retired model selections and retired identifiers in an independent product registry ✅ pass live Public model-index and receiver transcripts
Accept native catalog-listed models and refuse IDs proven absent ✅ pass live Real Claude native catalog; Real omp native catalog; Public model-index and receiver transcripts
Reject concatenated indexes without treating malformed catalog exports as contradictory evidence ✅ pass live Public model-index and receiver transcripts
Publish an index edit to a seeded lane and consume the new model through unchanged dispatch ✅ pass live Live config-push publication, refusals, mutation, and absence
Withhold catalog-invalid or removed-role routing while continuing unrelated inheritance ✅ pass live Live config-push publication, refusals, mutation, and absence
Mutate routing sources after validation without changing the generation published ✅ pass live Live config-push publication, refusals, mutation, and absence
Remove both inherited routing files when both sources are absent and jq is unavailable ✅ pass live Live config-push publication, refusals, mutation, and absence
Publish routing through the real receiver and refuse unsafe sibling or oversized requests ✅ pass live Public model-index and receiver transcripts
Launch and relaunch an indexed worker with the concrete model supplied to the real harness ✅ pass live Real indexed worker launch and relaunch
Use a resolved role for a real model response without changing the machine login ✅ pass live Real Claude response using the resolved role
Deliver the mid-session reread nudge to a running lane ⏸️ untested no The prior payload did not establish a live result for reread delivery. Actual pushes were attempted from stock marked lab homes with native receivers on private sockets. Publication worked, but nested…
  • bash tests/fm-model-index.test.sh — completed with exit 0.
  • FM_MODEL_CATALOG_DIR=<missing disposable export> bash tests/fm-worker-account.test.sh — completed with exit 0 after correcting the runner deadline and restoring the suite's supported temporary-fixture placement.
  • bin/fm-model-index.sh model, profiles, and check-registry against disposable indexes, dispatch profiles, and registries.
  • bin/fm-model-index.sh check against real Claude initialization and real omp models --json listings, accepting listed IDs and refusing deliberately absent IDs.
  • Public CLI checks for concatenated index documents, concatenated catalog exports, missing catalog evidence, retired identifiers, and explicit stand-ins.
  • Real bin/fm-config-push.sh publication into product-seeded lane homes, followed by the destination's real profiles consumer.
  • Real config-push refusals for a catalog-absent model and a removed dispatch role; verified preservation of both routing files and convergence of unrelated configuration.
  • Paused only the disposable config-push process after native validation, mutated its source pair, resumed it, and verified publication of the previously validated bytes.
  • Ran real config-push with both routing sources absent and a dependency PATH containing no jq; verified removal of both destination routing files.
  • Drove bin/fm-remote-inherit.sh put, check, and absent against an isolated receiver, including linked-sibling and oversized-payload refusals.
  • Launched and relaunched a real omp worker through fm-spawn.sh and fm-control.sh on private tmux sockets; inspected native panes, persisted metadata, and actual process arguments.
  • Resolved a Claude role and passed its selector to the real machine-login Claude CLI in print mode; received FM_MODEL_INDEX_LIVE_OK without tool access or session persistence.
  • Stopped all owned private tmux servers, tore down their helper-owned sockets, and removed disposable workspace materials.

✅ No issues found.

  • Live validation: ✅ go - 13 of 14 scenarios driven live against the product
Scenario Result Live Evidence
Edit only the index and resolve unchanged dispatch profiles to new models ✅ pass live model-selection-live.json: before/after model and profiles commands, with identical dispatch hashes.
Check a model against the installed Claude harness's native catalog ✅ pass live model-selection-live.json: real SDK initialization and native check commands.
Refuse absent catalog entries and retired selections without inventing unavailable evidence ✅ pass live model-selection-live.json: readable, missing, malformed, and concatenated catalog cases; retired literal and alias-target cases.
Check a separate product registry for retired identifiers without modifying it ✅ pass live model-selection-live.json: check-registry output and before/after registry hashes.
Reject malformed index and dispatch envelopes before selecting a profile ✅ pass live model-selection-live.json and typed-intake-guards.json.
Propagate an edited routing pair and resolve the destination's roles ✅ pass live propagation-live.json: shared-valid persisted files and destination profiles/model output.
Withhold incoherent or malformed routing pairs while continuing unrelated inheritance ✅ pass live propagation-live.json: removed-role, missing-index, empty-dispatch, and concatenated-dispatch shared API and public config-push cases.
Respect destination filesystem guards and preserve documented local link behavior ✅ pass live propagation-live.json: tracked/nonregular destination guards and local source/destination link cases.
Mirror absence of both routing sources without requiring jq ✅ pass live propagation-live.json: shared-both-absent-without-jq.
Apply routing payloads through the real receiver and refuse unsafe transfers ✅ pass live propagation-live.json: direct receiver put/check/absent commands and persisted state.
Launch real workers using roles and stand-ins, then launch again after one index edit ✅ pass live launch-live.json and launch-live.text: live-role, live-edit, and live-standin.
Mutate the source index after selection and retain the staged launch generation ✅ pass live launch-live.json: concurrent_source_mutation and synchronized staging/edit timestamps.
Refuse a catalog-absent spawn or replacement before creating or stopping an agent ✅ pass live launch-live.json and launch-live.text: prelaunch and pre-stop refusal records.
Replace a default-model local secondmate despite unsafe routing sources ⏸️ untested no Actual public launch attempts rejected both a home nested under the active lab and a sibling fixture nested under the source root. Every permitted fixture location is inside that root, while the produ…
  • bin/fm-model-index.sh model, profiles, check, and check-registry against disposable indices, dispatch files, catalog exports, and registries.
  • Real Claude SDK initialization with claude -p --input-format stream-json --output-format stream-json --verbose --no-session-persistence --setting-sources '', followed by native-catalog acceptance and absence checks.
  • bin/fm-dispatch-resolve.sh <brief> with empty, concatenated, nonobject, and valid single-object dispatch inputs; network proxies restricted to loopback.
  • propagate_secondmate_inheritance through the actual shared executable API, checking persisted destination configuration, pair retention, filesystem guards, link compatibility, and absence without jq.
  • bin/fm-config-push.sh against isolated source and destination homes for removed-role, missing-index, malformed-dispatch, tracked-file, and nonregular-destination cases.
  • bin/fm-remote-inherit.sh check|put|absent directly against disposable homes, including payload integrity, generation conflicts, byte bounds, and linked or nonregular destinations; no live SSH transport claimed.
  • Real private-tmux Claude primary and bin/fm-spawn.sh live-role|live-edit|live-standin <project> --scout --harness claude --model role:strong|stand-in:strong, with concrete launch commands and metadata captured.
  • Synchronized source-index mutation after actual spawn staging, followed by inspection of the launched model and a catalog-refused bin/fm-control.sh live-race relaunch --harness claude --model role:strong --note 'Remain idle.'.
  • Selected executable regressions from tests/fm-control-relaunch.test.sh: model-index pre-stop refusal, frozen generations across authentication and replacement, unknown account context, and default-secondmate unsafe routing sources.
  • bash tests/fm-remote-secondmate-relaunch.test.sh and contained bash -x tests/fm-model-index.test.sh as supplemental non-live regression checks.
  • bin/fm-teardown.sh <lab-task> --force, private-socket server shutdown, and removal of disposable homes, projects, pools, drivers, and transient fixtures.
✅ **Document** - passed

✅ No issues found.

✅ No issues found.

⚠️ **Lint** - 0 issues

✅ No issues found.

  • ⚠️ linter found issues (exit code 1)

🔧 Fix applied.
✅ Re-checked - no issues remain.

✅ **Push** - passed

✅ No issues found.

✅ No issues found.

@MrGTV-love MrGTV-love changed the title feat(bin): add fleet model index with role-based dispatch and catalog checks feat: add a fleet model index for role-based dispatch Oct 6, 2026
@MrGTV-love MrGTV-love changed the title feat: add a fleet model index for role-based dispatch feat: add a shared fleet model index for role-based dispatch Oct 7, 2026
…fixed both. The other two did not, so I changed nothing for them. 1. Lint 1 (fixed). ShellCheck SC2034: `tests/fm-control-relaunch.test.sh` line 766 declared a variable, `catalogs`, that nothing used. I removed it from that line. The earlier local lint missed it because local changed-file mode skips SC2034. A full lint run on all changed shell files, with the changed files named so SC2034 runs, was still running when I sent this result, so it is not confirmed. 2. Behavior portable serial 5 (fixed). The bootstrap test "array profile with malformed model is flagged" failed. The cause was `retired_guard` in `bin/fm-model-index.sh`. It called `retired()` on any `model` value. When `model` was the number 5, jq crashed with "number (5) cannot be matched". That crash message replaced the expected "must be non-empty strings" error. The fix runs the retired check only when `.model` is a string. A non-string model still reaches the normal checks, which reject it. Results after the fix: - `tests/fm-bootstrap.test.sh`: rc=0. - `tests/fm-model-index.test.sh`: rc=0. - `tests/fm-dispatch-resolve.test.sh`: rc=0. - `tests/fm-spawn-dispatch-profile.test.sh`: had 54 passes and 0 failures, but had not finished when I sent this result. 3. Behavior portable serial 2 ("rendered export DOM violated the Calm conversation boundary") did not come from this PR. This PR changes no Pi or Calm files. The same test fails in the same shard on PR 12 (run 36781504338) and PR 11 (run 36778047014). This looks like a change in the external Pi version, but I did not confirm that. 4. Behavior portable serial 4 ("Pi outcomes rendering consumers must preserve stock behavior") also did not come from this PR. It fails on PR 12 and PR 11 in the same way, with the same likely cause
…e serial 8 failed because the secondmate test fixture allowed crew-dispatch.json inheritance but not its coupled model-index.json destination. Reproduced the exact CI assertion locally, then corrected all three positive selective-ignore fixtures while preserving intentional refusal cases and production guards. Fixed Lint 1 by quoting the literal model="pane-only" without changing behavior. Updated the existing verification documentation. Verification passed: focused secondmate-harness, bootstrap, model-index, and worker-account suites (total=4, failed=0); canonical full-analysis lint for all four changed test scripts; two-home production inheritance smoke covering dispatch copy and absence convergence; documentation audience check. One optional Cursor case skipped because its executable was unavailable. The full CI shard and live Herdr suite were not rerun
@MrGTV-love MrGTV-love changed the title feat: add a shared fleet model index for role-based dispatch feat: centralize fleet model selection in a role-based index Oct 7, 2026
…l six named suites without changing production behavior, CI timeouts, skips, or shard placement. Remote spawn now uses five coverage-preserving selector/boundary launches instead of eight and avoids unrelated inherited-config transfers. Model-index fixtures retain all distinct guard, coherence, source-safety, mutation, schema, account-context, and no-jq cases while narrowing inheritance fan-out, removing 72 redundant receiver calls, 20 redundant remote recovery pushes, and 12 redundant local recovery calls. Bootstrap shares immutable setup while retaining all 14 caller/state executions. Dispatch removes two duplicate resolver calls and zero-duration sleep processes. Relaunch retains every refusal’s survivor notification proof, removes six repeated post-clear bootstrap runs, shares selection fixtures, and reduces the cwd-race fixture from ten fixed delays to one finite first-read delay. Contributor guidance now documents focused inheritance fixtures. Verification: all six suites passed through `bash bin/fm-test-run.sh --jobs 1` with zero failed suites and zero gate-skipped suites. Local durations: model-index 222.4s, remote compact-adviser spawn 349.8s, bootstrap 175.8s, dispatch-resolve 55.9s, control-relaunch 524.1s, remote-secondmate-relaunch 86.7s. The same-host model-index baseline was 436.2s, giving an observed reduction of approximately 49%. Dispatch’s existing EPOCHREALTIME capability case skipped under stock Bash. Full pinned ShellCheck 0.11.0 passed for all six changed roots, including the corrected race fixture; documentation ownership checks passed. Darwin timings do not establish Linux shard headroom. No pipeline control, push, or CI rerun was invoked; the outer executor owns the provider rerun
@MrGTV-love
MrGTV-love merged commit fe15702 into main Oct 7, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant