feat(orchestrate): configurable model catalog via ORCHESTRATE_MODEL_CATALOG (#171)

orchestrate/README.md

@@ -10,6 +10,61 @@ The skill itself lives in [`skills/orchestrate/SKILL.md`](./skills/orchestrate/S
 - A Cursor API key in `CURSOR_API_KEY`.
 - Optional Slack app and bot token if you want a Slack thread mirroring the run.
 
+## Model catalog (optional)
+
+`ORCHESTRATE_MODEL_CATALOG` replaces the built-in model catalog with your own. When it is set, that list is the complete menu: it is what planners choose `tasks[].model` from, what `bun cli.ts models` prints, and where each task type's default comes from. Nothing is merged with the built-in catalog, so what you write is exactly what runs. Use it to steer cost without editing the plugin.
+
+The value is a JSON array in the same shape as the built-in catalog, validated against [`skills/orchestrate/schemas/model-catalog.schema.json`](./skills/orchestrate/schemas/model-catalog.schema.json). Start from the built-in list rather than writing entries by hand:
+
+```bash
+bun skills/orchestrate/scripts/cli.ts models --json > catalog.json
+# edit catalog.json: drop what you don't want, move defaultFor where you want it
+export ORCHESTRATE_MODEL_CATALOG="$(cat catalog.json)"
+```
+
+Every entry needs `slug`, `selection`, `summary`, `strengths`, `speed`, and `use`. `defaultFor` and `selection.params` are optional:
+
+```json
+[
+  {
+    "slug": "house-worker",
+    "selection": { "id": "composer-2.5", "params": [{ "id": "fast", "value": "true" }] },
+    "summary": "Cheap, fast worker.",
+    "strengths": ["throughput", "well-scoped implementation"],
+    "speed": "fast",
+    "use": "Use for all bounded implementation work.",
+    "defaultFor": ["worker"]
+  },
+  {
+    "slug": "house-planner",
+    "selection": { "id": "claude-opus-4-8" },
+    "summary": "Frontier judgment for decomposition and acceptance checks.",
+    "strengths": ["judgment", "ambiguity resolution"],
+    "speed": "slow",
+    "use": "Use when the work needs design decisions rather than execution.",
+    "defaultFor": ["subplanner", "verifier"]
+  }
+]
+```
+
+`summary`, `strengths`, and `use` are required because planners select by capability, not by model name. An entry with thin prose tends to get passed over. `speed` is a free-form string, so new model vocabulary doesn't need a plugin release.
+
+Each of `worker`, `subplanner`, and `verifier` needs a `defaultFor` somewhere in the list. Root planners are not part of the catalog; they take their model from kickoff `--model`, which defaults to `claude-opus-4-8`.
+
+### Precedence
+
+1. Explicit `tasks[].model` in the plan
+2. The `defaultFor` entry for that task's type
+
+Run `bun cli.ts models` to print the catalog in effect, and `bun cli.ts models --check` to probe every entry against `/v1/agents`. Invalid config exits 2 at startup, naming the offending entry and field, rather than failing mid-run:
+
+```
+ORCHESTRATE_MODEL_CATALOG failed zod validation:
+  [0].summary: Required
+```
+
+Two caveats. This shapes what planners choose from, but a planner can still write any model id into `tasks[].model`, so it is guidance rather than a spend ceiling. And each spawned agent reads its own environment: set the variable as a Cursor Cloud secret for the repo so subplanners and workers inherit it, not just in the dispatcher's local shell.
+
 ## Cursor API key
 
 1. Open [https://cursor.com/dashboard/integrations](https://cursor.com/dashboard/integrations).

orchestrate/skills/orchestrate/SKILL.md

@@ -14,6 +14,7 @@ An explicit `/orchestrate <goal>` fans out a large task across parallel Cursor c
 
 - `CURSOR_API_KEY` must be a personal/user key. Create it from [Cursor Dashboard > Integrations](https://cursor.com/dashboard/integrations), then read `cursor-sdk` Auth before using it.
 - `SLACK_BOT_TOKEN` is optional. When set, pass `--slack-channel <id>` to `kickoff` or the first `run --root`, or set `SLACK_CHANNEL_ID`. The script stores the channel in `plan.slackChannel`, posts the kickoff thread there, mirrors task status, and reads Andon reactions. When the token is unset, the script logs once and runs without Slack visibility; correctness does not change.
+- `ORCHESTRATE_MODEL_CATALOG` is optional. When set, its JSON array replaces the built-in model catalog outright: it becomes the list planners pick `tasks[].model` from, and its `defaultFor` entries supply each task type's default. It is validated against `schemas/model-catalog.schema.json`; `bun cli.ts models` prints whichever catalog is in effect and `--json` emits it in that shape. See the plugin README.
 
 ## Core principles
 

orchestrate/skills/orchestrate/prompts/subplanner.md

@@ -30,7 +30,7 @@ Paths you must NOT modify (owned by siblings):
 
 Acceptance criteria for your subtree:
 {{accept}}{{verifyPlan}}{{upstream}}
-Model selection: pick `tasks[].model` per task by capability. Available models:
+Model selection: pick `tasks[].model` per task by capability, choosing only from the list below. That list is this repo's effective catalog, not a generic menu. Omit `tasks[].model` to accept the marked default for that task type; set it explicitly when the task needs a different capability.
 
 {{modelCatalog}}
 

orchestrate/skills/orchestrate/references/dispatcher.md

@@ -16,6 +16,8 @@ One-time setup: run `bun install` inside this skill's `scripts/` directory if `n
 bun cli.ts kickoff "<goal>" [--repo <url>] [--ref main] [--model claude-opus-4-8] [--slack-channel C123] [--dispatcher-name "Alex"]
 ```
 
+A repo can replace the model catalog planners choose from, including each task type's default, with `ORCHESTRATE_MODEL_CATALOG` (see the plugin README). That does not cover the root planner: pass `--model` to set it, otherwise it stays `claude-opus-4-8`. Run `bun cli.ts models` to print the catalog in effect; config the CLI can't read exits 2 with the offending entry named.
+
 The CLI reads `CURSOR_API_KEY`, auto-detects the repo from `git config --get remote.origin.url`, builds the spawn prompt, spawns via `cursor-sdk`, and prints `{ agentId, runId, status, url, dispatcherFirstName }` JSON. Slack is optional. If `SLACK_BOT_TOKEN` is set, also pass `--slack-channel <id>` or set `SLACK_CHANNEL_ID`; otherwise kickoff fails before spawning. If the token is unset, Slack stays disabled.
 
 ## Dispatcher identity

orchestrate/skills/orchestrate/schemas/model-catalog.schema.json

@@ -0,0 +1,92 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "$id": "https://cursor/orchestrate/model-catalog.schema.json",
+  "title": "orchestrate ORCHESTRATE_MODEL_CATALOG",
+  "description": "Optional operator-authored model catalog. When ORCHESTRATE_MODEL_CATALOG is set, it replaces the built-in catalog planners choose `tasks[].model` from.",
+  "type": "array",
+  "items": {
+    "type": "object",
+    "properties": {
+      "slug": {
+        "type": "string",
+        "minLength": 1,
+        "description": "Authoring name planners write into `tasks[].model`."
+      },
+      "selection": {
+        "type": "object",
+        "properties": {
+          "id": {
+            "type": "string",
+            "minLength": 1,
+            "description": "Model id as accepted by the Cursor API."
+          },
+          "params": {
+            "type": "array",
+            "items": {
+              "type": "object",
+              "properties": {
+                "id": {
+                  "type": "string"
+                },
+                "value": {
+                  "type": "string"
+                }
+              },
+              "required": [
+                "id",
+                "value"
+              ],
+              "additionalProperties": false
+            },
+            "description": "Model parameters, e.g. reasoning, effort, thinking, fast."
+          }
+        },
+        "required": [
+          "id"
+        ],
+        "additionalProperties": false,
+        "description": "Canonical SDK selection passed to `Agent.create({ model })`."
+      },
+      "summary": {
+        "type": "string",
+        "description": "One-line description planners read."
+      },
+      "strengths": {
+        "type": "array",
+        "items": {
+          "type": "string"
+        },
+        "description": "Capability keywords planners match a task against."
+      },
+      "speed": {
+        "type": "string",
+        "description": "Relative latency, e.g. fast, medium, slow."
+      },
+      "use": {
+        "type": "string",
+        "description": "When a planner should pick this model."
+      },
+      "defaultFor": {
+        "type": "array",
+        "items": {
+          "type": "string",
+          "enum": [
+            "worker",
+            "subplanner",
+            "verifier"
+          ]
+        },
+        "description": "Task types that use this model when `tasks[].model` is omitted."
+      }
+    },
+    "required": [
+      "slug",
+      "selection",
+      "summary",
+      "strengths",
+      "speed",
+      "use"
+    ],
+    "additionalProperties": false
+  }
+}

orchestrate/skills/orchestrate/scripts/__tests__/models-catalog.test.ts

@@ -1,12 +1,27 @@
-import { describe, expect, test } from "bun:test";
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
 
 import {
   defaultModelForType,
   isKnownModel,
   MODEL_CATALOG,
+  MODEL_ENV_CATALOG,
   resolveModelSelection,
 } from "../models.ts";
 
+let savedCatalogEnv: string | undefined;
+
+// These assertions describe the built-in catalog, so an env-provided catalog
+// from the surrounding shell must not leak in.
+beforeEach(() => {
+  savedCatalogEnv = process.env[MODEL_ENV_CATALOG];
+  delete process.env[MODEL_ENV_CATALOG];
+});
+
+afterEach(() => {
+  if (savedCatalogEnv === undefined) delete process.env[MODEL_ENV_CATALOG];
+  else process.env[MODEL_ENV_CATALOG] = savedCatalogEnv;
+});
+
 describe("MODEL_CATALOG", () => {
   test("every catalog entry passes isKnownModel", () => {
     for (const profile of MODEL_CATALOG) {

orchestrate/skills/orchestrate/scripts/__tests__/models-env-catalog.test.ts

@@ -0,0 +1,172 @@
+import { afterEach, beforeEach, describe, expect, test } from "bun:test";
+
+import { PlanValidationError } from "../errors.ts";
+import {
+  assertModelEnvConfig,
+  defaultModelForType,
+  effectiveModelCatalog,
+  isKnownModel,
+  MODEL_CATALOG,
+  MODEL_ENV_CATALOG,
+  renderModelCatalog,
+  resolveModelSelection,
+} from "../models.ts";
+
+let saved: string | undefined;
+
+/** A minimally complete entry; every field the schema requires. */
+function entry(
+  overrides: Record<string, unknown> = {}
+): Record<string, unknown> {
+  return {
+    slug: "house-worker",
+    selection: { id: "composer-2.5" },
+    summary: "House worker model.",
+    strengths: ["throughput"],
+    speed: "fast",
+    use: "Use for all bounded implementation work.",
+    ...overrides,
+  };
+}
+
+function setCatalog(entries: unknown): void {
+  process.env[MODEL_ENV_CATALOG] = JSON.stringify(entries);
+}
+
+beforeEach(() => {
+  saved = process.env[MODEL_ENV_CATALOG];
+  delete process.env[MODEL_ENV_CATALOG];
+});
+
+afterEach(() => {
+  if (saved === undefined) delete process.env[MODEL_ENV_CATALOG];
+  else process.env[MODEL_ENV_CATALOG] = saved;
+});
+
+describe("ORCHESTRATE_MODEL_CATALOG unset", () => {
+  test("the built-in catalog is in effect", () => {
+    expect(effectiveModelCatalog()).toBe(MODEL_CATALOG);
+    expect(defaultModelForType("worker")).toBe("gpt-5.5-high-fast");
+    expect(renderModelCatalog()).not.toContain("exact model menu");
+  });
+
+  test("whitespace-only value is treated as unset", () => {
+    process.env[MODEL_ENV_CATALOG] = "   ";
+    expect(effectiveModelCatalog()).toBe(MODEL_CATALOG);
+  });
+});
+
+describe("ORCHESTRATE_MODEL_CATALOG replaces the built-in catalog", () => {
+  test("only the listed models are published", () => {
+    setCatalog([entry({ defaultFor: ["worker", "subplanner", "verifier"] })]);
+    expect(effectiveModelCatalog().map(m => m.slug)).toEqual(["house-worker"]);
+    expect(isKnownModel("gpt-5.5-high-fast")).toBe(false);
+    expect(isKnownModel("house-worker")).toBe(true);
+  });
+
+  test("entries supply every task type's default", () => {
+    setCatalog([
+      entry({ defaultFor: ["worker"] }),
+      entry({
+        slug: "house-planner",
+        selection: { id: "claude-opus-4-8" },
+        defaultFor: ["subplanner", "verifier"],
+      }),
+    ]);
+    expect(defaultModelForType("worker")).toBe("house-worker");
+    expect(defaultModelForType("subplanner")).toBe("house-planner");
+    expect(defaultModelForType("verifier")).toBe("house-planner");
+  });
+
+  test("a slug resolves to its full selection, params included", () => {
+    setCatalog([
+      entry({
+        selection: {
+          id: "composer-2.5",
+          params: [{ id: "fast", value: "true" }],
+        },
+        defaultFor: ["worker", "subplanner", "verifier"],
+      }),
+    ]);
+    expect(resolveModelSelection("house-worker")).toEqual({
+      id: "composer-2.5",
+      params: [{ id: "fast", value: "true" }],
+    });
+  });
+
+  test("a model outside the catalog still passes through as a bare id", () => {
+    setCatalog([entry({ defaultFor: ["worker", "subplanner", "verifier"] })]);
+    expect(resolveModelSelection("gpt-5.5")).toEqual({ id: "gpt-5.5" });
+  });
+
+  test("the rendered catalog is what planners see", () => {
+    setCatalog([entry({ defaultFor: ["worker"] })]);
+    const text = renderModelCatalog();
+    expect(text).toContain("exact model menu");
+    expect(text).toContain("`house-worker` — House worker model.");
+    expect(text).toContain("(default for worker)");
+    expect(text).toContain("speed: fast; strengths: throughput");
+  });
+
+  // `speed` is a free-form string so new model vocabulary doesn't require a
+  // plugin release.
+  test("unrecognized speed values are passed through", () => {
+    setCatalog([
+      entry({
+        speed: "blistering",
+        defaultFor: ["worker", "subplanner", "verifier"],
+      }),
+    ]);
+    expect(renderModelCatalog()).toContain("speed: blistering");
+    expect(() => assertModelEnvConfig()).not.toThrow();
+  });
+
+  test("the built-in catalog round-trips through the schema", () => {
+    // `bun cli.ts models --json` is documented as a starting point, so its
+    // output has to be valid input.
+    setCatalog(MODEL_CATALOG);
+    expect(effectiveModelCatalog()).toEqual(MODEL_CATALOG);
+    expect(() => assertModelEnvConfig()).not.toThrow();
+  });
+});
+
+describe("catalog config errors", () => {
+  test("a missing task-type default fails fast at startup", () => {
+    setCatalog([entry({ defaultFor: ["worker"] })]);
+    expect(() => assertModelEnvConfig()).toThrow(PlanValidationError);
+    expect(() => assertModelEnvConfig()).toThrow(
+      /no subplanner default.*"defaultFor": \["subplanner"\]/s
+    );
+  });
+
+  test("assertModelEnvConfig passes when every task type resolves", () => {
+    setCatalog([entry({ defaultFor: ["worker", "subplanner", "verifier"] })]);
+    expect(() => assertModelEnvConfig()).not.toThrow();
+  });
+
+  test("malformed JSON is rejected", () => {
+    process.env[MODEL_ENV_CATALOG] = "[{slug:}]";
+    expect(() => effectiveModelCatalog()).toThrow(/is not valid JSON/);
+  });
+
+  test("an incomplete entry is rejected with the offending field", () => {
+    const { summary, ...withoutSummary } = entry();
+    expect(summary).toBeDefined();
+    setCatalog([withoutSummary]);
+    expect(() => effectiveModelCatalog()).toThrow(PlanValidationError);
+    expect(() => effectiveModelCatalog()).toThrow(/\[0\]\.summary/);
+  });
+
+  test("a bad selection or defaultFor is rejected", () => {
+    setCatalog([entry({ selection: { id: "" } })]);
+    expect(() => effectiveModelCatalog()).toThrow(/\[0\]\.selection\.id/);
+
+    setCatalog([entry({ defaultFor: ["planner"] })]);
+    expect(() => effectiveModelCatalog()).toThrow(/\[0\]\.defaultFor/);
+  });
+
+  test("a non-array value is rejected", () => {
+    process.env[MODEL_ENV_CATALOG] = '{"slug":"x"}';
+    expect(() => effectiveModelCatalog()).toThrow(PlanValidationError);
+  });
+});