feat(copilot): synthesize GitHub Copilot models dynamically from CAPI catalog

[flora131]

Summary

Replaces the plan to hardcode claude-sonnet-5 and mai-code-flash-1 as static built-in Copilot models with a general mechanism: github-copilot models are now synthesized dynamically from GitHub's live CAPI /models catalog, so any picker-enabled, non-disabled, plain-id chat model (including these two, and any future additions) is automatically exposed as a github-copilot/* model without a code change or hardcoded metadata list. The active session's model and thinking level are also kept live-synced when the catalog loads or refreshes mid-session.

Key Changes

• Dynamic model synthesis (src/core/copilot-model-synthesis.ts, new): synthesizeCopilotCatalogModels() walks the live CAPI catalog and builds a Model<Api> for every entry that is model_picker_enabled, not policy-disabled, type: "chat", has a plain (non-namespaced) id, and maps to a supported endpoint (/v1/messages → anthropic-messages, /responses → openai-responses, /chat/completions → openai-completions, in that preference order). Reasoning/thinking metadata, vision input, cost (zeroed, billed by Copilot), and context-window options are derived from catalog supports/limits rather than hand-written per model. copilotThinkingLevelMapFor() maps CAPI's advertised reasoningEffortLevels (or adaptiveThinking as a fallback) onto Atomic's thinking-level map, gating an off level by API and adaptive-thinking support. Namespaced enterprise deployment ids (e.g. org/deployment/model) are skipped, and ids already present in the upstream pi-ai built-ins always win over a synthesized entry.
• Catalog parsing (src/core/copilot-model-catalog.ts): CopilotModelContext now also carries displayName, vendor, supportedEndpoints, supports (adaptive thinking, reasoning effort/levels, vision, tool calls, etc.), limits, modelPickerEnabled, policyState, and type, all parsed from CAPI's /models response and round-tripped through the on-disk cache (cache schema bumped to version 5).
• Wiring (src/core/model-registry-builtins.ts): loadBuiltInModels() now calls withDynamicGitHubCopilotModels(), which dedupes against the upstream catalog by id and appends any freshly synthesized Copilot models using a connection template (baseUrl/headers) derived from an existing sibling github-copilot model. A new withCopilotThinkingLevelMap() also reapplies CAPI-derived reasoning-effort gating to existing (non-synthesized) Copilot models so their thinking levels stay in sync with the live catalog.
• Live session refresh (src/core/agent-session-extension-bindings.ts, src/core/agent-session-methods.ts, src/modes/interactive/interactive-model-routing.ts): adds a public AgentSession.refreshCurrentModelFromRegistry() (wrapping the existing internal _refreshCurrentModelFromRegistry) that re-resolves the active model against the current registry, clamps the active thinking level to whatever the refreshed model now supports via setThinkingLevel(), and emits a model_changed event (source: "restore") alongside any context_window_changed/thinking_level_changed events. loadCopilotModelCatalog now calls this after applying a freshly fetched or cached Copilot catalog, so a session that started on a fallback/stale Copilot model definition picks up the live catalog's metadata (e.g. corrected thinking levels) without a restart — while leaving the active model untouched if the registry still can't resolve it.
• Tests: new test/copilot-model-synthesis.test.ts (endpoint→API mapping, capability/thinking-level derivation, gating of non-picker/disabled/non-chat/namespaced/duplicate ids, deterministic endpoint preference) and new test/agent-session-copilot-catalog-refresh.test.ts (session adopts refreshed catalog metadata and clamps stale thinking levels, leaves the session untouched when the registry can't resolve the active model, and loadCopilotModelCatalog triggers the session refresh after applying a cached catalog), plus updated test/copilot-model-catalog.test.ts and test/model-registry-context-window.suite.ts covering the richer catalog metadata (including claude-sonnet-5 and mai-code-1-flash-picker fixtures) end-to-end through CAPI parsing, disk-cache round-tripping, and the registry overlay.
• Docs: docs/models.md, docs/providers.md, docs/rpc.md, docs/sdk.md, and docs/settings.md updated to describe dynamic catalog-driven model population (vs. the prior fixed allowlist) and to use claude-sonnet-5 / mai-code-1-flash-picker as examples.
• Changelog: [Unreleased] → ### Added documents the dynamic population behavior and catalog-driven thinking-level gating.

Notes

• No patching/vendoring of @earendil-works/pi-ai; no version bumps.
• Because model creation is now catalog-driven, GitHub adding, removing, or retiering Copilot models (including claude-sonnet-5 and mai-code-flash-1) is reflected automatically on the next catalog fetch — no follow-up PR needed once GitHub's CAPI lists them.
• Validation: bun run typecheck, bun run lint, bun run check:file-length, and AGENT=1 bun run test:unit all green.

AI-assisted (Claude Fable 5 via Atomic goal workflow).

[claude]

Code Review — feat(copilot): add claude-sonnet-5 and mai-code-flash-1 models

Overall this is a clean, well-scoped augmentation. The dedup-by-id design (withExtraGitHubCopilotModels) means the fork-side entries drop out automatically once pi-ai ships these natively, there's no vendoring or version bumping, and the changelog/docs/tests were all updated. Nice work. A few things worth addressing before merge, mostly around DRY and doc accuracy.

🔴 Duplicated constants that already exist in the codebase (DRY / drift risk)

src/core/model-registry-builtins.ts:17 — GITHUB_COPILOT_BASE_URL = "https://api.individual.githubcopilot.com" is byte-identical to DEFAULT_COPILOT_API_BASE_URL, which is already imported into this very file (line 10) and used at line 93. Please reuse the imported constant instead of redeclaring the literal:

// drop GITHUB_COPILOT_BASE_URL; use DEFAULT_COPILOT_API_BASE_URL directly
baseUrl: DEFAULT_COPILOT_API_BASE_URL,

src/core/model-registry-builtins.ts:18-23 — GITHUB_COPILOT_HEADERS duplicates the exported COPILOT_CATALOG_HEADERS in copilot-model-catalog.ts:67-73 (same User-Agent/Editor-Version/Editor-Plugin-Version/Copilot-Integration-Id, minus the API-version header that withGitHubCopilotApiVersionHeader injects separately). Those editor/plugin version strings (0.35.0, vscode/1.107.0) are now pinned in two places and will silently drift the next time someone bumps them for the catalog reads. Prefer importing and deriving from the existing constant, e.g. strip the API-version key rather than re-typing the values.

🟡 Docs assert unverified tier numbers as fact

The PR notes candidly state the live CAPI tier values for these two models "could not be verified without Copilot auth," yet docs/models.md now states as fact that github-copilot/mai-code-flash-1 "resolve[s] to a 272k default / 922k long window" and claude-sonnet-5 to "200k default / 936k long." Those numbers are copied from the gpt-5.5 / claude-opus-4.8 fixtures. Since behavior is catalog-driven at runtime, if GitHub tiers these models differently the docs become wrong with no code change to catch it. Consider hedging (e.g. "tiers follow GitHub's live /models metadata") rather than baking in specific token counts for the two new, unverified models.

🟢 Minor / nits

• Test value: the new CAPI fixtures for claude-sonnet-5 / mai-code-flash-1 (copilot-model-catalog.test.ts) are exact copies of the existing claude-opus-4.8 / gpt-5.5 fixtures, so those parseCopilotModelCatalog assertions mostly re-exercise generic overlay parsing already covered. That's harmless, and the ModelRegistry.find(...) metadata test in model-registry-context-window.suite.ts is genuinely useful (proves the IDs resolve with correct api/reasoning/input). Just noting the fixture assertions add little beyond confirming the ID appears in the map.
• Shared header object: both models reference the same GITHUB_COPILOT_HEADERS object literal. It's safe today (the version-header helper spreads rather than mutates), but a shared mutable object across model entries is a latent footgun — Object.freeze(...) or per-model literals would be safer.
• Offline scalar mismatch: the built-in contextWindow: 400_000 scalar for mai-code-flash-1 (used only offline/unauthenticated) matches neither advertised tier (272k default / 1.05m long). Intentional per the offline-fallback design, but slightly surprising — worth a one-line comment.
• 👍 Good catch removing the duplicate describeModelRegistry import in model-registry-context-window.suite.ts (leaves a stray double blank line, trivial).

Not blocking

Cost metadata looks internally consistent (Sonnet-tier 3/15 with 3.75 cache-write = 1.25× write multiplier; mai-code-flash-1 cacheWrite: 0 matches OpenAI's free cache writes). Changelog entry correctly lives under [Unreleased] → ### Added. I could not run bun run typecheck / test:unit in this environment (no installed node_modules), so please confirm the reported green run stands after addressing the DRY items.

[claude]

Code Review — PR #1583: feat(copilot): add claude-sonnet-5 and mai-code-flash-1 builtin models

Thanks for this — the pivot from a hardcoded model list to dynamic synthesis from the live CAPI catalog is the right architectural call. It generalizes cleanly (new Copilot models appear without a fork edit), the gating logic is careful, and the test coverage is strong. A few things worth addressing before merge, none blocking on their own.

🔴 PR description no longer matches the implementation

The description documents an EXTRA_GITHUB_COPILOT_MODELS static list merged via withExtraGitHubCopilotModels() that "deduped by id against the upstream pi-ai catalog." Neither symbol exists in the diff:

$ grep -rn "EXTRA_GITHUB_COPILOT_MODELS|withExtraGitHubCopilotModels" packages/coding-agent/
NOT FOUND

The actual mechanism is withDynamicGitHubCopilotModels() -> synthesizeCopilotCatalogModels(), driven entirely by the CAPI /models catalog (the CHANGELOG entry is accurate). This is a meaningfully different design — please update the PR body so reviewers and future git blame readers aren't misled. It also changes the failure mode worth calling out: these two models only appear when the user has Copilot authenticated and the catalog fetch/cache succeeds — there is no offline/static fallback, unlike what the description implies.

🟠 Domain docs stripped to fit under the 500-line gate

copilot-model-catalog.ts is now 498/500 lines after removing the extensive header block and per-field JSDoc that explained the non-obvious CAPI semantics (max_prompt_tokens vs max_context_window_tokens vs billing.token_prices.<tier>.context_max, why the data isn't baked into a static map, the branded-total-vs-input-cap distinction). That knowledge is exactly the kind that's expensive to reconstruct from the code alone. Two concerns:

1. Trading away that documentation to squeeze under the file-length gate is a net maintainability loss.
2. At 498 lines the file is effectively frozen — the next change breaks check:file-length.

Consider splitting the file (e.g. extract the disk-cache read/write/sanitize block into a copilot-model-catalog-cache.ts) and restoring the semantic docs, rather than paying for the new metadata parsing by deleting comments.

🟡 cost: ZERO_COST on every synthesized model

synthesizeCopilotCatalogModels hardcodes { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }. If the upstream pi-ai builtin github-copilot/* models carry non-zero cost, synthesized models will diverge and any cost display/estimation will read $0 for them. If Copilot is uniformly treated as zero-cost (subscription billing), this is fine — but please confirm it matches the sibling builtins so behavior is consistent (I couldn't verify locally; deps aren't installed in the review env).

🟡 maxTokens fallback can land on contextWindow

maxTokens: entry.maxTokens ?? entry.limits?.maxOutputTokens ?? entry.maxInputTokens ?? entry.contextWindow,

The final fallback uses an input budget (contextWindow, e.g. 128k–200k) as the output cap. In the normal case CAPI always advertises max_output_tokens so this never triggers, but if a future entry omits it, the synthesized model gets an implausibly large output cap that could produce request rejections. Consider a conservative literal default (matching the COPILOT_CONTEXT_WINDOW_FALLBACK style) as the last resort instead of contextWindow.

🟢 Minor / nits

• Redundant catalog application: synthesized models already have contextWindow/contextWindowOptions set from the catalog, then flow through withCopilotContextWindowOptions() in loadBuiltInModels, which re-looks-up the same catalog entry and re-applies. Idempotent, so not a bug — just a double lookup per model.
• policyState?.toLowerCase() === "disabled" only filters the exact "disabled" state. If CAPI ever uses other blocking states (e.g. "unauthorized"), they'd pass the gate. Fine as-is given the current catalog shape; worth a comment noting the allowlist-vs-denylist choice.
• Nice touch bumping COPILOT_CATALOG_CACHE_VERSION 4->5 so old caches lacking the new metadata fields are invalidated rather than partially rehydrated.

✅ Strengths

• canSynthesizeCopilotModel gating (picker-enabled, type === "chat", non-disabled, non-namespaced ids, mappable endpoint) is thorough and the octodemo/.../DeepSeek namespaced-skip case is explicitly tested.
• Deterministic endpoint->API preference (/v1/messages -> /responses -> /chat/completions) with a dedicated test.
• Dedup against existing builtin ids means pi-ai definitions correctly win once they ship natively.
• Docs (models.md/providers.md/rpc.md/sdk.md/settings.md) and CHANGELOG are updated consistently.
• Re: the import { describe, test } from "vitest" in the new test — that's correct for this package (283 test files use vitest; the bun:test rule in CLAUDE.md applies to the companion raw-TS packages, not the upstream-derived coding-agent), so no change needed there.

Overall: solid, well-tested work. The must-fix items are (1) syncing the PR description to the real implementation and (2) restoring the stripped catalog docs (ideally via a file split). The cost/maxTokens items are worth a quick confirm.

Reviewed by Claude (Opus 4.8).

[claude]

Code Review — dynamic GitHub Copilot model synthesis

Nice refactor. Replacing a hardcoded two-model allowlist with a general, catalog-driven synthesis mechanism is the right call, and the gating (picker-enabled + chat + non-disabled + plain-id + mappable endpoint) is appropriately conservative. Built-ins winning over synthesized entries, dedup-by-id, and the endpoint→API preference order are all sound. Test coverage is genuinely good — endpoint mapping, thinking-level derivation, the full gating matrix (non-picker/disabled/non-chat/namespaced/duplicate), and deterministic endpoint preference are all exercised. Docs and changelog are thorough.

A few things worth a look:

Potential bug — maxTokens can fall back to the full context window

copilot-model-synthesis.ts:91

maxTokens: entry.maxTokens ?? entry.limits?.maxOutputTokens ?? entry.maxInputTokens ?? entry.contextWindow,

Both entry.maxTokens and entry.limits?.maxOutputTokens derive from CAPI's max_output_tokens. If a catalog entry omits that field, this chain falls through to maxInputTokens (e.g. 936k) or contextWindow (e.g. 1M) and sets max output tokens to a prompt/context-sized value. A client that then requests output up to that budget will likely hit a server-side rejection. Since this is now the generic path for any future Copilot model, a saner terminal fallback (a modest capped default rather than the input/context window) would be safer than trusting every entry to advertise max_output_tokens.

Maintainability — duplicated capability/limit parsing

copilot-model-catalog.ts has two near-identical parsers that differ only in key casing:

• parseCopilotSupports (snake_case, adaptive_thinking) vs sanitizeCachedSupports (camelCase, adaptiveThinking)
• parseCopilotLimits vs sanitizeCachedLimits

Adding a new supports flag or limit now means editing two functions with different casings — easy to update one and silently drop the field on the cache round-trip. Worth factoring the field list into a single shared table keyed by (wire-name, cache-name) so both paths stay in sync.

File-length gate proximity

copilot-model-catalog.ts is now 498 lines against the repo's hard 500-line gate. It'll pass CI today, but the next small addition to this module will trip check:file-length with no headroom. Consider proactively splitting the disk-cache/sanitization half into its own module now.

Minor / discussion

• Zeroed cost for synthesized models is documented as intentional (Copilot bills separately), but note this is a behavior change from the first commit's hardcoded real costs — any cost display/estimate for these models will now read $0. Fine if that's the accepted tradeoff.
• Gating depends on CAPI reliably populating capabilities.type, model_picker_enabled, and supported_endpoints. The !== true / !== \"chat\" checks mean that if CAPI ever omits one of these for a legitimately selectable chat model, it silently won't be synthesized. The conservative default is defensible, but it does make the feature's coverage dependent on CAPI field completeness — worth a code comment so the next reader knows the exclusion is intentional, not a bug.
• Good catch removing the duplicate describeModelRegistry import in the context-window suite.

Notes

• Test framework: the new suite imports from vitest, which matches the rest of packages/coding-agent (upstream pi layout) — consistent, despite CLAUDE.md's bun:test guidance for the workspace at large.
• I was unable to execute vitest/typecheck in this review sandbox, so I'm relying on the PR's stated green validation for those. The 498-line count I confirmed directly.

Overall this is a solid, well-tested change. The maxTokens fallback is the one item I'd want addressed (or explicitly reasoned about) before merge; the rest are maintainability/robustness suggestions.

[claude]

Review: dynamic GitHub Copilot model synthesis

Nice work — this is a well-structured change. The catalog-driven approach (zero hardcoded model ids), the disk-cache schema bump to v5, host-matching + TTL, and the defensive re-sanitization of cached data are all solid. The stale-active-session fix (refreshCurrentModelFromRegistry after the catalog lands) is a good catch, and test coverage across synthesis, gating, cache round-trip, endpoint precedence, and the session-refresh regression is genuinely thorough. Comments below, roughly ordered by impact.

Bugs / correctness

1. maxTokens output cap can fall back to the full context window. In copilot-model-synthesis.ts:91:

maxTokens: entry.maxTokens ?? entry.limits?.maxOutputTokens ?? entry.maxInputTokens ?? entry.contextWindow,

When CAPI advertises neither max_output_tokens nor a per-context maxTokens, the synthesized model claims an output budget equal to its input/context window (e.g. 256k output tokens) — not a physically valid output cap, and it could produce bad request-shaping downstream. #1584's max_output_tokens plumbing makes this rare in practice, but a conservative literal default (matching what the builtin copilot models use) would be safer than ?? entry.maxInputTokens ?? entry.contextWindow.

2. _refreshCurrentModelFromRegistry bypasses the _emitModelChanged guard. At agent-session-extension-bindings.ts:124 it emits directly:

this._emit({ type: "model_changed", model: replay.model, previousModel, source: "restore" });

whereas _emitModelChanged (agent-session-models.ts:45) skips the emit when modelsAreEqual(previousModel, nextModel). The refreshedModel === currentModel early-return at line 111 only covers reference equality, so a registry refresh that produces a structurally-equal but new object will now fire a spurious model_changed on every registerProvider/unregisterProvider. Consider routing through _emitModelChanged for the dedupe. (This path also emits the internal model_changed but not the extension model_select hook that setModel fires — presumably intentional for a restore, but worth a one-line comment so it doesn't read as an oversight.)

Maintainability

3. copilot-model-catalog.ts is at 498/500 lines — one small addition from breaking the check:file-length gate. Given how much it grew (parsing + supports/limits sanitization + active state + disk cache), splitting the disk-cache half (read/write/seed/sanitize*) into a copilot-model-catalog-cache.ts sibling would clear the gate and improve cohesion.

4. Dropped JSDoc on nontrivial logic. The diff removes the detailed rationale comments (the max_prompt_tokens vs max_context_window_tokens context-window explanation, resolveCopilotModelContext, readCopilotCatalogCache, etc.). The branded-total-vs-input-cap logic is subtle and future readers will want that reasoning; if this was to stay under 500 lines, that's another argument for #3 rather than for deleting the docs.

Minor

• withCopilotThinkingLevelMap (builtins) recomputes copilotThinkingLevelMapFor for synthesized models that already set it during synthesis, and withCopilotContextWindowOptions re-applies from the same catalog entry. Idempotent, so harmless — just redundant passes.
• Test convention is slightly mixed: copilot-model-synthesis.test.ts uses vitest (the package's dominant convention) while agent-session-copilot-catalog-refresh.test.ts uses bun:test. Not wrong, just inconsistent within one PR.

Security

No concerns — no new secrets/logging, catalog fetch reuses the existing auth-gated token flow, cache is host-scoped, and cached fields are re-validated on read (sanitizeCachedContext/Supports/Limits). Cost is correctly zeroed for subscription billing.

Overall a clean, well-tested change. #1 (output-cap fallback) is the only item I'd consider blocking; the rest are polish.

_{🤖 Automated review by Claude Opus 4.8}

[claude]

Review: Dynamic GitHub Copilot model synthesis from CAPI catalog

Overall this is a clean, well-structured change that meaningfully improves on the "hardcode two model ids" alternative. The catalog-driven synthesis is nicely factored (copilot-model-synthesis.ts is small and focused), the endpoint→API preference is deterministic, and test coverage is genuinely strong — endpoint mapping, capability/thinking-level derivation, all the gating cases (non-picker/disabled/non-chat/namespaced/duplicate), disk-cache round-tripping, and the live session-refresh path are all exercised. Nice work.

A few things worth a look before merge:

1. Session refresh persists a clamped thinking level into global settings (medium)

_refreshCurrentModelFromRegistry() now calls this.setThinkingLevel(previousThinkingLevel) to re-clamp (agent-session-extension-bindings.ts:122). setThinkingLevel() (agent-session-models.ts:198-203) persists on any change via settingsManager.setDefaultThinkingLevel(), which writes the global defaultThinkingLevel to disk.

So when the live catalog restricts a Copilot model to fewer levels than the fallback offered, the automatic refresh silently rewrites the user's global thinking preference. Concretely: user launches with defaultThinkingLevel: "xhigh" on a Copilot model whose live catalog tops out at high → refresh clamps to high → high is persisted globally → switching to an Anthropic/Cursor model that does support xhigh now inherits the downgraded high, and the original preference is gone.

Note the asymmetry: this refresh path deliberately avoids persisting the model change (unlike the explicit /model switch flows at agent-session-models.ts:132-134,164-166), but it does persist the thinking-level clamp as a side effect. Clamping in explicit user-initiated switches is intended; doing it automatically in a background catalog refresh is probably not. Consider clamping the in-session agent.state.thinkingLevel without going through the persisting setThinkingLevel(), so the on-disk global default is left intact.

2. `model_changed` now fires on unrelated `unregisterProvider` calls (low)

_refreshCurrentModelFromRegistry() now unconditionally emits model_changed (source: "restore") whenever refreshedModel !== currentModel by reference (agent-session-extension-bindings.ts:124). For the intended Copilot catalog-apply path this is exactly right. But this method is also invoked from refreshCurrentModelFromRegistry() on every extension unregisterProvider(), and ModelRegistry.unregisterProvider() → refresh() → loadModels() rebuilds the entire models array, so every active model gets a fresh object identity — even one belonging to a completely unrelated provider that didn't change. Previously this path emitted no model_changed; now an unrelated provider teardown will emit a spurious model-change event, which downstream UI/listeners may treat as a real switch. (registerProvider is fine — it preserves identity for unrelated providers via filter.) Consider gating the emit on a semantic diff (id/provider/contextWindow/thinkingLevelMap) rather than reference inequality.

3. Policy gating only excludes exactly `"disabled"` (low)

canSynthesizeCopilotModel() skips a model only when policyState?.toLowerCase() === "disabled" (copilot-model-synthesis.ts:574). If CAPI ever reports other non-active states (e.g. unconfigured, blocked, pending), those models would still be synthesized and exposed. If the intent is "only expose usable models," an allowlist (enabled / undefined) is safer than a denylist of one string.

4. `copilot-model-catalog.ts` is now 498 lines (nit)

Right up against the 500-line check:file-length gate — the next edit to this file will trip it. The newly added sanitizeCached* helpers are natural candidates to move into a sibling module (e.g. copilot-model-catalog-cache.ts) proactively.

5. Removed field-level JSDoc (nit)

The PR trims the per-field doc comments on CopilotModelContext and CopilotModelLimits. The CAPI limit semantics (max_prompt_tokens vs max_context_window_tokens vs context_max, and which becomes maxInputTokens vs the displayed window) are genuinely non-obvious; those comments were carrying real weight. The consolidated module docstring covers the why but the field-level mapping notes were useful — consider keeping the trickiest ones.

Minor

• Synthesized models re-run withCopilotContextWindowOptions/withCopilotThinkingLevelMap in loadBuiltInModels() even though they were already built from the same catalog entry — harmless no-op, but a small redundancy.

None of these block the core mechanism, which is solid. #1 is the one I'd most want addressed since it silently mutates a user preference across providers.