diff --git a/packages/coding-agent/CHANGELOG.md b/packages/coding-agent/CHANGELOG.md index 83aa191c6..9421521e9 100644 --- a/packages/coding-agent/CHANGELOG.md +++ b/packages/coding-agent/CHANGELOG.md @@ -2,9 +2,15 @@ ## [Unreleased] +### Added + +- Added dynamic GitHub Copilot model population from the live CAPI `/models` catalog: picker-enabled, non-disabled plain chat ids are synthesized from catalog metadata (endpoints, capabilities, limits, and display names) while built-in `pi-ai` definitions still win, namespaced enterprise deployments such as `org/deployment/model` are skipped, and cached catalog metadata enables the same models on cold start. +- Added catalog-driven thinking-level gating for GitHub Copilot models so dynamically synthesized entries and bundled `pi-ai` Copilot models only offer the reasoning levels advertised by CAPI's `capabilities.supports.reasoning_effort` arrays, while models without an effort array keep their existing thinking behavior. + ### Fixed - Fixed GitHub Copilot models to use the live `max_output_tokens` value from the Copilot model catalog, preventing `github-copilot/claude-opus-4.8` from being capped by Atomic's stale built-in output-token limit after compaction ([#1582](https://github.com/bastani-inc/atomic/issues/1582)). +- Fixed active GitHub Copilot sessions to adopt live catalog model metadata as soon as the catalog loads, so fallback models refresh their supported reasoning levels without requiring a restart. ## [0.9.4-alpha.6] - 2026-07-01 diff --git a/packages/coding-agent/docs/models.md b/packages/coding-agent/docs/models.md index c1de984d6..d28ee230b 100644 --- a/packages/coding-agent/docs/models.md +++ b/packages/coding-agent/docs/models.md @@ -271,15 +271,15 @@ Users can select a supported context window independently from thinking level: atomic --model custom/long-context-model --thinking high --context-window 1m ``` -In interactive mode, run `/model` and pick a model; when the chosen model exposes more than one window, Atomic immediately prompts for the context window as a follow-up step — a GitHub Copilot CLI-style picker that lists numbered `Default` and `Long context` tiers with their token counts (for example `272k tokens` / `922k tokens` for `github-copilot/gpt-5.5`, or `200k tokens` / `936k tokens` for the Claude/Gemini long-context models) — so you can choose one of the active model's supported budgets. Persisted interactive selections are stored per model under `defaultContextWindows["provider/modelId"]` (raw token counts and compact labels such as `400k` or `1m` are accepted), so a Copilot-specific prompt cap does not leak into Anthropic, Cursor, or other providers. GitHub Copilot long-context requests treat `1m` as a branded budget request and resolve it to the model's largest advertised long-context tier not exceeding the request (for example `936k` for Copilot Claude Opus), while other providers continue to require one of their own exact supported windows or use their natural scalar default. Successful explicit startup selections are recorded as `context_window_change` entries even when the chosen value equals the scalar default, preserving the user's explicit budget choice across future settings changes and resume. +In interactive mode, run `/model` and pick a model; when the chosen model exposes more than one window, Atomic immediately prompts for the context window as a follow-up step — a GitHub Copilot CLI-style picker that lists numbered `Default` and `Long context` tiers with their token counts (for example `272k tokens` / `922k tokens` for `github-copilot/gpt-5.5`, or `200k tokens` / `936k tokens` for Claude/Gemini long-context models such as `github-copilot/claude-sonnet-5`) — so you can choose one of the active model's supported budgets. Persisted interactive selections are stored per model under `defaultContextWindows["provider/modelId"]` (raw token counts and compact labels such as `400k` or `1m` are accepted), so a Copilot-specific prompt cap does not leak into Anthropic, Cursor, or other providers. GitHub Copilot long-context requests treat `1m` as a branded budget request and resolve it to the model's largest advertised long-context tier not exceeding the request (for example `936k` for Copilot Claude models), while other providers continue to require one of their own exact supported windows or use their natural scalar default. Successful explicit startup selections are recorded as `context_window_change` entries even when the chosen value equals the scalar default, preserving the user's explicit budget choice across future settings changes and resume. Use larger context windows deliberately. Some providers charge more for larger windows, and Atomic preserves each model's default unless the user explicitly opts in through `--context-window`, the `/model` selection flow, per-model `defaultContextWindows`, or the optional global `defaultContextWindow` fallback. #### GitHub Copilot context windows -GitHub Copilot context windows are measured in **input (prompt) tokens**, exactly like every other provider's `contextWindow`, and are derived **dynamically from GitHub's live CAPI model catalog** (`GET {baseUrl}/models`) rather than a hardcoded model list — so models GitHub adds, removes, or retiers are reflected automatically. Atomic fetches the catalog only when you actually have the GitHub Copilot provider authenticated and caches it on disk for 30 minutes. The same catalog also supplies Copilot output-token caps: when CAPI advertises `capabilities.limits.max_output_tokens`, Atomic uses that live value as the model's `maxTokens` instead of the bundled fallback. +GitHub Copilot context windows are measured in **input (prompt) tokens**, exactly like every other provider's `contextWindow`, and are derived **dynamically from GitHub's live CAPI model catalog** (`GET {baseUrl}/models`) rather than a hardcoded model list — so plain Copilot model ids GitHub adds, removes, or retiers are reflected automatically. Atomic fetches the catalog only when you actually have the GitHub Copilot provider authenticated, caches it on disk for 30 minutes, and refreshes the active interactive session's model metadata as soon as that catalog is applied. Dynamic model creation is intentionally limited to picker-enabled, non-disabled `chat` entries with plain, non-namespaced ids; enterprise/org-deployed catalog ids containing `/` (for example `octodemo/Octodemo_Foundry/DeepSeek-V3.2`) are skipped instead of being exposed as `github-copilot/*` models. The same catalog also supplies Copilot output-token caps and selectable thinking levels: when CAPI advertises `capabilities.limits.max_output_tokens`, Atomic uses that live value as the model's `maxTokens` instead of the bundled fallback; when CAPI advertises `capabilities.supports.reasoning_effort` as an array, Atomic hides unsupported thinking levels for both dynamically synthesized Copilot models and bundled `pi-ai` Copilot models. Models that only advertise thinking budgets or a boolean reasoning-effort flag keep their existing thinking-level behavior. -Each selectable Copilot window is a prompt/input budget. Atomic reads `capabilities.limits.max_prompt_tokens` for the full prompt cap, `capabilities.limits.max_output_tokens` for the maximum response/output cap, and treats `capabilities.limits.max_context_window_tokens` as the model's total context capacity (prompt plus output reserve) and a compatibility fallback only when the prompt cap is absent. Models with tiered pricing expose their per-tier prompt budgets through `billing.token_prices..context_max`: the `default` tier becomes the base window and a larger `long_context` tier is offered as a selectable option. For example `github-copilot/gpt-5.5` resolves to a `272k` default / `922k` long window, and the Claude/Gemini long-context models resolve to `200k` default / `936k` long. When the request is a rounded budget such as `1m`, Atomic selects the largest advertised Copilot long-context prompt tier at or below that budget instead of falling back to the base `200k`/`272k` window. Offline, unauthenticated, or non-Copilot sessions leave the built-in scalar window and output-token cap untouched and show no picker. +Each selectable Copilot window is a prompt/input budget. Atomic reads `capabilities.limits.max_prompt_tokens` for the full prompt cap, `capabilities.limits.max_output_tokens` for the maximum response/output cap, and treats `capabilities.limits.max_context_window_tokens` as the model's total context capacity (prompt plus output reserve) and a compatibility fallback only when the prompt cap is absent. Models with tiered pricing expose their per-tier prompt budgets through `billing.token_prices..context_max`: the `default` tier becomes the base window and a larger `long_context` tier is offered as a selectable option. For example `github-copilot/gpt-5.5` resolves to a `272k` default / `922k` long prompt budget, `github-copilot/mai-code-1-flash-picker` advertises a `128k` prompt cap with a `256k` total context window, and `github-copilot/claude-sonnet-5` plus other Claude/Gemini long-context models resolve to `200k` default / `936k` long. When the request is a rounded budget such as `1m`, Atomic selects the largest advertised Copilot long-context prompt tier at or below that budget instead of falling back to the base `200k`/`272k` window. Offline, unauthenticated, or non-Copilot sessions leave the built-in scalar window and output-token cap untouched and show no picker. Selecting the long-context window does two client-side things: diff --git a/packages/coding-agent/docs/providers.md b/packages/coding-agent/docs/providers.md index 97a407b54..4c3fbf331 100644 --- a/packages/coding-agent/docs/providers.md +++ b/packages/coding-agent/docs/providers.md @@ -40,7 +40,7 @@ Anthropic subscription auth is active for Claude Pro/Max accounts. Third-party h - Press Enter for github.com, or enter your GitHub Enterprise Server domain - When using `COPILOT_GITHUB_TOKEN` instead of `/login`, Atomic uses the token's `proxy-ep` when present, honors `COPILOT_API_TARGET` or `GITHUB_COPILOT_BASE_URL` overrides, derives `copilot-api..ghe.com` from `GITHUB_SERVER_URL=*.ghe.com`, derives `https://api.enterprise.githubcopilot.com` from other non-`github.com` server URLs, and otherwise falls back to the public Copilot routing hub `https://api.githubcopilot.com` instead of the account-specific individual endpoint. - If you get "model not supported", enable it in VS Code: Copilot Chat → model selector → select model → "Enable" -- Supported built-in Copilot long-context models, including `github-copilot/gpt-5.5`, `github-copilot/claude-opus-4.8`, and `github-copilot/gemini-3.1-pro-preview`, expose an opt-in long-context choice through `--context-window`, the `/model` selection flow, per-model `defaultContextWindows`, SDK, and RPC controls. The long-context option advertises the model's full context window (for example `1m` or `1.05m` — GitHub's `max_context_window_tokens`), matching how the native `openai/*` and `anthropic/*` providers report these models and what the chat footer shows. GitHub's lower server-side prompt cap (`max_prompt_tokens`, for example `936k` or `922k`) is retained internally as the effective input budget that drives compaction thresholds and overflow recovery, and GitHub's live output cap (`max_output_tokens`) replaces Atomic's bundled `maxTokens` fallback for provider requests. This lets Atomic display the branded context window and request the catalog-advertised output budget without overrunning server limits. +- GitHub Copilot models are populated dynamically from Copilot's live CAPI `/models` catalog when Copilot auth is available. Atomic synthesizes only picker-enabled, non-disabled `chat` entries with plain ids (for example `github-copilot/claude-sonnet-5` and `github-copilot/mai-code-1-flash-picker`); namespaced enterprise deployments containing `/` are skipped rather than exposed as `github-copilot/*` models. Models that advertise long-context limits, such as `github-copilot/gpt-5.5`, `github-copilot/claude-opus-4.8`, and `github-copilot/gemini-3.1-pro-preview`, expose an opt-in long-context choice through `--context-window`, the `/model` selection flow, per-model `defaultContextWindows`, SDK, and RPC controls. The long-context option advertises the model's full context window (for example `1m` or `1.05m` — GitHub's `max_context_window_tokens`), matching how the native `openai/*` and `anthropic/*` providers report these models and what the chat footer shows. GitHub's lower server-side prompt cap (`max_prompt_tokens`, for example `936k` or `922k`) is retained internally as the effective input budget that drives compaction thresholds and overflow recovery, and GitHub's live output cap (`max_output_tokens`) replaces Atomic's bundled `maxTokens` fallback for provider requests. If CAPI advertises `capabilities.supports.reasoning_effort` as an array, Atomic also gates `/model` and thinking-level cycling to only those live levels for both dynamic Copilot models and bundled `pi-ai` Copilot models; budget-only or boolean-only reasoning metadata leaves the existing thinking map untouched. Active interactive sessions refresh from this metadata as soon as the catalog is applied, so a startup fallback model does not keep stale reasoning levels until restart. This lets Atomic display the branded context window, request the catalog-advertised output budget, and avoid offering unsupported Copilot reasoning levels. - Selecting long context sets Atomic's displayed window to the model's full capacity while compaction triggers against the effective prompt-token budget, and makes Copilot requests include `X-GitHub-Api-Version: 2026-06-01`. Atomic does not send a body field, `contextTier`, or model-id variant; GitHub automatically applies the server-side `long_context` tier when prompt tokens exceed the default budget. - Long-context Copilot requests consume more AI credits and require Copilot long-context/usage-based billing entitlement. A prompt that reaches the model's normal prompt cap is compacted and retried automatically. Only when GitHub rejects a prompt *below* that cap — for example because the account lacks the long-context/usage-based billing entitlement and is dropped to a smaller server tier — does Atomic surface a friendly entitlement/server-cap/cost hint rather than silently truncating context. - **Gemini models** (`github-copilot/gemini-3.1-pro-preview`, `github-copilot/gemini-3.5-flash`, …) are served through Copilot's CAPI gateway, which re-translates the OpenAI request into Google's GenAI format and enforces Gemini's stricter `FunctionDeclaration` schema (it rejects a tool-parameter `anyOf`/`oneOf` whose branch is a complex object, returning `400 invalid request body`). Atomic automatically sanitizes outbound tool/function JSON Schemas for these models into the supported subset — resolving object/array-bearing unions to their most expressive branch, converting `const`/literal unions to `enum`, collapsing nullable unions to `nullable`, and dropping non-portable keywords such as `additionalProperties`, `patternProperties`, `format`, and numeric/length bounds. Gemini also serializes array/object tool-call **arguments** as flattened indexed keys (`keywords[0]`, `keywords[1]`, …); Atomic reconstructs these back into proper arrays/objects before validation so tool calls (including `structured_output` and MCP tools) don't fail and loop. Both transforms are transparent and scoped to GitHub Copilot Gemini models only; no configuration is required and other providers/models are unaffected. diff --git a/packages/coding-agent/docs/rpc.md b/packages/coding-agent/docs/rpc.md index 8aef4c74b..52a0df252 100644 --- a/packages/coding-agent/docs/rpc.md +++ b/packages/coding-agent/docs/rpc.md @@ -335,7 +335,7 @@ Unsupported or malformed selections return the standard RPC error response: } ``` -Larger provider context windows may consume more credits/cost. For allowlisted GitHub Copilot long-context models (including `github-copilot/gpt-5.5` and `github-copilot/gemini-3.1-pro-preview`), selecting `1m` raises Atomic's local budget and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub applies the long-context billing tier server-side by prompt token count. That tier consumes more Copilot AI credits and requires Copilot long-context/usage-based billing entitlement, otherwise requests over GitHub's server cap are rejected with a friendly hint. +Larger provider context windows may consume more credits/cost. For catalog-advertised GitHub Copilot long-context models (including `github-copilot/gpt-5.5`, `github-copilot/claude-sonnet-5`, and `github-copilot/gemini-3.1-pro-preview`), selecting `1m` raises Atomic's local budget and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub applies the long-context billing tier server-side by prompt token count. That tier consumes more Copilot AI credits and requires Copilot long-context/usage-based billing entitlement, otherwise requests over GitHub's server cap are rejected with a friendly hint. ### Thinking @@ -1042,7 +1042,7 @@ Emitted when the active context-window token budget changes through RPC `set_con } ``` -Larger provider context windows may consume more credits/cost. Prefer the model default unless the additional repository/session context is useful for the current task. For allowlisted GitHub Copilot long-context models such as `github-copilot/gpt-5.5` and `github-copilot/gemini-3.1-pro-preview`, a `1m` selection raises Atomic's local budget and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub applies the long-context billing tier server-side by prompt size, consumes more Copilot AI credits, and requires long-context/usage-based billing entitlement. +Larger provider context windows may consume more credits/cost. Prefer the model default unless the additional repository/session context is useful for the current task. For catalog-advertised GitHub Copilot long-context models such as `github-copilot/gpt-5.5`, `github-copilot/claude-sonnet-5`, and `github-copilot/gemini-3.1-pro-preview`, a `1m` selection raises Atomic's local budget and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub applies the long-context billing tier server-side by prompt size, consumes more Copilot AI credits, and requires long-context/usage-based billing entitlement. ### compaction_start / compaction_end diff --git a/packages/coding-agent/docs/sdk.md b/packages/coding-agent/docs/sdk.md index a704caf94..3fe8dc3ff 100644 --- a/packages/coding-agent/docs/sdk.md +++ b/packages/coding-agent/docs/sdk.md @@ -439,7 +439,7 @@ If no model is provided: Context-window selection is independent from `thinkingLevel`. `contextWindow` accepts a raw token count such as `400_000` or `1_000_000`; for most providers the value must be present in the model's supported context windows (`model.contextWindowOptions` plus the scalar default). GitHub Copilot is the only provider with rounded long-context budget handling: when a tiered Copilot model advertises a long tier below the branded request (for example `936_000` for a `1_000_000` request), Atomic selects the largest advertised Copilot long tier at or below the request instead of falling back to the short tier. Settings lookup first checks the selected model's `defaultContextWindows["provider/modelId"]` entry, then the optional global `defaultContextWindow` fallback; unsupported model-specific settings keep the model default and return `contextWindowWarning`, while unsupported global fallback values are ignored silently as not applicable to the active model. When you pass `contextWindowStrict: true`, an unsupported explicit selection is reported as `contextWindowError` so callers can fail before prompting. A successful explicit `contextWindow` startup option is journaled as a `context_window_change` entry even when it equals the scalar model default, so the user's explicit budget choice survives future settings changes and resume. -At runtime, use `session.getAvailableContextWindows()` to inspect supported values, `session.supportsContextWindowSelection()` to check whether more than one value is selectable, and `session.setContextWindow(tokens, { persistDefault })` to change the active model budget. `setContextWindow()` journals a `context_window_change` entry only when the active value changes. Passing `{ persistDefault: true }` also writes the effective selected budget to `defaultContextWindows["provider/modelId"]` in settings instead of the global fallback, so a Copilot prompt cap such as `936k` does not leak into Anthropic, Cursor, or other providers. Tree navigation replays the target branch's `context_window_change` state into the active model without adding another journal entry or changing settings. Larger provider context windows may consume more credits/cost, so opt into larger values deliberately. For allowlisted GitHub Copilot long-context models (including `github-copilot/gpt-5.5` and `github-copilot/gemini-3.1-pro-preview`), selecting `1m` raises Atomic's local budget to the model's advertised `922k`/`936k` tier and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub applies the long-context tier server-side by prompt token count, consumes more Copilot AI credits, and requires long-context/usage-based billing entitlement. +At runtime, use `session.getAvailableContextWindows()` to inspect supported values, `session.supportsContextWindowSelection()` to check whether more than one value is selectable, and `session.setContextWindow(tokens, { persistDefault })` to change the active model budget. `setContextWindow()` journals a `context_window_change` entry only when the active value changes. Passing `{ persistDefault: true }` also writes the effective selected budget to `defaultContextWindows["provider/modelId"]` in settings instead of the global fallback, so a Copilot prompt cap such as `936k` does not leak into Anthropic, Cursor, or other providers. Tree navigation replays the target branch's `context_window_change` state into the active model without adding another journal entry or changing settings. Larger provider context windows may consume more credits/cost, so opt into larger values deliberately. For catalog-advertised GitHub Copilot long-context models (including `github-copilot/gpt-5.5`, `github-copilot/claude-sonnet-5`, and `github-copilot/gemini-3.1-pro-preview`), selecting `1m` raises Atomic's local budget to the model's advertised `922k`/`936k` tier and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub applies the long-context tier server-side by prompt token count, consumes more Copilot AI credits, and requires long-context/usage-based billing entitlement. The package root exports the same context-window helpers and types used by the runtime: `parseContextWindowValue()`, `formatContextWindow()`, `validateContextWindowValue()`, `normalizeContextWindowOptions()`, `getModelDefaultContextWindow()`, `getSupportedContextWindows()`, `withContextWindowOptions()`, `selectContextWindow()`, `ContextWindowParseResult`, `ContextWindowSelection`, `ContextWindowSelectionError`, and `ContextWindowSelectionOptions`. Importing from `@bastani/atomic` also includes the `@earendil-works/pi-ai` `Model` augmentation for `contextWindowOptions` and `defaultContextWindow`, so SDK consumers can use the helper types without importing internal source paths. diff --git a/packages/coding-agent/docs/settings.md b/packages/coding-agent/docs/settings.md index 975e97f62..0d7c5b23b 100644 --- a/packages/coding-agent/docs/settings.md +++ b/packages/coding-agent/docs/settings.md @@ -250,12 +250,13 @@ When multiple sources specify a session directory, precedence is `--session-dir` "defaultContextWindow": "1m", "defaultContextWindows": { "github-copilot/claude-opus-4.8": "936k", + "github-copilot/claude-sonnet-5": "936k", "github-copilot/gpt-5.5": "922k" } } ``` -Context-window settings are independent of `defaultThinkingLevel`: selecting a larger context window does not change reasoning effort. Interactive users can change the active model's budget through the `/model` selection flow, which prompts for a context window whenever the chosen model supports more than one window and persists the effective selection under `defaultContextWindows["provider/modelId"]`. Atomic treats `defaultContextWindow` as a broad fallback only: if the active model does not support that value, the model's own default is used without a startup warning; targeted `defaultContextWindows` entries still warn when they become unsupported for their exact model. Larger provider context windows can carry higher usage cost. For GitHub Copilot allowlisted long-context models (including `github-copilot/gpt-5.5` and `github-copilot/gemini-3.1-pro-preview`), selecting `1m` raises Atomic's local prompt budget to the largest advertised long-context tier at or below that rounded request (for example `922k` or `936k`) and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub then applies the long-context tier server-side by prompt token count. That tier consumes more Copilot AI credits and requires Copilot long-context/usage-based billing entitlement, otherwise requests over the server cap are rejected with a friendly hint. Custom providers and explicit model overrides can still declare their own selectable `contextWindowOptions`. +Context-window settings are independent of `defaultThinkingLevel`: selecting a larger context window does not change reasoning effort. Interactive users can change the active model's budget through the `/model` selection flow, which prompts for a context window whenever the chosen model supports more than one window and persists the effective selection under `defaultContextWindows["provider/modelId"]`. Atomic treats `defaultContextWindow` as a broad fallback only: if the active model does not support that value, the model's own default is used without a startup warning; targeted `defaultContextWindows` entries still warn when they become unsupported for their exact model. Larger provider context windows can carry higher usage cost. For catalog-advertised GitHub Copilot long-context models (including dynamically populated plain catalog ids such as `github-copilot/claude-sonnet-5`, while namespaced enterprise deployment ids containing `/` are skipped), selecting `1m` raises Atomic's local prompt budget to the largest advertised long-context tier at or below that rounded request (for example `922k` or `936k`) and sends `X-GitHub-Api-Version: 2026-06-01`; GitHub then applies the long-context tier server-side by prompt token count. That tier consumes more Copilot AI credits and requires Copilot long-context/usage-based billing entitlement, otherwise requests over the server cap are rejected with a friendly hint. Custom providers and explicit model overrides can still declare their own selectable `contextWindowOptions`. ### Markdown diff --git a/packages/coding-agent/src/core/agent-session-extension-bindings.ts b/packages/coding-agent/src/core/agent-session-extension-bindings.ts index 176b9f23b..c878f4062 100644 --- a/packages/coding-agent/src/core/agent-session-extension-bindings.ts +++ b/packages/coding-agent/src/core/agent-session-extension-bindings.ts @@ -97,6 +97,10 @@ export function _applyExtensionBindings(this: AgentSession, runner: ExtensionRun } +export function refreshCurrentModelFromRegistry(this: AgentSession): void { + this._refreshCurrentModelFromRegistry(); +} + export function _refreshCurrentModelFromRegistry(this: AgentSession): void { const currentModel = this.model; if (!currentModel) { @@ -108,12 +112,16 @@ export function _refreshCurrentModelFromRegistry(this: AgentSession): void { return; } + const previousModel = currentModel; + const previousThinkingLevel = this.thinkingLevel; const replay = this._getResumeContextWindowReplayForModel(refreshedModel); this.agent.state.model = replay.model; if (currentModel.contextWindow !== replay.contextWindow) { this._emit({ type: "context_window_changed", contextWindow: replay.contextWindow }); } + this.setThinkingLevel(previousThinkingLevel); this._refreshBaseSystemPromptFromActiveTools(); + this._emit({ type: "model_changed", model: replay.model, previousModel, source: "restore" }); } @@ -220,11 +228,11 @@ export function _bindExtensionCore(this: AgentSession, runner: ExtensionRunner): { registerProvider: (name, config) => { this._modelRegistry.registerProvider(name, config); - this._refreshCurrentModelFromRegistry(); + this.refreshCurrentModelFromRegistry(); }, unregisterProvider: (name) => { this._modelRegistry.unregisterProvider(name); - this._refreshCurrentModelFromRegistry(); + this.refreshCurrentModelFromRegistry(); }, }, ); @@ -269,6 +277,7 @@ export const agentSessionExtensionBindingsMethods = { buildExtensionResourcePaths, getExtensionSourceLabel, _applyExtensionBindings, + refreshCurrentModelFromRegistry, _refreshCurrentModelFromRegistry, _bindExtensionCore, reload, diff --git a/packages/coding-agent/src/core/agent-session-methods.ts b/packages/coding-agent/src/core/agent-session-methods.ts index a9bd0c403..99e51e537 100644 --- a/packages/coding-agent/src/core/agent-session-methods.ts +++ b/packages/coding-agent/src/core/agent-session-methods.ts @@ -210,6 +210,7 @@ export interface AgentSessionMethodSurface { getExtensionSourceLabel(extensionPath: string): string; _applyExtensionBindings(runner: ExtensionRunner): void; _refreshCurrentModelFromRegistry(): void; + refreshCurrentModelFromRegistry(): void; _bindExtensionCore(runner: ExtensionRunner): void; _refreshToolRegistry(options?: { activeToolNames?: string[]; includeAllExtensionTools?: boolean }): void; _buildRuntime(options: RuntimeBuildOptions): void; @@ -301,6 +302,7 @@ export interface AgentSessionPublicSurface extends Pick.context_max` is a prompt-token billing/selection threshold. The - * `default` tier is the short prompt budget (e.g. gpt-5.5 272k, Claude 200k); a - * `long_context` tier adds a selectable larger prompt budget (e.g. gpt-5.5 922k, Claude 936k). - * - * Atomic shows the model's full context window for the selectable long tier (the - * `max_context_window_tokens` total, e.g. 1_000_000/1_050_000), matching how the native `openai/*` - * and `anthropic/*` providers advertise these models. Because GitHub enforces a lower server-side - * prompt cap (`max_prompt_tokens`, e.g. 936k/922k) below that total, the prompt cap is retained as - * an internal effective input budget (`CopilotModelContext.maxInputTokens`) that drives compaction - * thresholds and the overflow-recovery guard, so the branded total can be displayed without - * overrunning the server limit. The default (short) tier stays at the `default` billing tier's - * prompt budget. - * - * This data is intentionally NOT baked into a static map: GitHub adds/removes models and retiers - * windows over time (e.g. a model that disappears from the catalog), so a hardcoded snapshot goes - * stale. Instead the catalog is fetched live (gated on the user actually having the GitHub Copilot - * provider) and cached on disk for a short TTL, exactly like the Copilot CLI. - */ +/** GitHub Copilot CAPI model catalog parsing, active state, and disk cache. */ import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; -/** Resolved context and output-token limits for a single Copilot model. */ +/** Resolved input-token context window(s) and optional synthesis metadata for a single Copilot model. */ export interface CopilotModelContext { - /** - * Base/displayed context window — shown in the footer. The default tier's `context_max`, or the - * model-level `max_prompt_tokens` fallback otherwise. - */ contextWindow: number; - /** - * Selectable windows (`[default, long]`) when the model exposes a `long_context` tier larger than - * its default; absent for single-window models. The long entry is the model's full - * `max_context_window_tokens` (total capacity) when advertised, matching `openai/*` and - * `anthropic/*`. - */ contextWindowOptions?: readonly number[]; - /** - * Hard prompt/input cap (`max_prompt_tokens`) when it sits below the displayed long window. Used - * as the effective input budget for compaction thresholds and overflow recovery so the branded - * total can be shown without overrunning GitHub's server-side prompt limit. Absent when the - * displayed window already equals the input cap. - */ maxInputTokens?: number; - /** - * Maximum output tokens (`max_output_tokens`) advertised by Copilot CAPI. When present, this - * replaces the static built-in `Model.maxTokens` so requests use GitHub's live output cap. - */ maxTokens?: number; + displayName?: string; + vendor?: string; + supportedEndpoints?: readonly string[]; + supports?: CopilotModelSupports; + limits?: CopilotModelLimits; + modelPickerEnabled?: boolean; + policyState?: string; + type?: string; +} + +export interface CopilotModelSupports { + adaptiveThinking?: boolean; + maxThinkingBudget?: boolean; + minThinkingBudget?: boolean; + parallelToolCalls?: boolean; + reasoningEffort?: boolean; + reasoningEffortLevels?: readonly string[]; + streaming?: boolean; + structuredOutputs?: boolean; + toolCalls?: boolean; + vision?: boolean; } -/** Map of model id → resolved input-token context window(s). */ +/** Map of model id → resolved context window(s) plus optional synthesis metadata. */ export type CopilotModelCatalog = ReadonlyMap; /** Safety fallback when a model reports neither `max_prompt_tokens` nor `max_context_window_tokens`. */ @@ -137,18 +112,8 @@ export function copilotTokenFromEnvironment(env: CopilotEnvironment = process.en export const COPILOT_CATALOG_CACHE_TTL_MS = 30 * 60 * 1000; /** Current on-disk cache schema version. */ -export const COPILOT_CATALOG_CACHE_VERSION = 4 as const; +export const COPILOT_CATALOG_CACHE_VERSION = 5 as const; -/** - * Resolve the Copilot CAPI base URL. - * - * Copilot access tokens embed a `proxy-ep=proxy.` segment; the API host is the same host with - * `proxy.` swapped for `api.`. Env-token routing resolves explicit `COPILOT_API_TARGET` / - * `GITHUB_COPILOT_BASE_URL` overrides, then `GITHUB_SERVER_URL` (`*.ghe.com` -> - * `copilot-api..ghe.com`, other non-github.com -> `api.enterprise.githubcopilot.com`), - * then the public Copilot routing hub `api.githubcopilot.com` for `COPILOT_GITHUB_TOKEN`. Stored - * OAuth credentials still fall back to the generated individual host when no token route is known. - */ export function copilotApiBaseUrlFromToken( token: string | undefined, enterpriseDomain?: string, @@ -181,28 +146,56 @@ function toPositiveInt(value: unknown): number | undefined { return typeof value === "number" && Number.isInteger(value) && value > 0 ? value : undefined; } -/** Raw token limits parsed from a CAPI model entry. */ export interface CopilotModelLimits { - /** `capabilities.limits.max_prompt_tokens` — maximum prompt/input budget (the hard input cap). */ maxPromptTokens?: number; - /** `capabilities.limits.max_context_window_tokens` — total context capacity (the displayed long tier). */ maxContextWindowTokens?: number; - /** `capabilities.limits.max_output_tokens` — output reserve; derives the input cap when `max_prompt_tokens` is absent. */ maxOutputTokens?: number; - /** `billing.token_prices.default.context_max` — default-tier prompt threshold. */ defaultContextMax?: number; - /** `billing.token_prices.long_context.context_max` — long-context prompt threshold. */ longContextMax?: number; } -/** - * Resolve a model's input-token context window(s) from its CAPI limits. - * - * `contextWindow` is the model's base input budget — the default tier's `context_max` when tiered, - * otherwise `max_prompt_tokens ?? max_context_window_tokens ?? 128_000`. A `long_context` tier that - * is larger than the base adds a second selectable window. Returns `undefined` when the entry - * carries no usable limit signal at all. - */ +function stringArray(value: unknown): readonly string[] | undefined { + if (!Array.isArray(value)) return undefined; + const strings = value.filter((item): item is string => typeof item === "string" && item.length > 0); + return strings.length > 0 ? strings : undefined; +} + +function supportedFlag(record: Record | undefined, key: string): boolean | undefined { + const value = record?.[key]; + if (typeof value === "boolean") return value; + if (typeof value === "number") return Number.isFinite(value) && value > 0; + if (Array.isArray(value)) return value.length > 0; + return undefined; +} + +function parseCopilotSupports(value: unknown): CopilotModelSupports | undefined { + const record = asRecord(value); + if (!record) return undefined; + const supports: CopilotModelSupports = { + adaptiveThinking: supportedFlag(record, "adaptive_thinking"), + maxThinkingBudget: supportedFlag(record, "max_thinking_budget"), + minThinkingBudget: supportedFlag(record, "min_thinking_budget"), + parallelToolCalls: supportedFlag(record, "parallel_tool_calls"), + reasoningEffort: supportedFlag(record, "reasoning_effort"), + reasoningEffortLevels: stringArray(record.reasoning_effort), + streaming: supportedFlag(record, "streaming"), + structuredOutputs: supportedFlag(record, "structured_outputs"), + toolCalls: supportedFlag(record, "tool_calls"), + vision: supportedFlag(record, "vision"), + }; + return Object.values(supports).some((flag) => flag !== undefined) ? supports : undefined; +} + +function parseCopilotLimits(limits: Record | undefined, prices: Record | undefined): CopilotModelLimits { + return { + maxPromptTokens: toPositiveInt(limits?.max_prompt_tokens), + maxContextWindowTokens: toPositiveInt(limits?.max_context_window_tokens), + maxOutputTokens: toPositiveInt(limits?.max_output_tokens), + defaultContextMax: toPositiveInt(asRecord(prices?.default)?.context_max), + longContextMax: toPositiveInt(asRecord(prices?.long_context)?.context_max), + }; +} + export function resolveCopilotModelContext(limits: CopilotModelLimits): CopilotModelContext | undefined { const hasSignal = limits.maxPromptTokens !== undefined || @@ -230,10 +223,9 @@ export function resolveCopilotModelContext(limits: CopilotModelLimits): CopilotM limits.longContextMax; // Only carry the cap when the displayed long window actually exceeds it (the branded-total // case); when they coincide there is no gap and the input budget is just the window. - const resolved: CopilotModelContext = - longWindow > inputCap - ? { contextWindow: base, contextWindowOptions: [base, longWindow], maxInputTokens: inputCap } - : { contextWindow: base, contextWindowOptions: [base, longWindow] }; + const resolved: CopilotModelContext = longWindow > inputCap + ? { contextWindow: base, contextWindowOptions: [base, longWindow], maxInputTokens: inputCap } + : { contextWindow: base, contextWindowOptions: [base, longWindow] }; return limits.maxOutputTokens !== undefined ? { ...resolved, maxTokens: limits.maxOutputTokens } : resolved; } return limits.maxOutputTokens !== undefined ? { contextWindow: base, maxTokens: limits.maxOutputTokens } : { contextWindow: base }; @@ -253,16 +245,25 @@ export function parseCopilotModelCatalog(body: unknown): CopilotModelCatalog { const id = record.id; if (typeof id !== "string" || id.length === 0) continue; - const limits = asRecord(asRecord(record.capabilities)?.limits); + const capabilities = asRecord(record.capabilities); + const limitsRecord = asRecord(capabilities?.limits); const prices = asRecord(asRecord(record.billing)?.token_prices); - const context = resolveCopilotModelContext({ - maxPromptTokens: toPositiveInt(limits?.max_prompt_tokens), - maxContextWindowTokens: toPositiveInt(limits?.max_context_window_tokens), - maxOutputTokens: toPositiveInt(limits?.max_output_tokens), - defaultContextMax: toPositiveInt(asRecord(prices?.default)?.context_max), - longContextMax: toPositiveInt(asRecord(prices?.long_context)?.context_max), - }); - if (context) catalog.set(id, context); + const limits = parseCopilotLimits(limitsRecord, prices); + const context = resolveCopilotModelContext(limits); + if (context) { + const policy = asRecord(record.policy); + catalog.set(id, { + ...context, + displayName: typeof record.name === "string" && record.name.length > 0 ? record.name : undefined, + vendor: typeof record.vendor === "string" && record.vendor.length > 0 ? record.vendor : undefined, + supportedEndpoints: stringArray(record.supported_endpoints), + supports: parseCopilotSupports(capabilities?.supports), + limits, + modelPickerEnabled: typeof record.model_picker_enabled === "boolean" ? record.model_picker_enabled : undefined, + policyState: typeof policy?.state === "string" ? policy.state : undefined, + type: typeof capabilities?.type === "string" ? capabilities.type : undefined, + }); + } } return catalog; @@ -357,6 +358,37 @@ export interface ReadCopilotCatalogCacheOptions { ttlMs?: number; } +function sanitizeCachedSupports(value: unknown): CopilotModelSupports | undefined { + const record = asRecord(value); + if (!record) return undefined; + const supports: CopilotModelSupports = { + adaptiveThinking: supportedFlag(record, "adaptiveThinking"), + maxThinkingBudget: supportedFlag(record, "maxThinkingBudget"), + minThinkingBudget: supportedFlag(record, "minThinkingBudget"), + parallelToolCalls: supportedFlag(record, "parallelToolCalls"), + reasoningEffort: supportedFlag(record, "reasoningEffort"), + reasoningEffortLevels: stringArray(record.reasoningEffortLevels), + streaming: supportedFlag(record, "streaming"), + structuredOutputs: supportedFlag(record, "structuredOutputs"), + toolCalls: supportedFlag(record, "toolCalls"), + vision: supportedFlag(record, "vision"), + }; + return Object.values(supports).some((flag) => flag !== undefined) ? supports : undefined; +} + +function sanitizeCachedLimits(value: unknown): CopilotModelLimits | undefined { + const record = asRecord(value); + if (!record) return undefined; + const limits: CopilotModelLimits = { + maxPromptTokens: toPositiveInt(record.maxPromptTokens), + maxContextWindowTokens: toPositiveInt(record.maxContextWindowTokens), + maxOutputTokens: toPositiveInt(record.maxOutputTokens), + defaultContextMax: toPositiveInt(record.defaultContextMax), + longContextMax: toPositiveInt(record.longContextMax), + }; + return Object.values(limits).some((limit) => limit !== undefined) ? limits : undefined; +} + function sanitizeCachedContext(value: unknown): CopilotModelContext | undefined { const record = asRecord(value); const contextWindow = toPositiveInt(record?.contextWindow); @@ -364,20 +396,29 @@ function sanitizeCachedContext(value: unknown): CopilotModelContext | undefined const maxInputTokens = toPositiveInt(record?.maxInputTokens); const maxTokens = toPositiveInt(record?.maxTokens); const rawOptions = record?.contextWindowOptions; + const base: CopilotModelContext = maxInputTokens !== undefined ? { contextWindow, maxInputTokens } : { contextWindow }; + if (maxTokens !== undefined) base.maxTokens = maxTokens; if (Array.isArray(rawOptions)) { const options = rawOptions.map(toPositiveInt).filter((n): n is number => n !== undefined); - if (options.length > 1) { - const context = maxInputTokens !== undefined - ? { contextWindow, contextWindowOptions: options, maxInputTokens } - : { contextWindow, contextWindowOptions: options }; - return maxTokens !== undefined ? { ...context, maxTokens } : context; - } + if (options.length > 1) base.contextWindowOptions = options; } - const context = maxInputTokens !== undefined ? { contextWindow, maxInputTokens } : { contextWindow }; - return maxTokens !== undefined ? { ...context, maxTokens } : context; + const displayName = record && typeof record.displayName === "string" && record.displayName.length > 0 ? record.displayName : undefined; + const vendor = record && typeof record.vendor === "string" && record.vendor.length > 0 ? record.vendor : undefined; + const policyState = record && typeof record.policyState === "string" ? record.policyState : undefined; + const type = record && typeof record.type === "string" ? record.type : undefined; + return { + ...base, + displayName, + vendor, + supportedEndpoints: stringArray(record?.supportedEndpoints), + supports: sanitizeCachedSupports(record?.supports), + limits: sanitizeCachedLimits(record?.limits), + modelPickerEnabled: typeof record?.modelPickerEnabled === "boolean" ? record.modelPickerEnabled : undefined, + policyState, + type, + }; } -/** Read a fresh, host-matching catalog from the cache file, or `undefined` if missing/stale/invalid. */ export function readCopilotCatalogCache( path: string, options: ReadCopilotCatalogCacheOptions, @@ -405,7 +446,6 @@ export function readCopilotCatalogCache( return catalog; } -/** Write the catalog to the cache file (creating parent dirs). Best-effort; never throws. */ export function writeCopilotCatalogCache( path: string, baseUrl: string, @@ -426,12 +466,10 @@ export function writeCopilotCatalogCache( } } -/** Host component of a base URL, for matching {@link readCopilotCatalogCache} `host`. */ export function copilotCatalogCacheHost(baseUrl: string): string { return hostFromBaseUrl(baseUrl); } -/** Standard on-disk cache path for the Copilot model catalog under an agent directory. */ export function copilotCatalogCachePath(agentDir: string): string { return join(agentDir, "cache", "copilot-models.json"); } diff --git a/packages/coding-agent/src/core/copilot-model-synthesis.ts b/packages/coding-agent/src/core/copilot-model-synthesis.ts new file mode 100644 index 000000000..798926ca5 --- /dev/null +++ b/packages/coding-agent/src/core/copilot-model-synthesis.ts @@ -0,0 +1,99 @@ +import type { Api, Model } from "@earendil-works/pi-ai/compat"; +import { COPILOT_CATALOG_HEADERS, DEFAULT_COPILOT_API_BASE_URL, type CopilotModelContext } from "./copilot-model-catalog.ts"; +import { withContextWindowOptions } from "./context-window.ts"; + +const GITHUB_COPILOT_PROVIDER = "github-copilot"; +const ENDPOINT_API_PREFERENCE = [ + ["/v1/messages", "anthropic-messages"], + ["/responses", "openai-responses"], + ["/chat/completions", "openai-completions"], +] as const satisfies readonly (readonly [string, Api])[]; + +const ZERO_COST: Model["cost"] = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }; +const ADAPTIVE_THINKING_LEVEL_MAP: Model["thinkingLevelMap"] = { minimal: "low", xhigh: "max" }; + +export interface CopilotModelTemplate { + baseUrl: string; + headers?: Record; +} + +export function copilotThinkingLevelMapFor(entry: CopilotModelContext, api: Api): Model["thinkingLevelMap"] | undefined { + const advertised = entry.supports?.reasoningEffortLevels; + if (!advertised || advertised.length === 0) { + return entry.supports?.adaptiveThinking ? ADAPTIVE_THINKING_LEVEL_MAP : undefined; + } + + const levels = new Set(advertised); + const map: Model["thinkingLevelMap"] = { + minimal: levels.has("minimal") ? "minimal" : null, + low: levels.has("low") ? "low" : null, + medium: levels.has("medium") ? "medium" : null, + high: levels.has("high") ? "high" : null, + }; + if (levels.has("none")) map.off = "none"; + else if (api !== "anthropic-messages" || !entry.supports?.adaptiveThinking) map.off = null; + if (levels.has("xhigh")) map.xhigh = "xhigh"; + else if (levels.has("max") && entry.supports?.adaptiveThinking) map.xhigh = "max"; + return map; +} + +function mapCopilotApi(entry: CopilotModelContext): Api | undefined { + const endpoints = new Set(entry.supportedEndpoints ?? []); + return ENDPOINT_API_PREFERENCE.find(([endpoint]) => endpoints.has(endpoint))?.[1]; +} + +function hasReasoning(entry: CopilotModelContext): boolean { + return Boolean( + entry.supports?.reasoningEffort || entry.supports?.adaptiveThinking || entry.supports?.minThinkingBudget || entry.supports?.maxThinkingBudget, + ); +} + +function canSynthesizeCopilotModel(id: string, entry: CopilotModelContext): boolean { + if (id.includes("/")) return false; + if (entry.modelPickerEnabled !== true) return false; + if (entry.type !== "chat") return false; + if (entry.policyState?.toLowerCase() === "disabled") return false; + return mapCopilotApi(entry) !== undefined; +} + +export function copilotTemplateFromModels(models: readonly Model[]): CopilotModelTemplate { + const sibling = models.find((model) => model.provider === GITHUB_COPILOT_PROVIDER); + return { + baseUrl: sibling?.baseUrl ?? DEFAULT_COPILOT_API_BASE_URL, + headers: sibling?.headers ?? COPILOT_CATALOG_HEADERS, + }; +} + +export function synthesizeCopilotCatalogModels( + catalog: ReadonlyMap, + existingIds: ReadonlySet, + template: CopilotModelTemplate, +): Model[] { + const synthesized: Model[] = []; + for (const [id, entry] of catalog) { + if (existingIds.has(id) || !canSynthesizeCopilotModel(id, entry)) continue; + const api = mapCopilotApi(entry); + if (!api) continue; + let model: Model = { + id, + name: entry.displayName ?? id, + api, + provider: GITHUB_COPILOT_PROVIDER, + baseUrl: template.baseUrl, + headers: template.headers, + reasoning: hasReasoning(entry), + thinkingLevelMap: copilotThinkingLevelMapFor(entry, api), + compat: entry.supports?.adaptiveThinking ? ({ forceAdaptiveThinking: true } as Model["compat"]) : undefined, + input: entry.supports?.vision ? ["text", "image"] : ["text"], + cost: ZERO_COST, + maxInputTokens: entry.maxInputTokens, + contextWindow: entry.contextWindow, + maxTokens: entry.maxTokens ?? entry.limits?.maxOutputTokens ?? entry.maxInputTokens ?? entry.contextWindow, + }; + if (entry.contextWindowOptions && entry.contextWindowOptions.length > 1) { + model = withContextWindowOptions(model, entry.contextWindowOptions); + } + synthesized.push(model); + } + return synthesized; +} diff --git a/packages/coding-agent/src/core/model-registry-builtins.ts b/packages/coding-agent/src/core/model-registry-builtins.ts index 2812e9f6f..e7177330d 100644 --- a/packages/coding-agent/src/core/model-registry-builtins.ts +++ b/packages/coding-agent/src/core/model-registry-builtins.ts @@ -8,12 +8,21 @@ import { } from "@earendil-works/pi-ai/compat"; import { normalizeContextWindowOptions, withContextWindowOptions } from "./context-window.ts"; import { copilotApiBaseUrlFromToken, copilotTokenFromEnvironment, DEFAULT_COPILOT_API_BASE_URL, getActiveCopilotModelCatalog } from "./copilot-model-catalog.ts"; +import { copilotTemplateFromModels, copilotThinkingLevelMapFor, synthesizeCopilotCatalogModels } from "./copilot-model-synthesis.ts"; import type { ModelOverride } from "./model-registry-schemas.ts"; import type { ProviderCompat, ProviderOverride } from "./model-registry-types.ts"; const GITHUB_COPILOT_API_VERSION_HEADER = "X-GitHub-Api-Version"; const GITHUB_COPILOT_API_VERSION = "2026-06-01"; +function withDynamicGitHubCopilotModels(provider: string, models: Model[]): Model[] { + if (provider !== "github-copilot") return models; + const existingIds = new Set(models.map((model) => model.id)); + const template = copilotTemplateFromModels(models); + const dynamicModels = synthesizeCopilotCatalogModels(getActiveCopilotModelCatalog(), existingIds, template); + return dynamicModels.length === 0 ? models : [...models, ...dynamicModels]; +} + function hasHeader(headers: Record | undefined, headerName: string): boolean { if (!headers) return false; const normalizedHeaderName = headerName.toLowerCase(); @@ -37,16 +46,19 @@ function withCopilotEnvironmentBaseUrl(model: Model): Model { return { ...model, baseUrl: resolvedBaseUrl }; } +function withCopilotThinkingLevelMap(model: Model): Model { + if (model.provider !== "github-copilot") return model; + const context = getActiveCopilotModelCatalog().get(model.id); + if (!context?.supports?.reasoningEffortLevels) return model; + const thinkingLevelMap = copilotThinkingLevelMapFor(context, model.api); + return thinkingLevelMap ? { ...model, thinkingLevelMap } : model; +} + function withCopilotContextWindowOptions(model: Model): Model { if (model.provider !== "github-copilot") return model; const context = getActiveCopilotModelCatalog().get(model.id); if (!context) return model; - const base = { - ...model, - contextWindow: context.contextWindow, - maxInputTokens: context.maxInputTokens, - maxTokens: context.maxTokens ?? model.maxTokens, - }; + const base = { ...model, contextWindow: context.contextWindow, maxInputTokens: context.maxInputTokens, maxTokens: context.maxTokens ?? model.maxTokens }; if (context.contextWindowOptions && context.contextWindowOptions.length > 1) { return withContextWindowOptions(base, context.contextWindowOptions); } @@ -130,7 +142,7 @@ export function loadBuiltInModels( modelOverrides: Map>, ): Model[] { return getProviders().flatMap((provider) => { - const models = getModels(provider as KnownProvider) as Model[]; + const models = withDynamicGitHubCopilotModels(provider, getModels(provider as KnownProvider) as Model[]); const providerOverride = overrides.get(provider); const perModelOverrides = modelOverrides.get(provider); @@ -145,7 +157,7 @@ export function loadBuiltInModels( }; } - model = withCopilotContextWindowOptions(model); + model = withCopilotThinkingLevelMap(withCopilotContextWindowOptions(model)); const modelOverride = perModelOverrides?.get(m.id); return modelOverride ? applyModelOverride(model, modelOverride) : model; }); diff --git a/packages/coding-agent/src/modes/interactive/interactive-model-routing.ts b/packages/coding-agent/src/modes/interactive/interactive-model-routing.ts index 76f0cb26e..4eb0ec98e 100644 --- a/packages/coding-agent/src/modes/interactive/interactive-model-routing.ts +++ b/packages/coding-agent/src/modes/interactive/interactive-model-routing.ts @@ -73,6 +73,7 @@ InteractiveModeBase.prototype.loadCopilotModelCatalog = async function(this: Int } setActiveCopilotModelCatalog(catalog); registry.refresh(); + this.session.refreshCurrentModelFromRegistry(); this.copilotCatalogApplied = true; } catch { // Best-effort: leave the active catalog as-is on any failure (offline, auth, parse). diff --git a/packages/coding-agent/test/agent-session-copilot-catalog-refresh.test.ts b/packages/coding-agent/test/agent-session-copilot-catalog-refresh.test.ts new file mode 100644 index 000000000..24ced007a --- /dev/null +++ b/packages/coding-agent/test/agent-session-copilot-catalog-refresh.test.ts @@ -0,0 +1,182 @@ +import { existsSync, mkdirSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import assert from "node:assert/strict"; +import { afterEach, test } from "vitest"; +import { getSupportedThinkingLevels, type Api, type Model } from "@earendil-works/pi-ai/compat"; +import { AuthStorage } from "../src/core/auth-storage.ts"; +import { + clearActiveCopilotModelCatalog, + copilotCatalogCachePath, + type CopilotModelCatalog, + setActiveCopilotModelCatalog, + writeCopilotCatalogCache, +} from "../src/core/copilot-model-catalog.ts"; +import { ModelRegistry } from "../src/core/model-registry.ts"; +import { resolveCliModel } from "../src/core/model-resolver.ts"; +import { createAgentSession } from "../src/core/sdk.ts"; +import { SessionManager } from "../src/core/session-manager.ts"; +import { SettingsManager } from "../src/core/settings-manager.ts"; +import { InteractiveModeBase } from "../src/modes/interactive/interactive-mode-base.ts"; +import "../src/modes/interactive/interactive-model-routing.ts"; + +const MAI_CODE_FLASH_ID = "mai-code-1-flash-picker"; +const COPILOT_TOKEN = "tid=x;proxy-ep=proxy.individual.githubcopilot.com"; +const COPILOT_BASE_URL = "https://api.individual.githubcopilot.com"; + +const maiCodeCatalog: CopilotModelCatalog = new Map([ + [ + MAI_CODE_FLASH_ID, + { + contextWindow: 128_000, + maxInputTokens: 128_000, + maxTokens: 128_000, + displayName: "MAI-Code-1-Flash", + supportedEndpoints: ["/responses"], + supports: { reasoningEffort: true, reasoningEffortLevels: ["low", "medium", "high"], toolCalls: true }, + limits: { maxPromptTokens: 128_000, maxOutputTokens: 128_000, maxContextWindowTokens: 256_000 }, + modelPickerEnabled: true, + policyState: "enabled", + type: "chat", + }, + ], +]); + +let tempDirs: string[] = []; +const previousAgentDir = process.env.ATOMIC_CODING_AGENT_DIR; + +function makeTempDir(prefix: string): string { + const dir = join(tmpdir(), `${prefix}-${Date.now()}-${Math.random().toString(36).slice(2)}`); + mkdirSync(dir, { recursive: true }); + tempDirs.push(dir); + return dir; +} + +function createCopilotRegistry(agentDir: string): ModelRegistry { + const authStorage = AuthStorage.create(join(agentDir, "auth.json")); + authStorage.setRuntimeApiKey("github-copilot", COPILOT_TOKEN); + return ModelRegistry.create(authStorage, join(agentDir, "models.json")); +} + +function resolveFallbackModel(registry: ModelRegistry, modelId: string): Model { + const resolved = resolveCliModel({ + cliProvider: "github-copilot", + cliModel: modelId, + modelRegistry: registry, + }); + assert.equal(resolved.error, undefined); + assert.ok(resolved.model); + return resolved.model; +} + +afterEach(() => { + clearActiveCopilotModelCatalog(); + if (previousAgentDir === undefined) { + delete process.env.ATOMIC_CODING_AGENT_DIR; + } else { + process.env.ATOMIC_CODING_AGENT_DIR = previousAgentDir; + } + for (const dir of tempDirs) { + if (existsSync(dir)) rmSync(dir, { recursive: true, force: true }); + } + tempDirs = []; +}); + +test("refreshCurrentModelFromRegistry adopts catalog metadata and clamps stale Copilot thinking", async () => { + const tempDir = makeTempDir("atomic-copilot-session-refresh"); + const agentDir = join(tempDir, "agent"); + mkdirSync(agentDir, { recursive: true }); + const registry = createCopilotRegistry(agentDir); + const fallbackModel = resolveFallbackModel(registry, MAI_CODE_FLASH_ID); + + const fallbackLevels = getSupportedThinkingLevels(fallbackModel); + assert.ok(fallbackLevels.includes("minimal")); + assert.ok(fallbackLevels.includes("xhigh")); + assert.equal(fallbackLevels.includes("off"), false); + + const { session } = await createAgentSession({ + cwd: tempDir, + agentDir, + model: fallbackModel, + modelRegistry: registry, + settingsManager: SettingsManager.create(tempDir, agentDir), + sessionManager: SessionManager.inMemory(tempDir), + thinkingLevel: "xhigh", + }); + + assert.deepEqual(session.getAvailableThinkingLevels(), fallbackLevels); + assert.equal(session.thinkingLevel, "xhigh"); + const emittedEventTypes: string[] = []; + const unsubscribe = session.subscribe((event) => emittedEventTypes.push(event.type)); + + setActiveCopilotModelCatalog(maiCodeCatalog); + registry.refresh(); + session.refreshCurrentModelFromRegistry(); + unsubscribe(); + + assert.notEqual(session.model, fallbackModel); + assert.equal(session.model?.provider, "github-copilot"); + assert.equal(session.model?.id, MAI_CODE_FLASH_ID); + assert.deepEqual(session.getAvailableThinkingLevels(), ["low", "medium", "high"]); + assert.equal(session.thinkingLevel, "high"); + assert.ok(emittedEventTypes.includes("thinking_level_changed")); + assert.ok(emittedEventTypes.includes("model_changed")); + session.dispose(); +}); + +test("refreshCurrentModelFromRegistry leaves the active fallback untouched when registry cannot resolve it", async () => { + const tempDir = makeTempDir("atomic-copilot-session-refresh-missing"); + const agentDir = join(tempDir, "agent"); + mkdirSync(agentDir, { recursive: true }); + const registry = createCopilotRegistry(agentDir); + const fallbackModel = resolveFallbackModel(registry, "future-copilot-model"); + const { session } = await createAgentSession({ + cwd: tempDir, + agentDir, + model: fallbackModel, + modelRegistry: registry, + settingsManager: SettingsManager.create(tempDir, agentDir), + sessionManager: SessionManager.inMemory(tempDir), + thinkingLevel: "xhigh", + }); + + setActiveCopilotModelCatalog(maiCodeCatalog); + registry.refresh(); + session.refreshCurrentModelFromRegistry(); + + assert.equal(session.model, fallbackModel); + assert.equal(session.thinkingLevel, "xhigh"); + session.dispose(); +}); + +test("loadCopilotModelCatalog refreshes the active session after applying a cached catalog", async () => { + const tempDir = makeTempDir("atomic-copilot-routing-refresh"); + const agentDir = join(tempDir, "agent"); + mkdirSync(agentDir, { recursive: true }); + process.env.ATOMIC_CODING_AGENT_DIR = agentDir; + writeCopilotCatalogCache(copilotCatalogCachePath(agentDir), COPILOT_BASE_URL, maiCodeCatalog); + + let registryRefreshCount = 0; + let sessionRefreshCount = 0; + const harness = { + copilotCatalogApplied: false, + session: { + modelRegistry: { + getApiKeyForProvider: async (provider: string) => provider === "github-copilot" ? COPILOT_TOKEN : undefined, + refresh: () => { + registryRefreshCount += 1; + }, + }, + refreshCurrentModelFromRegistry: () => { + sessionRefreshCount += 1; + }, + }, + }; + const loadCatalog = InteractiveModeBase.prototype.loadCopilotModelCatalog as (this: typeof harness) => Promise; + + await loadCatalog.call(harness); + + assert.equal(harness.copilotCatalogApplied, true); + assert.equal(registryRefreshCount, 1); + assert.equal(sessionRefreshCount, 1); +}); diff --git a/packages/coding-agent/test/copilot-model-catalog.test.ts b/packages/coding-agent/test/copilot-model-catalog.test.ts index 81827c345..dd16bae25 100644 --- a/packages/coding-agent/test/copilot-model-catalog.test.ts +++ b/packages/coding-agent/test/copilot-model-catalog.test.ts @@ -10,6 +10,7 @@ import { COPILOT_CONTEXT_WINDOW_FALLBACK, copilotApiBaseUrlFromToken, copilotCatalogCachePath, + type CopilotModelContext, fetchCopilotModelCatalog, getActiveCopilotModelCatalog, parseCopilotModelCatalog, @@ -20,25 +21,63 @@ import { writeCopilotCatalogCache, } from "../src/core/copilot-model-catalog.ts"; -// Minimal CAPI /models fixture mirroring the live shape. Every window is INPUT (prompt) tokens: -// gpt-5.5 / claude -> tiered (default + long_context), different default budgets -// gpt-5.3-codex -> default tier only, no long_context (single window) -// gpt-4o -> no tiered pricing; window comes from max_prompt_tokens -// mystery-model -> no max_prompt_tokens; window falls back to max_context_window_tokens -// bare-model -> no usable limit signal at all (skipped) +function contextOnly(context: CopilotModelContext | undefined): CopilotModelContext | undefined { + if (!context) return undefined; + const result: CopilotModelContext = { contextWindow: context.contextWindow }; + if (context.contextWindowOptions) result.contextWindowOptions = context.contextWindowOptions; + if (context.maxInputTokens) result.maxInputTokens = context.maxInputTokens; + return result; +} + +// Minimal CAPI /models fixture mirroring the live shape. Every window is INPUT (prompt) tokens. function capiBody() { return { data: [ { id: "gpt-5.5", - capabilities: { limits: { max_output_tokens: 128_000, max_prompt_tokens: 922_000, max_context_window_tokens: 1_050_000 } }, + name: "GPT-5.5", + model_picker_enabled: true, + policy: { state: "enabled" }, + capabilities: { + type: "chat", + limits: { max_output_tokens: 128_000, max_prompt_tokens: 922_000, max_context_window_tokens: 1_050_000 }, + supports: { reasoning_effort: ["low", "medium", "high", "xhigh"], vision: true, tool_calls: true }, + }, + supported_endpoints: ["/responses"], billing: { token_prices: { default: { context_max: 272_000 }, long_context: { context_max: 922_000 } } }, }, { id: "claude-opus-4.8", - capabilities: { limits: { max_output_tokens: 128_000, max_prompt_tokens: 936_000, max_context_window_tokens: 1_000_000 } }, + capabilities: { limits: { max_output_tokens: 64_000, max_prompt_tokens: 936_000, max_context_window_tokens: 1_000_000 } }, + billing: { token_prices: { default: { context_max: 200_000 }, long_context: { context_max: 936_000 } } }, + }, + { + id: "claude-sonnet-5", + name: "Claude Sonnet 5", + vendor: "Anthropic", + model_picker_enabled: true, + policy: { state: "enabled" }, + capabilities: { + type: "chat", + limits: { max_output_tokens: 64_000, max_prompt_tokens: 936_000, max_context_window_tokens: 1_000_000 }, + supports: { adaptive_thinking: true, reasoning_effort: ["low", "medium", "high", "xhigh", "max"], min_thinking_budget: 1024, max_thinking_budget: 64_000, vision: true, tool_calls: true }, + }, + supported_endpoints: ["/v1/messages", "/chat/completions"], billing: { token_prices: { default: { context_max: 200_000 }, long_context: { context_max: 936_000 } } }, }, + { + id: "mai-code-1-flash-picker", + name: "MAI-Code-1-Flash", + vendor: "Microsoft", + model_picker_enabled: true, + policy: { state: "enabled" }, + capabilities: { + type: "chat", + limits: { max_output_tokens: 128_000, max_prompt_tokens: 128_000, max_context_window_tokens: 256_000 }, + supports: { reasoning_effort: ["low", "medium", "high"], tool_calls: true }, + }, + supported_endpoints: ["/responses"], + }, { id: "gpt-5.3-codex", capabilities: { limits: { max_output_tokens: 128_000, max_prompt_tokens: 272_000, max_context_window_tokens: 400_000 } }, @@ -127,26 +166,49 @@ describe("resolveCopilotModelContext", () => { describe("parseCopilotModelCatalog", () => { test("includes every model with a usable input budget", () => { const catalog = parseCopilotModelCatalog(capiBody()); - assert.deepEqual([...catalog.keys()].sort(), ["claude-opus-4.8", "gpt-4o", "gpt-5.3-codex", "gpt-5.5", "mystery-model"]); + assert.deepEqual([...catalog.keys()].sort(), [ + "claude-opus-4.8", + "claude-sonnet-5", + "gpt-4o", + "gpt-5.3-codex", + "gpt-5.5", + "mai-code-1-flash-picker", + "mystery-model", + ]); }); test("resolves windows per model: full total long tier with the prompt cap as effective budget", () => { const catalog = parseCopilotModelCatalog(capiBody()); - assert.deepEqual(catalog.get("gpt-5.5"), { + assert.deepEqual(contextOnly(catalog.get("gpt-5.5")), { contextWindow: 272_000, contextWindowOptions: [272_000, 1_050_000], maxInputTokens: 922_000, - maxTokens: 128_000, }); - assert.deepEqual(catalog.get("claude-opus-4.8"), { + assert.deepEqual(contextOnly(catalog.get("claude-opus-4.8")), { + contextWindow: 200_000, + contextWindowOptions: [200_000, 1_000_000], + maxInputTokens: 936_000, + }); + assert.deepEqual(contextOnly(catalog.get("claude-sonnet-5")), { contextWindow: 200_000, contextWindowOptions: [200_000, 1_000_000], maxInputTokens: 936_000, - maxTokens: 128_000, }); - assert.deepEqual(catalog.get("gpt-5.3-codex"), { contextWindow: 272_000, maxTokens: 128_000 }); - assert.deepEqual(catalog.get("gpt-4o"), { contextWindow: 64_000, maxTokens: 16_384 }); - assert.deepEqual(catalog.get("mystery-model"), { contextWindow: 256_000, maxTokens: 16_384 }); + assert.deepEqual(contextOnly(catalog.get("mai-code-1-flash-picker")), { contextWindow: 128_000 }); + assert.deepEqual(contextOnly(catalog.get("gpt-5.3-codex")), { contextWindow: 272_000 }); + assert.deepEqual(contextOnly(catalog.get("gpt-4o")), { contextWindow: 64_000 }); + assert.deepEqual(contextOnly(catalog.get("mystery-model")), { contextWindow: 256_000 }); + assert.equal(catalog.get("claude-sonnet-5")?.displayName, "Claude Sonnet 5"); + assert.deepEqual(catalog.get("claude-sonnet-5")?.supportedEndpoints, ["/v1/messages", "/chat/completions"]); + assert.equal(catalog.get("claude-sonnet-5")?.supports?.adaptiveThinking, true); + assert.deepEqual(catalog.get("claude-sonnet-5")?.supports?.reasoningEffortLevels, ["low", "medium", "high", "xhigh", "max"]); + assert.equal(catalog.get("claude-sonnet-5")?.supports?.minThinkingBudget, true); + assert.equal(catalog.get("claude-sonnet-5")?.supports?.maxThinkingBudget, true); + assert.equal(catalog.get("mai-code-1-flash-picker")?.supports?.reasoningEffort, true); + assert.deepEqual(catalog.get("mai-code-1-flash-picker")?.supports?.reasoningEffortLevels, ["low", "medium", "high"]); + assert.equal(catalog.get("claude-sonnet-5")?.modelPickerEnabled, true); + assert.equal(catalog.get("claude-sonnet-5")?.policyState, "enabled"); + assert.equal(catalog.get("claude-sonnet-5")?.type, "chat"); }); test("tolerates malformed bodies", () => { @@ -223,11 +285,10 @@ describe("fetchCopilotModelCatalog", () => { assert.equal(capturedHeaders.Authorization, "Bearer tid=abc;proxy-ep=proxy.individual.githubcopilot.com"); assert.equal(capturedHeaders["X-GitHub-Api-Version"], COPILOT_CATALOG_HEADERS["X-GitHub-Api-Version"]); assert.equal(capturedHeaders["Copilot-Integration-Id"], "vscode-chat"); - assert.deepEqual(catalog.get("gpt-5.5"), { + assert.deepEqual(contextOnly(catalog.get("gpt-5.5")), { contextWindow: 272_000, contextWindowOptions: [272_000, 1_050_000], maxInputTokens: 922_000, - maxTokens: 128_000, }); }); @@ -249,7 +310,7 @@ describe("active catalog overlay", () => { test("set/get/clear round-trips", () => { assert.equal(getActiveCopilotModelCatalog().size, 0); setActiveCopilotModelCatalog(parseCopilotModelCatalog(capiBody())); - assert.equal(getActiveCopilotModelCatalog().size, 5); + assert.equal(getActiveCopilotModelCatalog().size, 7); clearActiveCopilotModelCatalog(); assert.equal(getActiveCopilotModelCatalog().size, 0); }); @@ -271,14 +332,17 @@ describe("disk cache", () => { const catalog = parseCopilotModelCatalog(capiBody()); writeCopilotCatalogCache(path, baseUrl, catalog, 1_000); const read = readCopilotCatalogCache(path, { host, now: 1_000 + COPILOT_CATALOG_CACHE_TTL_MS - 1 }); - assert.deepEqual(read?.get("gpt-5.5"), { + assert.deepEqual(contextOnly(read?.get("gpt-5.5")), { contextWindow: 272_000, contextWindowOptions: [272_000, 1_050_000], maxInputTokens: 922_000, - maxTokens: 128_000, }); - assert.deepEqual(read?.get("gpt-5.3-codex"), { contextWindow: 272_000, maxTokens: 128_000 }); - assert.deepEqual(read?.get("mystery-model"), { contextWindow: 256_000, maxTokens: 16_384 }); + assert.deepEqual(contextOnly(read?.get("gpt-5.3-codex")), { contextWindow: 272_000 }); + assert.deepEqual(contextOnly(read?.get("mystery-model")), { contextWindow: 256_000 }); + assert.equal(read?.get("claude-sonnet-5")?.displayName, "Claude Sonnet 5"); + assert.deepEqual(read?.get("claude-sonnet-5")?.supportedEndpoints, ["/v1/messages", "/chat/completions"]); + assert.equal(read?.get("claude-sonnet-5")?.maxTokens, 64_000); + assert.deepEqual(read?.get("claude-sonnet-5")?.supports?.reasoningEffortLevels, ["low", "medium", "high", "xhigh", "max"]); }); test("ignores a stale catalog", () => { @@ -321,11 +385,10 @@ describe("seedActiveCopilotModelCatalogFromCache", () => { writeCopilotCatalogCache(cachePath, baseUrl, parseCopilotModelCatalog(capiBody()), 1_000); assert.equal(getActiveCopilotModelCatalog().size, 0); assert.equal(seedActiveCopilotModelCatalogFromCache(token, cachePath), true); - assert.deepEqual(getActiveCopilotModelCatalog().get("gpt-5.5"), { + assert.deepEqual(contextOnly(getActiveCopilotModelCatalog().get("gpt-5.5")), { contextWindow: 272_000, contextWindowOptions: [272_000, 1_050_000], maxInputTokens: 922_000, - maxTokens: 128_000, }); }); @@ -333,7 +396,7 @@ describe("seedActiveCopilotModelCatalogFromCache", () => { writeCopilotCatalogCache(cachePath, baseUrl, parseCopilotModelCatalog(capiBody()), 0); // Far beyond COPILOT_CATALOG_CACHE_TTL_MS: the seed must still apply (validation only). assert.equal(seedActiveCopilotModelCatalogFromCache(token, cachePath, COPILOT_CATALOG_CACHE_TTL_MS * 1_000), true); - assert.equal(getActiveCopilotModelCatalog().size, 5); + assert.equal(getActiveCopilotModelCatalog().size, 7); }); test("no-ops without a token, on host mismatch, or with no cache file", () => { diff --git a/packages/coding-agent/test/copilot-model-synthesis.test.ts b/packages/coding-agent/test/copilot-model-synthesis.test.ts new file mode 100644 index 000000000..db7fddfa8 --- /dev/null +++ b/packages/coding-agent/test/copilot-model-synthesis.test.ts @@ -0,0 +1,135 @@ +import assert from "node:assert/strict"; +import { describe, test } from "vitest"; +import { getSupportedThinkingLevels, type Api, type Model } from "@earendil-works/pi-ai/compat"; +import { synthesizeCopilotCatalogModels } from "../src/core/copilot-model-synthesis.ts"; +import type { CopilotModelContext } from "../src/core/copilot-model-catalog.ts"; + +const template = { + baseUrl: "https://api.enterprise.githubcopilot.com", + headers: { "User-Agent": "test-agent" }, +}; + +function chatEntry(overrides: Partial = {}): CopilotModelContext { + return { + contextWindow: 128_000, + displayName: "Fixture Model", + supportedEndpoints: ["/responses"], + supports: { reasoningEffort: true, reasoningEffortLevels: ["low", "medium", "high"], toolCalls: true }, + limits: { maxPromptTokens: 128_000, maxOutputTokens: 64_000, maxContextWindowTokens: 192_000 }, + modelPickerEnabled: true, + policyState: "enabled", + type: "chat", + ...overrides, + }; +} + +function supportedThinkingLevels(model: Model | undefined): readonly string[] { + return model ? getSupportedThinkingLevels(model) : []; +} + +describe("synthesizeCopilotCatalogModels", () => { + test("maps endpoints and capability metadata without model-name special cases", () => { + const catalog = new Map([ + [ + "claude-sonnet-5", + chatEntry({ + displayName: "Claude Sonnet 5", + contextWindow: 200_000, + contextWindowOptions: [200_000, 1_000_000], + maxInputTokens: 936_000, + supportedEndpoints: ["/v1/messages", "/chat/completions"], + supports: { adaptiveThinking: true, reasoningEffort: true, reasoningEffortLevels: ["low", "medium", "high", "xhigh", "max"], minThinkingBudget: true, maxThinkingBudget: true, vision: true, toolCalls: true }, + limits: { maxPromptTokens: 936_000, maxOutputTokens: 64_000, maxContextWindowTokens: 1_000_000 }, + }), + ], + [ + "mai-code-1-flash-picker", + chatEntry({ + displayName: "MAI-Code-1-Flash", + contextWindow: 128_000, + maxInputTokens: 128_000, + supportedEndpoints: ["/responses"], + supports: { reasoningEffort: true, reasoningEffortLevels: ["low", "medium", "high"], toolCalls: true }, + limits: { maxPromptTokens: 128_000, maxOutputTokens: 128_000, maxContextWindowTokens: 256_000 }, + }), + ], + ]); + + const models = synthesizeCopilotCatalogModels(catalog, new Set(), template); + const claude = models.find((model) => model.id === "claude-sonnet-5"); + const mai = models.find((model) => model.id === "mai-code-1-flash-picker"); + + assert.equal(claude?.api, "anthropic-messages"); + assert.deepEqual(claude?.input, ["text", "image"]); + assert.equal(claude?.reasoning, true); + assert.deepEqual(claude?.compat, { forceAdaptiveThinking: true }); + assert.deepEqual(supportedThinkingLevels(claude), ["off", "low", "medium", "high", "xhigh"]); + assert.deepEqual(claude?.thinkingLevelMap?.xhigh, "xhigh"); + assert.equal(claude?.contextWindow, 200_000); + assert.deepEqual(claude?.contextWindowOptions, [200_000, 1_000_000]); + assert.equal(claude?.maxInputTokens, 936_000); + assert.equal(claude?.maxTokens, 64_000); + assert.deepEqual(claude?.cost, { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }); + + assert.equal(mai?.api, "openai-responses"); + assert.deepEqual(mai?.input, ["text"]); + assert.equal(mai?.reasoning, true); + assert.deepEqual(supportedThinkingLevels(mai), ["low", "medium", "high"]); + assert.equal(mai?.maxTokens, 128_000); + }); + + test("gates selectable thinking levels by advertised CAPI reasoning_effort arrays", () => { + const models = synthesizeCopilotCatalogModels( + new Map([ + ["gpt-5.5-style", chatEntry({ supports: { reasoningEffort: true, reasoningEffortLevels: ["none", "low", "medium", "high", "xhigh"] } })], + ["gemini-3.5-flash-style", chatEntry({ supports: { reasoningEffort: true, reasoningEffortLevels: ["minimal", "low", "medium", "high"] } })], + ["claude-opus-4.6-style", chatEntry({ supportedEndpoints: ["/v1/messages"], supports: { adaptiveThinking: true, reasoningEffort: true, reasoningEffortLevels: ["low", "medium", "high", "max"] } })], + ]), + new Set(), + template, + ); + + assert.deepEqual(supportedThinkingLevels(models.find((model) => model.id === "gpt-5.5-style")), ["off", "low", "medium", "high", "xhigh"]); + assert.deepEqual(supportedThinkingLevels(models.find((model) => model.id === "gemini-3.5-flash-style")), ["minimal", "low", "medium", "high"]); + const adaptiveMax = models.find((model) => model.id === "claude-opus-4.6-style"); + assert.deepEqual(supportedThinkingLevels(adaptiveMax), ["off", "low", "medium", "high", "xhigh"]); + assert.equal(adaptiveMax?.thinkingLevelMap?.xhigh, "max"); + }); + + test("gates out non-picker, non-chat, disabled, unmapped, namespaced, and duplicate entries", () => { + const catalog = new Map([ + ["plain-good", chatEntry()], + ["exec-agent-a", chatEntry({ modelPickerEnabled: false })], + ["chamomile", chatEntry({ modelPickerEnabled: false })], + ["gpt-4o-2024-11-20", chatEntry({ modelPickerEnabled: false })], + ["text-embedding-3-small", chatEntry({ type: "embeddings" })], + ["disabled-model", chatEntry({ policyState: "disabled" })], + ["endpointless-model", chatEntry({ supportedEndpoints: [] })], + ["octodemo/Octodemo_Foundry/DeepSeek-V3.2", chatEntry({ displayName: "DeepSeek-V3.2" })], + ["builtin-wins", chatEntry()], + ]); + + const models = synthesizeCopilotCatalogModels(catalog, new Set(["builtin-wins"]), template); + assert.deepEqual(models.map((model) => model.id), ["plain-good"]); + }); + + test("prefers CAPI endpoints in deterministic API order", () => { + const models = synthesizeCopilotCatalogModels( + new Map([ + ["messages", chatEntry({ supportedEndpoints: ["/responses", "/v1/messages"] })], + ["responses", chatEntry({ supportedEndpoints: ["/chat/completions", "/responses"] })], + ["completions", chatEntry({ supportedEndpoints: ["/chat/completions"] })], + ]), + new Set(), + template, + ); + assert.deepEqual( + models.map((model) => [model.id, model.api]), + [ + ["messages", "anthropic-messages"], + ["responses", "openai-responses"], + ["completions", "openai-completions"], + ], + ); + }); +}); diff --git a/packages/coding-agent/test/model-registry-context-window.suite.ts b/packages/coding-agent/test/model-registry-context-window.suite.ts index fc91b2aec..d837fc5e7 100644 --- a/packages/coding-agent/test/model-registry-context-window.suite.ts +++ b/packages/coding-agent/test/model-registry-context-window.suite.ts @@ -1,3 +1,4 @@ +import { getSupportedThinkingLevels } from "@earendil-works/pi-ai/compat"; import { afterEach, describe, expect, test } from "vitest"; import { getSupportedContextWindows, selectContextWindow } from "../src/core/context-window.ts"; import { @@ -25,9 +26,38 @@ describeModelRegistry((context) => { // The live CAPI catalog (only populated when the user has the GitHub Copilot provider) drives // which Copilot models expose a selectable long-context window. Seed it like a successful fetch. const copilotCatalog = new Map([ - ["gpt-5.5", { contextWindow: 272_000, contextWindowOptions: [272_000, 1_050_000], maxInputTokens: 922_000, maxTokens: 128_000 }], - ["claude-opus-4.8", { contextWindow: 200_000, contextWindowOptions: [200_000, 1_000_000], maxInputTokens: 936_000, maxTokens: 128_000 }], + ["gpt-5.5", { contextWindow: 272_000, contextWindowOptions: [272_000, 1_050_000], maxInputTokens: 922_000, maxTokens: 128_000, supports: { reasoningEffort: true, reasoningEffortLevels: ["none", "low", "medium", "high", "xhigh"] } }], + ["claude-opus-4.8", { contextWindow: 200_000, contextWindowOptions: [200_000, 1_000_000], maxInputTokens: 936_000, maxTokens: 64_000 }], ["gemini-3.1-pro-preview", { contextWindow: 200_000, contextWindowOptions: [200_000, 1_000_000], maxInputTokens: 936_000, maxTokens: 64_000 }], + [ + "claude-sonnet-5", + { + contextWindow: 200_000, + contextWindowOptions: [200_000, 1_000_000], + maxInputTokens: 936_000, + displayName: "Claude Sonnet 5", + supportedEndpoints: ["/v1/messages", "/chat/completions"], + supports: { adaptiveThinking: true, reasoningEffort: true, reasoningEffortLevels: ["low", "medium", "high", "xhigh", "max"], minThinkingBudget: true, maxThinkingBudget: true, vision: true, toolCalls: true }, + limits: { maxPromptTokens: 936_000, maxOutputTokens: 64_000, maxContextWindowTokens: 1_000_000 }, + modelPickerEnabled: true, + policyState: "enabled", + type: "chat", + }, + ], + [ + "mai-code-1-flash-picker", + { + contextWindow: 128_000, + maxInputTokens: 128_000, + displayName: "MAI-Code-1-Flash", + supportedEndpoints: ["/responses"], + supports: { reasoningEffort: true, reasoningEffortLevels: ["low", "medium", "high"], toolCalls: true }, + limits: { maxPromptTokens: 128_000, maxOutputTokens: 128_000, maxContextWindowTokens: 256_000 }, + modelPickerEnabled: true, + policyState: "enabled", + type: "chat", + }, + ], ["gpt-4.1", { contextWindow: 200_000 }], ]); @@ -52,12 +82,77 @@ describeModelRegistry((context) => { expect(claude?.defaultContextWindow).toBe(200_000); expect(claude?.contextWindowOptions).toEqual([200_000, 1_000_000]); expect(claude?.maxInputTokens).toBe(936_000); - expect(claude?.maxTokens).toBe(128_000); + expect(claude?.maxTokens).toBe(64_000); const gemini31 = registry.find("github-copilot", "gemini-3.1-pro-preview"); expect(gemini31?.contextWindowOptions).toEqual([200_000, 1_000_000]); expect(gemini31?.maxInputTokens).toBe(936_000); - expect(gemini31?.maxTokens).toBe(64_000); + }); + + test("synthesizes picker-enabled github-copilot catalog models with metadata-driven fields", () => { + setActiveCopilotModelCatalog(copilotCatalog); + const registry = ModelRegistry.create(context.authStorage, context.modelsJsonPath); + + const claudeSonnet5 = registry.find("github-copilot", "claude-sonnet-5"); + if (!claudeSonnet5) throw new Error("Missing dynamic github-copilot/claude-sonnet-5 model"); + expect(claudeSonnet5.name).toBe("Claude Sonnet 5"); + expect(claudeSonnet5.provider).toBe("github-copilot"); + expect(claudeSonnet5.api).toBe("anthropic-messages"); + expect(claudeSonnet5.reasoning).toBe(true); + expect(claudeSonnet5.compat).toEqual({ forceAdaptiveThinking: true }); + expect(getSupportedThinkingLevels(claudeSonnet5)).toEqual(["off", "low", "medium", "high", "xhigh"]); + expect(claudeSonnet5.thinkingLevelMap?.xhigh).toBe("xhigh"); + expect(claudeSonnet5.input).toEqual(["text", "image"]); + expect(claudeSonnet5.cost).toEqual({ input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }); + expect(claudeSonnet5.contextWindowOptions).toEqual([200_000, 1_000_000]); + expect(claudeSonnet5.maxInputTokens).toBe(936_000); + expect(claudeSonnet5.maxTokens).toBe(64_000); + + const maiCodeFlash = registry.find("github-copilot", "mai-code-1-flash-picker"); + if (!maiCodeFlash) throw new Error("Missing dynamic github-copilot/mai-code-1-flash-picker model"); + expect(maiCodeFlash.name).toBe("MAI-Code-1-Flash"); + expect(maiCodeFlash.provider).toBe("github-copilot"); + expect(maiCodeFlash.api).toBe("openai-responses"); + expect(maiCodeFlash.reasoning).toBe(true); + expect(getSupportedThinkingLevels(maiCodeFlash)).toEqual(["low", "medium", "high"]); + expect(maiCodeFlash.input).toEqual(["text"]); + expect(maiCodeFlash.contextWindow).toBe(128_000); + expect(maiCodeFlash.contextWindowOptions).toBeUndefined(); + expect(maiCodeFlash.maxInputTokens).toBe(128_000); + expect(maiCodeFlash.maxTokens).toBe(128_000); + }); + + test("overlays builtin github-copilot thinking levels from catalog effort arrays", () => { + setActiveCopilotModelCatalog(copilotCatalog); + const registry = ModelRegistry.create(context.authStorage, context.modelsJsonPath); + const gpt55 = registry.find("github-copilot", "gpt-5.5"); + + if (!gpt55) throw new Error("Missing built-in github-copilot/gpt-5.5 model"); + expect(getSupportedThinkingLevels(gpt55)).toEqual(["off", "low", "medium", "high", "xhigh"]); + expect(gpt55.thinkingLevelMap?.minimal).toBe(null); + }); + + test("leaves builtin copilot thinking maps untouched without effort arrays", () => { + const baseline = ModelRegistry.create(context.authStorage, context.modelsJsonPath).find("github-copilot", "gpt-5-mini"); + setActiveCopilotModelCatalog(new Map([["gpt-5-mini", { contextWindow: 128_000, supports: { reasoningEffort: true } }]])); + const overlaid = ModelRegistry.create(context.authStorage, context.modelsJsonPath).find("github-copilot", "gpt-5-mini"); + + expect(overlaid?.thinkingLevelMap).toEqual(baseline?.thinkingLevelMap); + }); + + test("user model thinkingLevelMap overrides win over catalog overlays", () => { + setActiveCopilotModelCatalog(copilotCatalog); + writeRawModelsJson({ + "github-copilot": { + modelOverrides: { "gpt-5.5": { thinkingLevelMap: { minimal: "minimal", xhigh: null } } }, + }, + }); + const registry = ModelRegistry.create(context.authStorage, context.modelsJsonPath); + const gpt55 = registry.find("github-copilot", "gpt-5.5"); + + expect(gpt55?.thinkingLevelMap?.minimal).toBe("minimal"); + expect(gpt55?.thinkingLevelMap?.xhigh).toBe(null); + expect(gpt55 ? getSupportedThinkingLevels(gpt55) : []).toEqual(["off", "minimal", "low", "medium", "high"]); }); test("overrides contextWindow (input tokens) without options for single-window catalog models", () => { @@ -102,10 +197,7 @@ describeModelRegistry((context) => { writeCopilotCatalogCache( copilotCatalogCachePath(context.tempDir), "https://api.individual.githubcopilot.com", - new Map([[ - "claude-opus-4.8", - { contextWindow: 200_000, contextWindowOptions: [200_000, 1_000_000], maxInputTokens: 936_000, maxTokens: 128_000 }, - ]]), + new Map([["claude-opus-4.8", { contextWindow: 200_000, contextWindowOptions: [200_000, 1_000_000], maxInputTokens: 936_000 }]]), 1_000, ); @@ -115,7 +207,6 @@ describeModelRegistry((context) => { expect(claude?.contextWindow).toBe(200_000); expect(claude?.contextWindowOptions).toEqual([200_000, 1_000_000]); expect(claude?.maxInputTokens).toBe(936_000); - expect(claude?.maxTokens).toBe(128_000); // The previously selected long window now validates instead of warning/resetting. const selected = claude ? selectContextWindow(claude, 1_000_000) : { error: "missing" }; expect("error" in selected).toBe(false); @@ -146,7 +237,6 @@ describeModelRegistry((context) => { expect(selected.model.defaultContextWindow).toBe(272_000); // The prompt cap rides along so compaction/overflow still respect the real input budget. expect(selected.model.maxInputTokens).toBe(922_000); - expect(selected.model.maxTokens).toBe(128_000); expect(getSupportedContextWindows(selected.model)).toEqual([272_000, 1_050_000]); } });