feat(copilot): Anthropic Fast Mode for opus/sonnet/haiku 4.x + synthetic -fast model ids - #52105
feat(copilot): Anthropic Fast Mode for opus/sonnet/haiku 4.x + synthetic -fast model ids#52105arminanton wants to merge 1 commit into
Conversation
…tic -fast model ids
Unlock GitHub Copilot's Anthropic Fast Mode (extra_body.speed="fast", ~2.5x
output throughput) for the opus/sonnet/haiku 4.x families, make it selectable
three ways, and fix the TUI status-bar label so "fast" isn't printed twice.
Background
----------
Native Anthropic only honors speed=fast on Opus 4.6 (4.7/4.8 explicitly 400).
But GitHub Copilot's /v1/messages proxy (api.githubcopilot.com) accepts
speed=fast on the whole opus/sonnet/haiku 4.x line and passes it upstream
(empirically verified). The previous gate was hard-coded to Opus-4.6-only, so
Copilot users couldn't reach Fast Mode on the newer models at all.
What changed
------------
(a) Endpoint-aware gate
- agent/anthropic_adapter._supports_fast_mode now takes base_url. Native
Anthropic stays Opus-4.6-only; Copilot's proxy returns True for opus/sonnet/
haiku 4.x. A small _is_copilot_base_url(base_url) helper centralizes the
api.githubcopilot.com check.
- hermes_cli.models._is_anthropic_fast_model widened to the same families so
the /fast toggle and resolve_fast_mode_overrides apply to them.
(b) Auto-knob — a `-fast` model id just works
- normalize_copilot_model_id strips a trailing "-fast" to the real base id on
the wire (claude-opus-4.8-fast -> claude-opus-4.8); the speed param is
attached separately by the adapter.
- selecting a `-fast` id is an implicit /fast toggle (sets request_overrides
even when the toggle is off).
(c) Visibility — `-fast` variants in the catalog
- fetch_github_model_catalog injects synthetic "<model>-fast" companion
entries (name "<Name> (Fast)") for fast-capable Claude models, so the Fast
variant is selectable directly from the model picker (the way OpenRouter
exposes …-fast). Gated off by HERMES_COPILOT_HIDE_FAST_VARIANTS.
(d) TUI label fix
- ui-tui appChrome.shortModelLabel strips a trailing "-fast"/"_fast" suffix so
the model name is the base model and fast mode is shown only by the badge.
The label is now identical whether fast came from the -fast id or the /fast
toggle ("opus 4.8 fast" instead of "opus 4.8 fast fast"). modelLabel is
exported for the regression test.
Tests
-----
- tests/cli/test_fast_command.py: endpoint-aware policy + TestSyntheticFastVariant
(normalize, override resolution, catalog injection, copilot-yes/native-no
adapter gates).
- tests/hermes_cli/test_model_validation.py: probe-catalog assertion includes a
-fast variant.
- ui-tui appChromeModelLabel.test.ts: double-print regression + toggle/suffix
parity + fast-off + sonnet/haiku variants.
Relationship to NousResearch#49184
----------------------
Fast Mode is only reachable because Claude-on-Copilot is routed to /v1/messages
(NousResearch#49184). This PR is self-contained (it carries the small _is_copilot_base_url
helper and the catalog-injection logic it needs), but it is best reviewed/merged
after NousResearch#49184 since they share the Copilot Anthropic-proxy surface.
Wire-model strip (the load-bearing fix)
----------------------------------------
build_anthropic_kwargs now strips a trailing "-fast" from the model id it puts
on the wire (new _strip_fast_suffix helper, reused by _supports_fast_mode). The
"-fast" id is the Fast Mode KNOB, not a real model — sending it verbatim 400s
"model not supported". Selecting claude-opus-4.8-fast in the TUI hit exactly this
(the picker's per-session model override reaches the adapter without the catalog
normalizer). Stripping at the adapter boundary fixes every path regardless of how
the id was selected. Regression test added (test_fast_suffix_stripped_from_wire_model).
teknium1
left a comment
There was a problem hiding this comment.
Thanks for mapping the adapter, picker, and label paths. The Copilot-specific fast-mode gap remains on current main, but this draft needs routing and surface-completeness work before it can safely provide the advertised behavior.
Problems
- The new adapter path is unreachable for Copilot Claude on current main:
hermes_cli/runtime_provider.py:320-335derives Copilot mode throughcopilot_model_api_mode, whose non-GPT path ischat_completions(hermes_cli/models.py:3293-3300); the changed adapter is reached throughanthropic_messages(agent/transports/anthropic.py:41-78). #49184 is still open. - The implicit
-fastoverride is added only inhermes_cli/cli_agent_setup_mixin.py; gateway turns return{}when/fastis off (gateway/run.py:3924-3933), and TUI creation only enables fast whenparams.fastis supplied (tui_gateway/server.py:5225-5228). - The unconditional wire stripping conflicts with main’s separate
anthropic/claude-opus-4.8-fastcatalog/pricing entry (hermes_cli/models.py:44,agent/usage_pricing.py:154-180). Scope the transform to Copilot synthetic IDs. - Please replace the new
HERMES_COPILOT_HIDE_FAST_VARIANTSsetting withconfig.yaml, perAGENTS.md:102-107, and avoid the exact catalog snapshot assertion intests/hermes_cli/test_model_validation.py.
Automated hermes-sweeper review.
| @@ -2463,7 +2508,7 @@ def _to_oauth_wire_name(name: str) -> str: | |||
| pass # tool_result uses ID, not name | |||
There was a problem hiding this comment.
This strips -fast for every Anthropic-Messages call, but main treats anthropic/claude-opus-4.8-fast as a distinct real OpenRouter model (hermes_cli/models.py:44; agent/usage_pricing.py:172-180). Restrict stripping to a positively identified Copilot synthetic variant.
| # A synthetic "-fast" model id is itself an opt-in to fast mode: selecting | ||
| # e.g. ``copilot/claude-opus-4.8-fast`` MEANS speed=fast on the base model, | ||
| # even if the /fast toggle was never flipped. So treat a -fast id as an | ||
| # implicit fast toggle. |
There was a problem hiding this comment.
This implements suffix-as-fast only for the classic CLI. Gateway resolution still returns no override when /fast is off (gateway/run.py:3924-3933), and the TUI create path only enables fast from params.fast (tui_gateway/server.py:5225-5228); centralize this behavior before advertising picker support.
| @@ -2807,6 +2812,39 @@ def fetch_github_model_catalog( | |||
| continue | |||
There was a problem hiding this comment.
HERMES_COPILOT_HIDE_FAST_VARIANTS is a new user-facing non-secret HERMES_* behavior switch. AGENTS.md:102-107 requires behavioral configuration to live in config.yaml; use a config setting if this opt-out is needed.
|
|
||
| assert mock_urlopen.call_args[0][0].full_url == "https://api.githubcopilot.com/models" | ||
| assert probe["models"] == ["gpt-5.4", "claude-sonnet-4.6"] | ||
| # claude-sonnet-4.6 also surfaces its synthetic "-fast" variant (Anthropic Fast |
There was a problem hiding this comment.
Avoid expanding this exact catalog snapshot. Assert the relationship instead: the returned Claude item has exactly one -fast companion while non-Claude entries do not, so routine catalog changes do not break the test.
|
Closing this. Re-verified against Copilot's live This PR applies the native-Anthropic mechanism to Copilot: it strips Current
Since the model this PR targets is already merged and the mechanism here is counter-productive, there's nothing left to salvage. Anything still worth doing (a guard ensuring Thanks for the review. |
feat(copilot): Anthropic Fast Mode for opus/sonnet/haiku 4.x + synthetic -fast model ids
Unlock GitHub Copilot's Anthropic Fast Mode (extra_body.speed="fast", ~2.5x
output throughput) for the opus/sonnet/haiku 4.x families, make it selectable
three ways, and fix the TUI status-bar label so "fast" isn't printed twice.
Background
Native Anthropic only honors speed=fast on Opus 4.6 (4.7/4.8 explicitly 400).
But GitHub Copilot's /v1/messages proxy (api.githubcopilot.com) accepts
speed=fast on the whole opus/sonnet/haiku 4.x line and passes it upstream
(empirically verified). The previous gate was hard-coded to Opus-4.6-only, so
Copilot users couldn't reach Fast Mode on the newer models at all.
What changed
(a) Endpoint-aware gate
Anthropic stays Opus-4.6-only; Copilot's proxy returns True for opus/sonnet/
haiku 4.x. A small _is_copilot_base_url(base_url) helper centralizes the
api.githubcopilot.com check.
the /fast toggle and resolve_fast_mode_overrides apply to them.
(b) Auto-knob — a
-fastmodel id just worksthe wire (claude-opus-4.8-fast -> claude-opus-4.8); the speed param is
attached separately by the adapter.
-fastid is an implicit /fast toggle (sets request_overrideseven when the toggle is off).
(c) Visibility —
-fastvariants in the catalogentries (name " (Fast)") for fast-capable Claude models, so the Fast
variant is selectable directly from the model picker (the way OpenRouter
exposes …-fast). Gated off by HERMES_COPILOT_HIDE_FAST_VARIANTS.
(d) TUI label fix
the model name is the base model and fast mode is shown only by the badge.
The label is now identical whether fast came from the -fast id or the /fast
toggle ("opus 4.8 fast" instead of "opus 4.8 fast fast"). modelLabel is
exported for the regression test.
Tests
(normalize, override resolution, catalog injection, copilot-yes/native-no
adapter gates).
-fast variant.
parity + fast-off + sonnet/haiku variants.
Relationship to #49184
Fast Mode is only reachable because Claude-on-Copilot is routed to /v1/messages
(#49184). This PR is self-contained (it carries the small _is_copilot_base_url
helper and the catalog-injection logic it needs), but it is best reviewed/merged
after #49184 since they share the Copilot Anthropic-proxy surface.
Wire-model strip (the load-bearing fix)
build_anthropic_kwargs now strips a trailing "-fast" from the model id it puts
on the wire (new _strip_fast_suffix helper, reused by _supports_fast_mode). The
"-fast" id is the Fast Mode KNOB, not a real model — sending it verbatim 400s
"model not supported". Selecting claude-opus-4.8-fast in the TUI hit exactly this
(the picker's per-session model override reaches the adapter without the catalog
normalizer). Stripping at the adapter boundary fixes every path regardless of how
the id was selected. Regression test added (test_fast_suffix_stripped_from_wire_model).