Skip to content

feat(copilot): Anthropic Fast Mode for opus/sonnet/haiku 4.x + synthetic -fast model ids - #52105

Closed
arminanton wants to merge 1 commit into
NousResearch:mainfrom
arminanton:feat/copilot-fast-mode-models
Closed

feat(copilot): Anthropic Fast Mode for opus/sonnet/haiku 4.x + synthetic -fast model ids#52105
arminanton wants to merge 1 commit into
NousResearch:mainfrom
arminanton:feat/copilot-fast-mode-models

Conversation

@arminanton

Copy link
Copy Markdown
Contributor

feat(copilot): Anthropic Fast Mode for opus/sonnet/haiku 4.x + synthetic -fast model ids

Unlock GitHub Copilot's Anthropic Fast Mode (extra_body.speed="fast", ~2.5x
output throughput) for the opus/sonnet/haiku 4.x families, make it selectable
three ways, and fix the TUI status-bar label so "fast" isn't printed twice.

Background

Native Anthropic only honors speed=fast on Opus 4.6 (4.7/4.8 explicitly 400).
But GitHub Copilot's /v1/messages proxy (api.githubcopilot.com) accepts
speed=fast on the whole opus/sonnet/haiku 4.x line and passes it upstream
(empirically verified). The previous gate was hard-coded to Opus-4.6-only, so
Copilot users couldn't reach Fast Mode on the newer models at all.

What changed

(a) Endpoint-aware gate

  • agent/anthropic_adapter._supports_fast_mode now takes base_url. Native
    Anthropic stays Opus-4.6-only; Copilot's proxy returns True for opus/sonnet/
    haiku 4.x. A small _is_copilot_base_url(base_url) helper centralizes the
    api.githubcopilot.com check.
  • hermes_cli.models._is_anthropic_fast_model widened to the same families so
    the /fast toggle and resolve_fast_mode_overrides apply to them.

(b) Auto-knob — a -fast model id just works

  • normalize_copilot_model_id strips a trailing "-fast" to the real base id on
    the wire (claude-opus-4.8-fast -> claude-opus-4.8); the speed param is
    attached separately by the adapter.
  • selecting a -fast id is an implicit /fast toggle (sets request_overrides
    even when the toggle is off).

(c) Visibility — -fast variants in the catalog

  • fetch_github_model_catalog injects synthetic "-fast" companion
    entries (name " (Fast)") for fast-capable Claude models, so the Fast
    variant is selectable directly from the model picker (the way OpenRouter
    exposes …-fast). Gated off by HERMES_COPILOT_HIDE_FAST_VARIANTS.

(d) TUI label fix

  • ui-tui appChrome.shortModelLabel strips a trailing "-fast"/"_fast" suffix so
    the model name is the base model and fast mode is shown only by the badge.
    The label is now identical whether fast came from the -fast id or the /fast
    toggle ("opus 4.8 fast" instead of "opus 4.8 fast fast"). modelLabel is
    exported for the regression test.

Tests

  • tests/cli/test_fast_command.py: endpoint-aware policy + TestSyntheticFastVariant
    (normalize, override resolution, catalog injection, copilot-yes/native-no
    adapter gates).
  • tests/hermes_cli/test_model_validation.py: probe-catalog assertion includes a
    -fast variant.
  • ui-tui appChromeModelLabel.test.ts: double-print regression + toggle/suffix
    parity + fast-off + sonnet/haiku variants.

Relationship to #49184

Fast Mode is only reachable because Claude-on-Copilot is routed to /v1/messages
(#49184). This PR is self-contained (it carries the small _is_copilot_base_url
helper and the catalog-injection logic it needs), but it is best reviewed/merged
after #49184 since they share the Copilot Anthropic-proxy surface.

Wire-model strip (the load-bearing fix)

build_anthropic_kwargs now strips a trailing "-fast" from the model id it puts
on the wire (new _strip_fast_suffix helper, reused by _supports_fast_mode). The
"-fast" id is the Fast Mode KNOB, not a real model — sending it verbatim 400s
"model not supported". Selecting claude-opus-4.8-fast in the TUI hit exactly this
(the picker's per-session model override reaches the adapter without the catalog
normalizer). Stripping at the adapter boundary fixes every path regardless of how
the id was selected. Regression test added (test_fast_suffix_stripped_from_wire_model).

…tic -fast model ids

Unlock GitHub Copilot's Anthropic Fast Mode (extra_body.speed="fast", ~2.5x
output throughput) for the opus/sonnet/haiku 4.x families, make it selectable
three ways, and fix the TUI status-bar label so "fast" isn't printed twice.

Background
----------
Native Anthropic only honors speed=fast on Opus 4.6 (4.7/4.8 explicitly 400).
But GitHub Copilot's /v1/messages proxy (api.githubcopilot.com) accepts
speed=fast on the whole opus/sonnet/haiku 4.x line and passes it upstream
(empirically verified). The previous gate was hard-coded to Opus-4.6-only, so
Copilot users couldn't reach Fast Mode on the newer models at all.

What changed
------------
(a) Endpoint-aware gate
  - agent/anthropic_adapter._supports_fast_mode now takes base_url. Native
    Anthropic stays Opus-4.6-only; Copilot's proxy returns True for opus/sonnet/
    haiku 4.x. A small _is_copilot_base_url(base_url) helper centralizes the
    api.githubcopilot.com check.
  - hermes_cli.models._is_anthropic_fast_model widened to the same families so
    the /fast toggle and resolve_fast_mode_overrides apply to them.

(b) Auto-knob — a `-fast` model id just works
  - normalize_copilot_model_id strips a trailing "-fast" to the real base id on
    the wire (claude-opus-4.8-fast -> claude-opus-4.8); the speed param is
    attached separately by the adapter.
  - selecting a `-fast` id is an implicit /fast toggle (sets request_overrides
    even when the toggle is off).

(c) Visibility — `-fast` variants in the catalog
  - fetch_github_model_catalog injects synthetic "<model>-fast" companion
    entries (name "<Name> (Fast)") for fast-capable Claude models, so the Fast
    variant is selectable directly from the model picker (the way OpenRouter
    exposes …-fast). Gated off by HERMES_COPILOT_HIDE_FAST_VARIANTS.

(d) TUI label fix
  - ui-tui appChrome.shortModelLabel strips a trailing "-fast"/"_fast" suffix so
    the model name is the base model and fast mode is shown only by the badge.
    The label is now identical whether fast came from the -fast id or the /fast
    toggle ("opus 4.8 fast" instead of "opus 4.8 fast fast"). modelLabel is
    exported for the regression test.

Tests
-----
- tests/cli/test_fast_command.py: endpoint-aware policy + TestSyntheticFastVariant
  (normalize, override resolution, catalog injection, copilot-yes/native-no
  adapter gates).
- tests/hermes_cli/test_model_validation.py: probe-catalog assertion includes a
  -fast variant.
- ui-tui appChromeModelLabel.test.ts: double-print regression + toggle/suffix
  parity + fast-off + sonnet/haiku variants.

Relationship to NousResearch#49184
----------------------
Fast Mode is only reachable because Claude-on-Copilot is routed to /v1/messages
(NousResearch#49184). This PR is self-contained (it carries the small _is_copilot_base_url
helper and the catalog-injection logic it needs), but it is best reviewed/merged
after NousResearch#49184 since they share the Copilot Anthropic-proxy surface.

Wire-model strip (the load-bearing fix)
----------------------------------------
build_anthropic_kwargs now strips a trailing "-fast" from the model id it puts
on the wire (new _strip_fast_suffix helper, reused by _supports_fast_mode). The
"-fast" id is the Fast Mode KNOB, not a real model — sending it verbatim 400s
"model not supported". Selecting claude-opus-4.8-fast in the TUI hit exactly this
(the picker's per-session model override reaches the adapter without the catalog
normalizer). Stripping at the adapter boundary fixes every path regardless of how
the id was selected. Regression test added (test_fast_suffix_stripped_from_wire_model).
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/tui Terminal UI (ui-tui/ + tui_gateway/) provider/copilot GitHub Copilot (ACP + Chat) provider/anthropic Anthropic native Messages API P3 Low — cosmetic, nice to have labels Jun 24, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for mapping the adapter, picker, and label paths. The Copilot-specific fast-mode gap remains on current main, but this draft needs routing and surface-completeness work before it can safely provide the advertised behavior.

Problems

  • The new adapter path is unreachable for Copilot Claude on current main: hermes_cli/runtime_provider.py:320-335 derives Copilot mode through copilot_model_api_mode, whose non-GPT path is chat_completions (hermes_cli/models.py:3293-3300); the changed adapter is reached through anthropic_messages (agent/transports/anthropic.py:41-78). #49184 is still open.
  • The implicit -fast override is added only in hermes_cli/cli_agent_setup_mixin.py; gateway turns return {} when /fast is off (gateway/run.py:3924-3933), and TUI creation only enables fast when params.fast is supplied (tui_gateway/server.py:5225-5228).
  • The unconditional wire stripping conflicts with main’s separate anthropic/claude-opus-4.8-fast catalog/pricing entry (hermes_cli/models.py:44, agent/usage_pricing.py:154-180). Scope the transform to Copilot synthetic IDs.
  • Please replace the new HERMES_COPILOT_HIDE_FAST_VARIANTS setting with config.yaml, per AGENTS.md:102-107, and avoid the exact catalog snapshot assertion in tests/hermes_cli/test_model_validation.py.

Automated hermes-sweeper review.

@@ -2463,7 +2508,7 @@ def _to_oauth_wire_name(name: str) -> str:
pass # tool_result uses ID, not name

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This strips -fast for every Anthropic-Messages call, but main treats anthropic/claude-opus-4.8-fast as a distinct real OpenRouter model (hermes_cli/models.py:44; agent/usage_pricing.py:172-180). Restrict stripping to a positively identified Copilot synthetic variant.

# A synthetic "-fast" model id is itself an opt-in to fast mode: selecting
# e.g. ``copilot/claude-opus-4.8-fast`` MEANS speed=fast on the base model,
# even if the /fast toggle was never flipped. So treat a -fast id as an
# implicit fast toggle.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implements suffix-as-fast only for the classic CLI. Gateway resolution still returns no override when /fast is off (gateway/run.py:3924-3933), and the TUI create path only enables fast from params.fast (tui_gateway/server.py:5225-5228); centralize this behavior before advertising picker support.

Comment thread hermes_cli/models.py
@@ -2807,6 +2812,39 @@ def fetch_github_model_catalog(
continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HERMES_COPILOT_HIDE_FAST_VARIANTS is a new user-facing non-secret HERMES_* behavior switch. AGENTS.md:102-107 requires behavioral configuration to live in config.yaml; use a config setting if this opt-out is needed.


assert mock_urlopen.call_args[0][0].full_url == "https://api.githubcopilot.com/models"
assert probe["models"] == ["gpt-5.4", "claude-sonnet-4.6"]
# claude-sonnet-4.6 also surfaces its synthetic "-fast" variant (Anthropic Fast

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid expanding this exact catalog snapshot. Assert the relationship instead: the returned Claude item has exactly one -fast companion while non-Claude entries do not, so routine catalog changes do not break the test.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026
@arminanton

Copy link
Copy Markdown
Contributor Author

Closing this.

Re-verified against Copilot's live /v1/messages catalog and current upstream main: on Copilot, fast mode is a real entitled catalog model id (claude-opus-4.8-fast, billed 2×), selected by sending that id verbatim — the inverse of native Anthropic (where you strip -fast and attach speed:"fast").

This PR applies the native-Anthropic mechanism to Copilot: it strips -fast on the wire, attaches an inert speed param, and synthesizes <family>-fast companions (claude-sonnet-*-fast, claude-haiku-*-fast) that don't exist in the Copilot catalog and 400 with model_not_supported. That would actively break the working path.

Current main already does the right thing by default:

  • claude-opus-4.8-fast is a real catalog + pricing entry — added and merged in feat: add claude-opus-4.8 and claude-opus-4.8-fast #34003 (hermes_cli/models.py:89, agent/usage_pricing.py:215-222, 2× billing).
  • Non-GPT Copilot models return the id verbatim, so an entitled -fast id already reaches the proxy unchanged and bills correctly.

Since the model this PR targets is already merged and the mechanism here is counter-productive, there's nothing left to salvage. Anything still worth doing (a guard ensuring -fast is never stripped/mangled on the Copilot path) is a much smaller, separate change; I'll open a focused PR if a real gap remains rather than rework this one.

Thanks for the review.

@arminanton arminanton closed this Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have provider/anthropic Anthropic native Messages API provider/copilot GitHub Copilot (ACP + Chat) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants