Skip to content

fix(runtime): exchange Copilot raw OAuth token in pool resolution path - #24546

Open
lucvan wants to merge 1 commit into
NousResearch:mainfrom
lucvan:pr/copilot-pool-token-exchange
Open

fix(runtime): exchange Copilot raw OAuth token in pool resolution path#24546
lucvan wants to merge 1 commit into
NousResearch:mainfrom
lucvan:pr/copilot-pool-token-exchange

Conversation

@lucvan

@lucvan lucvan commented May 12, 2026

Copy link
Copy Markdown

Problem

_resolve_runtime_from_pool_entry's Copilot branch returns getattr(entry, "runtime_api_key", "") directly. The sibling api-key path (resolve_api_key_provider_credentials("copilot") in hermes_cli/auth.py) correctly calls get_copilot_api_token() to exchange the raw gho_* / ghu_* GitHub OAuth token via /copilot_internal/v2/token for the short-lived tid=... Bearer that Copilot's /chat/completions endpoint actually requires.

When the gateway-init fallback path picks Copilot via the pool branch, the raw token goes on the wire. GitHub then maps the raw OAuth grant to whatever integrator the originating OAuth app declared as default — typically copilot-language-server for Business/Enterprise seats — which has a restricted model allowlist that excludes Claude:

HTTP 400: The requested model is not available for integrator
"copilot-language-server"

even though copilot_request_headers() correctly advertises Copilot-Integration-Id: vscode-chat (verified by adding an httpx.Client.send wrapper and inspecting outgoing headers). The header is ignored when the Bearer is a raw OAuth token rather than an exchanged Copilot API token.

The bug is partially masked because the secondary _try_activate_fallback path constructs its own client via resolve_provider_client() which DOES exchange — so the next-tier fallback (e.g. gpt-4.1) succeeds, hiding the first-attempt failure.

Fix

When the pool branch resolves Copilot, run the same get_copilot_api_token() exchange before assigning the returned api_key. Guarded against a string already in exchanged form (tid=...) so a future caller that pre-exchanges the token is not double-exchanged.

Strictly additive — no signature changes, only adds the missing exchange step to one specific provider branch. Other providers (openai-codex, anthropic, nous, etc.) are unaffected.

Reproduction

  1. Configure fallback_providers::
    fallback_providers:
      - provider: copilot
        model: claude-sonnet-4.6
  2. Force the primary to fail at gateway init (e.g. remove Codex auth so the gateway falls back at _resolve_runtime_agent_kwargs).
  3. Send any inbound message.
  4. Without the fix: chat-completions request fails with HTTP 400: The requested model is not available for integrator "copilot-language-server". With the fix: the request succeeds.

The 40-character ghu_* vs ~400-character tid=... Bearer-token length difference is observable in httpx.Client.send if you wrap it temporarily.

Test plan

  • Existing _resolve_runtime_from_pool_entry tests pass.
  • Add a test asserting that _resolve_runtime_from_pool_entry({provider: "copilot", runtime_api_key: "gho_..."}) returns an api_key that starts with tid= (mocking get_copilot_api_token to return a sentinel exchanged token).
  • Add a test asserting double-exchange is skipped when the entry's runtime_api_key already starts with tid=.

Note

This is a separate root cause from #17622 (which targets the same function for the api_mode calculation but doesn't touch the returned api_key). The two fixes are complementary; both are needed for the gateway-init Copilot fallback to work for Claude on Business seats.

🤖 Generated with Claude Code

…g api_key

The credential pool resolution path in `_resolve_runtime_from_pool_entry`
returns the raw GitHub OAuth token (`gho_*`/`ghu_*`) straight from the
pool entry's `runtime_api_key`.  The sibling `resolve_api_key_provider_
credentials("copilot")` path correctly calls `get_copilot_api_token()`
to exchange the raw token via `/copilot_internal/v2/token` for the
short-lived `tid=...` Bearer that Copilot's `/chat/completions`
endpoint actually expects.

When the gateway-init fallback path picks Copilot through the pool
branch, the raw token is sent unexchanged, and GitHub maps it to
whatever integrator the originating OAuth app declared as default --
typically ``copilot-language-server`` for Business seats -- which has
a restricted model allowlist that excludes Claude.  Result:

    HTTP 400: The requested model is not available for integrator
    "copilot-language-server"

even though the headers correctly advertise
``Copilot-Integration-Id: vscode-chat``.  The bug is partially masked
because the secondary `_try_activate_fallback` path constructs its
own client via `resolve_provider_client()` which DOES exchange -- so
the next-tier fallback (e.g. gpt-4.1) succeeds, hiding the first-
attempt failure.

Fix: when the pool branch resolves Copilot, run the same
`get_copilot_api_token()` exchange before assigning the returned
`api_key`.  Guarded against a string already in exchanged form
(``tid=...``) so a future caller that pre-exchanges the token is not
double-exchanged.

Reproducer: configure `fallback_providers: [{provider: copilot,
model: claude-sonnet-4.6}]`, force the primary to fail at gateway
init, observe the first chat-completions call ships
`Authorization: Bearer ghu_*` (raw, 40 chars) instead of
`Bearer tid=...` (exchanged, several hundred chars).
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/copilot GitHub Copilot (ACP + Chat) labels May 12, 2026
@lucvan
lucvan marked this pull request as ready for review May 13, 2026 06:09
@SvichkarevAnatoly

Copy link
Copy Markdown

Thanks for spotting this root cause 🙏 — the raw ghu_ token being sent to api.githubcopilot.com is exactly why most Copilot models return 400 model_not_available_for_integrator (GitHub pins the request to integrator copilot-language-server and ignores Copilot-Integration-Id: vscode-chat).

One heads-up on this diff: get_copilot_api_token() returns a tuple (api_token, base_url), so

api_key = get_copilot_api_token(api_key)

assigns the whole tuple to api_key, and a stringified ('tid=…', 'https://…') then goes on the wire. I applied this branch on current main and every model fails with HTTP 400: invalid token: invalid whitespace.

I opened #58830 building on your finding — it unpacks the tuple, also adopts the account-specific base_url from the exchange (Business/Enterprise tenants), covers both the pool and explicit resolution paths, and adds network-free regression tests. Verified live: every model that previously failed now returns 200. Credit to you for identifying the underlying issue — happy to consolidate however the maintainers prefer.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the Copilot pool-resolution gap; current main still returns the pool key unchanged from hermes_cli/runtime_provider.py:410 through the Copilot branch at :445-447.

Problems

  • The added assignment treats get_copilot_api_token() as returning a string, but it returns (api_token, base_url) at hermes_cli/copilot_auth.py:415-434. The tuple would be returned as api_key by hermes_cli/runtime_provider.py:524-532.
  • The equivalent explicit-key path remains unhandled: hermes_cli/runtime_provider.py:1474-1503 returns a Copilot explicit_api_key without exchange.
  • This PR changes only hermes_cli/runtime_provider.py; add regression coverage for the pool and explicit paths. Existing coverage only tests the auth resolver at tests/hermes_cli/test_copilot_token_exchange.py:153-166.

Suggested changes

  • Unpack token and base URL from the exchange, preserve fallback behavior, and use the account-specific exchanged endpoint where appropriate.
  • Cover raw, pre-exchanged, pool, and explicit-key cases with network-free tests.

Automated hermes-sweeper review.

if api_key and not api_key.startswith("tid="):
try:
from hermes_cli.copilot_auth import get_copilot_api_token
api_key = get_copilot_api_token(api_key)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_copilot_api_token() returns (api_token, base_url), not a token string (hermes_cli/copilot_auth.py:415-434). Unpack the result here; otherwise this tuple becomes the runtime bearer value.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/copilot GitHub Copilot (ACP + Chat) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants