fix(copilot): exchange token in env seeding, catalog-aware API mode, ACP guard

[khpawan]

What does this PR do?

Fixes four related GitHub Copilot failures, all reproduced and verified against a live enterprise Copilot account. Three of them are independent of any model choice — they made Copilot look intermittently or completely broken.

The headline one is a credential-seeding order bug that caused intermittent HTTP 403s on every Copilot request.

1. _seed_from_env overwrote the exchanged token with the raw one → intermittent 403

load_pool("copilot") runs two seeders in sequence against the same env:COPILOT_GITHUB_TOKEN source key:

1. _seed_from_singletons correctly exchanges the raw ghu_/gho_ GitHub token for a short-lived tid=… Copilot token, and records the endpoint the exchange advertises (business/enterprise accounts get a dedicated proxy, e.g. api.enterprise.githubcopilot.com).
2. _seed_from_env then upserts the same key — and clobbered that with the raw GitHub token plus the generic api.githubcopilot.com.

GitHub rejects that pairing intermittently. Measured on a live enterprise account:

Credential written to the pool	Result
exchanged tid= + account endpoint	10/10 HTTP 200
raw ghu_ + api.githubcopilot.com	~1 in 3 requests HTTP 403

Because the persisted entry stores only a fingerprint and re-hydrates from env, auth.json looked healthy while requests failed at random. _seed_from_env now exchanges for copilot, mirroring the existing kimi-coding / zai base-URL hooks in the same loop. The exchange is memoised in copilot_auth._jwt_cache, so it adds no round-trip. An explicit COPILOT_API_BASE_URL still wins, and a failed exchange still degrades to the previous raw-token behaviour.

2. Responses-only non-GPT models routed to /chat/completions → HTTP 400

copilot_model_api_mode classified models by the ^gpt-N name pattern only. Copilot also ships responses-only models under other vendor prefixes — grok-4.5 advertises exactly ["/responses"] — so they got chat_completions and returned 400 "not accessible via the /chat/completions endpoint".

The docstring already promised a supported_endpoints fallback that was never implemented; this implements it, conservatively: upgrade only when /responses is present and /chat/completions is absent, so dual-endpoint models (Claude, gpt-5-mini) keep their current routing. A 1-hour catalog cache mirrors the existing _copilot_context_cache convention.

3. resolve_provider_client bypassed that decision

It called the name-pattern helper directly, so fix 2 never reached the main client-construction path. It now uses copilot_model_api_mode, resolved before the per-request headers are built so the catalog request cannot interleave with header construction.

4. ACP providers upgraded into a Responses call they cannot serve

CopilotACPClient speaks chat-shaped JSON-RPC over a spawned subprocess and exposes only .chat — there is no .responses attribute. A gpt-5.x model reached over copilot-acp matched the "GPT-5 → Responses API" rule and died with:

AttributeError: 'CopilotACPClient' object has no attribute 'responses'

agent_init.py guards the primary path by provider/base_url, but _try_activate_fallback in chat_completion_helpers.py recomputes the mode from scratch — so a Copilot ACP entry configured as a fallback still crashed, and its explicit api_mode: chat_completions was ignored. Per AGENTS.md ("fix the whole bug class including sibling call paths"), the rule now lives in the shared AIAgent._provider_model_requires_responses_api helper, which covers both call sites, plus an acp:// / acp+tcp:// branch in the fallback path.

Relationship to existing PRs

I searched open PRs before filing. Flagging the overlap explicitly so maintainers can dedupe:

• fix(models): route Responses-only Copilot models to the Responses API #58835 (fix(models): route Responses-only Copilot models to the Responses API) proposes the same supported_endpoints fallback as my fix 2, for mai-code-1-flash-picker. Open since 2026-07-05. If fix(models): route Responses-only Copilot models to the Responses API #58835 lands first I'll happily drop my fix 2 and rebase — fix 3 then simply consumes their version. Fixes 1, 3 and 4 are independent of it.
• fix(copilot): thread target_model and route Claude/Grok via catalog endpoints #57625 touches copilot_model_api_mode for Claude/Grok catalog routing; overlapping area, different scope (gateway target_model threading).
• fix(copilot): refresh base_url via live token exchange during credential rotation #62689 / fix(run_agent): refresh Copilot token and base_url together during credential rotation #61754 / fix(copilot): keep rotated credentials on account endpoint #61757 address the same symptom as my fix 1 (stale public endpoint → 403) but at credential rotation time in run_agent/gateway. This PR fixes the earlier root cause: the pool entry is seeded wrong in the first place, in _seed_from_env. They are complementary, not duplicates.

Related Issue

Fixes #

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• agent/credential_pool.py — _seed_from_env exchanges the raw GitHub token for copilot and uses the account endpoint from the exchange.
• hermes_cli/models.py — implement the documented supported_endpoints fallback in copilot_model_api_mode; add _cached_github_model_catalog (1h TTL).
• agent/auxiliary_client.py — resolve_provider_client uses copilot_model_api_mode; decision resolved before header construction.
• run_agent.py — _provider_model_requires_responses_api returns False for copilot-acp.
• agent/chat_completion_helpers.py — fallback activation pins chat_completions for acp:// / acp+tcp://.
• Tests: 15 new behavior-contract tests across 3 files.

How to Test

Bug 1 (403s) — before/after, no mocks:

from agent.credential_pool import load_pool
e = list(load_pool("copilot").entries())[0]
print(e.base_url, e.access_token[:4])
# before: https://api.githubcopilot.com ghu_
# after:  https://api.enterprise.githubcopilot.com tid=

Then issue ~10 /models requests with the pool credential: before the fix roughly a third return 403; after, 10/10 return 200. auth.json self-repairs on the next load_pool().

Bugs 2–4 — live routing:

from agent.auxiliary_client import resolve_provider_client
for p, m in [("copilot","grok-4.5"), ("copilot","claude-sonnet-5"),
             ("copilot","gpt-5-mini"), ("copilot-acp","gpt-5.6-sol")]:
    c, mm = resolve_provider_client(provider=p, model=m)[:2]
    print(p, m, type(c).__name__)

Expected after the fix (all four complete a real turn):

copilot grok-4.5        CodexAuxiliaryClient   # was HTTP 400
copilot claude-sonnet-5 OpenAI                 # unchanged
copilot gpt-5-mini      OpenAI                 # unchanged
copilot-acp gpt-5.6-sol CopilotACPClient       # was AttributeError

Tests:

scripts/run_tests.sh tests/agent/test_credential_pool_copilot_env_exchange.py \
                     tests/agent/test_acp_never_uses_responses_api.py \
                     tests/hermes_cli/test_copilot_model_api_mode_endpoints.py

Each new test was verified to fail without its corresponding fix (reverting each hunk individually turns the relevant assertions red), so none are tautological.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits
• I searched for existing PRs to make sure this isn't a duplicate — see "Relationship to existing PRs" above
• My PR contains only changes related to this fix
• I've run the test suite — tests/agent + tests/hermes_cli: 17202 passed. (The 3 unrelated failures observed under 40-worker load — test_kanban_db_init, test_relay_shared_metrics, test_early_recovery — reproduce on unpatched main in the same environment or pass when re-run serially.)
• I've added tests for my changes
• I've tested on my platform: Ubuntu 24.04 aarch64 (NVIDIA DGX Spark), Python 3.11

Documentation & Housekeeping

• I've updated relevant documentation (docstrings on the changed functions) — the copilot_model_api_mode docstring already described the supported_endpoints fallback this PR implements
• N/A — no config keys added or changed
• N/A — no architecture or workflow changes
• Cross-platform: all four changes are pure Python control flow with no platform-specific behaviour
• N/A — no tool descriptions/schemas changed

Screenshots / Logs

Original failures from ~/.hermes/logs/errors.log:

provider=copilot-acp base_url=acp://copilot model=gpt-5.6-sol
  AttributeError: 'CopilotACPClient' object has no attribute 'responses'   (x3 retries)

provider=copilot model=grok-4.5
  HTTP 400 'model "grok-4.5" is not accessible via the /chat/completions endpoint'

provider=github-copilot base_url=https://api.githubcopilot.com model=claude-opus-5
  HTTP 403

All three are silent after this change.

[teknium1]

Thanks for tracing the Copilot credential and transport failures. The current-main premises are present, but two routing details need revision before this is safe to salvage.

Problems

• agent/chat_completion_helpers.py:1856-1863 resolves fallback mode through AIAgent._provider_model_requires_responses_api(). Current run_agent.py:1532-1540 uses only the GPT-name Copilot predicate. This PR adds the ACP exclusion, but a regular non-GPT Copilot fallback that advertises only /responses still becomes chat_completions.
• The proposed _cached_github_model_catalog(api_key=...) is one global cache, although fetch_github_model_catalog() returns a catalog “for this account” (hermes_cli/models.py:3355-3387). A catalog must not be reused across credentials without account/credential scoping.
• The new ACP regression test calls the helper directly rather than exercising the fallback activation path described in the PR.

Suggested changes

• Route fallback Copilot mode through the same catalog-aware resolver and add a non-GPT /responses-only fallback test.
• Key the catalog cache by a non-secret credential fingerprint, and test sequential distinct credentials.
• Add an ACP fallback activation test asserting chat_completions.

Automated hermes-sweeper review.

[teknium1]

This checks the helper only. Please add a try_activate_fallback() regression test: the reported failure is the fallback path recomputing API mode after resolve_provider_client() has returned the ACP client.

[fanyangCS]

Independent reproduction, live Enterprise Copilot account

Hit this in production (Hermes at d54a8f707) — gateway down with model provider failed after retries on every turn. Item 1 of this PR is the cause.

Same account, same moment:

Credential → /responses	Result
raw ghu_ from COPILOT_GITHUB_TOKEN	403 ToS
exchanged tid=… @ api.enterprise.githubcopilot.com	200

/chat/completions returns 200 with either credential, so the break is specific to /responses — where api_mode: codex_responses sends everything. Token was fine: GET /user → 200.

Easy to misdiagnose: the pool persists only secret_fingerprint and re-hydrates from env, so auth.json looks healthy while every request fails, and _seed_from_env re-clobbers the exchanged entry on each load_pool(). With GitHub's ToS wording on the 403, the natural first read is "Responses API blocked for third-party clients" — which points at downgrading the model instead. Probing with the raw token reinforces it: gpt-5.6-sol/-terra also return 400 "not accessible via the /chat/completions endpoint".

Mechanism, agent/credential_pool.py: _seed_from_singletons (~L1923) exchanges correctly; _seed_from_env (~L2198) then upserts the same source with the raw token — its base_url hooks only cover kimi-coding and zai.

Applying only item 1 restored service:

source=env:COPILOT_GITHUB_TOKEN
token_kind=EXCHANGED(tid=)
base_url=https://api.enterprise.githubcopilot.com
status=None

403 count since: 0. No config.yaml change needed.

The review findings above all concern items 2–3. Item 1 is independently reproducible and sufficient on its own — happy to open a narrow credential-seeding-only PR if useful.