Skip to content

fix(models): route Responses-only Copilot models to the Responses API - #58835

Open
SvichkarevAnatoly wants to merge 1 commit into
NousResearch:mainfrom
SvichkarevAnatoly:fix/copilot-responses-only-api-mode
Open

fix(models): route Responses-only Copilot models to the Responses API#58835
SvichkarevAnatoly wants to merge 1 commit into
NousResearch:mainfrom
SvichkarevAnatoly:fix/copilot-responses-only-api-mode

Conversation

@SvichkarevAnatoly

Copy link
Copy Markdown

What does this PR do?

Routes Responses-only Copilot models to the Responses API so they stop failing with HTTP 400 "model … is not accessible via the /chat/completions endpoint".

copilot_model_api_mode() picks the Copilot API mode primarily from a model-id regex (_should_use_copilot_responses_api) that only recognises GPT-5+ models as Responses-API models. A non-GPT model that is Responses-only — e.g. mai-code-1-flash-picker — falls through to chat_completions and the request fails.

This PR keeps the opencode-parity regex first (so GPT-5+ reasoning models stay on their native Responses endpoint even when the catalog also advertises /chat/completions), then consults the catalog's supported_endpoints to rescue models the regex doesn't cover: when a model does not advertise /chat/completions at all, it is routed by its declared endpoint — /responsescodex_responses, /v1/messagesanthropic_messages. Models that do advertise /chat/completions (Claude, Gemini on Copilot) keep using it, so their behaviour is unchanged.

Model supported_endpoints before after
mai-code-1-flash-picker /responses chat_completions → 400 codex_responses
claude-* / gemini-* (Copilot) /chat/completions, … chat_completions chat_completions (unchanged)
Messages-only model /v1/messages anthropic_messages anthropic_messages (unchanged)

Follow-up to #58830 (raw-token exchange). #58830 is required first — without it every Copilot model fails with the integrator error, so this change is only observable once tokens are exchanged correctly. Together they make every model advertised by a Copilot seat usable.

Related Issue

Relates to #45813

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • hermes_cli/models.py: in copilot_model_api_mode, replace the "messages-only" catalog check with a general "no /chat/completions" fallback that honours /responses (→ codex_responses) and /v1/messages (→ anthropic_messages).
  • tests/hermes_cli/test_model_validation.py: add 3 unit tests — Responses-only non-GPT model → codex_responses, Messages-only → anthropic_messages, chat-capable non-GPT stays chat_completions.

How to Test

  1. On top of fix(runtime): exchange raw Copilot OAuth token in runtime resolution #58830, configure the Copilot provider and select mai-code-1-flash-picker.
  2. On main: HTTP 400 … not accessible via the /chat/completions endpoint.
  3. With this patch: returns 200 via the Responses API and the model answers. All other Copilot models keep their previous api_mode.
  4. pytest tests/hermes_cli/test_model_validation.py -q → 90 passed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(models):)
  • I searched for existing PRs (no duplicate; follow-up to fix(runtime): exchange raw Copilot OAuth token in runtime resolution #58830)
  • My PR contains only changes related to this fix
  • I've run the tests and they pass (tests/hermes_cli/test_model_validation.py: 90 passed)
  • I've added tests for my changes
  • I've tested on my platform: macOS 15

Documentation & Housekeeping

  • No user-facing docs/config keys changed — N/A for README / cli-config.yaml.example
  • No architecture/workflow change — N/A for CONTRIBUTING.md / AGENTS.md
  • Cross-platform safe (pure-Python catalog/string handling)

`copilot_model_api_mode` decides the Copilot API mode primarily from a
model-id regex that only recognises GPT-5+ models as Responses-API
models. A non-GPT model that is Responses-only — e.g.
`mai-code-1-flash-picker` — falls through to `chat_completions` and the
request fails with HTTP 400 "model ... is not accessible via the
/chat/completions endpoint".

Keep the opencode-parity regex first (so GPT-5+ reasoning models stay on
their native Responses endpoint even when the catalog also advertises
/chat/completions), then consult the catalog's `supported_endpoints` to
rescue models the regex doesn't cover: when a model does NOT advertise
/chat/completions at all, route it by its declared endpoint
(`/responses` -> codex_responses, `/v1/messages` -> anthropic_messages).
Models that DO advertise /chat/completions (Claude, Gemini on Copilot)
keep using it, so behaviour for them is unchanged.

Verified live against a Business seat: `mai-code-1-flash-picker` now
returns 200 via the Responses API and self-identifies correctly, while
all other Copilot models keep their previous api_mode.

Follow-up to NousResearch#58830 (raw-token exchange); together they make every model
advertised by a Copilot seat usable. Relates to NousResearch#45813.

Signed-off-by: Anatoly Svichkarev <6915620+SvichkarevAnatoly@users.noreply.github.com>
@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard provider/copilot GitHub Copilot (ACP + Chat) P2 Medium — degraded but workaround exists labels Jul 5, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Comment (LGTM)

Routing fix: routes Responses-only Copilot models to the Responses API. Correctly distinguishes between models that use the Responses API endpoint vs. the Chat Completions endpoint. 2 files changed — model routing logic and tests.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the catalog-routing gap. The primary defect is present on current main: hermes_cli/models.py:3408-3425 routes non-GPT catalog entries through a fallback that recognizes only Messages-only models, so a Responses-only entry reaches chat_completions. The proposed catalog fallback addresses that path while retaining the existing GPT-first rule.

Problems

  • agent/chat_completion_helpers.py:1503-1534 independently computes fallback API mode. Its Copilot branch ultimately uses the GPT-name-only predicate in run_agent.py:1430-1433, so a non-GPT Copilot fallback advertising only /responses still activates as chat_completions.

Suggested changes

  • Reuse copilot_model_api_mode() or a shared catalog-aware resolver in the fallback route, and add a fallback-activation regression test for a non-GPT /responses-only catalog entry.

Automated hermes-sweeper review.

Comment thread hermes_cli/models.py
@@ -3310,9 +3315,11 @@ def copilot_model_api_mode(
for endpoint in (catalog_entry.get("supported_endpoints") or [])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes the primary resolver, but fallback activation still derives Copilot mode separately in agent/chat_completion_helpers.py:1503-1534 through the GPT-name-only predicate. Please route that sibling path through the same catalog-aware decision and add a fallback regression test; otherwise a non-GPT /responses-only fallback remains on Chat Completions.

@intellectronica

Copy link
Copy Markdown
Contributor

I reproduced this with Copilot Grok 4.6 and opened #88538 by cherry-picking 0df0765 (preserving your authorship), then adding the catalog-aware auxiliary and fallback activation paths requested in the maintainer review. Thank you for the original fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists provider/copilot GitHub Copilot (ACP + Chat) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants