Skip to content

fix(github-copilot): route per-model on /v1/responses based on model info - #19650

Closed
codgician wants to merge 1 commit into
BerriAI:litellm_oss_stagingfrom
codgician:ghc-responses
Closed

fix(github-copilot): route per-model on /v1/responses based on model info#19650
codgician wants to merge 1 commit into
BerriAI:litellm_oss_stagingfrom
codgician:ghc-responses

Conversation

@codgician

@codgician codgician commented Jan 23, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

Fixes #20103

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

Note: failing tests and lints are not caused by this change.

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🐛 Bug Fix

Changes

Problem

GitHub Copilot's /v1/responses endpoint is per-model: only some models (e.g. gpt-5.5, gpt-5.4, gpt-5.4-mini) support it natively upstream. The GithubCopilotResponsesAPIConfig provider config was registered for github_copilot provider-wide, so every model was forced to native /v1/responses dispatch. Chat-only models like gemini-3.1-pro-preview therefore failed upstream with model X does not support Responses API.

Fix

  • litellm/llms/github_copilot/responses/transformation.py: add github_copilot_supports_responses_api(model) helper. The helper reads mode and supported_endpoints from the merged model info (which incorporates user-supplied per-deployment model_info via the router's litellm.register_model call) using a 4-tier resolution:

    1. mode == "responses" → native dispatch (positive opt-in)
    2. mode == "chat" → chat-completions translation bridge (explicit opt-out, lets users force the bridge for dual-endpoint models)
    3. "/v1/responses" in supported_endpoints → native dispatch
    4. Otherwise → bridge (conservative default; the bridge always works because every Copilot model supports /chat/completions)

    Catalog lookup raising (model not registered) → bridge (conservative).

  • litellm/utils.py: ProviderConfigManager.get_provider_responses_api_config now calls the helper for the github_copilot branch and returns None for non-supporting models, letting the existing dispatcher in litellm/responses/main.py fall back to the chat-completions translation bridge.

  • model_prices_and_context_window.json: refresh github_copilot/* entries from upstream Copilot /models (replace 31 stale entries with 24 fresh ones). Each model now declares mode and (where applicable) supported_endpoints accurately, including mode: "responses" for Responses-capable models like gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.2, gpt-5-mini, and the codex variants. I wrote a small script on GitHub gist for retrieving raw info and converted litellm convention to achieve this.

Behavior

Model Before After
github_copilot/gemini-3.1-pro-preview on /v1/responses HTTP 400 from upstream HTTP 200 via chat-completions bridge
github_copilot/gpt-5.5 on /v1/responses HTTP 200 (native) HTTP 200 (native, unchanged)
github_copilot/gpt-5.4 on /v1/responses HTTP 200 (native) HTTP 200 (native, unchanged)

Tests

  • Added 10 unit tests covering the gate logic in tests/test_litellm/llms/github_copilot/responses/test_github_copilot_responses_transformation.py: every resolution tier (positive mode, negative mode, endpoint-list fallback, conservative defaults), model=None follow-up operations, and realistic catalog-shape fixtures for both chat-only and Responses-only Copilot models.
  • Updated 1 pre-existing test to use a model unaffected by the gate (model=None path).
  • All 28 tests in the file pass; sibling regression tests in test_responses_api_bridge_flag.py continue to pass.

Before the change, all GitHub Copilot models would be routed for GitHub Copilot Responses API when you call Responses API in LiteLLM. For models that does not support Responses API (like Claude), the API call would fail:

image

After the fix, such models would fallback to the chat completion translation bridge automatically, resulting in successful API calls:

image

@vercel

vercel Bot commented Jan 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
litellm Ready Ready Preview, Comment Jan 24, 2026 7:52am

Request Review

@Rilomilo

Copy link
Copy Markdown

hope this fix can be merged sooner

@codgician codgician changed the title fix(github-copilot): route only supported GPT-5 models to native responses API fix(github-copilot): route per-model on /v1/responses based on model info May 2, 2026
@codspeed-hq

codspeed-hq Bot commented May 2, 2026

Copy link
Copy Markdown
Contributor

Congrats! CodSpeed is installed 🎉

🆕 16 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


Open in CodSpeed

@codecov

codecov Bot commented May 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codgician
codgician changed the base branch from main to litellm_oss_staging May 2, 2026 03:44
@codgician

Copy link
Copy Markdown
Contributor Author

Updated the implementation to avoid hard-coding models and changed to targeting the correct branch for external contributors.

@codgician

Copy link
Copy Markdown
Contributor Author

@greptileai request review

@greptile-apps

greptile-apps Bot commented May 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes GitHub Copilot's /v1/responses routing by adding a per-model gate (github_copilot_supports_responses_api) that reads mode and supported_endpoints from the merged model catalog, so chat-only Copilot models (e.g. gemini-3.1-pro-preview) fall back to the chat-completions translation bridge instead of failing with a 400 from upstream. The catalog is also refreshed from 31 stale entries to 24 current ones, with responses-capable models correctly marked mode: \"responses\".

Confidence Score: 5/5

Safe to merge; the gate logic is correct, model=None path is preserved, and all resolution tiers are covered by tests.

No P0 or P1 findings. The one P2 note (removed catalog entries for models that may still be in user configs) is a documentation/migration concern rather than a code defect—users hitting removed models already get upstream errors regardless of LiteLLM.

model_prices_and_context_window.json — verify the removed entries are genuinely no longer available upstream before merging.

Important Files Changed

Filename Overview
litellm/llms/github_copilot/responses/transformation.py Adds github_copilot_supports_responses_api helper that reads mode/supported_endpoints from the merged model catalog to gate per-model native dispatch; logic and exception handling are correct.
litellm/utils.py Routes github_copilot through the per-model gate via a lazy import; model=None path preserved for follow-up GET/DELETE operations; consistent with existing provider-branch pattern in this function.
model_prices_and_context_window.json Refreshes 31 stale GitHub Copilot entries with 24 fresh ones; correctly marks responses-capable models with mode: "responses" and chat-only models with mode: "chat". Removes models like gpt-4, gpt-4o-mini, gpt-5.1-codex-max that may still be in user configs.
tests/test_litellm/llms/github_copilot/responses/test_github_copilot_responses_transformation.py Adds 10 new unit tests covering all four resolution tiers, catalog fixtures, model=None path, and exception fallback; autouse fixture pins catalog to local backup so no network calls are made; replaces deprecated gpt-5.1-codex with gpt-5.3-codex in the existing integration test.

Reviews (2): Last reviewed commit: "fix(github-copilot): route per-model on ..." | Re-trigger Greptile

@codgician

Copy link
Copy Markdown
Contributor Author

@greptileai request re-review

@mubashir1osmani

Copy link
Copy Markdown
Collaborator

resolve merge conflicts

@mateo-berri
mateo-berri deleted the branch BerriAI:litellm_oss_staging May 18, 2026 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: GitHub Copilot: unable to call Claude models via Responses API

4 participants