fix(github-copilot): route per-model on /v1/responses based on model info - #19650
fix(github-copilot): route per-model on /v1/responses based on model info#19650codgician wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
52f2820 to
a5bd11d
Compare
a5bd11d to
6f14437
Compare
6f14437 to
e93137f
Compare
|
hope this fix can be merged sooner |
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Updated the implementation to avoid hard-coding models and changed to targeting the correct branch for external contributors. |
|
@greptileai request review |
Greptile SummaryThis PR fixes GitHub Copilot's Confidence Score: 5/5Safe to merge; the gate logic is correct, model=None path is preserved, and all resolution tiers are covered by tests. No P0 or P1 findings. The one P2 note (removed catalog entries for models that may still be in user configs) is a documentation/migration concern rather than a code defect—users hitting removed models already get upstream errors regardless of LiteLLM. model_prices_and_context_window.json — verify the removed entries are genuinely no longer available upstream before merging.
|
| Filename | Overview |
|---|---|
| litellm/llms/github_copilot/responses/transformation.py | Adds github_copilot_supports_responses_api helper that reads mode/supported_endpoints from the merged model catalog to gate per-model native dispatch; logic and exception handling are correct. |
| litellm/utils.py | Routes github_copilot through the per-model gate via a lazy import; model=None path preserved for follow-up GET/DELETE operations; consistent with existing provider-branch pattern in this function. |
| model_prices_and_context_window.json | Refreshes 31 stale GitHub Copilot entries with 24 fresh ones; correctly marks responses-capable models with mode: "responses" and chat-only models with mode: "chat". Removes models like gpt-4, gpt-4o-mini, gpt-5.1-codex-max that may still be in user configs. |
| tests/test_litellm/llms/github_copilot/responses/test_github_copilot_responses_transformation.py | Adds 10 new unit tests covering all four resolution tiers, catalog fixtures, model=None path, and exception fallback; autouse fixture pins catalog to local backup so no network calls are made; replaces deprecated gpt-5.1-codex with gpt-5.3-codex in the existing integration test. |
Reviews (2): Last reviewed commit: "fix(github-copilot): route per-model on ..." | Re-trigger Greptile
|
@greptileai request re-review |
|
resolve merge conflicts |
Relevant issues
Fixes #20103
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitNote: failing tests and lints are not caused by this change.
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🐛 Bug Fix
Changes
Problem
GitHub Copilot's
/v1/responsesendpoint is per-model: only some models (e.g.gpt-5.5,gpt-5.4,gpt-5.4-mini) support it natively upstream. TheGithubCopilotResponsesAPIConfigprovider config was registered forgithub_copilotprovider-wide, so every model was forced to native/v1/responsesdispatch. Chat-only models likegemini-3.1-pro-previewtherefore failed upstream withmodel X does not support Responses API.Fix
litellm/llms/github_copilot/responses/transformation.py: addgithub_copilot_supports_responses_api(model)helper. The helper readsmodeandsupported_endpointsfrom the merged model info (which incorporates user-supplied per-deploymentmodel_infovia the router'slitellm.register_modelcall) using a 4-tier resolution:mode == "responses"→ native dispatch (positive opt-in)mode == "chat"→ chat-completions translation bridge (explicit opt-out, lets users force the bridge for dual-endpoint models)"/v1/responses"insupported_endpoints→ native dispatch/chat/completions)Catalog lookup raising (model not registered) → bridge (conservative).
litellm/utils.py:ProviderConfigManager.get_provider_responses_api_confignow calls the helper for thegithub_copilotbranch and returnsNonefor non-supporting models, letting the existing dispatcher inlitellm/responses/main.pyfall back to the chat-completions translation bridge.model_prices_and_context_window.json: refreshgithub_copilot/*entries from upstream Copilot/models(replace 31 stale entries with 24 fresh ones). Each model now declaresmodeand (where applicable)supported_endpointsaccurately, includingmode: "responses"for Responses-capable models likegpt-5.5,gpt-5.4,gpt-5.4-mini,gpt-5.2,gpt-5-mini, and the codex variants. I wrote a small script on GitHub gist for retrieving raw info and converted litellm convention to achieve this.Behavior
github_copilot/gemini-3.1-pro-previewon/v1/responsesgithub_copilot/gpt-5.5on/v1/responsesgithub_copilot/gpt-5.4on/v1/responsesTests
tests/test_litellm/llms/github_copilot/responses/test_github_copilot_responses_transformation.py: every resolution tier (positivemode, negativemode, endpoint-list fallback, conservative defaults),model=Nonefollow-up operations, and realistic catalog-shape fixtures for both chat-only and Responses-only Copilot models.model=Nonepath).test_responses_api_bridge_flag.pycontinue to pass.Before the change, all GitHub Copilot models would be routed for GitHub Copilot Responses API when you call Responses API in LiteLLM. For models that does not support Responses API (like Claude), the API call would fail:
After the fix, such models would fallback to the chat completion translation bridge automatically, resulting in successful API calls: