Repository navigation
fix: answer get_api_base for github_copilot and chatgpt without running the login flow - #42602
Conversation
…ng the login flow
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
| def _api_base_without_login(provider: str) -> str | None: | ||
| match provider: | ||
| case "github_copilot": | ||
| return litellm.GithubCopilotConfig().api_base_without_login() | ||
| case "chatgpt": | ||
| return litellm.ChatGPTConfig().api_base_without_login() | ||
| case _: | ||
| return None | ||
|
|
There was a problem hiding this comment.
Provider Logic Outside Adapters
This shared utility directly branches on GitHub Copilot and ChatGPT and constructs their configurations. Repository rules require provider-specific logic to remain under litellm/llms/, so this requirement must be satisfied before merging
Rule Used: What: Avoid writing provider-specific code outside of the llms/ directory. Why: This practice ensures better maintainability and reduces complexity over time. Good: ```python # Handle provider-specific logic within llms/vertex_ai/transformation.py ... (source)
Knowledge Base Used: Provider adapters and capabilities
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
The provider knowledge lives in each adapter's api_base_without_login under llms/. This util only dispatches on the provider name, like the gemini and openai branch below
There was a problem hiding this comment.
You're right — this utility only dispatches to each adapter's api_base_without_login(); the provider-specific URL logic remains under litellm/llms/. That is consistent with the existing Gemini/OpenAI default-base dispatch in this file, so the comment does not apply. I'll retract it.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit bcdcb8a. Configure here.
TLDR
Problem this solves:
litellm.get_api_basefor agithub_copilot/orchatgpt/model ran the OAuth device loginHow it solves it:
User Flow
Before: a developer with no stored GitHub Copilot login asks the SDK which endpoint a Copilot model will hit and gets three device-code prompts, a 3-minute hang, and None
litellm.get_api_base(model="github_copilot/gpt-5.3-codex", optional_params={})Please visit https://github.com/login/device and enter code XXXX-XXXX to authenticate.and waits about a minuteTimed out waiting for user to authorize the devicewarningNoneAfter: the same call returns the Copilot endpoint at once with no login prompt
litellm.get_api_base(model="github_copilot/gpt-5.3-codex", optional_params={})https://api.githubcopilot.comat once (20 ms in the run below) with no device-code promptRelevant issues
Affected release
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup for both sides: one detached worktree per commit (d7c27cd is the merge base, bcdcb8a is this PR's tip), one shared virtualenv, no database. Every proxy boots with
python litellm/proxy/proxy_cli.py --config <config> --port <random free port> --num_workers 2 --detailed_debugand gets killed after its leg. The no-login cases pointGITHUB_COPILOT_TOKEN_DIRat an empty directory; the chatgpt case pointsCHATGPT_TOKEN_DIRat a directory holding a real stored login. Both configs setrouter_settings.num_retries: 0The SDK probe (case 1) is this file, run as
GITHUB_COPILOT_TOKEN_DIR=<empty dir> python sdk_probe.py github_copilot/gpt-5.3-codex:Case 4's curl, sent to the wildcard proxy, is
and case 3 sends the same prompt to
/v1/chat/completionswith"stream":true, to/v1/responseswith"input":"Reply with the single word pong"non-stream and stream, and to/v1/messageswith"max_tokens":32, all againstchatgpt-terraBefore (d7c27cd)
1. SDK:
litellm.get_api_basefor a Copilot model with no stored loginFails: three device-code prompts, three timeouts,
Noneafter 190 sGITHUB_COPILOT_TOKEN_DIRpointing at an empty directory2. The rerank unit test that used to sit in that login flow
Fails: the test spends 191 s in GitHub's device flow
pytest 'tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution' --durations=0 -qwith the same empty token directory3. Proxy, 2 workers: chatgpt with a stored login, every unified endpoint
Passes: streaming chat answers
pongfromhttps://chatgpt.com/backend-api/codex;/v1/responsesand/v1/messagesfail the same way they do after (pre-existing, see caveats)config-chatgpt.yamlwithCHATGPT_TOKEN_DIRholding the stored loginx-litellm-model-api-baseheader, and the bodies4. Proxy, 2 workers: a Copilot request with no stored login
Fails the request either way: 400 after three login rounds (about ten minutes), identical to After
GITHUB_COPILOT_TOKEN_DIRpointing at an empty directoryAfter (bcdcb8a)
1. SDK:
litellm.get_api_basefor a Copilot model with no stored loginPasses:
https://api.githubcopilot.comin well under a second, no promptGITHUB_COPILOT_TOKEN_DIRpointing at an empty directory2. The rerank unit test that used to sit in that login flow
Passes: the test finishes in well under a second
pytest 'tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution' --durations=0 -qwith the same empty token directory3. Proxy, 2 workers: chatgpt with a stored login, every unified endpoint
Passes: same statuses, header, and bodies as Before
config-chatgpt.yamlwithCHATGPT_TOKEN_DIRholding the stored loginx-litellm-model-api-baseheader, and the bodies4. Proxy, 2 workers: a Copilot request with no stored login
Unchanged: 400 after three login rounds, the same count of device-code prompts
GITHUB_COPILOT_TOKEN_DIRpointing at an empty directoryType
🐛 Bug Fix
Caveats (if any)
Low
model_liststill logs in at proxy boot, same as baseget_api_basenow returns the provider default for a declared copilot or chatgpt model where base returned None after a failed login; that is the fix/v1/responsesand/v1/messagesfail identically on both trees; tracked in [Bug]: chatgpt/* non-streaming chat completions fail with "Unknown items in responses API response: []" (streaming works) #37039, chatgpt/ provider: gpt-5.6-sol response.completed has empty output[] despite streamed content, raises "Unknown items in responses API response: []" #41009, [Bug]: chatgpt/gpt-5.4 returns empty final Responses output, and completion() bridge fails with "Unknown items in responses API response: []" #25429, [Bug]: chatgpt/* ignores the client's stream:false since v1.90.0; /v1/responses returns raw SSE and /chat/completions raises "Unknown items in responses API response: []" #34094, chatgpt/gpt-5.4 returns empty output=[] with status='completed' via litellm.responses() #26179, [Bug]:chatgpt/gpt-5.4throw exception when stream isfalse#26309get_api_basewere not driven live; they receive the same value the SDK case showsget_api_base, so no fix here:test_secret_detection::test_scan_message_stays_linear_on_adversarial_credential_lines[assignment-flood], a call-count ratio that failed on 7 of the last 12 main runs (90056, 90023, 90021, 89979, 89977, 89976, 89974)usageActivityTabs.spec.ts"Model Activity can name its models by deployment instead of by public name", the same Playwright failure as main run 90101using_litellm_on_windows, a 10-minute no-output timeout duringuv syncon the Windows executor, which also killed main runs 90068, 90060, 90023, and 90021test_cost_tracking::test_case_bills_expected_cost[gpt-5.6-input_text], a 20-second spend-rollup poll that passed on the single rerun-from-failed (workflow e5d8b8ae)Live PR risk (bcdcb8a)
Verdict: passes. No caller of the changed function loses a value it had before, and the one observable change is the one this PR exists for
Breaking: none.
get_api_base(model, optional_params)keeps its signature and itsstr | Nonereturn, anddeclared_authenticating_provideris only read here, not changedBackward incompatible: for a model declared as
github_copilot/...orchatgpt/...with noapi_baseset,get_api_basenow returns the provider's default endpoint (or the endpoint stored on disk for Copilot) where it used to run the device login and returnNoneonce that login timed out. With a stored login the value is the same as before. This is the requested fix and staysRegression risk: the readers of this value are Slack alerting, the hanging-request check, cooldown callbacks, exception mapping, the
x-litellm-model-api-baseresponse header, and both streaming iterators. Each gets the provider default where it gotNoneafter a 3-minute hang, or the same string as before when a login is stored. Case 3 above drives the header path live on both sides; the other readers only display the stringDependency graph:
litellm.get_api_baseis called fromlitellm/interactions/streaming_iterator.py,litellm/responses/streaming_iterator.py,litellm/integrations/SlackAlerting/slack_alerting.py,litellm/integrations/SlackAlerting/hanging_request_check.py,litellm/litellm_core_utils/exception_mapping_utils.py,litellm/litellm_core_utils/llm_response_utils/response_metadata.py,litellm/litellm_core_utils/streaming_handler.py, andlitellm/router_utils/cooldown_callbacks.py.declared_authenticating_provideralready gates the same login inlitellm/utils.py,litellm/router.py,litellm/proxy/proxy_server.py,litellm/rerank_api/main.py,litellm/litellm_core_utils/get_supported_openai_params.py,litellm/router_utils/pattern_match_deployments.py,litellm/router_strategy/savings_baseline.py, andlitellm/router_strategy/complexity_router/complexity_router.py, so the new branch mirrors a dispatch those paths already take. No config key, env var, DB column, or wire field changed, andGithubCopilotConfig().api_base_without_login()andChatGPTConfig().api_base_without_login()already existedNot verified: the chatgpt no-login SDK call (its device flow blocks up to 15 minutes; the unit test pins it), the alerting and cooldown readers driven live (they only display the string), and CircleCI's legacy suites: red only on tests main already fails (the #42435 reference pipeline 90098 plus main runs 90101, 90068, 90060, 90056, 90023, and 90021) and one spend-rollup poll that passed on its single rerun, recorded as a Low caveat
Final Attestation
The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
bcdcb8a passes /live-pr-risk