Skip to content

fix: answer get_api_base for github_copilot and chatgpt without running the login flow - #42602

Merged
mateo-berri merged 2 commits into
mainfrom
litellm_get_api_base_skip_authenticating_provider_resolution
Sep 23, 2026
Merged

mateo-berri merged 2 commits into
mainfrom
litellm_get_api_base_skip_authenticating_provider_resolution

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • litellm.get_api_base for a github_copilot/ or chatgpt/ model ran the OAuth device login
  • With no stored login it printed device-code prompts, blocked about 3 minutes, then returned None
  • The rerank unit test on that path polled GitHub's device flow live for 3 minutes in CI

How it solves it:

  • A model that already names github_copilot or chatgpt answers from the stored token or provider default, no login
  • The rerank test's 300 s timeout marker is gone; it finishes in well under a second

User Flow

Before: a developer with no stored GitHub Copilot login asks the SDK which endpoint a Copilot model will hit and gets three device-code prompts, a 3-minute hang, and None

  1. In a Python shell they run litellm.get_api_base(model="github_copilot/gpt-5.3-codex", optional_params={})
  2. The shell prints Please visit https://github.com/login/device and enter code XXXX-XXXX to authenticate. and waits about a minute
  3. The same prompt repeats twice more, each followed by a Timed out waiting for user to authorize the device warning
  4. After 190 s the call returns None

After: the same call returns the Copilot endpoint at once with no login prompt

  1. In a Python shell they run litellm.get_api_base(model="github_copilot/gpt-5.3-codex", optional_params={})
  2. It returns https://api.githubcopilot.com at once (20 ms in the run below) with no device-code prompt

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup for both sides: one detached worktree per commit (d7c27cd is the merge base, bcdcb8a is this PR's tip), one shared virtualenv, no database. Every proxy boots with python litellm/proxy/proxy_cli.py --config <config> --port <random free port> --num_workers 2 --detailed_debug and gets killed after its leg. The no-login cases point GITHUB_COPILOT_TOKEN_DIR at an empty directory; the chatgpt case points CHATGPT_TOKEN_DIR at a directory holding a real stored login. Both configs set router_settings.num_retries: 0

# config-wild.yaml (case 4)
model_list:
  - model_name: "*"
    litellm_params:
      model: "*"
router_settings:
  num_retries: 0
# config-chatgpt.yaml (case 3)
model_list:
  - model_name: chatgpt-terra
    litellm_params:
      model: chatgpt/gpt-5.6-terra
router_settings:
  num_retries: 0

The SDK probe (case 1) is this file, run as GITHUB_COPILOT_TOKEN_DIR=<empty dir> python sdk_probe.py github_copilot/gpt-5.3-codex:

import sys
import time

import litellm

model = sys.argv[1]
print(f'>>> litellm.get_api_base(model="{model}", optional_params={{}})')
start = time.time()
print("->", litellm.get_api_base(model=model, optional_params={}))
print(f"elapsed {time.time() - start:.2f}s")

Case 4's curl, sent to the wildcard proxy, is

curl -s -m 1500 http://127.0.0.1:$PORT/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
  -d '{"model":"github_copilot/gpt-5.3-codex","messages":[{"role":"user","content":"Reply with the single word pong"}]}' -w 'status=%{http_code} time=%{time_total}s\n'

and case 3 sends the same prompt to /v1/chat/completions with "stream":true, to /v1/responses with "input":"Reply with the single word pong" non-stream and stream, and to /v1/messages with "max_tokens":32, all against chatgpt-terra

Before (d7c27cd)

1. SDK: litellm.get_api_base for a Copilot model with no stored login

Fails: three device-code prompts, three timeouts, None after 190 s

  1. Run the probe with GITHUB_COPILOT_TOKEN_DIR pointing at an empty directory
  2. Read what it prints and how long it took
15:38:24 - LiteLLM:WARNING: authenticator.py:145 - Error reading API endpoint from file: [Errno 2] No such file or directory: '<qa dir>/copilot-sdk-base/api-key.json'
15:38:24 - LiteLLM:WARNING: authenticator.py:101 - No API key file found or error opening file
15:38:24 - LiteLLM:WARNING: authenticator.py:58 - No existing access token found or error reading file
>>> litellm.get_api_base(model="github_copilot/gpt-5.3-codex", optional_params={})
Please visit https://github.com/login/device and enter code XXXX-XXXX to authenticate.
15:39:27 - LiteLLM:WARNING: authenticator.py:71 - Failed attempt 1: Timed out waiting for user to authorize the device
Please visit https://github.com/login/device and enter code XXXX-XXXX to authenticate.
15:40:30 - LiteLLM:WARNING: authenticator.py:71 - Failed attempt 2: Timed out waiting for user to authorize the device
Please visit https://github.com/login/device and enter code XXXX-XXXX to authenticate.
15:41:34 - LiteLLM:WARNING: authenticator.py:71 - Failed attempt 3: Timed out waiting for user to authorize the device
-> None
elapsed 190.41s

2. The rerank unit test that used to sit in that login flow

Fails: the test spends 191 s in GitHub's device flow

  1. Run pytest 'tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution' --durations=0 -q with the same empty token directory
  2. Read the duration line
.                                                                        [100%]
============================== slowest durations ===============================
191.11s call     tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution
0.23s setup    tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution
0.04s teardown tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution
1 passed in 192.23s (0:03:12)

3. Proxy, 2 workers: chatgpt with a stored login, every unified endpoint

Passes: streaming chat answers pong from https://chatgpt.com/backend-api/codex; /v1/responses and /v1/messages fail the same way they do after (pre-existing, see caveats)

  1. Boot the proxy from config-chatgpt.yaml with CHATGPT_TOKEN_DIR holding the stored login
  2. Send the four requests described in the setup
  3. Read each status, the x-litellm-model-api-base header, and the bodies
== chat completions, stream
status=200 time=7.770696s
== responses, non-stream
status=400 time=0.951704s
== responses, stream
status=400 time=0.568778s
== messages, non-stream
status=500 time=6.541157s

-- chat-stream response header and first chunk
x-litellm-model-api-base: https://chatgpt.com/backend-api/codex
data: {"id":"resp_08d97262321b386b016ab300dabc6487d19c07cac0de632c09","object":"chat.completion.chunk","created":1790116059,"model":"chatgpt-terra","choices":[{"index":0,"delta":{"role":"assistant","content":"pong"}}]}

-- responses body (both requests)
{"error":{"message":"litellm.BadRequestError: ChatgptException - {\"detail\":\"Input must be a list\"}\n\nLiteLLM: model group 'chatgpt-terra' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

-- messages body
{"type":"error","error":{"type":"api_error","message":"litellm.APIConnectionError: APIConnectionError: ChatgptException - Unknown items in responses API response: []\n\nLiteLLM: model group 'chatgpt-terra' failed with the error above. No fallback was attempted."}}

4. Proxy, 2 workers: a Copilot request with no stored login

Fails the request either way: 400 after three login rounds (about ten minutes), identical to After

  1. Boot the wildcard proxy with GITHUB_COPILOT_TOKEN_DIR pointing at an empty directory
  2. Send the case 4 curl and count the login rounds the proxy log shows while it runs
== copilot no-login case, started 22:27:33
status=400 time=580.884667s
login flows during boot: 0
login flows during the request: 3
device code prompts in proxy log (3 per flow): 9
{"error":{"message":"litellm.BadRequestError: GetLLMProvider Exception - Failed to get access token after 3 attempts\n\noriginal model: github_copilot/gpt-5.3-codex\n\nLiteLLM: model group 'github_copilot/gpt-5.3-codex' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

After (bcdcb8a)

1. SDK: litellm.get_api_base for a Copilot model with no stored login

Passes: https://api.githubcopilot.com in well under a second, no prompt

  1. Run the probe with GITHUB_COPILOT_TOKEN_DIR pointing at an empty directory
  2. Read what it prints and how long it took
16:35:13 - LiteLLM:WARNING: authenticator.py:145 - Error reading API endpoint from file: [Errno 2] No such file or directory: '<qa dir>/copilot-sdk-after/api-key.json'
>>> litellm.get_api_base(model="github_copilot/gpt-5.3-codex", optional_params={})
-> https://api.githubcopilot.com
elapsed 0.02s

2. The rerank unit test that used to sit in that login flow

Passes: the test finishes in well under a second

  1. Run pytest 'tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution' --durations=0 -q with the same empty token directory
  2. Read the duration line
.                                                                        [100%]
============================== slowest durations ===============================
1.27s setup    tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution
0.06s call     tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution
0.01s teardown tests/test_litellm/rerank_api/test_main.py::test_arerank_declared_authenticating_provider_skips_resolution
1 passed in 1.82s

3. Proxy, 2 workers: chatgpt with a stored login, every unified endpoint

Passes: same statuses, header, and bodies as Before

  1. Boot the proxy from config-chatgpt.yaml with CHATGPT_TOKEN_DIR holding the stored login
  2. Send the four requests described in the setup
  3. Read each status, the x-litellm-model-api-base header, and the bodies
== chat completions, stream
status=200 time=4.464650s
== responses, non-stream
status=400 time=4.304136s
== responses, stream
status=400 time=0.560556s
== messages, non-stream
status=500 time=3.375635s

-- chat-stream response header and first chunk
x-litellm-model-api-base: https://chatgpt.com/backend-api/codex
data: {"id":"resp_08559671aaecd0be016ab31140e72087d291712e2148c55599","object":"chat.completion.chunk","created":1790120257,"model":"chatgpt-terra","choices":[{"index":0,"delta":{"role":"assistant","content":"pong"}}]}

-- responses body (both requests)
{"error":{"message":"litellm.BadRequestError: ChatgptException - {\"detail\":\"Input must be a list\"}\n\nLiteLLM: model group 'chatgpt-terra' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

-- messages body
{"type":"error","error":{"type":"api_error","message":"litellm.APIConnectionError: APIConnectionError: ChatgptException - Unknown items in responses API response: []\n\nLiteLLM: model group 'chatgpt-terra' failed with the error above. No fallback was attempted."}}

4. Proxy, 2 workers: a Copilot request with no stored login

Unchanged: 400 after three login rounds, the same count of device-code prompts

  1. Boot the wildcard proxy with GITHUB_COPILOT_TOKEN_DIR pointing at an empty directory
  2. Send the case 4 curl and count the login rounds the proxy log shows while it runs
== copilot no-login case, started 23:51:46
status=400 time=591.225481s
login flows during boot: 0
login flows during the request: 3
device code prompts in proxy log (3 per flow): 9
{"error":{"message":"litellm.BadRequestError: GetLLMProvider Exception - Failed to get access token after 3 attempts\n\noriginal model: github_copilot/gpt-5.3-codex\n\nLiteLLM: model group 'github_copilot/gpt-5.3-codex' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

Type

🐛 Bug Fix

Caveats (if any)

Low

Live PR risk (bcdcb8a)

Verdict: passes. No caller of the changed function loses a value it had before, and the one observable change is the one this PR exists for

Breaking: none. get_api_base(model, optional_params) keeps its signature and its str | None return, and declared_authenticating_provider is only read here, not changed

Backward incompatible: for a model declared as github_copilot/... or chatgpt/... with no api_base set, get_api_base now returns the provider's default endpoint (or the endpoint stored on disk for Copilot) where it used to run the device login and return None once that login timed out. With a stored login the value is the same as before. This is the requested fix and stays

Regression risk: the readers of this value are Slack alerting, the hanging-request check, cooldown callbacks, exception mapping, the x-litellm-model-api-base response header, and both streaming iterators. Each gets the provider default where it got None after a 3-minute hang, or the same string as before when a login is stored. Case 3 above drives the header path live on both sides; the other readers only display the string

Dependency graph: litellm.get_api_base is called from litellm/interactions/streaming_iterator.py, litellm/responses/streaming_iterator.py, litellm/integrations/SlackAlerting/slack_alerting.py, litellm/integrations/SlackAlerting/hanging_request_check.py, litellm/litellm_core_utils/exception_mapping_utils.py, litellm/litellm_core_utils/llm_response_utils/response_metadata.py, litellm/litellm_core_utils/streaming_handler.py, and litellm/router_utils/cooldown_callbacks.py. declared_authenticating_provider already gates the same login in litellm/utils.py, litellm/router.py, litellm/proxy/proxy_server.py, litellm/rerank_api/main.py, litellm/litellm_core_utils/get_supported_openai_params.py, litellm/router_utils/pattern_match_deployments.py, litellm/router_strategy/savings_baseline.py, and litellm/router_strategy/complexity_router/complexity_router.py, so the new branch mirrors a dispatch those paths already take. No config key, env var, DB column, or wire field changed, and GithubCopilotConfig().api_base_without_login() and ChatGPTConfig().api_base_without_login() already existed

Not verified: the chatgpt no-login SDK call (its device flow blocks up to 15 minutes; the unit test pins it), the alerting and cooldown readers driven live (they only display the string), and CircleCI's legacy suites: red only on tests main already fails (the #42435 reference pipeline 90098 plus main runs 90101, 90068, 90060, 90056, 90023, and 90021) and one spend-rollup poll that passed on its single rerun, recorded as a Low caveat

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

  • bcdcb8a passes /live-pr-risk

@devin-ai-integration

devin-ai-integration Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness or repository-rule issues

Findings

  1. P2 Provider Logic Outside Adapters ▶

Summary

This PR lets API-base reporting for GitHub Copilot and ChatGPT avoid authentication login flows while preserving adapter-owned endpoint selection

  • Detects declared authenticating providers before general provider resolution
  • Exposes login-free API-base lookup through the provider adapters
  • Preserves explicit and enterprise Copilot endpoints
  • Adds regression coverage for login-free resolution and fixed provider defaults
  • Updates rerank error-path coverage for the additional resolver reference

Reviews (3) · Last reviewed commit: "refactor(get_api_base): dispatch the pro..."

Comment on lines +13 to +21
def _api_base_without_login(provider: str) -> str | None:
match provider:
case "github_copilot":
return litellm.GithubCopilotConfig().api_base_without_login()
case "chatgpt":
return litellm.ChatGPTConfig().api_base_without_login()
case _:
return None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Provider Logic Outside Adapters

This shared utility directly branches on GitHub Copilot and ChatGPT and constructs their configurations. Repository rules require provider-specific logic to remain under litellm/llms/, so this requirement must be satisfied before merging

Rule Used: What: Avoid writing provider-specific code outside of the llms/ directory. Why: This practice ensures better maintainability and reduces complexity over time. Good: ```python # Handle provider-specific logic within llms/vertex_ai/transformation.py ... (source)

Knowledge Base Used: Provider adapters and capabilities

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provider knowledge lives in each adapter's api_base_without_login under llms/. This util only dispatches on the provider name, like the gemini and openai branch below

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right — this utility only dispatches to each adapter's api_base_without_login(); the provider-specific URL logic remains under litellm/llms/. That is consistent with the existing Gemini/OpenAI default-base dispatch in this file, so the comment does not apply. I'll retract it.

@codspeed

codspeed Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_get_api_base_skip_authenticating_provider_resolution (bcdcb8a) with main (392e807)

Open in CodSpeed

Comment thread litellm/litellm_core_utils/llm_response_utils/get_api_base.py Fixed
Comment thread litellm/litellm_core_utils/llm_response_utils/get_api_base.py Fixed
@codecov

codecov Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ellm_core_utils/llm_response_utils/get_api_base.py 94.44% 1 Missing ⚠️
litellm/llms/chatgpt/chat/transformation.py 66.66% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit bcdcb8a. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mateo-berri
mateo-berri merged commit ca95fc2 into main Sep 23, 2026
138 of 150 checks passed
@mateo-berri
mateo-berri deleted the litellm_get_api_base_skip_authenticating_provider_resolution branch September 23, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants