Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/my-website/docs/providers/chatgpt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# ChatGPT Subscription

Use ChatGPT Pro/Max subscription models through LiteLLM with OAuth device flow authentication.

| Property | Details |
|-------|-------|
| Description | ChatGPT subscription access (Codex + GPT-5.2 family) via ChatGPT backend API |
| Provider Route on LiteLLM | `chatgpt/` |
| Supported Endpoints | `/responses`, `/chat/completions` (bridged to Responses for supported models) |
| API Reference | https://chatgpt.com |

ChatGPT subscription access is native to the Responses API. Chat Completions requests are bridged to Responses for supported models (for example `chatgpt/gpt-5.2`).

Notes:
- The ChatGPT subscription backend rejects token limit fields (`max_tokens`, `max_output_tokens`, `max_completion_tokens`) and `metadata`. LiteLLM strips these fields for this provider.
- `/v1/chat/completions` honors `stream`. When `stream` is false (default), LiteLLM aggregates the Responses stream into a single JSON response.

## Authentication

ChatGPT subscription access uses an OAuth device code flow:

1. LiteLLM prints a device code and verification URL
2. Open the URL, sign in, and enter the code
3. Tokens are stored locally for reuse

## Usage - LiteLLM Python SDK

### Responses (recommended for Codex models)

```python showLineNumbers title="ChatGPT Responses"
import litellm

response = litellm.responses(
model="chatgpt/gpt-5.2-codex",
input="Write a Python hello world"
)

print(response)
```

### Chat Completions (bridged to Responses)

```python showLineNumbers title="ChatGPT Chat Completions"
import litellm

response = litellm.completion(
model="chatgpt/gpt-5.2",
messages=[{"role": "user", "content": "Write a Python hello world"}]
)

print(response)
```

## Usage - LiteLLM Proxy

```yaml showLineNumbers title="config.yaml"
model_list:
- model_name: chatgpt/gpt-5.2
model_info:
mode: responses
litellm_params:
model: chatgpt/gpt-5.2
- model_name: chatgpt/gpt-5.2-codex
model_info:
mode: responses
litellm_params:
model: chatgpt/gpt-5.2-codex
```

```bash showLineNumbers title="Start LiteLLM Proxy"
litellm --config config.yaml
```

## Configuration

### Environment Variables

- `CHATGPT_TOKEN_DIR`: Custom token storage directory
- `CHATGPT_AUTH_FILE`: Auth file name (default: `auth.json`)
- `CHATGPT_API_BASE`: Override API base (default: `https://chatgpt.com/backend-api/codex`)
- `OPENAI_CHATGPT_API_BASE`: Alias for `CHATGPT_API_BASE`
- `CHATGPT_ORIGINATOR`: Override the `originator` header value
- `CHATGPT_USER_AGENT`: Override the `User-Agent` header value
- `CHATGPT_USER_AGENT_SUFFIX`: Optional suffix appended to the `User-Agent` header
1 change: 1 addition & 0 deletions docs/my-website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -691,6 +691,7 @@ const sidebars = {
"providers/galadriel",
"providers/github",
"providers/github_copilot",
"providers/chatgpt",
"providers/gradient_ai",
"providers/groq",
"providers/helicone",
Expand Down
6 changes: 6 additions & 0 deletions litellm/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -557,6 +557,7 @@ def identify(event_details):
amazon_nova_models: Set = set()
stability_models: Set = set()
github_copilot_models: Set = set()
chatgpt_models: Set = set()
minimax_models: Set = set()
aws_polly_models: Set = set()
gigachat_models: Set = set()
Expand Down Expand Up @@ -812,6 +813,8 @@ def add_known_models():
stability_models.add(key)
elif value.get("litellm_provider") == "github_copilot":
github_copilot_models.add(key)
elif value.get("litellm_provider") == "chatgpt":
chatgpt_models.add(key)
elif value.get("litellm_provider") == "minimax":
minimax_models.add(key)
elif value.get("litellm_provider") == "aws_polly":
Expand Down Expand Up @@ -1025,6 +1028,7 @@ def add_known_models():
"amazon_nova": amazon_nova_models,
"stability": stability_models,
"github_copilot": github_copilot_models,
"chatgpt": chatgpt_models,
"minimax": minimax_models,
"aws_polly": aws_polly_models,
"gigachat": gigachat_models,
Expand Down Expand Up @@ -1458,6 +1462,8 @@ def set_global_gitlab_config(config: Dict[str, Any]) -> None:
from .llms.github_copilot.chat.transformation import GithubCopilotConfig as GithubCopilotConfig
from .llms.github_copilot.responses.transformation import GithubCopilotResponsesAPIConfig as GithubCopilotResponsesAPIConfig
from .llms.github_copilot.embedding.transformation import GithubCopilotEmbeddingConfig as GithubCopilotEmbeddingConfig
from .llms.chatgpt.chat.transformation import ChatGPTConfig as ChatGPTConfig
from .llms.chatgpt.responses.transformation import ChatGPTResponsesAPIConfig as ChatGPTResponsesAPIConfig
from .llms.gigachat.chat.transformation import GigaChatConfig as GigaChatConfig
from .llms.gigachat.embedding.transformation import GigaChatEmbeddingConfig as GigaChatEmbeddingConfig
from .llms.nebius.chat.transformation import NebiusConfig as NebiusConfig
Expand Down
4 changes: 4 additions & 0 deletions litellm/_lazy_imports_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,8 @@
"IBMWatsonXAudioTranscriptionConfig",
"GithubCopilotConfig",
"GithubCopilotResponsesAPIConfig",
"ChatGPTConfig",
"ChatGPTResponsesAPIConfig",
"ManusResponsesAPIConfig",
"GithubCopilotEmbeddingConfig",
"NebiusConfig",
Expand Down Expand Up @@ -648,6 +650,8 @@
"GithubCopilotConfig": (".llms.github_copilot.chat.transformation", "GithubCopilotConfig"),
"GithubCopilotResponsesAPIConfig": (".llms.github_copilot.responses.transformation", "GithubCopilotResponsesAPIConfig"),
"GithubCopilotEmbeddingConfig": (".llms.github_copilot.embedding.transformation", "GithubCopilotEmbeddingConfig"),
"ChatGPTConfig": (".llms.chatgpt.chat.transformation", "ChatGPTConfig"),
"ChatGPTResponsesAPIConfig": (".llms.chatgpt.responses.transformation", "ChatGPTResponsesAPIConfig"),
"NebiusConfig": (".llms.nebius.chat.transformation", "NebiusConfig"),
"WandbConfig": (".llms.wandb.chat.transformation", "WandbConfig"),
"GigaChatConfig": (".llms.gigachat.chat.transformation", "GigaChatConfig"),
Expand Down
108 changes: 107 additions & 1 deletion litellm/completion_extras/litellm_responses_transformation/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@
Handler for transforming /chat/completions api requests to litellm.responses requests
"""

from typing import TYPE_CHECKING, Any, Coroutine, Union
from typing import TYPE_CHECKING, Any, Coroutine, Optional, Union

from typing_extensions import TypedDict

if TYPE_CHECKING:
from litellm import CustomStreamWrapper, LiteLLMLoggingObj, ModelResponse
from litellm.types.llms.openai import ResponsesAPIResponse


class ResponsesToCompletionBridgeHandlerInputKwargs(TypedDict):
Expand All @@ -28,6 +29,77 @@ def __init__(self):
super().__init__()
self.transformation_handler = LiteLLMResponsesTransformationHandler()

@staticmethod
def _resolve_stream_flag(optional_params: dict, litellm_params: dict) -> bool:
stream = optional_params.get("stream")
if stream is None:
stream = litellm_params.get("stream", False)
return bool(stream)

@staticmethod
def _coerce_response_object(
response_obj: Any,
hidden_params: Optional[dict],
) -> "ResponsesAPIResponse":
from litellm.types.llms.openai import ResponsesAPIResponse

if isinstance(response_obj, ResponsesAPIResponse):
response = response_obj
elif isinstance(response_obj, dict):
try:
response = ResponsesAPIResponse(**response_obj)
except Exception:
response = ResponsesAPIResponse.model_construct(**response_obj)
else:
raise ValueError("Unexpected responses stream payload")

if hidden_params:
existing = getattr(response, "_hidden_params", None)
if not isinstance(existing, dict) or not existing:
setattr(response, "_hidden_params", dict(hidden_params))
else:
for key, value in hidden_params.items():
existing.setdefault(key, value)
return response

def _collect_response_from_stream(
self, stream_iter: Any
) -> "ResponsesAPIResponse":
from litellm.types.llms.openai import ResponsesAPIResponse

for _ in stream_iter:
pass

completed = getattr(stream_iter, "completed_response", None)
response_obj = getattr(completed, "response", None) if completed else None
if response_obj is None:
raise ValueError("Stream ended without a completed response")

hidden_params = getattr(stream_iter, "_hidden_params", None)
response = self._coerce_response_object(response_obj, hidden_params)
if not isinstance(response, ResponsesAPIResponse):
raise ValueError("Stream completed response is invalid")
return response

async def _collect_response_from_stream_async(
self, stream_iter: Any
) -> "ResponsesAPIResponse":
from litellm.types.llms.openai import ResponsesAPIResponse
Comment thread
msexxeta marked this conversation as resolved.
Resolved

async for _ in stream_iter:
pass

completed = getattr(stream_iter, "completed_response", None)
response_obj = getattr(completed, "response", None) if completed else None
if response_obj is None:
raise ValueError("Stream ended without a completed response")

hidden_params = getattr(stream_iter, "_hidden_params", None)
response = self._coerce_response_object(response_obj, hidden_params)
if not isinstance(response, ResponsesAPIResponse):
raise ValueError("Stream completed response is invalid")
return response

def validate_input_kwargs(
self, kwargs: dict
) -> ResponsesToCompletionBridgeHandlerInputKwargs:
Expand Down Expand Up @@ -113,6 +185,7 @@ def completion(self, *args, **kwargs) -> Union[
**request_data,
)

stream = self._resolve_stream_flag(optional_params, litellm_params)
if isinstance(result, ResponsesAPIResponse):
return self.transformation_handler.transform_response(
model=model,
Expand All @@ -127,6 +200,21 @@ def completion(self, *args, **kwargs) -> Union[
api_key=kwargs.get("api_key"),
json_mode=kwargs.get("json_mode"),
)
elif not stream:
responses_api_response = self._collect_response_from_stream(result)
return self.transformation_handler.transform_response(
model=model,
raw_response=responses_api_response,
model_response=model_response,
logging_obj=logging_obj,
request_data=request_data,
messages=messages,
optional_params=optional_params,
litellm_params=litellm_params,
encoding=kwargs.get("encoding"),
api_key=kwargs.get("api_key"),
json_mode=kwargs.get("json_mode"),
)
else:
completion_stream = self.transformation_handler.get_model_response_iterator(
streaming_response=result, # type: ignore
Expand Down Expand Up @@ -175,6 +263,7 @@ async def acompletion(
aresponses=True,
)

stream = self._resolve_stream_flag(optional_params, litellm_params)
if isinstance(result, ResponsesAPIResponse):
return self.transformation_handler.transform_response(
model=model,
Expand All @@ -189,6 +278,23 @@ async def acompletion(
api_key=kwargs.get("api_key"),
json_mode=kwargs.get("json_mode"),
)
elif not stream:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the api only allows stream=true, so this is a component that buffers the streamed output and returns a regular json/http result from it, if the user requests stream=false.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we already handle stream=false. for other providers. Can you take a look? I don't think this would be needed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn’t find any other places where stream=False is handled by buffering a streaming iterator. The Responses API path only returns a streaming iterator when stream=True at the request layer, and otherwise does a non‑stream HTTP call that yields a ResponsesAPIResponse directly.

  • The core Responses handler only instantiates streaming iterators when stream=True.
  • Other Responses configs (OpenAI, Manus) just parse JSON and don’t consume SSE.
  • MockResponsesAPIStreamingIterator goes the opposite direction (non‑stream → stream).

So the “buffer stream to JSON” pattern exists only in the ChatGPT Responses config, and I didn’t find other providers doing it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. one final thing, can you do ruff, mypy and lint checks of all files in the PR? Because we are good to merge then

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Ruff: 0/10 errors on changed lines; all 10 are F811s in litellm/utils.py but outside PR hunks at litellm/
    utils.py:1188, litellm/utils.py:1347, litellm/utils.py:1917, litellm/utils.py:2978, litellm/utils.py:3784, litellm/
    utils.py:4956, litellm/utils.py:5600, litellm/utils.py:7622, litellm/utils.py:8340, litellm/utils.py:8481.

  • MyPy: 0/20 errors on changed lines; all are outside PR hunks in untouched files: litellm/passthrough/utils.py:21,
    litellm/caching/redis_cache.py:556, litellm/caching/redis_cache.py:556, litellm/caching/redis_cache.py:577, litellm/
    caching/redis_cache.py:1170, litellm/caching/redis_cache.py:1173, litellm/caching/redis_cache.py:1175, litellm/
    caching/redis_cache.py:1324, litellm/caching/redis_cache.py:1325, litellm/integrations/custom_guardrail.py:553,
    litellm/vector_stores/main.py:196, litellm/llms/anthropic/files/handler.py:94, litellm/responses/
    litellm_completion_transformation/handler.py:25, litellm/responses/litellm_completion_transformation/handler.py:94,
    litellm/llms/custom_httpx/llm_http_handler.py:7058, litellm/main.py:5287, litellm/litellm_core_utils/
    token_counter.py:709, litellm/proxy/openai_files_endpoints/storage_backend_service.py:237, litellm/proxy/
    openai_files_endpoints/storage_backend_service.py:237, litellm/proxy/agent_endpoints/a2a_endpoints.py:263.

    lint ran in the pipeline (timed out on the most recent run but was good before)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@krrishdholakia Feel free to merge it in today's staging branch

responses_api_response = await self._collect_response_from_stream_async(
result
)
return self.transformation_handler.transform_response(
model=model,
raw_response=responses_api_response,
model_response=model_response,
logging_obj=logging_obj,
request_data=request_data,
messages=messages,
optional_params=optional_params,
litellm_params=litellm_params,
encoding=kwargs.get("encoding"),
api_key=kwargs.get("api_key"),
json_mode=kwargs.get("json_mode"),
)
else:
completion_stream = self.transformation_handler.get_model_response_iterator(
streaming_response=result, # type: ignore
Expand Down
2 changes: 2 additions & 0 deletions litellm/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,7 @@
"galadriel",
"gradient_ai",
"github_copilot", # GitHub Copilot Chat API
"chatgpt", # ChatGPT subscription API
"novita",
"meta_llama",
"featherless_ai",
Expand Down Expand Up @@ -603,6 +604,7 @@
"lm_studio",
"galadriel",
"github_copilot", # GitHub Copilot Chat API
"chatgpt", # ChatGPT subscription API
"novita",
"meta_llama",
"publicai", # PublicAI - JSON-configured provider
Expand Down
8 changes: 8 additions & 0 deletions litellm/litellm_core_utils/get_llm_provider_logic.py
Original file line number Diff line number Diff line change
Expand Up @@ -768,6 +768,14 @@ def _get_openai_compatible_provider_info( # noqa: PLR0915
) = litellm.GithubCopilotConfig()._get_openai_compatible_provider_info(
model, api_base, api_key, custom_llm_provider
)
elif custom_llm_provider == "chatgpt":
(
api_base,
dynamic_api_key,
custom_llm_provider,
) = litellm.ChatGPTConfig()._get_openai_compatible_provider_info(
model, api_base, api_key, custom_llm_provider
)
elif custom_llm_provider == "novita":
api_base = (
api_base
Expand Down
Loading
Loading