Repository navigation
Add ChatGPT subscription support and responses bridge #19030
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 4 commits
0116748
45fa567
efb481c
e810f07
11c02ae
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # ChatGPT Subscription | ||
|
|
||
| Use ChatGPT Pro/Max subscription models through LiteLLM with OAuth device flow authentication. | ||
|
|
||
| | Property | Details | | ||
| |-------|-------| | ||
| | Description | ChatGPT subscription access (Codex + GPT-5.2 family) via ChatGPT backend API | | ||
| | Provider Route on LiteLLM | `chatgpt/` | | ||
| | Supported Endpoints | `/responses`, `/chat/completions` (bridged to Responses for supported models) | | ||
| | API Reference | https://chatgpt.com | | ||
|
|
||
| ChatGPT subscription access is native to the Responses API. Chat Completions requests are bridged to Responses for supported models (for example `chatgpt/gpt-5.2`). | ||
|
|
||
| Notes: | ||
| - The ChatGPT subscription backend rejects token limit fields (`max_tokens`, `max_output_tokens`, `max_completion_tokens`) and `metadata`. LiteLLM strips these fields for this provider. | ||
| - `/v1/chat/completions` honors `stream`. When `stream` is false (default), LiteLLM aggregates the Responses stream into a single JSON response. | ||
|
|
||
| ## Authentication | ||
|
|
||
| ChatGPT subscription access uses an OAuth device code flow: | ||
|
|
||
| 1. LiteLLM prints a device code and verification URL | ||
| 2. Open the URL, sign in, and enter the code | ||
| 3. Tokens are stored locally for reuse | ||
|
|
||
| ## Usage - LiteLLM Python SDK | ||
|
|
||
| ### Responses (recommended for Codex models) | ||
|
|
||
| ```python showLineNumbers title="ChatGPT Responses" | ||
| import litellm | ||
|
|
||
| response = litellm.responses( | ||
| model="chatgpt/gpt-5.2-codex", | ||
| input="Write a Python hello world" | ||
| ) | ||
|
|
||
| print(response) | ||
| ``` | ||
|
|
||
| ### Chat Completions (bridged to Responses) | ||
|
|
||
| ```python showLineNumbers title="ChatGPT Chat Completions" | ||
| import litellm | ||
|
|
||
| response = litellm.completion( | ||
| model="chatgpt/gpt-5.2", | ||
| messages=[{"role": "user", "content": "Write a Python hello world"}] | ||
| ) | ||
|
|
||
| print(response) | ||
| ``` | ||
|
|
||
| ## Usage - LiteLLM Proxy | ||
|
|
||
| ```yaml showLineNumbers title="config.yaml" | ||
| model_list: | ||
| - model_name: chatgpt/gpt-5.2 | ||
| model_info: | ||
| mode: responses | ||
| litellm_params: | ||
| model: chatgpt/gpt-5.2 | ||
| - model_name: chatgpt/gpt-5.2-codex | ||
| model_info: | ||
| mode: responses | ||
| litellm_params: | ||
| model: chatgpt/gpt-5.2-codex | ||
| ``` | ||
|
|
||
| ```bash showLineNumbers title="Start LiteLLM Proxy" | ||
| litellm --config config.yaml | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
| ### Environment Variables | ||
|
|
||
| - `CHATGPT_TOKEN_DIR`: Custom token storage directory | ||
| - `CHATGPT_AUTH_FILE`: Auth file name (default: `auth.json`) | ||
| - `CHATGPT_API_BASE`: Override API base (default: `https://chatgpt.com/backend-api/codex`) | ||
| - `OPENAI_CHATGPT_API_BASE`: Alias for `CHATGPT_API_BASE` | ||
| - `CHATGPT_ORIGINATOR`: Override the `originator` header value | ||
| - `CHATGPT_USER_AGENT`: Override the `User-Agent` header value | ||
| - `CHATGPT_USER_AGENT_SUFFIX`: Optional suffix appended to the `User-Agent` header |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,12 +2,13 @@ | |
| Handler for transforming /chat/completions api requests to litellm.responses requests | ||
| """ | ||
|
|
||
| from typing import TYPE_CHECKING, Any, Coroutine, Union | ||
| from typing import TYPE_CHECKING, Any, Coroutine, Optional, Union | ||
|
|
||
| from typing_extensions import TypedDict | ||
|
|
||
| if TYPE_CHECKING: | ||
| from litellm import CustomStreamWrapper, LiteLLMLoggingObj, ModelResponse | ||
| from litellm.types.llms.openai import ResponsesAPIResponse | ||
|
|
||
|
|
||
| class ResponsesToCompletionBridgeHandlerInputKwargs(TypedDict): | ||
|
|
@@ -28,6 +29,77 @@ def __init__(self): | |
| super().__init__() | ||
| self.transformation_handler = LiteLLMResponsesTransformationHandler() | ||
|
|
||
| @staticmethod | ||
| def _resolve_stream_flag(optional_params: dict, litellm_params: dict) -> bool: | ||
| stream = optional_params.get("stream") | ||
| if stream is None: | ||
| stream = litellm_params.get("stream", False) | ||
| return bool(stream) | ||
|
|
||
| @staticmethod | ||
| def _coerce_response_object( | ||
| response_obj: Any, | ||
| hidden_params: Optional[dict], | ||
| ) -> "ResponsesAPIResponse": | ||
| from litellm.types.llms.openai import ResponsesAPIResponse | ||
|
|
||
| if isinstance(response_obj, ResponsesAPIResponse): | ||
| response = response_obj | ||
| elif isinstance(response_obj, dict): | ||
| try: | ||
| response = ResponsesAPIResponse(**response_obj) | ||
| except Exception: | ||
| response = ResponsesAPIResponse.model_construct(**response_obj) | ||
| else: | ||
| raise ValueError("Unexpected responses stream payload") | ||
|
|
||
| if hidden_params: | ||
| existing = getattr(response, "_hidden_params", None) | ||
| if not isinstance(existing, dict) or not existing: | ||
| setattr(response, "_hidden_params", dict(hidden_params)) | ||
| else: | ||
| for key, value in hidden_params.items(): | ||
| existing.setdefault(key, value) | ||
| return response | ||
|
|
||
| def _collect_response_from_stream( | ||
| self, stream_iter: Any | ||
| ) -> "ResponsesAPIResponse": | ||
| from litellm.types.llms.openai import ResponsesAPIResponse | ||
|
|
||
| for _ in stream_iter: | ||
| pass | ||
|
|
||
| completed = getattr(stream_iter, "completed_response", None) | ||
| response_obj = getattr(completed, "response", None) if completed else None | ||
| if response_obj is None: | ||
| raise ValueError("Stream ended without a completed response") | ||
|
|
||
| hidden_params = getattr(stream_iter, "_hidden_params", None) | ||
| response = self._coerce_response_object(response_obj, hidden_params) | ||
| if not isinstance(response, ResponsesAPIResponse): | ||
| raise ValueError("Stream completed response is invalid") | ||
| return response | ||
|
|
||
| async def _collect_response_from_stream_async( | ||
| self, stream_iter: Any | ||
| ) -> "ResponsesAPIResponse": | ||
| from litellm.types.llms.openai import ResponsesAPIResponse | ||
|
|
||
| async for _ in stream_iter: | ||
| pass | ||
|
|
||
| completed = getattr(stream_iter, "completed_response", None) | ||
| response_obj = getattr(completed, "response", None) if completed else None | ||
| if response_obj is None: | ||
| raise ValueError("Stream ended without a completed response") | ||
|
|
||
| hidden_params = getattr(stream_iter, "_hidden_params", None) | ||
| response = self._coerce_response_object(response_obj, hidden_params) | ||
| if not isinstance(response, ResponsesAPIResponse): | ||
| raise ValueError("Stream completed response is invalid") | ||
| return response | ||
|
|
||
| def validate_input_kwargs( | ||
| self, kwargs: dict | ||
| ) -> ResponsesToCompletionBridgeHandlerInputKwargs: | ||
|
|
@@ -113,6 +185,7 @@ def completion(self, *args, **kwargs) -> Union[ | |
| **request_data, | ||
| ) | ||
|
|
||
| stream = self._resolve_stream_flag(optional_params, litellm_params) | ||
| if isinstance(result, ResponsesAPIResponse): | ||
| return self.transformation_handler.transform_response( | ||
| model=model, | ||
|
|
@@ -127,6 +200,21 @@ def completion(self, *args, **kwargs) -> Union[ | |
| api_key=kwargs.get("api_key"), | ||
| json_mode=kwargs.get("json_mode"), | ||
| ) | ||
| elif not stream: | ||
| responses_api_response = self._collect_response_from_stream(result) | ||
| return self.transformation_handler.transform_response( | ||
| model=model, | ||
| raw_response=responses_api_response, | ||
| model_response=model_response, | ||
| logging_obj=logging_obj, | ||
| request_data=request_data, | ||
| messages=messages, | ||
| optional_params=optional_params, | ||
| litellm_params=litellm_params, | ||
| encoding=kwargs.get("encoding"), | ||
| api_key=kwargs.get("api_key"), | ||
| json_mode=kwargs.get("json_mode"), | ||
| ) | ||
| else: | ||
| completion_stream = self.transformation_handler.get_model_response_iterator( | ||
| streaming_response=result, # type: ignore | ||
|
|
@@ -175,6 +263,7 @@ async def acompletion( | |
| aresponses=True, | ||
| ) | ||
|
|
||
| stream = self._resolve_stream_flag(optional_params, litellm_params) | ||
| if isinstance(result, ResponsesAPIResponse): | ||
| return self.transformation_handler.transform_response( | ||
| model=model, | ||
|
|
@@ -189,6 +278,23 @@ async def acompletion( | |
| api_key=kwargs.get("api_key"), | ||
| json_mode=kwargs.get("json_mode"), | ||
| ) | ||
| elif not stream: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we need this?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the api only allows stream=true, so this is a component that buffers the streamed output and returns a regular json/http result from it, if the user requests stream=false.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But we already handle stream=false. for other providers. Can you take a look? I don't think this would be needed
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn’t find any other places where stream=False is handled by buffering a streaming iterator. The Responses API path only returns a streaming iterator when stream=True at the request layer, and otherwise does a non‑stream HTTP call that yields a ResponsesAPIResponse directly.
So the “buffer stream to JSON” pattern exists only in the ChatGPT Responses config, and I didn’t find other providers doing it.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ok. one final thing, can you do ruff, mypy and lint checks of all files in the PR? Because we are good to merge then
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @krrishdholakia Feel free to merge it in today's staging branch |
||
| responses_api_response = await self._collect_response_from_stream_async( | ||
| result | ||
| ) | ||
| return self.transformation_handler.transform_response( | ||
| model=model, | ||
| raw_response=responses_api_response, | ||
| model_response=model_response, | ||
| logging_obj=logging_obj, | ||
| request_data=request_data, | ||
| messages=messages, | ||
| optional_params=optional_params, | ||
| litellm_params=litellm_params, | ||
| encoding=kwargs.get("encoding"), | ||
| api_key=kwargs.get("api_key"), | ||
| json_mode=kwargs.get("json_mode"), | ||
| ) | ||
| else: | ||
| completion_stream = self.transformation_handler.get_model_response_iterator( | ||
| streaming_response=result, # type: ignore | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.