Repository navigation
feat(clinepass): add ClinePass (Cline API) as a provider #44379
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
djbclark
wants to merge
24
commits into
BerriAI:main
Choose a base branch
from
djbclark:clinepass-upstream
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
2553cbd
feat(clinepass): add ClinePass provider
djbclark e688210
fix(clinepass): correct the model namespace, harden truncation detection
djbclark 9a91da3
feat(clinepass): add Admin UI credential fields entry
djbclark 7d9ef15
test(clinepass): move the provider tests into the CI-selected tree
djbclark b9d159f
fix(clinepass): stop unsupported endpoints reaching the provider with…
djbclark bf18f22
fix(clinepass): address external review -- logging, unicode, dead cod…
djbclark 69ecca0
feat(clinepass): register the model in the price/context catalogue
djbclark edf3132
fix(clinepass): report the upstream finish reason instead of inferrin…
djbclark a6c9f01
test(clinepass): cover streaming, usage, auth headers and 429
djbclark 51214ed
fix(clinepass): address CI — mirrored cost map, logoless set, formatting
djbclark d33e84b
fix(clinepass): satisfy ruff — drop a redundant pass, narrow two raises
djbclark cb9aeab
fix(clinepass): type logging_obj and encoding instead of Any
djbclark 6d3a53f
fix(clinepass): register the provider with litellm's own guards
djbclark 0e756bf
test(clinepass): expect context-window and content-policy recognition
djbclark abadd88
fix(clinepass): never borrow the global litellm.api_key for chat
djbclark 08408fe
test(clinepass): state what the extra_body assertions actually pin
djbclark 304d7a2
fix(clinepass): justify the dict-typed overrides for the type-discipl…
djbclark ace7be6
fix(clinepass): reject realtime credential fallback and isolate provi…
djbclark 497dc8a
fix(clinepass): isolate moderation and responses websocket credentials
djbclark 0bc31c0
fix(clinepass): pin authenticated websocket models and type moderatio…
djbclark 886b145
Merge upstream/main into clinepass-upstream
djbclark ba51099
fix(clinepass): drop typing.Any from the runtime logging-object fallback
djbclark 6506040
fix(clinepass): satisfy the new Final, rebind and basedpyright ceilings
djbclark 838f7bd
Merge upstream/main into clinepass-upstream
djbclark File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,281 @@ | ||
| """ | ||
| Support for ClinePass (the Cline API) `/v1/chat/completions` endpoint. | ||
|
|
||
| ClinePass is OpenAI-compatible apart from two quirks, both handled here: | ||
|
|
||
| 1. Non-streaming completions are nested under a ``data`` envelope -- | ||
| ``{"data": {"choices": [...]}, "success": true}`` -- rather than returning | ||
| ``choices`` at the top level. Streaming responses are *not* wrapped, so the | ||
| inherited SSE handling needs no change. | ||
| 2. A bare model id is rejected with HTTP 400 ``invalid model format. Expected | ||
| format: modelType/model``, but LiteLLM strips its own ``clinepass/`` routing | ||
| prefix before the request is built, so a qualifier has to be restored. | ||
|
|
||
| Documentation: https://docs.cline.bot/ | ||
|
|
||
| Credentials come only from the request's api_key or CLINEPASS_API_KEY. | ||
| Moderation and realtime endpoints are unsupported and rejected before dispatch. | ||
| """ | ||
|
|
||
| import json | ||
| from collections.abc import Mapping | ||
| from typing import TYPE_CHECKING, Final, NoReturn | ||
|
|
||
| import httpx | ||
| from pydantic import TypeAdapter, ValidationError | ||
|
|
||
| from litellm.llms.base_llm.chat.transformation import BaseLLMException | ||
| from litellm.secret_managers.main import get_secret_str | ||
| from litellm.types.llms.openai import AllMessageValues | ||
| from litellm.types.utils import ModelResponse | ||
|
|
||
| from ...openai.chat.gpt_transformation import OpenAIGPTConfig | ||
| from ..common_utils import ClinePassException | ||
|
|
||
| # Needed only for annotations; importing litellm_logging at runtime from a | ||
| # provider module risks a circular import. | ||
| if TYPE_CHECKING: | ||
| from litellm.litellm_core_utils.litellm_logging import Logging | ||
| from litellm.litellm_core_utils.tokenizer import Encoding as Tokenizer | ||
|
|
||
| CLINEPASS_API_BASE: Final = "https://api.cline.bot/api/v1" | ||
|
|
||
| # ClinePass nests the completion under this key on non-streaming responses. | ||
| CLINEPASS_RESPONSE_ENVELOPE_KEY: Final = "data" | ||
|
|
||
| # The qualifier ClinePass expects on outbound model ids. | ||
| # | ||
| # Note the hyphen: the catalog namespace is ``cline-pass/``, not ``clinepass/`` | ||
| # (the latter is LiteLLM's own routing prefix, which is stripped before the | ||
| # request is built). The API only validates the *shape* of a model id -- any | ||
| # ``<segment>/<model>`` is accepted with HTTP 200 -- so an incorrect namespace | ||
| # fails silently rather than loudly. It is not inert, though: for at least one | ||
| # model an unrecognised namespace resolves to a different, date-pinned snapshot | ||
| # (``cline-pass/deepseek-v4-flash`` -> ``deepseek/deepseek-v4-flash``, while | ||
| # ``clinepass/deepseek-v4-flash`` -> ``deepseek/deepseek-v4-flash-0731``). | ||
| CLINEPASS_MODEL_PREFIX: Final = "cline-pass/" | ||
|
|
||
| # Headers that describe the original byte stream and would be wrong once the | ||
| # body is rewritten by _unwrap_response_envelope(). | ||
| _BODY_SPECIFIC_HEADERS: Final = ("content-length", "content-encoding") | ||
|
|
||
| _JSON_OBJECT: Final = TypeAdapter(Mapping[str, object]) | ||
|
|
||
|
|
||
| def _as_json_object(value: object) -> Mapping[str, object] | None: | ||
| try: | ||
| return _JSON_OBJECT.validate_python(value) | ||
| except ValidationError: | ||
| return None | ||
|
|
||
|
|
||
| def _unwrap_response_envelope(raw_response: httpx.Response) -> httpx.Response: | ||
| """Strip ClinePass's ``data`` wrapper off a JSON completion body. | ||
|
|
||
| The OpenAI transforms read ``raw_response.json()`` directly, so rebuild the | ||
| response around the inner object rather than duplicating their bodies here. | ||
|
|
||
| Returns the original response untouched whenever the body does not look like | ||
| a wrapped completion, so an already-OpenAI-shaped body -- or an error nested | ||
| under the same key -- is not mistaken for one. | ||
| """ | ||
| try: | ||
| payload: Final = _JSON_OBJECT.validate_json(raw_response.content) | ||
| except ValidationError: | ||
| # Not a JSON object body -- there is no envelope to strip. | ||
| return raw_response | ||
|
|
||
| if "choices" in payload: | ||
| return raw_response | ||
|
|
||
| inner: Final = _as_json_object(payload.get(CLINEPASS_RESPONSE_ENVELOPE_KEY)) | ||
| if inner is None or "choices" not in inner: | ||
| return raw_response | ||
|
|
||
| return httpx.Response( | ||
| status_code=raw_response.status_code, | ||
| headers={k: v for k, v in raw_response.headers.items() if k.lower() not in _BODY_SPECIFIC_HEADERS}, | ||
| content=json.dumps(inner, ensure_ascii=False).encode("utf-8"), | ||
| request=_attached_request(raw_response), | ||
| ) | ||
|
|
||
|
|
||
| def _attached_request(raw_response: httpx.Response) -> httpx.Request | None: | ||
| # httpx.Response.request raises RuntimeError rather than returning None when | ||
| # no request is attached, so ask for it defensively instead of reaching for | ||
| # the private attribute behind it. | ||
| try: | ||
| return raw_response.request | ||
| except RuntimeError: | ||
| return None | ||
|
|
||
|
|
||
| def _apply_model_prefix(data: dict[str, object]) -> dict[str, object]: # mutable-ok: dict-typed base transform_request | ||
| """Restore the ``modelType/model`` qualifier on the outbound model id. | ||
|
|
||
| Only prefix ids that lost their qualifier, so a cross-provider id | ||
| (``clinepass/openrouter/foo`` -> ``openrouter/foo``) is forwarded unchanged. | ||
| """ | ||
| model: Final = data.get("model") | ||
| if isinstance(model, str) and "/" not in model: | ||
| return {**data, "model": f"{CLINEPASS_MODEL_PREFIX}{model}"} | ||
| return data | ||
|
|
||
|
|
||
| class ClinePassConfig(OpenAIGPTConfig): | ||
| """ | ||
| ClinePass configuration, inheriting the OpenAI chat transforms. | ||
|
|
||
| Overrides only the request/response points where ClinePass diverges; see the | ||
| module docstring for the two quirks. | ||
| """ | ||
|
|
||
| @staticmethod | ||
| def get_realtime_http_config(model: str) -> NoReturn: | ||
| from litellm.exceptions import BadRequestError | ||
|
|
||
| raise BadRequestError( | ||
| message="ClinePass does not support realtime endpoints", | ||
| model=model, | ||
| llm_provider="clinepass", | ||
| ) | ||
|
|
||
| @staticmethod | ||
| def validate_moderation(model: str | None, custom_llm_provider: str | None = None) -> None: | ||
| if custom_llm_provider != "clinepass" and not (model or "").startswith("clinepass/"): | ||
| return | ||
| from litellm.exceptions import BadRequestError | ||
|
|
||
| raise BadRequestError( | ||
| message="ClinePass does not support moderation endpoints", | ||
| model=model or "", | ||
| llm_provider="clinepass", | ||
| ) | ||
|
|
||
| def _get_openai_compatible_provider_info( | ||
| self, api_base: str | None, api_key: str | None | ||
| ) -> tuple[str | None, str | None]: | ||
| return ( | ||
| api_base or get_secret_str("CLINEPASS_API_BASE") or CLINEPASS_API_BASE, | ||
| api_key or get_secret_str("CLINEPASS_API_KEY"), | ||
| ) | ||
|
|
||
| def get_openai_compatible_provider_info( | ||
| self, | ||
| api_base: str | None, | ||
| api_key: str | None, | ||
| ) -> tuple[str | None, str | None]: | ||
| return self._get_openai_compatible_provider_info(api_base, api_key) | ||
|
|
||
| def get_complete_url( | ||
| self, | ||
| api_base: str | None, | ||
| api_key: str | None, | ||
| model: str, | ||
| optional_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| litellm_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| stream: bool | None = None, | ||
| ) -> str: | ||
| base: Final = (api_base or CLINEPASS_API_BASE).rstrip("/") | ||
| if base.endswith("/chat/completions"): | ||
| return base | ||
|
|
||
| return f"{base}/chat/completions" | ||
|
|
||
| def get_models( | ||
| self, api_key: str | None = None, api_base: str | None = None | ||
| ) -> list[str]: # mutable-ok: matches the dict-typed base-class signature | ||
| """ClinePass exposes no model catalog. | ||
|
|
||
| ``GET https://api.cline.bot/api/v1/models`` returns HTTP 404, and the | ||
| inherited OpenAI implementation would additionally ask for it at the | ||
| wrong path -- it rewrites the base URL down to scheme+host and appends | ||
| ``/v1/models``. Return an empty catalog rather than making a request | ||
| that is known to fail. | ||
| """ | ||
| return [] | ||
|
|
||
| def map_openai_params( | ||
| self, | ||
| non_default_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| optional_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| model: str, | ||
| drop_params: bool, | ||
| ) -> dict[str, object]: # mutable-ok: matches the dict-typed base-class signature | ||
| """ClinePass takes the legacy ``max_tokens`` spelling only.""" | ||
| mapped_params: Final[dict[str, object]] # mutable-ok: dict-typed base map_openai_params | ||
| mapped_params = super().map_openai_params( # pyright: ignore[reportUnknownVariableType, reportUnknownMemberType] # base signature takes bare dict | ||
| non_default_params=non_default_params, | ||
| optional_params=optional_params, | ||
| model=model, | ||
| drop_params=drop_params, | ||
| ) | ||
| if "max_completion_tokens" in mapped_params: | ||
| mapped_params["max_tokens"] = mapped_params.pop("max_completion_tokens") | ||
| return mapped_params | ||
|
|
||
| def transform_request( | ||
| self, | ||
| model: str, | ||
| messages: list[AllMessageValues], # mutable-ok: matches the dict-typed base-class signature | ||
| optional_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| litellm_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| headers: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| ) -> dict[str, object]: # mutable-ok: matches the dict-typed base-class signature | ||
| # BaseLLMHTTPHandler builds the body with this synchronous method on | ||
| # both the sync and the async path, so there is deliberately no | ||
| # async_transform_request() override -- it would never be called. | ||
| data: Final[dict[str, object]] # mutable-ok: dict-typed base transform_request | ||
| data = super().transform_request( # pyright: ignore[reportUnknownVariableType, reportUnknownMemberType] # base signature takes bare dict | ||
| model=model, | ||
| messages=messages, | ||
| optional_params=optional_params, | ||
| litellm_params=litellm_params, | ||
| headers=headers, | ||
| ) | ||
| return _apply_model_prefix(data) | ||
|
|
||
| def transform_response( | ||
| self, | ||
| model: str, | ||
| raw_response: httpx.Response, | ||
| model_response: ModelResponse, | ||
| logging_obj: "Logging", | ||
| request_data: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| messages: list[AllMessageValues], # mutable-ok: matches the dict-typed base-class signature | ||
| optional_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| litellm_params: dict[str, object], # mutable-ok: matches the dict-typed base-class signature | ||
| encoding: "Tokenizer | None", | ||
| api_key: str | None = None, | ||
| json_mode: bool | None = None, | ||
| ) -> ModelResponse: | ||
| # ClinePass was once observed returning finish_reason "stop" on a completion | ||
| # cut off by max_tokens. Follow-up probes on 2026-08-22 did not reproduce it. | ||
| # The provider therefore reports the upstream finish reason unmodified: inferring | ||
| # truncation from usage equalling the cap produces false positives on natural | ||
| # completions that happen to land exactly on the cap. | ||
| return super().transform_response( # pyright: ignore[reportUnknownMemberType] # base signature takes bare dict | ||
| model=model, | ||
| raw_response=_unwrap_response_envelope(raw_response), | ||
| model_response=model_response, | ||
| logging_obj=logging_obj, | ||
| request_data=request_data, | ||
| messages=messages, | ||
| optional_params=optional_params, | ||
| litellm_params=litellm_params, | ||
| encoding=encoding, | ||
| api_key=api_key, | ||
| json_mode=json_mode, | ||
| ) | ||
|
|
||
| def get_error_class( | ||
| self, | ||
| error_message: str, | ||
| status_code: int, | ||
| headers: dict[str, object] | httpx.Headers, # mutable-ok: matches the dict-typed base-class signature | ||
| ) -> BaseLLMException: | ||
| return ClinePassException( | ||
| message=error_message, | ||
| status_code=status_code, | ||
| headers=headers, | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| from litellm.llms.base_llm.chat.transformation import BaseLLMException | ||
|
|
||
|
|
||
| class ClinePassException(BaseLLMException): | ||
| """ClinePass exception handling class""" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PR description says the linked ClinePass documentation page is only drafted for a separate repository. If this change merges before that page is published, readers following the new provider link will reach a missing page. Publish the companion page before exposing the link.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!