feat(openai_like): add Responses API support to JSON providers

[Chesars]

Relevant issues

Enables JSON-declared providers to support /v1/responses without Python code. Simplifies Perplexity responses config (410 → 40 lines).

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🆕 New Feature
🧹 Refactoring

Changes

Responses API infrastructure for JSON providers

• Add supported_endpoints field to SimpleProviderConfig (default: [])
• Add supports_responses_api() to JSONProviderRegistry
• Create OpenAILikeResponsesConfig base class in litellm/llms/openai_like/responses/
• Add create_responses_config_class() for dynamic config generation from JSON
• ProviderConfigManager.get_provider_responses_api_config() now accepts Union[LlmProviders, str] and checks JSON providers first

Perplexity (410 → 40 lines)

• Move cost dict→float parsing to generic validators (ResponseAPIUsage.parse_cost + Usage.__init__)
• Simplify PerplexityResponsesConfig: only preset/ model handling remains, everything else inherited from OpenAIResponsesAPIConfig

How to use

Add supported_endpoints to any JSON provider in providers.json:

{
  "your_provider": {
    "base_url": "https://api.provider.com/v1",
    "api_key_env": "PROVIDER_API_KEY",
    "supported_endpoints": ["/v1/chat/completions", "/v1/responses"]
  }
}

Tests

• 19 unit tests covering: supported_endpoints defaults, supports_responses_api(), dynamic class generation (URL, headers, inheritance), ProviderConfigManager integration

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 11, 2026 3:14am

[greptile-apps]

Greptile Summary

This PR introduces Responses API (/v1/responses) support for JSON-declared OpenAI-compatible providers, enabling zero-code configuration via a supported_endpoints field in providers.json. It also simplifies PerplexityResponsesConfig from ~410 to ~40 lines by inheriting from OpenAIResponsesAPIConfig, with restored status error checking and proper message type handling for Perplexity's list-input requirements.

Key changes:

• SimpleProviderConfig gains supported_endpoints: list (default [])
• JSONProviderRegistry.supports_responses_api() checks for "/v1/responses" in that list
• create_responses_config_class() dynamically generates a JSONProviderResponsesConfig class (cached per slug)
• ProviderConfigManager.get_provider_responses_api_config() now accepts Union[LlmProviders, str] and checks Python classes first, then falls back to JSON providers
• ResponseAPIUsage.parse_cost field validator added to handle providers (like Perplexity) that return cost as a dict instead of a float
• All six responses/main.py call sites updated to pass raw provider string directly, enabling JSON-only providers

Issue found:

• The dynamic get_complete_url appends only /responses to base_url, meaning providers must have base_url ending in /v1 for the generated URL to be correct — this convention is documented but not enforced in code, creating risk of silent configuration errors.

Confidence Score: 4/5

• The PR introduces well-tested Responses API support for JSON providers with one actionable issue about URL validation that should be addressed before merge.
• The core infrastructure is sound with comprehensive test coverage. The architecture correctly prioritizes Python-class providers over JSON configs, and the simplified Perplexity config properly delegates to the base class while adding necessary overrides. One verified issue remains: the URL construction for the Responses API endpoint assumes base_url ends with /v1 but doesn't enforce this at runtime, creating risk of silent configuration errors. This should be addressed with validation. The docstring concerning provider-specific references was already corrected, and the edge case about pydantic model inputs is too theoretical to warrant blocking. Overall quality is good with proper test coverage for the happy path.
• litellm/llms/openai_like/dynamic_config.py — needs URL validation to prevent silent configuration errors

_{Last reviewed commit: da76e17}

[greptile-apps]

_{13 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[Chesars]

@greptile, could u re-review please?

[greptile-apps]

Greptile Summary

This PR adds Responses API (/v1/responses) support to the JSON provider system, allowing OpenAI-compatible providers declared in providers.json to support the Responses API without writing Python code. It also simplifies the Perplexity responses config from ~410 to ~40 lines by inheriting from OpenAIResponsesAPIConfig.

• Adds supported_endpoints field to SimpleProviderConfig and supports_responses_api() to JSONProviderRegistry for declaring endpoint support in JSON
• Creates OpenAILikeResponsesConfig base class and create_responses_config_class() for dynamic config generation, with class-level caching
• Refactors ProviderConfigManager.get_provider_responses_api_config() to accept Union[LlmProviders, str] and check JSON providers as fallback after Python classes
• Moves Perplexity's dict-to-float cost parsing into generic validators (ResponseAPIUsage.parse_cost and Usage.__init__), benefiting any future provider with similar behavior
• Simplifies PerplexityResponsesConfig to only retain preset model handling and restricted param list, delegating everything else to the OpenAI base
• Includes 19 mock-only unit tests covering the full feature surface

Confidence Score: 4/5

• This PR is safe to merge with one minor issue in the base class URL handling that should be addressed.
• The architecture is well-designed with proper separation of concerns, class caching, and Python-class-priority over JSON fallback. The Perplexity simplification is a good refactor that moves provider-specific cost parsing into generic validators. One logic issue exists: the base class OpenAILikeResponsesConfig.get_complete_url silently produces an invalid URL when no base URL is configured, though this is mitigated by the dynamic config's override that raises a ValueError. Tests are comprehensive and mock-only.
• litellm/llms/openai_like/responses/transformation.py - get_complete_url can produce invalid URL '/responses' when api_base is not configured

Important Files Changed

Filename	Overview
litellm/llms/openai_like/dynamic_config.py	Adds create_responses_config_class() with proper caching (addresses the prior review concern). Minor: sends empty bearer token when no API key is configured.
litellm/llms/openai_like/json_loader.py	Adds supported_endpoints field and supports_responses_api() registry method. Clean, minimal additions with sensible defaults.
litellm/llms/openai_like/responses/transformation.py	New base class for OpenAI-like responses. get_complete_url falls back to empty string when no base URL is configured, producing an invalid /responses URL instead of raising an error.
litellm/llms/perplexity/responses/transformation.py	Significantly simplified from ~410 to ~40 lines. Custom tool transformation, response parsing, and streaming logic removed in favor of OpenAI base class inheritance. Only preset model handling and supported params remain.
litellm/responses/main.py	Removes LlmProviders() enum wrapping from custom_llm_provider before passing to get_provider_responses_api_config, allowing string-based JSON providers to be resolved. Clean, minimal change across 6 call sites.
litellm/utils.py	Refactors get_provider_responses_api_config to accept Union[LlmProviders, str], check Python classes first (priority), then fall back to JSON providers. Extracts existing logic into _get_python_responses_api_config. Well-structured with clear priority ordering.
litellm/types/llms/openai.py	Adds parse_cost field validator to ResponseAPIUsage to handle dict-format cost (e.g. Perplexity's {total_cost: 0.01}). Clean Pydantic validator approach.
litellm/types/utils.py	Adds dict-to-float cost parsing in Usage.__init__. Handles the case where dict lacks total_cost key by deleting the attribute (consistent with the else branch).
tests/test_litellm/llms/openai_like/responses/test_openai_like_responses.py	19 unit tests covering config defaults, registry methods, dynamic class generation, URL building, header validation, inheritance, and ProviderConfigManager integration. All mock-based, no network calls.

Flowchart

flowchart TD
    A["litellm.responses(model='provider/model')"] --> B["responses/main.py\nResolve custom_llm_provider"]
    B --> C["ProviderConfigManager\n.get_provider_responses_api_config()"]
    C --> D{"Is provider a\nknown LlmProviders enum?"}
    D -->|Yes| E["_get_python_responses_api_config()\nCheck Python class registry"]
    D -->|No| F["JSONProviderRegistry\n.exists() + .supports_responses_api()"]
    E -->|Found| G["Return Python config\ne.g. PerplexityResponsesConfig"]
    E -->|Not found| F
    F -->|Supported| H["create_responses_config_class()\nGenerate from JSON + cache"]
    H --> I["Return JSONProviderResponsesConfig\ninherits OpenAILikeResponsesConfig"]
    F -->|Not supported| J["Return None\nFallback to chat-based handler"]
    G --> K["Execute Responses API request"]
    I --> K

Loading

_{Last reviewed commit: df40a19}

[greptile-apps]

_{13 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[Chesars]

@greptile preset removal is a false positive — it was never a kwarg. It's handled via model name prefix: perplexity/preset/pro-search → transform_responses_api_request sends {"preset": "pro-search"}. No breaking change.

[greptile-apps]

Got it - thanks for the clarification. If preset is parsed from the model name (e.g., perplexity/preset/pro-search) and never exposed as a kwarg, then there's no breaking change. That's consistent with how LiteLLM handles provider-specific routing through model name prefixes.

[Chesars]

@greptile. preset and models do not work in main as explicit parameters — they were filtered out by ResponsesAPIOptionalRequestParams TypedDict in get_requested_response_api_optional_param. The refactoring doesn't introduce a regression. Verified by running the filtering logic with main.

[greptile-apps]

The generated URL appends only /responses to api_base. This silently produces an incorrect endpoint for any provider whose base_url does NOT already end with /v1 (e.g., "https://api.provider.com"). The documentation examples show base_url ending in /v1 (e.g., "https://api.yourprovider.com/v1"), but nothing in the code validates or enforces this convention.

If a provider sets base_url: "https://api.provider.com" (without /v1) and supported_endpoints: ["/v1/responses"], the generated URL becomes https://api.provider.com/responses — which fails at runtime with a confusing 404.

Consider adding a validation check to enforce the convention and provide a clear error message:

Suggested change

	def get_complete_url(
	self,
	api_base: Optional[str],
	litellm_params: dict,
	) -> str:
	if not api_base:
	if provider.api_base_env:
	api_base = get_secret_str(provider.api_base_env)
	if not api_base:
	api_base = provider.base_url
	if api_base is None:
	raise ValueError(
	f"api_base is required for provider {provider.slug}"
	)
	api_base = api_base.rstrip("/")
	return f"{api_base}/responses"
	if api_base is None:
	raise ValueError(
	f"api_base is required for provider {provider.slug}"
	)
	api_base = api_base.rstrip("/")
	# Validate that base_url ends with /v1 for responses endpoint
	if not api_base.endswith("/v1"):
	raise ValueError(
	f"base_url for provider '{provider.slug}' must end with '/v1' to use the Responses API "
	f"(got '{api_base}'). Example: 'https://api.yourprovider.com/v1'"
	)
	return f"{api_base}/responses"