feat(neosantara): add Neosantara integration as OpenAI-compatible pro… - #20641
feat(neosantara): add Neosantara integration as OpenAI-compatible pro…#20641ErRickow wants to merge 8 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile SummaryThis PR integrates Neosantara as an OpenAI-compatible provider across the LiteLLM codebase — registering it in the provider JSON, enum, constants, model pricing, documentation, and tests.
Confidence Score: 1/5
|
| Filename | Overview |
|---|---|
| litellm/main.py | Adds overly broad catch-all conditions to the embedding function that will break existing providers (hosted_vllm, llamafile, lm_studio, vercel_ai_gateway) by intercepting them before their dedicated handlers. |
| litellm/litellm_core_utils/get_llm_provider_logic.py | Adds hardcoded Neosantara auto-detection before the existing openai_compatible_endpoints loop, duplicating boilerplate instead of following the established pattern. |
| litellm/constants.py | Correctly adds neosantara to LITELLM_CHAT_PROVIDERS and openai_compatible_providers, but questionably adds it to LITELLM_EMBEDDING_PROVIDERS_SUPPORTING_INPUT_ARRAY_OF_TOKENS without evidence of support. |
| litellm/init.py | Adds model sets and registration logic for neosantara following existing patterns correctly. |
| litellm/types/utils.py | Adds NEOSANTARA to LlmProviders enum — straightforward, no issues. |
| litellm/llms/openai_like/providers.json | Registers neosantara with correct base URL and API key env var. Removes trailing newline (minor). |
| model_prices_and_context_window.json | Adds ~35 neosantara model entries with pricing/context info. Has unrelated formatting changes to vertex_ai entries. |
| provider_endpoints_support.json | Claims support for nearly every endpoint type including assistants, fine_tuning, realtime, vector_stores — highly inflated compared to similar providers. |
| tests/test_litellm/test_neosantara.py | Tests cover provider detection, completion, embedding, and bridge APIs. Uses mock patching at openai SDK level. Missing trailing newline. |
| docs/my-website/docs/providers/neosantara.md | Documentation follows the standard provider page format with usage examples and model listing. |
| docs/my-website/sidebars.js | Adds neosantara to sidebar in alphabetical order — no issues. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["embedding() called"] --> B{"custom_llm_provider == 'openai' OR\n'together_ai' OR 'nvidia_nim' OR\n'litellm_proxy' OR\n**in openai_compatible_providers** OR\n**JSONProviderRegistry.exists()**?"}
B -- "Yes (NEW catch-all)" --> C["Generic OpenAI embedding handler\napi_base defaults to api.openai.com"]
B -- "No" --> D{"custom_llm_provider == 'hosted_vllm'?"}
D -- "Yes" --> E["hosted_vllm handler\nHOSTED_VLLM_API_BASE"]
D -- "No" --> F{"custom_llm_provider == 'llamafile'\nOR 'lm_studio'?"}
F -- "Yes" --> G["openai_like handler\nOPENAI_LIKE_API_BASE"]
F -- "No" --> H{"custom_llm_provider ==\n'vercel_ai_gateway'?"}
H -- "Yes" --> I["vercel handler\nVERCEL_AI_GATEWAY_API_BASE"]
style B fill:#ff6b6b,color:#000
style C fill:#ff6b6b,color:#000
style E fill:#cccccc,color:#000
style G fill:#cccccc,color:#000
style I fill:#cccccc,color:#000
Last reviewed commit: 583f645
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| "neosantara/nusantara-base": { | ||
| "max_input_tokens": 64000, | ||
| "max_output_tokens": 2048, | ||
| "input_cost_per_token": 0.00000001875, |
There was a problem hiding this comment.
Duplicate model cost keys
This commit introduces duplicated Neosantara entries in model_prices_and_context_window.json (e.g., neosantara/nusantara-base, neosantara/archipelago-70b, etc. appear multiple times). In JSON, duplicate keys are undefined/implementation-dependent; whichever occurrence is last wins, which can silently change max token limits/costs and provider detection via add_known_models().
Fix: de-duplicate so each model key appears exactly once, and validate that the resulting map is deterministic.
| "neosantara": { | ||
| "display_name": "Neosantara (`neosantara`)", | ||
| "url": "https://docs.litellm.ai/docs/providers/neosantara", | ||
| "endpoints": { | ||
| "chat_completions": true, | ||
| "messages": true, | ||
| "responses": true, | ||
| "embeddings": true |
There was a problem hiding this comment.
Missing endpoint flags keys
The new neosantara entry’s endpoints object only includes a subset of keys (chat_completions/messages/responses/embeddings), while neighboring providers include the full flag set (e.g., image_generations, audio_transcriptions, moderations, batches, rerank, a2a, interactions, etc.). Any code that assumes these keys exist (rather than using .get(..., False)) will raise KeyError/fail when iterating providers.
Fix: include the full set of endpoint boolean keys with explicit true/false values, matching the schema used for other providers.
| def test_neosantara_responses_api_bridge(): | ||
| """ | ||
| Test that Neosantara works with litellm.responses() API bridge. | ||
| """ | ||
| os.environ["NEOSANTARA_API_KEY"] = "sk-1234" | ||
|
|
||
| with pytest.MonkeyPatch().context() as m: | ||
| def mock_send(self, request, **kwargs): | ||
| return httpx.Response( | ||
| 200, | ||
| content='{"choices": [{"message": {"content": "Hello from responses API"}, "finish_reason": "stop", "index": 0}], "usage": {"prompt_tokens": 10, "completion_tokens": 10, "total_tokens": 20}, "object": "chat.completion", "id": "resp-123", "created": 123456789}'.encode(), | ||
| request=request | ||
| ) | ||
|
|
||
| m.setattr("httpx.Client.send", mock_send) | ||
|
|
||
| response = litellm.responses( | ||
| model="neosantara/claude-3-haiku", | ||
| input="hi", | ||
| api_key="sk-1234" | ||
| ) | ||
|
|
||
| assert response.model == "neosantara/claude-3-haiku" | ||
| assert response.output[0].content[0].text == "Hello from responses API" | ||
|
|
There was a problem hiding this comment.
Bridge tests mock wrong schema
test_neosantara_responses_api_bridge() mocks an OpenAI chat.completion payload and then asserts on litellm.responses() output shape. responses() expects a Responses API object (with output blocks), and anthropic_messages() expects an Anthropic messages-shaped response. If the bridge code doesn’t transform chat.completion into those shapes, these tests will fail (or pass for the wrong reasons if the bridge returns raw dicts).
Fix: mock the actual wire-format each bridge expects/produces (Responses API + Anthropic Messages), or assert on the exact transformation behavior implemented in the bridge layer.
Additional Comments (1)
Fix: ensure (Also verify the file parses as JSON after the edit.) |
|
@greptile-apps can you re-review this? |
Greptile OverviewGreptile SummaryThis PR adds Neosantara as an OpenAI-compatible provider by registering a JSON provider config ( Provider routing is primarily handled via Tests and metadata files were added/updated to reflect supported endpoints and model pricing/context windows, but the new tests’ HTTP mocking and bridge-schema assertions do not appear aligned with LiteLLM’s actual request path and bridge output shapes. Confidence Score: 2/5
|
| Filename | Overview |
|---|---|
| docs/my-website/docs/providers/neosantara.md | Adds Neosantara provider documentation with SDK and proxy examples; no functional code changes. |
| docs/my-website/sidebars.js | Adds Neosantara doc to providers sidebar; ensure build tooling validates JS syntax in CI. |
| litellm/init.py | No Neosantara-specific logic changes observed; relies on existing provider loading and model registration paths. |
| litellm/constants.py | Registers 'neosantara' in openai-compatible lists and embedding providers; includes a likely duplicate entry ('baseten') pre-existing, and adds neosantara endpoint substring matching. |
| litellm/litellm_core_utils/get_llm_provider_logic.py | Adds api_base auto-detection for https://api.neosantara.xyz/v1 -> neosantara; bug: the endpoint constant string does not match the substring list entry, so auto-detection will not fire. |
| litellm/llms/openai_like/providers.json | Adds JSON-configured provider entry for neosantara with base_url and env var; used by JSONProviderRegistry. |
| litellm/main.py | No Neosantara-specific logic found in scanned sections; large file, but provider routing relies on get_llm_provider logic. |
| litellm/types/utils.py | Adds NEOSANTARA to LlmProviders enum; integrates into provider_list / validation paths. |
| model_prices_and_context_window.json | Adds Neosantara model pricing/context entries; prior duplicate-key issue was noted in review threads and should remain de-duplicated and deterministic. |
| provider_endpoints_support.json | Adds neosantara provider endpoint support flags; prior review notes missing full endpoint key set—ensure schema-complete endpoints map. |
| tests/test_litellm/test_neosantara.py | Adds tests for provider mapping, request formatting, and bridges; tests assume httpx.Client.send patching is sufficient and include bridge-shape assertions that likely don't match actual bridge outputs. |
Sequence Diagram
sequenceDiagram
participant U as Caller
participant GL as get_llm_provider()
participant JPR as JSONProviderRegistry
participant OH as OpenAI-like handler
participant NX as Neosantara API
U->>GL: "get_llm_provider(model=neosantara/..., api_base=None)"
GL->>JPR: "exists('neosantara')"
JPR-->>GL: "true"
GL-->>U: "provider=neosantara, api_base=https://api.neosantara.xyz/v1"
U->>GL: "get_llm_provider(model=claude-3-haiku, api_base=https://api.neosantara.xyz/v1)"
GL-->>U: "expected provider=neosantara"
Note over GL,U: "In this PR, substring list vs. endpoint comparison mismatch can prevent auto-detect"
U->>OH: "completion(model=neosantara/claude-3-haiku)"
OH->>NX: "POST /v1/chat/completions"
NX-->>OH: "chat.completion JSON"
OH-->>U: "ModelResponse"
| elif endpoint == "https://api.inference.wandb.ai/v1": | ||
| custom_llm_provider = "wandb" | ||
| dynamic_api_key = get_secret_str("WANDB_API_KEY") | ||
| elif endpoint == "https://api.neosantara.xyz/v1": | ||
| custom_llm_provider = "neosantara" | ||
| dynamic_api_key = get_secret_str("NEOSANTARA_API_KEY") | ||
|
|
||
| if api_base is not None and not isinstance(api_base, str): |
There was a problem hiding this comment.
Neosantara auto-detect never triggers
get_llm_provider() checks for endpoint in litellm.openai_compatible_endpoints: if endpoint in api_base, but the Neosantara entry in openai_compatible_endpoints is "https://api.neosantara.xyz/v1" while this branch compares against endpoint == "https://api.neosantara.xyz/v1" (note the extra https:// prefix in the list and also that other entries are often host/path substrings). As written, endpoint in api_base will not match when users pass api_base="https://api.neosantara.xyz/v1" unless the list and the elif endpoint == ... constant are aligned; this makes the documented auto-detection path fail (and your test_neosantara_auto_detection_api_base will fail in real execution).
There was a problem hiding this comment.
any suggestions how to fix?
There was a problem hiding this comment.
Looking at the code, the issue is that the auto-detection happens inside the loop that checks if endpoint in api_base, but your elif check uses exact equality == instead of the substring check.
The fix is simple - change your condition to match the pattern used by other providers:
| elif endpoint == "https://api.inference.wandb.ai/v1": | |
| custom_llm_provider = "wandb" | |
| dynamic_api_key = get_secret_str("WANDB_API_KEY") | |
| elif endpoint == "https://api.neosantara.xyz/v1": | |
| custom_llm_provider = "neosantara" | |
| dynamic_api_key = get_secret_str("NEOSANTARA_API_KEY") | |
| if api_base is not None and not isinstance(api_base, str): | |
| elif "api.neosantara.xyz" in endpoint: | |
| custom_llm_provider = "neosantara" | |
| dynamic_api_key = get_secret_str("NEOSANTARA_API_KEY") |
This way:
- The outer loop condition
if endpoint in api_basewill match whenapi_base="https://api.neosantara.xyz/v1" - Your inner check
"api.neosantara.xyz" in endpointwill then correctly identify it as Neosantara
This pattern matches how other providers like Fireworks and Together AI handle detection in the same file.
| # We mock the actual call to avoid network requests | ||
| with pytest.MonkeyPatch().context() as m: | ||
| def mock_send(self, request, **kwargs): | ||
| # Verify request URL and headers | ||
| assert "api.neosantara.xyz" in str(request.url) | ||
| assert request.headers["Authorization"] == "Bearer sk-1234" | ||
|
|
||
| # Verify request body is OpenAI format | ||
| body = json.loads(request.read()) | ||
| assert "messages" in body | ||
| assert body["model"] == "claude-3-haiku" | ||
|
|
||
| return httpx.Response( | ||
| 200, | ||
| content='{"choices": [{"message": {"content": "Hello world"}, "finish_reason": "stop", "index": 0}], "usage": {"prompt_tokens": 10, "completion_tokens": 10, "total_tokens": 20}, "object": "chat.completion"}'.encode(), | ||
| request=request | ||
| ) | ||
|
|
||
| m.setattr("httpx.Client.send", mock_send) | ||
|
|
||
| response = litellm.completion( | ||
| model="neosantara/claude-3-haiku", | ||
| messages=[{"role": "user", "content": "hi"}], | ||
| api_key="sk-1234" | ||
| ) | ||
|
|
||
| assert response.choices[0].message.content == "Hello world" | ||
| assert response._hidden_params["custom_llm_provider"] == "neosantara" | ||
| assert response._hidden_params["api_base"] == "https://api.neosantara.xyz/v1" | ||
|
|
There was a problem hiding this comment.
Monkeypatch may not intercept HTTP
These tests patch httpx.Client.send / httpx.AsyncClient.send, but LiteLLM often routes requests through its own HTTPHandler / AsyncHTTPHandler (or an AsyncClient instance), so patching the class method may not affect the actual call path. If the patched method isn’t used, the test can attempt a real network request (and/or fail for the wrong reason). Consider patching the concrete request function LiteLLM uses (e.g., the module-level client/handler used in OpenAI-like providers) so the tests deterministically intercept outbound traffic.
Also appears in: tests/test_litellm/test_neosantara.py:61-85, :105-131, :139-171.
| def test_neosantara_responses_api_bridge(): | ||
| """ | ||
| Test that Neosantara works with litellm.responses() API bridge. | ||
| """ | ||
| os.environ["NEOSANTARA_API_KEY"] = "sk-1234" | ||
|
|
||
| with pytest.MonkeyPatch().context() as m: | ||
| def mock_send(self, request, **kwargs): | ||
| # Verify the bridge transformed 'input' into OpenAI 'messages' | ||
| body = json.loads(request.read()) | ||
| assert "messages" in body | ||
| assert body["messages"][0]["content"] == "hi from responses" | ||
| assert "/v1/chat/completions" in str(request.url) | ||
|
|
||
| return httpx.Response( | ||
| 200, | ||
| content='{"choices": [{"message": {"content": "Hello from bridge"}, "finish_reason": "stop", "index": 0}], "usage": {"prompt_tokens": 10, "completion_tokens": 10, "total_tokens": 20}, "object": "chat.completion", "id": "chatcmpl-123", "created": 1677610602}'.encode(), | ||
| request=request | ||
| ) | ||
|
|
||
| m.setattr("httpx.Client.send", mock_send) | ||
|
|
||
| response = litellm.responses( | ||
| model="neosantara/claude-3-haiku", | ||
| input="hi from responses", | ||
| api_key="sk-1234" | ||
| ) | ||
|
|
||
| # Verify result is transformed to Responses API schema | ||
| assert response.model == "neosantara/claude-3-haiku" | ||
| assert response.output[0].content[0].text == "Hello from bridge" | ||
| assert hasattr(response, "created_at") |
There was a problem hiding this comment.
Bridge response schema mismatch
test_neosantara_responses_api_bridge() mocks a chat.completion-shaped response ({"choices": ...}) but then asserts litellm.responses() returns a Responses API object with output[0].content[0].text and created_at. Unless the bridge layer actually transforms chat.completions into Responses API output blocks, this will fail (or pass only if responses() is returning an unexpected raw/compat object). The mock payload/assertions need to match the real wire format and transformation implemented by the bridge.
Also appears in: tests/test_litellm/test_neosantara.py:133-170 for anthropic_messages().
|
@greptile-apps re review the changes |
Greptile OverviewGreptile SummaryAdded Neosantara as a new OpenAI-compatible LLM provider for the Indonesian market. Integration includes 42 models (chat and embeddings), provider auto-detection via Key changes:
Previous review concerns addressed:
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/llms/openai_like/providers.json | Added neosantara provider configuration with correct base_url, api_key_env, and param_mappings |
| litellm/constants.py | Registered neosantara in all required provider lists and endpoints with correct configurations |
| litellm/litellm_core_utils/get_llm_provider_logic.py | Added auto-detection logic for neosantara when api_base contains api.neosantara.xyz/v1 |
| litellm/main.py | Added neosantara to embedding condition (redundant with openai_compatible_providers check), also added broader compatibility checks |
| tests/test_litellm/test_neosantara.py | Comprehensive tests covering provider detection, completions, embeddings, and bridge APIs with proper mocking |
Sequence Diagram
sequenceDiagram
participant User
participant LiteLLM as litellm.completion()
participant GetProvider as get_llm_provider()
participant Registry as JSONProviderRegistry
participant OpenAI as OpenAILikeAPI
participant Neosantara as api.neosantara.xyz
User->>LiteLLM: completion(model="neosantara/claude-3-haiku")
LiteLLM->>GetProvider: get_llm_provider("neosantara/claude-3-haiku")
GetProvider->>GetProvider: Parse prefix "neosantara/"
GetProvider->>Registry: Check providers.json for "neosantara"
Registry-->>GetProvider: Found: base_url, api_key_env, param_mappings
GetProvider-->>LiteLLM: provider="neosantara", api_base="https://api.neosantara.xyz/v1"
alt Auto-detection via api_base
User->>LiteLLM: completion(model="claude-3-haiku", api_base="https://api.neosantara.xyz/v1")
LiteLLM->>GetProvider: get_llm_provider(model, api_base)
GetProvider->>GetProvider: Loop through openai_compatible_endpoints
GetProvider->>GetProvider: if "api.neosantara.xyz/v1" in api_base
GetProvider-->>LiteLLM: provider="neosantara", dynamic_api_key
end
LiteLLM->>OpenAI: Route to OpenAI-compatible handler
OpenAI->>OpenAI: Transform params (max_completion_tokens → max_tokens)
OpenAI->>Neosantara: POST /v1/chat/completions
Note over Neosantara: OpenAI-compatible endpoint
Neosantara-->>OpenAI: Chat completion response
OpenAI-->>LiteLLM: Parse and transform response
LiteLLM-->>User: ModelResponse object
| or custom_llm_provider == "together_ai" | ||
| or custom_llm_provider == "nvidia_nim" | ||
| or custom_llm_provider == "litellm_proxy" | ||
| or custom_llm_provider == "neosantara" |
There was a problem hiding this comment.
redundant - neosantara is already in openai_compatible_providers (checked on line 4760)
|
@jquinter hi, i was resolving the commented code review there |
|
@greptile-apps re review the pr |
Greptile OverviewGreptile SummaryThis PR adds Neosantara, an Indonesian LLM gateway, as a new OpenAI-compatible provider to LiteLLM. The implementation follows established patterns and includes all necessary components. Key changes:
Previous issues resolved:
The implementation is clean, follows LiteLLM conventions, and integrates seamlessly with existing OpenAI-compatible provider infrastructure. Confidence Score: 5/5
|
| Filename | Overview |
|---|---|
| litellm/llms/openai_like/providers.json | Adds Neosantara provider config with base URL, API key env var, and parameter mappings |
| litellm/constants.py | Registers neosantara in provider lists, endpoints, and embedding models |
| litellm/litellm_core_utils/get_llm_provider_logic.py | Adds auto-detection logic for Neosantara based on api_base URL |
| model_prices_and_context_window.json | Adds 21 Neosantara models with pricing and token limits (duplicates resolved) |
| provider_endpoints_support.json | Adds complete Neosantara endpoint flags with all required keys |
| tests/test_litellm/test_neosantara.py | Comprehensive test suite with provider detection, chat, embeddings, and bridge tests |
Sequence Diagram
sequenceDiagram
participant User
participant LiteLLM
participant get_llm_provider
participant JSONProviderRegistry
participant OpenAILike
participant Neosantara
User->>LiteLLM: completion(model="neosantara/claude-3-haiku")
LiteLLM->>get_llm_provider: get_llm_provider(model)
alt Model has neosantara/ prefix
get_llm_provider->>JSONProviderRegistry: Check providers.json
JSONProviderRegistry-->>get_llm_provider: base_url, api_key_env, param_mappings
get_llm_provider-->>LiteLLM: provider="neosantara", api_base="https://api.neosantara.xyz/v1"
else api_base contains "api.neosantara.xyz/v1"
get_llm_provider->>get_llm_provider: Auto-detect from openai_compatible_endpoints
get_llm_provider-->>LiteLLM: provider="neosantara", dynamic_api_key from NEOSANTARA_API_KEY
end
LiteLLM->>OpenAILike: Route to OpenAI-compatible handler
OpenAILike->>OpenAILike: Apply param_mappings (max_completion_tokens → max_tokens)
OpenAILike->>Neosantara: POST https://api.neosantara.xyz/v1/chat/completions
Neosantara-->>OpenAILike: OpenAI-compatible response
OpenAILike-->>LiteLLM: Formatted response
LiteLLM-->>User: ModelResponse
|
@jquinter any response? should i make new pull or should i follow existing provider pattern? |
|
Automated patch bundle from top-50 unresolved backlog scan.\nGenerated due limited direct branch-write access; please apply/cherry-pick the proposed minimal edits below.\n\n## PR #20641 — Unresolved threads summary
Concrete patch proposal
|
|
@giulio-leone Its will be great if i'm create the dir for Neosantara in I just fixed the integration but not yet push to the repo. I just realized my integration doesn't make on Though? before i push the changes i think |
… fix detection logic and update tests
|
@giulio-leone you can tests now, 7 tests passed |
| or custom_llm_provider in litellm.openai_compatible_providers | ||
| or JSONProviderRegistry.exists(custom_llm_provider) |
There was a problem hiding this comment.
Catch-all breaks existing embedding providers
These two new conditions (custom_llm_provider in litellm.openai_compatible_providers and JSONProviderRegistry.exists(custom_llm_provider)) are far broader than what's needed for neosantara. They will now intercept every openai-compatible provider and route it through the generic OpenAI embedding path — but several of those providers have their own dedicated elif blocks further down that will now never be reached:
hosted_vllm(line 4832) — has specificHOSTED_VLLM_API_BASE/HOSTED_VLLM_API_KEYenv var handlingllamafile/lm_studio(line 4856) — usesopenai_like_embeddinghandlervercel_ai_gateway(line 4964) — has specificVERCEL_AI_GATEWAY_API_BASE/VERCEL_AI_GATEWAY_API_KEYandVERCEL_OIDC_TOKENhandling
With this change, these providers will fall into the generic OpenAI handler, which defaults api_base to https://api.openai.com/v1 and api_key to OPENAI_API_KEY — completely wrong for them.
To only add neosantara embedding support without breaking existing providers, the condition should be narrowed. For example:
| or custom_llm_provider in litellm.openai_compatible_providers | |
| or JSONProviderRegistry.exists(custom_llm_provider) | |
| or custom_llm_provider in litellm.openai_compatible_providers | |
| or JSONProviderRegistry.exists(custom_llm_provider) |
Should instead be something like:
or custom_llm_provider == "neosantara"
Or, better yet, follow the existing pattern and add neosantara to the list at the top of this elif chain.
Context Used: Rule from dashboard - What: Avoid writing provider-specific code outside of the llms/ directory.
Why: This practice ensur... (source)
| if "api.neosantara.xyz/v1" in api_base: | ||
| custom_llm_provider = "neosantara" | ||
| dynamic_api_key = get_secret_str("NEOSANTARA_API_KEY") | ||
| if api_base is not None and not isinstance(api_base, str): | ||
| raise Exception( | ||
| "api base needs to be a string. api_base={}".format(api_base) | ||
| ) | ||
| if dynamic_api_key is not None and not isinstance(dynamic_api_key, str): | ||
| raise Exception( | ||
| "dynamic_api_key needs to be a string. dynamic_api_key={}".format( | ||
| dynamic_api_key | ||
| ) | ||
| ) | ||
| return model, custom_llm_provider, dynamic_api_key, api_base |
There was a problem hiding this comment.
Auto-detection bypasses established pattern
This hardcoded block is placed before the openai_compatible_endpoints loop, duplicating the validation/return logic that already exists inside that loop. Every other provider (Perplexity, Groq, Cerebras, etc.) adds its endpoint to openai_compatible_endpoints in constants.py and is then detected inside the loop with an elif clause.
Neosantara should follow the same pattern:
- Add
"api.neosantara.xyz/v1"toopenai_compatible_endpointsinconstants.py - Add an
elif "api.neosantara.xyz" in endpoint:clause inside the existing loop (around line 319)
This removes 14 lines of duplicated boilerplate and keeps the codebase consistent.
Context Used: Rule from dashboard - What: Avoid writing provider-specific code outside of the llms/ directory.
Why: This practice ensur... (source)
| "azure", | ||
| "hosted_vllm", | ||
| "nebius", | ||
| "neosantara", |
There was a problem hiding this comment.
Unsubstantiated token-array support claim
Adding neosantara to LITELLM_EMBEDDING_PROVIDERS_SUPPORTING_INPUT_ARRAY_OF_TOKENS means LiteLLM will allow callers to pass pre-tokenized integer arrays as embedding input. Only providers whose embedding endpoints actually accept token arrays (like OpenAI, Azure) should be in this list. There's no documentation or evidence that Neosantara's nusa-embedding-0001 endpoint supports this — if it doesn't, this will cause silent failures or incorrect behavior at the provider level.
| "neosantara": { | ||
| "display_name": "Neosantara (`neosantara`)", | ||
| "url": "https://docs.litellm.ai/docs/providers/neosantara", | ||
| "endpoints": { | ||
| "chat_completions": true, | ||
| "messages": true, | ||
| "responses": true, | ||
| "embeddings": true, | ||
| "image_generations": true, | ||
| "audio_transcriptions": true, | ||
| "audio_speech": true, | ||
| "moderations": true, | ||
| "batches": true, | ||
| "rerank": false, | ||
| "container": true, | ||
| "compact": true, | ||
| "a2a": true, | ||
| "interactions": true, | ||
| "vector_store_files": true, | ||
| "vector_stores_create": true, | ||
| "vector_stores_search": true, | ||
| "assistants": true, | ||
| "container_files": true, | ||
| "fine_tuning": true, | ||
| "image_variations": true, | ||
| "rag_ingest": true, | ||
| "rag_query": true, | ||
| "realtime": true, | ||
| "text_completion": true | ||
| } | ||
| }, |
There was a problem hiding this comment.
Inflated endpoint support claims
Neosantara's endpoint flags claim support for assistants, fine_tuning, realtime, vector_stores_*, rag_*, image_generations, audio_transcriptions, audio_speech, moderations, batches, container, container_files, and image_variations. For comparison, similar small OpenAI-compatible providers like Groq, Together AI, and SambaNova only claim chat_completions, messages, responses, and a few metadata flags.
Unless Neosantara actually supports all of these endpoints, these should be set to false to prevent LiteLLM from advertising or routing unsupported operations to this provider.
|
i think i need re-pull request and try to understand the repo first:) |
4 participants
Neosantara is an LLM Gateaway from Indonesian that provides OpenAI Compatible and Anthropic Compatibe endpoints
Maybe u need verify this changes at
litellm/main.pyIf this doesn't correct let me know and will be make a changes againRelevant issues
None
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
📖 Documentation
✅ Test
Changes
Registered Neosantara in [
litellm/llms/openai_like/providers.json].Added NEOSANTARA to LlmProviders enum in [
litellm/types/utils.py].Added neosantara to LITELLM_CHAT_PROVIDERS, openai_compatible_providers, and openai_compatible_endpoints in [
litellm/constants.py].Implemented auto-detection logic in
get_llm_provider_logic.pyto recognize the provider automatically when api_base="https://api.neosantara.xyz/v1" is used.Synchronized
model_prices_and_context_window.jsonwith Neosantara ModelsCreated comprehensive documentation in
docs/my-website/docs/providers/neosantara.md:Added Neosantara to the documentation sidebar.
Testing
Created
tests/test_litellm/test_neosantara.pywith 6 tests:litellm.responses()andlitellm.anthropic_messages().