feat(neosantara): add Neosantara integration as OpenAI-compatible pro…

docs/my-website/docs/providers/neosantara.md

@@ -0,0 +1,201 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Neosantara
+
+## Overview
+
+| Property | Details |
+|-------|-------|
+| Description | Neosantara is a unified LLM gateway designed for developers in Indonesia, providing a single OpenAI-compatible interface to multiple top-tier AI models (OpenAI, Anthropic, Gemini, etc.). |
+| Provider Route on LiteLLM | `neosantara/` |
+| Link to Provider Doc | [Neosantara Dashboard ↗](https://app.neosantara.xyz) |
+| Base URL | `https://api.neosantara.xyz/v1` |
+| Supported Operations | [`/chat/completions`](#sample-usage), [`/embeddings`](#embeddings) |
+
+<br />
+
+## What is Neosantara?
+
+Neosantara is a unified gateway that lets developers:
+- **Access Multiple LLM Providers**: Unified interface for OpenAI, Anthropic, Gemini, and more.
+- **Optimized for Indonesia**: Designed specifically for the needs of developers in the region.
+- **Unified Billing**: Pay-As-You-Go system with local payment support.
+- **OpenAI Compatible**: Seamlessly drop into existing OpenAI-based workflows.
+
+## Required Variables
+
+```python showLineNumbers title="Environment Variables"
+os.environ["NEOSANTARA_API_KEY"] = "your-neosantara-api-key"
+```
+
+Get your Neosantara API key from [app.neosantara.xyz](https://app.neosantara.xyz).
+
+## Usage - LiteLLM Python SDK
+
+<Tabs>
+<TabItem value="non-streaming" label="Non-streaming">
+
+```python showLineNumbers title="Neosantara Non-streaming Completion"
+import os
+import litellm
+from litellm import completion
+
+os.environ["NEOSANTARA_API_KEY"] = "your-neosantara-api-key"
+
+messages = [{"content": "What is the capital of Indonesia?", "role": "user"}]
+
+# Neosantara call
+response = completion(
+    model="neosantara/claude-3-haiku",
+    messages=messages
+)
+
+print(response)
+```
+
+</TabItem>
+<TabItem value="streaming" label="Streaming">
+
+```python showLineNumbers title="Neosantara Streaming Completion"
+import os
+import litellm
+from litellm import completion
+
+os.environ["NEOSANTARA_API_KEY"] = "your-neosantara-api-key"
+
+messages = [{"content": "Write a short poem about Jakarta", "role": "user"}]
+
+# Neosantara call with streaming
+response = completion(
+    model="neosantara/claude-3-haiku",
+    messages=messages,
+    stream=True
+)
+
+for chunk in response:
+    print(chunk)
+```
+
+</TabItem>
+<TabItem value="embeddings" label="Embeddings">
+
+```python showLineNumbers title="Neosantara Embeddings"
+import os
+import litellm
+from litellm import embedding
+
+os.environ["NEOSANTARA_API_KEY"] = "your-neosantara-api-key"
+
+# Neosantara call
+response = embedding(
+    model="neosantara/nusa-embedding-0001",
+    input=["Hello, how are you?"]
+)
+
+print(response)
+```
+
+</TabItem>
+</Tabs>
+
+## Usage - LiteLLM Proxy Server
+
+### 1. Set Neosantara Models on `config.yaml`
+
+```yaml
+model_list:
+  - model_name: neosantara-claude-3-haiku
+    litellm_params:
+      model: neosantara/claude-3-haiku
+      api_key: os.environ/NEOSANTARA_API_KEY
+```
+
+### 2. Start Proxy
+
+```bash
+litellm --config config.yaml
+```
+
+### 3. Test it
+
+<Tabs>
+<TabItem value="Curl" label="Curl Request">
+
+```shell
+curl --location 'http://0.0.0.0:4000/chat/completions' \
+--header 'Content-Type: application/json' \
+--header 'Authorization: Bearer sk-1234' \
+--data ' {
+      "model": "neosantara-claude-3-haiku",
+      "messages": [
+        {
+          "role": "user",
+          "content": "what llm are you"
+        }
+      ]
+    }
+'
+```
+</TabItem>
+<TabItem value="openai" label="OpenAI v1.0.0+">
+
+```python
+import openai
+client = openai.OpenAI(
+    api_key="anything",
+    base_url="http://0.0.0.0:4000"
+)
+
+response = client.chat.completions.create(
+    model="neosantara-claude-3-haiku", 
+    messages = [
+        {
+            "role": "user",
+            "content": "this is a test request, write a short poem"
+        }
+    ]
+)
+
+print(response)
+```
+</TabItem>
+</Tabs>
+
+## Supported Models
+
+We support a wide range of models optimized for the Indonesian context and high-performance tasks.
+
+| Model Name | Model ID (for LiteLLM) | Provider | Description |
+|------------|------------------------|----------|-------------|
+| **Nusantara Base** | `neosantara/nusantara-base` | Gemini | Flagship balanced model |
+| **Archipelago 70B** | `neosantara/archipelago-70b` | Llama 3.3 | Cultural context awareness |
+| **Garda Beta Mini** | `neosantara/garda-beta-mini` | Groq/Paxsenix | Fast & efficient Indonesian understanding |
+| **Claude 3 Haiku** | `neosantara/claude-3-haiku` | Bedrock | Near-instant responsiveness |
+| **Claude 3 Sonnet** | `neosantara/claude-3-sonnet` | Bedrock | Balance of intelligence and speed |
+| **Sahabat AI Llama v4** | `neosantara/sahabat-ai-llama-v4` | SahabatAI | Fine-tuned for Sahabat AI ecosystem |
+| **Nusa Embedding 0001**| `neosantara/nusa-embedding-0001` | Embedding | Optimized for Indonesian search |
+
+:::info
+**Note:** You can use any model supported by Neosantara by adding the `neosantara/` prefix to the model name in your LiteLLM calls.
+:::
+
+## Supported OpenAI Parameters
+
+Neosantara supports all standard OpenAI-compatible parameters:
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `messages` | array | **Required**. Array of message objects with 'role' and 'content' |
+| `model` | string | **Required**. Model ID (e.g., `claude-3-haiku`, `archipelago-70b`) |
+| `stream` | boolean | Optional. Enable streaming responses |
+| `temperature` | float | Optional. Sampling temperature |
+| `top_p` | float | Optional. Nucleus sampling parameter |
+| `max_tokens` | integer | Optional. Maximum tokens to generate |
+| `tools` | array | Optional. List of available tools/functions |
+| `tool_choice` | string/object | Optional. Control tool/function calling |
+
+## Additional Resources
+
+- [Neosantara Dashboard](https://app.neosantara.xyz)
+- [API Documentation](https://docs.neosantara.xyz)

docs/my-website/sidebars.js

@@ -790,6 +790,7 @@ const sidebars = {
         "providers/moonshot",
         "providers/morph",
         "providers/nebius",
+        "providers/neosantara",
         "providers/nlp_cloud",
         "providers/nano-gpt",
         "providers/novita",

litellm/__init__.py

@@ -543,6 +543,7 @@ def identify(event_details):
 nscale_models: Set = set()
 nebius_models: Set = set()
 nebius_embedding_models: Set = set()
+neosantara_embedding_models: Set = set()
 aiml_models: Set = set()
 deepgram_models: Set = set()
 elevenlabs_models: Set = set()
@@ -571,6 +572,7 @@ def identify(event_details):
 aws_polly_models: Set = set()
 gigachat_models: Set = set()
 llamagate_models: Set = set()
+neosantara_models: Set = set()
 
 
 def is_bedrock_pricing_only_model(key: str) -> bool:
@@ -832,6 +834,11 @@ def add_known_models():
             gigachat_models.add(key)
         elif value.get("litellm_provider") == "llamagate":
             llamagate_models.add(key)
+        elif value.get("litellm_provider") == "neosantara":
+            if value.get("mode") == "embedding":
+                neosantara_embedding_models.add(key)
+            else:
+                neosantara_models.add(key)
 
 
 add_known_models()
@@ -1042,6 +1049,7 @@ def add_known_models():
     "aws_polly": aws_polly_models,
     "gigachat": gigachat_models,
     "llamagate": llamagate_models,
+    "neosantara": neosantara_models | neosantara_embedding_models,
 }
 
 # mapping for those models which have larger equivalents
@@ -1077,6 +1085,7 @@ def add_known_models():
     | nebius_embedding_models
     | sambanova_embedding_models
     | ovhcloud_embedding_models
+    | neosantara_embedding_models
 )
 
 ####### IMAGE GENERATION MODELS ###################

litellm/constants.py

@@ -455,13 +455,15 @@
     "lemonade",
     "docker_model_runner",
     "amazon_nova",
+    "neosantara",
 ]
 
 LITELLM_EMBEDDING_PROVIDERS_SUPPORTING_INPUT_ARRAY_OF_TOKENS = [
     "openai",
     "azure",
     "hosted_vllm",
     "nebius",
+    "neosantara",
 ]
 
 
@@ -664,6 +666,7 @@
     "clarifai",
     "docker_model_runner",
     "ragflow",
+    "neosantara",
 ]
 openai_text_completion_compatible_providers: List = (
     [  # providers that support `/v1/completions`
@@ -904,6 +907,12 @@
     ]
 )
 
+neosantara_embedding_models: set = set(
+    [
+        "nusa-embedding-0001",
+    ]
+)
+
 WANDB_MODELS: set = set(
     [
         # openai models

litellm/litellm_core_utils/get_llm_provider_logic.py

@@ -195,6 +195,21 @@ def get_llm_provider(  # noqa: PLR0915
             return model, custom_llm_provider, dynamic_api_key, api_base
         # check if api base is a known openai compatible endpoint
         if api_base:
+            if "api.neosantara.xyz/v1" in api_base:
+                custom_llm_provider = "neosantara"
+                dynamic_api_key = get_secret_str("NEOSANTARA_API_KEY")
+                if api_base is not None and not isinstance(api_base, str):
+                    raise Exception(
+                        "api base needs to be a string. api_base={}".format(api_base)
+                    )
+                if dynamic_api_key is not None and not isinstance(dynamic_api_key, str):
+                    raise Exception(
+                        "dynamic_api_key needs to be a string. dynamic_api_key={}".format(
+                            dynamic_api_key
+                        )
+                    )
+                return model, custom_llm_provider, dynamic_api_key, api_base
+
             for endpoint in litellm.openai_compatible_endpoints:
                 if endpoint in api_base:
                     if endpoint == "api.perplexity.ai":

litellm/llms/openai_like/providers.json

@@ -86,5 +86,12 @@
     "headers": {
       "api-subscription-key": "{api_key}"
     }
+  },
+  "neosantara": {
+    "base_url": "https://api.neosantara.xyz/v1",
+    "api_key_env": "NEOSANTARA_API_KEY",
+    "param_mappings": {
+      "max_completion_tokens": "max_tokens"
+    }
   }
-}
+}

litellm/main.py

@@ -4756,6 +4756,8 @@ def embedding(  # noqa: PLR0915
             or custom_llm_provider == "nvidia_nim"
             or custom_llm_provider == "litellm_proxy"
             or (model in litellm.open_ai_embedding_models and custom_llm_provider is None)
+            or custom_llm_provider in litellm.openai_compatible_providers
+            or JSONProviderRegistry.exists(custom_llm_provider)
         ):
             api_base = (
                 api_base

litellm/types/utils.py

@@ -3084,6 +3084,7 @@ class LlmProviders(str, Enum):
     GRADIENT_AI = "gradient_ai"
     LLAMA = "meta_llama"
     NSCALE = "nscale"
+    NEOSANTARA = "neosantara"
     PG_VECTOR = "pg_vector"
     S3_VECTORS = "s3_vectors"
     HELICONE = "helicone"