-
+
@@ -37,7 +37,7 @@ LiteLLM manages:
- Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - [Router](https://docs.litellm.ai/docs/routing)
- Set Budgets & Rate limits per project, api key, model [LiteLLM Proxy Server (LLM Gateway)](https://docs.litellm.ai/docs/simple_proxy)
-[**Jump to LiteLLM Proxy (LLM Gateway) Docs**](https://github.com/BerriAI/litellm?tab=readme-ov-file#openai-proxy---docs) 
+
+
+## Authentication
+
+To use CometAPI models, you need to obtain an API key from [CometAPI Token Console](https://api.cometapi.com/console/token). CometAPI offers free tokens for new users - you can get your free API key instantly by registering.
+
+## Usage
+
+Set your CometAPI key as an environment variable and use the completion function:
+
+```python
+import os
+from litellm import completion
+
+# Set API key
+os.environ["COMETAPI_KEY"] = "your_comet_api_key_here"
+
+# Define messages
+messages = [{"content": "Hello, how are you?", "role": "user"}]
+
+# Method 1: Using environment variable (recommended)
+response = completion(
+ model="cometapi/gpt-5",
+ messages=messages
+)
+
+print(response.choices[0].message.content)
+```
+
+### Alternative Usage - Explicit API Key
+
+You can also pass the API key explicitly:
+
+```python
+import os
+from litellm import completion
+
+# Define messages
+messages = [{"content": "Hello, how are you?", "role": "user"}]
+
+# Method 2: Explicitly passing API key
+response = completion(
+ model="cometapi/gpt-4o",
+ messages=messages,
+ api_key="your_comet_api_key_here"
+)
+
+print(response.choices[0].message.content)
+```
+
+## Usage - Streaming
+
+Just set `stream=True` when calling completion:
+
+```python
+import os
+from litellm import completion
+
+os.environ["COMETAPI_KEY"] = "your_comet_api_key_here"
+
+messages = [{"content": "Hello, how are you?", "role": "user"}]
+
+response = completion(
+ model="cometapi/gpt-5",
+ messages=messages,
+ stream=True
+)
+
+for chunk in response:
+ print(chunk.choices[0].delta.content or "", end="")
+```
+
+## Usage - Async Streaming
+
+For async streaming, use `acompletion`:
+
+```python
+from litellm import acompletion
+import asyncio, os, traceback
+
+async def completion_call():
+ try:
+ os.environ["COMETAPI_KEY"] = "your_comet_api_key_here"
+
+ print("test acompletion + streaming")
+ response = await acompletion(
+ model="cometapi/chatgpt-4o-latest",
+ messages=[{"content": "Hello, how are you?", "role": "user"}],
+ stream=True
+ )
+ print(f"response: {response}")
+ async for chunk in response:
+ print(chunk)
+ except:
+ print(f"error occurred: {traceback.format_exc()}")
+ pass
+
+# Run the async function
+await completion_call()
+```
+
+## CometAPI Models
+
+CometAPI offers access to 500+ AI models through a unified API. Some popular models include:
+
+| Model Name | Function Call |
+|------------|---------------|
+| cometapi/gpt-5 | `completion('cometapi/gpt-5', messages)` |
+| cometapi/gpt-5-mini | `completion('cometapi/gpt-5-mini', messages)` |
+| cometapi/gpt-5-nano | `completion('cometapi/gpt-5-nano', messages)` |
+| cometapi/gpt-oss-20b | `completion('cometapi/gpt-oss-20b', messages)` |
+| cometapi/gpt-oss-120b | `completion('cometapi/gpt-oss-120b', messages)` |
+| cometapi/chatgpt-4o-latest | `completion('cometapi/chatgpt-4o-latest', messages)` |
+
+For a complete list of available models, visit the [CometAPI Models page](https://www.cometapi.com/model/).
+
+## Environment Variables
+
+| Variable | Description | Required |
+|----------|-------------|----------|
+| `COMETAPI_KEY` | Your CometAPI API key | Yes |
+
+## Error Handling
+
+```python
+import os
+from litellm import completion
+
+try:
+ os.environ["COMETAPI_KEY"] = "your_comet_api_key_here"
+
+ messages = [{"content": "Hello, how are you?", "role": "user"}]
+
+ response = completion(
+ model="cometapi/gpt-5",
+ messages=messages
+ )
+
+ print(response.choices[0].message.content)
+
+except Exception as e:
+ print(f"Error: {e}")
+```
diff --git a/docs/my-website/docs/providers/compactifai.md b/docs/my-website/docs/providers/compactifai.md
new file mode 100644
index 000000000000..1aa81463071a
--- /dev/null
+++ b/docs/my-website/docs/providers/compactifai.md
@@ -0,0 +1,223 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# CompactifAI
+https://docs.compactif.ai/
+
+CompactifAI offers highly compressed versions of leading language models, delivering up to **70% lower inference costs**, **4x throughput gains**, and **low-latency inference** with minimal quality loss (under 5%). CompactifAI's OpenAI-compatible API makes integration straightforward, enabling developers to build ultra-efficient, scalable AI applications with superior concurrency and resource efficiency.
+
+| Property | Details |
+|-------|-------|
+| Description | CompactifAI offers compressed versions of leading language models with up to 70% cost reduction and 4x throughput gains |
+| Provider Route on LiteLLM | `compactifai/` (add this prefix to the model name - e.g. `compactifai/cai-llama-3-1-8b-slim`) |
+| Provider Doc | [CompactifAI ↗](https://docs.compactif.ai/) |
+| API Endpoint for Provider | https://api.compactif.ai/v1 |
+| Supported Endpoints | `/chat/completions`, `/completions` |
+
+## Supported OpenAI Parameters
+
+CompactifAI is fully OpenAI-compatible and supports the following parameters:
+
+```
+"stream",
+"stop",
+"temperature",
+"top_p",
+"max_tokens",
+"presence_penalty",
+"frequency_penalty",
+"logit_bias",
+"user",
+"response_format",
+"seed",
+"tools",
+"tool_choice",
+"parallel_tool_calls",
+"extra_headers"
+```
+
+## API Key Setup
+
+CompactifAI API keys are available through AWS Marketplace subscription:
+
+1. Subscribe via [AWS Marketplace](https://aws.amazon.com/marketplace)
+2. Complete subscription verification (24-hour review process)
+3. Access MultiverseIAM dashboard with provided credentials
+4. Retrieve your API key from the dashboard
+
+```python
+import os
+
+os.environ["COMPACTIFAI_API_KEY"] = "your-api-key"
+```
+
+## Usage
+
+Best regards,
Your Team
Best regards,
Your Company Team
+ LiteLLM MCP Architecture: Use MCP tools with all LiteLLM supported models +
+ + +This release adds support for setting rate limits on individual members (including machine users) within a team. Teams can now give each agent its own rate limits—so that heavy-traffic agents don’t impact other agents or human users. + +Agents can authenticate with LiteLLM using JWT and the same team role as human users, while still enforcing per-agent rate limits. + + +## New Models / Updated Models + +#### New Model Support + +| Provider | Model | Context Window | Input ($/1M tokens) | Output ($/1M tokens) | Features | +| ----------- | -------------------------------------- | -------------- | ------------------- | -------------------- | -------- | +| Azure AI | `azure_ai/FLUX-1.1-pro` | - | - | $40/image | Image generation | +| Azure AI | `azure_ai/FLUX.1-Kontext-pro` | - | - | $40/image | Image generation | +| Vertex AI | `vertex_ai/deepseek-ai/deepseek-r1-0528-maas` | 65k | $1.35 | $5.4 | Chat completions + reasoning | +| OpenRouter | `openrouter/deepseek/deepseek-chat-v3-0324` | 65k | $0.14 | $0.28 | Chat completions | + + +#### Features + +- **[OpenAI](../../docs/providers/openai)** + - Added `reasoning_effort` parameter support for GPT-5 model family - [PR #13475](https://github.com/BerriAI/litellm/pull/13475), [Get Started](../../docs/providers/openai#openai-chat-completion-models) + - Support for `reasoning` parameter in Responses API - [PR #13475](https://github.com/BerriAI/litellm/pull/13475), [Get Started](../../docs/response_api) +- **[Azure OpenAI](../../docs/providers/azure/azure)** + - GPT-5 support with max_tokens and `reasoning` parameter - [PR #13510](https://github.com/BerriAI/litellm/pull/13510), [Get Started](../../docs/providers/azure/azure#gpt-5-models) +- **[AWS Bedrock](../../docs/providers/bedrock)** + - Streaming support for bedrock gpt-oss model family - [PR #13346](https://github.com/BerriAI/litellm/pull/13346), [Get Started](../../docs/providers/bedrock#openai-gpt-oss) + - `/messages` endpoint compatibility with `bedrock/converse/