Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
230 changes: 230 additions & 0 deletions docs/providers/meta.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Meta Model API

| Property | Details |
|-------|-------|
| Description | Meta's Model API provides access to Meta's Muse Spark family of reasoning models. |
| Provider Route on LiteLLM | `meta/` |
| Supported Endpoints | `/chat/completions`, `/responses` |
| API Reference | [Meta Model API Reference ↗](https://dev.meta.ai/docs) |

## Required Variables

```python showLineNumbers title="Environment Variables"
os.environ["META_API_KEY"] = "" # your Meta Model API key
```

Requests go to `https://api.meta.ai/v1` by default. Set `META_API_BASE` to override the API base.

## Supported Models

:::info
We actively maintain the list of models, pricing, token window, etc. [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json).
:::

| Model ID | Input context length | Input Modalities | Output Modalities |
| --- | --- | --- | --- |
| `muse-spark-1.1` | 1M | Text, Image, Video, PDF | Text |

`muse-spark-1.1` supports function calling, parallel function calling, structured outputs, prompt caching, web search grounding, and reasoning via `reasoning_effort` (`"minimal"` through `"xhigh"`).

## Usage - LiteLLM Python SDK

### Non-streaming

```python showLineNumbers title="Meta Model API Non-streaming Completion"
import os
import litellm
from litellm import completion

os.environ["META_API_KEY"] = "" # your Meta Model API key

messages = [{"content": "Hello, how are you?", "role": "user"}]

response = completion(model="meta/muse-spark-1.1", messages=messages)
```

### Streaming

```python showLineNumbers title="Meta Model API Streaming Completion"
import os
import litellm
from litellm import completion

os.environ["META_API_KEY"] = "" # your Meta Model API key

messages = [{"content": "Hello, how are you?", "role": "user"}]

response = completion(
model="meta/muse-spark-1.1",
messages=messages,
stream=True
)

for chunk in response:
print(chunk)
```

### Reasoning Effort

`muse-spark-1.1` accepts `reasoning_effort` values `"minimal"`, `"low"`, `"medium"`, `"high"`, and `"xhigh"`.

```python showLineNumbers title="Meta Model API Reasoning Effort"
import os
import litellm
from litellm import completion

os.environ["META_API_KEY"] = "" # your Meta Model API key

messages = [{"content": "What is 15% of 2840?", "role": "user"}]

response = completion(
model="meta/muse-spark-1.1",
messages=messages,
reasoning_effort="xhigh"
)

print(response.choices[0].message.content)
print(response.usage.completion_tokens_details.reasoning_tokens)
```

### Function Calling

```python showLineNumbers title="Meta Model API Function Calling"
import os
import litellm
from litellm import completion

os.environ["META_API_KEY"] = "" # your Meta Model API key

messages = [{"content": "What's the weather like in San Francisco?", "role": "user"}]

tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
}
}
]

response = completion(
model="meta/muse-spark-1.1",
messages=messages,
tools=tools,
tool_choice="auto"
)

print(response.choices[0].message.tool_calls)
```

## Usage - LiteLLM Proxy

Add the following to your LiteLLM Proxy configuration file:

```yaml showLineNumbers title="config.yaml"
model_list:
- model_name: muse-spark-1.1
litellm_params:
model: meta/muse-spark-1.1
api_key: os.environ/META_API_KEY
```

Start your LiteLLM Proxy server:

```bash showLineNumbers title="Start LiteLLM Proxy"
litellm --config config.yaml

# RUNNING on http://0.0.0.0:4000
```

<Tabs>
<TabItem value="openai-sdk" label="OpenAI SDK">

```python showLineNumbers title="Meta Model API via Proxy - Non-streaming"
from openai import OpenAI

client = OpenAI(
base_url="http://localhost:4000", # Your proxy URL
api_key="your-proxy-api-key" # Your proxy API key
)

response = client.chat.completions.create(
model="muse-spark-1.1",
messages=[{"role": "user", "content": "Write a short poem about AI."}],
reasoning_effort="minimal"
)

print(response.choices[0].message.content)
```

```python showLineNumbers title="Meta Model API via Proxy - Streaming"
from openai import OpenAI

client = OpenAI(
base_url="http://localhost:4000", # Your proxy URL
api_key="your-proxy-api-key" # Your proxy API key
)

response = client.chat.completions.create(
model="muse-spark-1.1",
messages=[{"role": "user", "content": "Write a short poem about AI."}],
stream=True
)

for chunk in response:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
```

</TabItem>

<TabItem value="litellm-sdk" label="LiteLLM SDK">

```python showLineNumbers title="Meta Model API via Proxy - LiteLLM SDK"
import litellm

response = litellm.completion(
model="litellm_proxy/muse-spark-1.1",
messages=[{"role": "user", "content": "Write a short poem about AI."}],
api_base="http://localhost:4000",
api_key="your-proxy-api-key"
)

print(response.choices[0].message.content)
```

</TabItem>

<TabItem value="curl" label="cURL">

```bash showLineNumbers title="Meta Model API via Proxy - cURL"
curl http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-proxy-api-key" \
-d '{
"model": "muse-spark-1.1",
"messages": [{"role": "user", "content": "Write a short poem about AI."}],
"reasoning_effort": "minimal"
}'
```

</TabItem>
</Tabs>
1 change: 1 addition & 0 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -1101,6 +1101,7 @@ const sidebars = {
"providers/llamagate",
"providers/lm_studio",
"providers/manus",
"providers/meta",
"providers/meta_llama",
"providers/milvus_vector_stores",
"providers/mistral",
Expand Down