Skip to content

[Feat]Add new claude 4-6 feat for v1/messages #20733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sameerlite merged 10 commits into from
Feb 9, 2026
Merged

[Feat]Add new claude 4-6 feat for v1/messages #20733

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
8cd8a01
Add compaction for vertex ai
Sameerlite Feb 9, 2026
d41df60
Add all new feat for v1/messages
Sameerlite Feb 9, 2026
3307f3d
Add inference_geo as supported messages param
Sameerlite Feb 9, 2026
20440bc
Add inference based costing
Sameerlite Feb 9, 2026
29e6efa
Add inference_geo as supported messages param
Sameerlite Feb 9, 2026
b822e2e
Add support for fast param
Sameerlite Feb 9, 2026
c3b1c0a
Add fast mode for other providers
Sameerlite Feb 9, 2026
319453d
Add documentation for Fast Mode
Sameerlite Feb 9, 2026
ef55d37
Merge branch 'main' into litellm_v1_messages_claude_4_6
Sameerlite Feb 9, 2026
5611974
Fix : litellm/tests/test_litellm/llms/bedrock/chat/invoke_transformat…
Sameerlite Feb 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
330 changes: 319 additions & 11 deletions docs/my-website/blog/claude_opus_4_6/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,11 +223,16 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
</TabItem>
</Tabs>

## Compaction
## Advanced Features

### Compaction

<Tabs>
<TabItem value="completions" label="/chat/completions">

Litellm supports enabling compaction for the new claude-opus-4-6.

### Enabling Compaction
**Enabling Compaction**

To enable compaction, add the `context_management` parameter with the `compact_20260112` edit type:

Expand Down Expand Up @@ -255,8 +260,43 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
```
All the parameters supported for context_management by anthropic are supported and can be directly added. Litellm automatically adds the `compact-2026-01-12` beta header in the request.

</TabItem>
<TabItem value="messages" label="/v1/messages">

Enable compaction to reduce context size while preserving key information. LiteLLM automatically adds the `compact-2026-01-12` beta header when compaction is enabled.

:::info
**Provider Support:** Compaction is supported on Anthropic, Azure AI, and Vertex AI. It is **not supported** on Bedrock (Invoke or Converse APIs).
:::

```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-6",
"max_tokens": 4096,
"messages": [
{
"role": "user",
"content": "Hi"
}
],
"context_management": {
"edits": [
{
"type": "compact_20260112"
}
]
}
}'
```

</TabItem>
</Tabs>

### Response with Compaction Block

**Response with Compaction Block**

The response will include the compaction summary in `provider_specific_fields.compaction_blocks`:

Expand Down Expand Up @@ -292,7 +332,7 @@ The response will include the compaction summary in `provider_specific_fields.co
}
```

### Using Compaction Blocks in Follow-up Requests
**Using Compaction Blocks in Follow-up Requests**

To continue the conversation with compaction, include the compaction block in the assistant message's `provider_specific_fields`:

Expand Down Expand Up @@ -340,15 +380,17 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
}'
```

### Streaming Support
**Streaming Support**

Compaction blocks are also supported in streaming mode. You'll receive:
- `compaction_start` event when a compaction block begins
- `compaction_delta` events with the compaction content
- The accumulated `compaction_blocks` in `provider_specific_fields`

### Adaptive Thinking

## Adaptive Thinking
<Tabs>
<TabItem value="completions" label="/chat/completions">

LiteLLM supports adaptive thinking through the `reasoning_effort` parameter:

Expand All @@ -368,7 +410,37 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
}'
```

## Effort Levels
</TabItem>
<TabItem value="messages" label="/v1/messages">

Use the `thinking` parameter with `type: "adaptive"` to enable adaptive thinking mode:

```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-6",
"max_tokens": 16000,
"thinking": {
"type": "adaptive"
},
"messages": [
{
"role": "user",
"content": "Explain why the sum of two even numbers is always even."
}
]
}'
```

</TabItem>
</Tabs>

### Effort Levels

<Tabs>
<TabItem value="completions" label="/chat/completions">

Four effort levels available: `low`, `medium`, `high` (default), and `max`. Pass directly via the `output_config` parameter:

Expand All @@ -387,17 +459,253 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
"output_config": {
"effort": "medium"
}

}'
```

You can use reasoning effort plus output_config to have more control on the model.

## 1M Token Context (Beta)
</TabItem>
<TabItem value="messages" label="/v1/messages">

Four effort levels available: `low`, `medium`, `high` (default), and `max`. Pass directly via the `output_config` parameter:

```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-6",
"max_tokens": 4096,
"messages": [
{
"role": "user",
"content": "Explain quantum computing"
}
],
"output_config": {
"effort": "medium"
}
}'
```

</TabItem>
</Tabs>

### 1M Token Context (Beta)

Opus 4.6 supports 1M token context. Premium pricing applies for prompts exceeding 200k tokens ($10/$37.50 per million input/output tokens). LiteLLM supports cost calculations for 1M token contexts.

## US-Only Inference
<Tabs>
<TabItem value="completions" label="/chat/completions">

To use the 1M token context window, you need to forward the `anthropic-beta` header from your client to the LLM provider.

**Step 1: Enable header forwarding in your config**

```yaml
general_settings:
forward_client_headers_to_llm_api: true
```

**Step 2: Send requests with the beta header**

```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--header 'anthropic-beta: context-1m-2025-08-07' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "Analyze this large document..."
}
]
}'
```

</TabItem>
<TabItem value="messages" label="/v1/messages">

To use the 1M token context window, you need to forward the `anthropic-beta` header from your client to the LLM provider.

**Step 1: Enable header forwarding in your config**

```yaml
general_settings:
forward_client_headers_to_llm_api: true
```

**Step 2: Send requests with the beta header**

```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'anthropic-beta: context-1m-2025-08-07' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-6",
"max_tokens": 16000,
"messages": [
{
"role": "user",
"content": "Analyze this large document..."
}
Comment on lines +551 to +554

@greptile-apps greptile-apps Bot Feb 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect provider support claim
This section states: Compaction is supported on Anthropic, Bedrock Invoke Azure AI, and Vertex AI. but /v1/messages compaction is typically not supported on Bedrock Invoke (it requires anthropic_beta support for the compact feature, which Bedrock Invoke filters/whitelists). Given the code in litellm/llms/bedrock/.../anthropic_claude3_transformation.py explicitly filters provider betas, this doc line is misleading and should be corrected (and also add the missing comma: “Bedrock Invoke, Azure AI”).

]
}'
```

:::tip
You can combine multiple beta headers by separating them with commas:
```bash
--header 'anthropic-beta: context-1m-2025-08-07,compact-2026-01-12'
```
:::

</TabItem>
</Tabs>

### US-Only Inference

Available at 1.1× token pricing. LiteLLM automatically tracks costs for US-only inference.

<Tabs>
<TabItem value="completions" label="/chat/completions">

Use the `inference_geo` parameter to specify US-only inference:

```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
],
"inference_geo": "us"
}'
```

LiteLLM will automatically apply the 1.1× pricing multiplier for US-only inference in cost tracking.

</TabItem>
<TabItem value="messages" label="/v1/messages">

Available at 1.1× token pricing. LiteLLM supports this pricing model.
Use the `inference_geo` parameter to specify US-only inference:

```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-6",
"max_tokens": 4096,
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
],
"inference_geo": "us"
}'
```

LiteLLM will automatically apply the 1.1× pricing multiplier for US-only inference in cost tracking.

</TabItem>
</Tabs>

### Fast Mode

:::info
Fast mode is **only supported on the Anthropic provider** (`anthropic/claude-opus-4-6`). It is not available on Azure AI, Vertex AI, or Bedrock.
:::

**Pricing:**
- Standard: $5 input / $25 output per MTok
- Fast: $30 input / $150 output per MTok (6× premium)

<Tabs>
<TabItem value="completions" label="/chat/completions">

```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-6",
"messages": [
{
"role": "user",
"content": "Refactor this module..."
}
],
"max_tokens": 4096,
"speed": "fast"
}'
```

**Using OpenAI SDK:**

```python
import openai

client = openai.OpenAI(
api_key="your-litellm-key",
base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
model="claude-opus-4-6",
messages=[{"role": "user", "content": "Refactor this module..."}],
max_tokens=4096,
extra_body={"speed": "fast"}
)
```

**Using LiteLLM SDK:**

```python
from litellm import completion

response = completion(
model="anthropic/claude-opus-4-6",
messages=[{"role": "user", "content": "Refactor this module..."}],
max_tokens=4096,
speed="fast"
)
```

LiteLLM automatically tracks the higher costs for fast mode in usage and cost calculations.

</TabItem>
<TabItem value="messages" label="/v1/messages">

```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-6",
"max_tokens": 4096,
"speed": "fast",
"messages": [
{
"role": "user",
"content": "Refactor this module..."
}
]
}'
```

LiteLLM automatically:
- Adds the `fast-mode-2026-02-01` beta header
- Tracks the 6× premium pricing in cost calculations

</TabItem>
</Tabs>
Loading
Loading