Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 16 additions & 9 deletions docs/my-website/docs/providers/voyage.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ All models listed here https://docs.voyageai.com/embeddings/#models-and-specific

| Model Name | Function Call |
|-------------------------|------------------------------------------------------------|
| voyage-4 | `embedding(model="voyage/voyage-4", input)` |
| voyage-4-large | `embedding(model="voyage/voyage-4-large", input)` |
| voyage-4-lite | `embedding(model="voyage/voyage-4-lite", input)` |
| voyage-3.5 | `embedding(model="voyage/voyage-3.5", input)` |
| voyage-3.5-lite | `embedding(model="voyage/voyage-3.5-lite", input)` |
| voyage-3-large | `embedding(model="voyage/voyage-3-large", input)` |
Expand All @@ -72,9 +75,9 @@ All models listed here https://docs.voyageai.com/embeddings/#models-and-specific
| voyage-lite-01 | `embedding(model="voyage/voyage-lite-01", input)` |
| voyage-lite-01-instruct | `embedding(model="voyage/voyage-lite-01-instruct", input)` |

## Contextual Embeddings (voyage-context-3)
## Contextual Embeddings (voyage-context-4)

VoyageAI's `voyage-context-3` model provides contextualized chunk embeddings, where each chunk is embedded with awareness of its surrounding document context. This significantly improves retrieval quality compared to standard context-agnostic embeddings.
VoyageAI's contextual models (`voyage-context-4`, and the previous generation `voyage-context-3`) provide contextualized chunk embeddings, where each chunk is embedded with awareness of its surrounding document context. This significantly improves retrieval quality compared to standard context-agnostic embeddings. `voyage-context-4` is the latest model and is recommended over `voyage-context-3`.

### Key Benefits
- Chunks understand their position and role within the full document
Expand All @@ -94,7 +97,7 @@ os.environ['VOYAGE_API_KEY'] = "your-api-key"

# Single document with multiple chunks
response = embedding(
model="voyage/voyage-context-3",
model="voyage/voyage-context-4",
input=[
[
"Chapter 1: Introduction to AI",
Expand All @@ -107,7 +110,7 @@ print(f"Number of chunk groups: {len(response.data)}")

# Multiple documents
response = embedding(
model="voyage/voyage-context-3",
model="voyage/voyage-context-4",
input=[
["Paris is the capital of France.", "It is known for the Eiffel Tower."],
["Tokyo is the capital of Japan.", "It is a major economic hub."]
Expand All @@ -117,17 +120,17 @@ print(f"Processed {len(response.data)} documents")
```

### Specifications
- Model: `voyage-context-3`
- Context length: 32,000 tokens per document
- Model: `voyage-context-4` (recommended), `voyage-context-3`
- Context length: 32,000 tokens per chunk
- Output dimensions: 256, 512, 1024 (default), or 2048
- Max inputs: 1,000 per request
- Max total tokens: 120,000
- Max chunks: 16,000
- Pricing: $0.18 per million tokens
- Pricing: `voyage-context-4` $0.12 / `voyage-context-3` $0.18 per million tokens

### When to Use Contextual Embeddings

**Use `voyage-context-3` when:**
**Use `voyage-context-4` when:**
- Processing long documents split into chunks
- Document structure and flow are important
- References between sections matter
Expand All @@ -143,13 +146,17 @@ print(f"Processed {len(response.data)} documents")

| Model | Best For | Context Length | Price/M Tokens |
|-------|----------|----------------|----------------|
| voyage-4 | General-purpose, multilingual | 32K | $0.06 |
| voyage-4-large | Best overall quality | 32K | $0.12 |
| voyage-4-lite | Latency-sensitive applications | 32K | $0.02 |
| voyage-3.5 | General-purpose, multilingual | 32K | $0.06 |
| voyage-3.5-lite | Latency-sensitive applications | 32K | $0.02 |
| voyage-3-large | Best overall quality | 32K | $0.18 |
| voyage-code-3 | Code retrieval and search | 32K | $0.18 |
| voyage-finance-2 | Financial documents | 32K | $0.12 |
| voyage-law-2 | Legal documents | 16K | $0.12 |
| voyage-context-3 | Contextual document embeddings | 32K | $0.18 |
| voyage-context-4 | Contextual document embeddings (recommended) | 32K/chunk | $0.12 |
| voyage-context-3 | Contextual document embeddings (previous gen) | 32K/chunk | $0.18 |

## Rerank

Expand Down
9 changes: 8 additions & 1 deletion litellm/llms/voyage/embedding/transformation_contextual.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,15 @@ def transform_embedding_request(
optional_params: dict,
headers: dict,
) -> dict:
# Voyage contextualized embeddings API accepts `inputs` as a list[str]
# (a single document's chunks). Pass list[list[str]] through unchanged
# only when the caller already provided that nested shape.
if isinstance(input, str):
inputs: Union[List[str], List[List[str]]] = [input]
else:
inputs = input
return {
"inputs": input,
"inputs": inputs,
"model": model,
**optional_params,
}
Expand Down
64 changes: 64 additions & 0 deletions litellm/model_prices_and_context_window_backup.json
Original file line number Diff line number Diff line change
Expand Up @@ -27345,6 +27345,38 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4": {
"input_cost_per_token": 6e-08,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-large": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-lite": {
"input_cost_per_token": 2e-08,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-nano": {
"input_cost_per_token": 0.0,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-code-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand All @@ -27369,6 +27401,14 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-context-4": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 120000,
"max_tokens": 120000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-finance-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand All @@ -27385,6 +27425,14 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-large-2-instruct": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 16000,
"max_tokens": 16000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-law-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand Down Expand Up @@ -27417,6 +27465,22 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-multimodal-3.5": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-multilingual-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"wandb/openai/gpt-oss-120b": {
"max_tokens": 131072,
"max_input_tokens": 131072,
Expand Down
64 changes: 64 additions & 0 deletions model_prices_and_context_window.json
Original file line number Diff line number Diff line change
Expand Up @@ -27345,6 +27345,38 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4": {
"input_cost_per_token": 6e-08,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-large": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-lite": {
"input_cost_per_token": 2e-08,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-nano": {
"input_cost_per_token": 0.0,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-code-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand All @@ -27369,6 +27401,14 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-context-4": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 120000,
"max_tokens": 120000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-finance-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand All @@ -27385,6 +27425,14 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-large-2-instruct": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 16000,
"max_tokens": 16000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-law-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand Down Expand Up @@ -27417,6 +27465,22 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-multimodal-3.5": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-multilingual-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"wandb/openai/gpt-oss-120b": {
"max_tokens": 131072,
"max_input_tokens": 131072,
Expand Down
28 changes: 28 additions & 0 deletions tests/llm_translation/test_voyage_ai.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ def test_contextual_embedding_model_detection(self):

# Test contextual model detection
assert config.is_contextualized_embeddings("voyage-context-3") is True
assert config.is_contextualized_embeddings("voyage-context-4") is True
assert config.is_contextualized_embeddings("voyage-context-2") is True
assert config.is_contextualized_embeddings("context-model") is True

Expand Down Expand Up @@ -198,6 +199,33 @@ def test_contextual_embedding_request_transformation(self):
assert transformed["model"] == "voyage-context-3"
assert transformed["encoding_format"] == "float"

def test_contextual_embedding_request_input_normalization(self):
"""Contextual API is called with list[str]; list[list[str]] only when received."""
from litellm.llms.voyage.embedding.transformation_contextual import (
VoyageContextualEmbeddingConfig,
)

config = VoyageContextualEmbeddingConfig()

# A bare string is normalized to a single-element list[str]
transformed = config.transform_embedding_request(
"voyage-context-4", "Hello", {}, {}
)
assert transformed["inputs"] == ["Hello"]

# A list[str] is passed through unchanged (not wrapped)
transformed = config.transform_embedding_request(
"voyage-context-4", ["Hello", "world"], {}, {}
)
assert transformed["inputs"] == ["Hello", "world"]

# A list[list[str]] is passed through unchanged
nested = [["Hello", "world"], ["Test"]]
transformed = config.transform_embedding_request(
"voyage-context-4", nested, {}, {}
)
assert transformed["inputs"] == nested

def test_contextual_embedding_response_transformation(self):
"""Test response transformation for contextual embeddings"""
from litellm.llms.voyage.embedding.transformation_contextual import (
Expand Down
Loading