Skip to content

feat(voyage): add voyage-context-4 + voyage-4 series models #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
fzowl wants to merge 1 commit into from
Closed

feat(voyage): add voyage-context-4 + voyage-4 series models #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 19 additions & 7 deletions docs/my-website/docs/providers/voyage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ All models listed here https://docs.voyageai.com/embeddings/#models-and-specific

| Model Name | Function Call |
|-------------------------|------------------------------------------------------------|
| voyage-4-large | `embedding(model="voyage/voyage-4-large", input)` |
| voyage-4 | `embedding(model="voyage/voyage-4", input)` |
| voyage-4-lite | `embedding(model="voyage/voyage-4-lite", input)` |
| voyage-context-4 | `embedding(model="voyage/voyage-context-4", input)` |
| voyage-context-3 | `embedding(model="voyage/voyage-context-3", input)` |
| voyage-multimodal-3.5 | `embedding(model="voyage/voyage-multimodal-3.5", input)` |
| voyage-multimodal-3 | `embedding(model="voyage/voyage-multimodal-3", input)` |
| voyage-3.5 | `embedding(model="voyage/voyage-3.5", input)` |
| voyage-3.5-lite | `embedding(model="voyage/voyage-3.5-lite", input)` |
| voyage-3-large | `embedding(model="voyage/voyage-3-large", input)` |
Expand All @@ -72,9 +79,9 @@ All models listed here https://docs.voyageai.com/embeddings/#models-and-specific
| voyage-lite-01 | `embedding(model="voyage/voyage-lite-01", input)` |
| voyage-lite-01-instruct | `embedding(model="voyage/voyage-lite-01-instruct", input)` |

## Contextual Embeddings (voyage-context-3)
## Contextual Embeddings (voyage-context-4)

VoyageAI's `voyage-context-3` model provides contextualized chunk embeddings, where each chunk is embedded with awareness of its surrounding document context. This significantly improves retrieval quality compared to standard context-agnostic embeddings.
VoyageAI's contextualized chunk embedding models (`voyage-context-4`, and the previous generation `voyage-context-3`) embed each chunk with awareness of its surrounding document context. This significantly improves retrieval quality compared to standard context-agnostic embeddings. `voyage-context-4` is the latest and recommended model.

### Key Benefits
- Chunks understand their position and role within the full document
Expand Down Expand Up @@ -117,17 +124,17 @@ print(f"Processed {len(response.data)} documents")
```

### Specifications
- Model: `voyage-context-3`
- Context length: 32,000 tokens per document
- Models: `voyage-context-4` (latest), `voyage-context-3`
- Per-chunk context window: 32,000 tokens
- Output dimensions: 256, 512, 1024 (default), or 2048
- Max inputs: 1,000 per request
- Max total tokens: 120,000
- Max chunks: 16,000
- Pricing: $0.18 per million tokens
- Pricing: `voyage-context-4` $0.12 / `voyage-context-3` $0.18 per million tokens

### When to Use Contextual Embeddings

**Use `voyage-context-3` when:**
**Use `voyage-context-4` (or `voyage-context-3`) when:**
- Processing long documents split into chunks
- Document structure and flow are important
- References between sections matter
Expand All @@ -143,13 +150,18 @@ print(f"Processed {len(response.data)} documents")

| Model | Best For | Context Length | Price/M Tokens |
|-------|----------|----------------|----------------|
| voyage-4-large | Best overall quality | 32K | $0.12 |
| voyage-4 | General-purpose | 32K | $0.06 |
| voyage-4-lite | Latency-sensitive applications | 32K | $0.02 |
| voyage-context-4 | Contextual document embeddings (latest) | 120K | $0.12 |
| voyage-context-3 | Contextual document embeddings | 120K | $0.18 |
| voyage-multimodal-3.5 | Multimodal (text + images) | 32K | $0.12 |
| voyage-3.5 | General-purpose, multilingual | 32K | $0.06 |
| voyage-3.5-lite | Latency-sensitive applications | 32K | $0.02 |
| voyage-3-large | Best overall quality | 32K | $0.18 |
| voyage-code-3 | Code retrieval and search | 32K | $0.18 |
| voyage-finance-2 | Financial documents | 32K | $0.12 |
| voyage-law-2 | Legal documents | 16K | $0.12 |
| voyage-context-3 | Contextual document embeddings | 32K | $0.18 |

## Rerank

Expand Down
22 changes: 21 additions & 1 deletion litellm/llms/voyage/embedding/transformation_contextual.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,26 @@ def validate_environment(
"Authorization": f"Bearer {api_key}",
}

@staticmethod
def _transform_input(
input: Union[AllEmbeddingInputValues, List[List[str]]],
) -> List[List[str]]:
"""
The Voyage contextualized embeddings API expects `inputs` to be a
list of lists of strings (``list[list[str]]``) - each inner
``list[str]`` is a single document made up of ordered chunks.

Callers may pass a plain string or a flat ``list[str]`` (the regular
embeddings input shape). Normalize those into ``list[list[str]]`` so
the API is always called with ``list[str]`` groups.
"""
if isinstance(input, str):
return [[input]]
if isinstance(input, list) and all(isinstance(i, str) for i in input):
# flat list[str] -> a single document of chunks
return [input] # type: ignore[list-item]
return input # type: ignore[return-value]

def transform_embedding_request(
self,
model: str,
Expand All @@ -106,7 +126,7 @@ def transform_embedding_request(
headers: dict,
) -> dict:
return {
"inputs": input,
"inputs": self._transform_input(input),
"model": model,
**optional_params,
}
Expand Down
40 changes: 40 additions & 0 deletions litellm/model_prices_and_context_window_backup.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27345,6 +27345,30 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4": {
"input_cost_per_token": 6e-08,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-large": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-4-lite": {
"input_cost_per_token": 2e-08,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-code-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand All @@ -27369,6 +27393,14 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-context-4": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 120000,
"max_tokens": 120000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-finance-2": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
Expand Down Expand Up @@ -27417,6 +27449,14 @@
"mode": "embedding",
"output_cost_per_token": 0.0
},
"voyage/voyage-multimodal-3.5": {
"input_cost_per_token": 1.2e-07,
"litellm_provider": "voyage",
"max_input_tokens": 32000,
"max_tokens": 32000,
"mode": "embedding",
"output_cost_per_token": 0.0
},
"wandb/openai/gpt-oss-120b": {
"max_tokens": 131072,
"max_input_tokens": 131072,
Expand Down
29 changes: 29 additions & 0 deletions tests/llm_translation/test_voyage_ai.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ def test_contextual_embedding_model_detection(self):
config = VoyageContextualEmbeddingConfig()

# Test contextual model detection
assert config.is_contextualized_embeddings("voyage-context-4") is True
assert config.is_contextualized_embeddings("voyage-context-3") is True
assert config.is_contextualized_embeddings("voyage-context-2") is True
assert config.is_contextualized_embeddings("context-model") is True
Expand Down Expand Up @@ -198,6 +199,34 @@ def test_contextual_embedding_request_transformation(self):
assert transformed["model"] == "voyage-context-3"
assert transformed["encoding_format"] == "float"

def test_contextual_embedding_flat_input_normalization(self):
"""Flat list[str] / str inputs are normalized to list[list[str]]"""
from litellm.llms.voyage.embedding.transformation_contextual import (
VoyageContextualEmbeddingConfig,
)

config = VoyageContextualEmbeddingConfig()

# flat list[str] -> single document of chunks
transformed = config.transform_embedding_request(
"voyage-context-4", ["Hello", "world"], {}, {}
)
assert transformed["inputs"] == [["Hello", "world"]]
assert transformed["model"] == "voyage-context-4"

# single string -> nested list[list[str]]
transformed = config.transform_embedding_request(
"voyage-context-4", "Hello", {}, {}
)
assert transformed["inputs"] == [["Hello"]]

# already nested list[list[str]] -> unchanged
nested = [["Hello", "world"], ["Test"]]
transformed = config.transform_embedding_request(
"voyage-context-4", nested, {}, {}
)
assert transformed["inputs"] == nested

def test_contextual_embedding_response_transformation(self):
"""Test response transformation for contextual embeddings"""
from litellm.llms.voyage.embedding.transformation_contextual import (
Expand Down
Loading