Skip to content

feat(voyage): add voyage-context-4 and voyage-4 family models - #2

Closed
fzowl wants to merge 3 commits into
mainfrom
feat/voyage-context-4
Closed

feat(voyage): add voyage-context-4 and voyage-4 family models#2
fzowl wants to merge 3 commits into
mainfrom
feat/voyage-context-4

Conversation

@fzowl

@fzowl fzowl commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Summary

Adds VoyageAI's latest models and fills in missing pricing/context data.

New models

  • voyage/voyage-context-4 — contextualized chunk embeddings (latest), $0.12/M, 120K max total tokens
  • voyage/voyage-4 — $0.06/M, 32K context
  • voyage/voyage-4-large — $0.12/M, 32K context
  • voyage/voyage-4-lite — $0.02/M, 32K context
  • voyage/voyage-4-nano — open-weight, 32K context

Added to both model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json.

Contextual embeddings input

The Voyage contextualized embeddings API expects inputs as List[str] or List[List[str]]. transform_embedding_request now normalizes a bare str into a List[str] so the API is always called with the shape it accepts; List[str] / List[List[str]] pass through unchanged.

Docs & tests

  • Updated docs/my-website/docs/providers/voyage.md with the new models and input formats.
  • Added tests in tests/llm_translation/test_voyage_ai.py for context-4 detection and string normalization.

To double-check

  • Pricing/context values pulled from Voyage docs (pricing + embeddings + contextualized-chunk pages).
  • voyage-4-nano is open-weight with no listed per-token price; included in model map without a price override.

fzowl and others added 3 commits July 26, 2026 00:37
- Add voyage-context-4 (contextual embeddings, $0.12/M, 120k ctx)
- Add voyage-4, voyage-4-large, voyage-4-lite and voyage-multimodal-3.5
- Normalize contextual embedding input so bare str becomes List[str]
- Update Voyage provider docs and tests
Open-weight voyage-4 family member (32k ctx). No public API price;
input/output cost set to 0.0.
The Voyage contextualized embeddings API requires inputs to be a
List[List[str]]. Normalize bare string and flat List[str] inputs into a
single-document List[List[str]] so the request always matches the API
contract. Update tests and docs accordingly.
@fzowl

fzowl commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

VERDICT:APPROVED

Reviewed model config, pricing, contextual-input handling, and lineup completeness against the current Voyage docs (embeddings, pricing, contextualized-chunk, multimodal pages). Solid PR — ships correctly.

Pricing / config — all correct

Model PR value Voyage docs
voyage-4 $0.06 (6e-08) $0.06
voyage-4-large $0.12 (1.2e-07) $0.12
voyage-4-lite $0.02 (2e-08) $0.02
voyage-4-nano 0.0 (open-weight) no public price
voyage-context-4 $0.12 (1.2e-07), 120K $0.12, 120K total
voyage-multimodal-3.5 $0.12 (1.2e-07), 32K $0.12/M text

Context windows consistent with existing siblings (context-4 = 120000 matches context-3; multimodal-3.5 = 32000 matches multimodal-3). Both JSON files updated identically.

Contextual input normalization — correct

API accepts List[List[str]] for pre-chunked docs. The PR normalizes str -> [[input]] and flat List[str] -> [input], passing List[List[str]] through unchanged. Produces a valid API shape; fixes the prior bug where a bare str would be sent as "inputs": "..." (invalid). is_contextualized_embeddings("voyage-context-4") matches via "context" in model. Tests cover all three shapes + detection.

Lineup — complete for embeddings

All current Voyage embedding models present: voyage-4 family, context-4, multimodal-3.5 added; code-3 / 3.5 / context-3 / multimodal-3 / finance-2 / law-2 already existed. No missing embedding models. (Rerank models out of scope.)

Non-blocking notes

  1. voyage-multimodal-3.5 is undisclosed. Added to both JSON files but absent from the PR body's "New models" list and from voyage.md docs. Real, current model — but mention it in the description and add a docs row for consistency.
  2. Flat List[str] -> single document. Defensible choice, but Voyage treats a flat List[str] as either auto-chunking (enable_auto_chunking=True) or multiple queries. Wrapping as one document's chunks is a reasonable default and yields valid output; worth a one-line doc note so callers know the semantics.
  3. voyage-4-nano cost 0.0. Fine given no public price (open-weight, not in Voyage's free-tier list). If it ever gets a served API price, cost tracking would undercount.

None block merge.

@fzowl

fzowl commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

Correct the followings:

  • lint fails
  • the transform_embedding_request's response should provide inputs as a list[str]! (unless list[list[str]] is received, in that case leave as it is)

@fzowl

fzowl commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

Starting fresh to test the v2 flow end-to-end.

@fzowl fzowl closed this Jul 26, 2026
@fzowl
fzowl deleted the feat/voyage-context-4 branch July 26, 2026 20:44
fzowl pushed a commit that referenced this pull request Jul 29, 2026
…api (BerriAI#33110)

* feat(guardrails): support streaming text transformation in generic_guardrail_api

* chore(guardrails): address PR review feedback

* fix(guardrails): fail closed on tool-call and prefix-rewrite leaks in streaming transform

* fix(guardrails): address Bugbot review on streaming transform correctness

* fix(guardrails): coerce holdback in handler for in-process guardrails

* fix(guardrails): harden streaming transform (holdback coercion, tool-call passthrough, n>1 finish_reason)

* test(guardrails): targeted _mode_matches coverage for all guardrail_mode shapes

* fix(guardrails): inspect streamed tool calls and harden incremental_diff edge cases

* test: move ComplianceChecker mode tests to the compliance PR

* fix(guardrails): strip content from tool-call passthrough so streamed text can't bypass the transform

* fix(guardrails): four correctness fixes for incremental_diff streaming path

Four bug fixes on top of the OSS PR's incremental_diff streaming text
transformation, all inside the incremental_diff code paths only. No
existing block_only, non-streaming, or pre_call behavior is touched.

Fix #1 — Mixed content+tool_call finish_reason ordering
  _tool_call_passthrough_chunk now takes an optional finish_reason_per_choice
  map. For a choice carrying both delta.content and delta.tool_calls,
  finish_reason is stripped from the passthrough and recorded on the map so
  the final synthetic text chunk delivers it. Without this, SSE-compliant
  clients stopping at finish_reason drop the guardrailed text — defeating
  the redaction the whole feature exists for. (Greptile P1 twice, Veria.)

Fix #2 — Choice index sort in _process_streaming_transform
  indices/texts_to_check were derived from dict insertion order. For n>1
  streams where choice 1 emits before choice 0, guardrail-returned texts
  aligned to the input order mapped back to the wrong choice indices on
  write-back — wrong text goes to wrong choice. Sort raw_by_index.keys()
  up front so realignment is deterministic. (Bugbot Medium.)

Fix #3 — Cross-chunk pre-tool-call text flush
  With default streaming_sampling_rate=5, text chunks followed by a pure
  tool-call chunk carrying finish_reason='tool_calls' would emit the
  passthrough with finish_reason before any transformed text delta had
  fired. Same failure mode as fix #1 but cross-chunk. Now we flush any
  accumulated text via _round(is_final=False) BEFORE yielding the
  tool-call passthrough. (Greptile P1.)

Fix #4 — Terminator chunk for deferred finish_reason on empty mutated_text
  _build_transform_chunk returned None early when mutated_text_per_choice
  was empty. If a mixed content+tool_call chunk had deferred its
  finish_reason (via fix #1) and the guardrail then suppressed the text
  (empty return), the deferred finish_reason was never delivered. Now on
  is_final=True with empty mutated_text_per_choice, we emit a terminator
  carrying finish_reason per choice from finish_reason_per_choice.
  (Bugbot High.)

Also normalized Optional[X] → X | None across the OSS PR's added surface
via ruff UP045 autofix to keep the strict-rule gate within budget. Pure
mechanical typing style change, no semantic effect.

Regression tests for all four fixes:
- test_mixed_chunk_finish_reason_arrives_after_transformed_text (#1)
- test_text_flush_precedes_tool_call_passthrough (#3)
- test_final_finish_reason_flushed_when_guardrail_suppresses_text (#4)
- test_transform_sends_texts_sorted_by_choice_index (#2)

All fixes reachable only when streaming_transform_mode == 'incremental_diff'
is configured (via _run_incremental_transform_stream) or when a
StreamTransformSink is present (via _process_streaming_transform). Verified
scope-clean: no changes to block_only, non-streaming, pre_call, moderation,
or sibling guardrails.

---------

Co-authored-by: Marton Schneider <marton@schneider.co.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants