feat(voyage): add voyage-context-4 and voyage-4 family models - #2
Conversation
- Add voyage-context-4 (contextual embeddings, $0.12/M, 120k ctx) - Add voyage-4, voyage-4-large, voyage-4-lite and voyage-multimodal-3.5 - Normalize contextual embedding input so bare str becomes List[str] - Update Voyage provider docs and tests
Open-weight voyage-4 family member (32k ctx). No public API price; input/output cost set to 0.0.
The Voyage contextualized embeddings API requires inputs to be a List[List[str]]. Normalize bare string and flat List[str] inputs into a single-document List[List[str]] so the request always matches the API contract. Update tests and docs accordingly.
|
VERDICT:APPROVED Reviewed model config, pricing, contextual-input handling, and lineup completeness against the current Voyage docs (embeddings, pricing, contextualized-chunk, multimodal pages). Solid PR — ships correctly. Pricing / config — all correct
Context windows consistent with existing siblings (context-4 = 120000 matches context-3; multimodal-3.5 = 32000 matches multimodal-3). Both JSON files updated identically. Contextual input normalization — correctAPI accepts Lineup — complete for embeddingsAll current Voyage embedding models present: voyage-4 family, context-4, multimodal-3.5 added; code-3 / 3.5 / context-3 / multimodal-3 / finance-2 / law-2 already existed. No missing embedding models. (Rerank models out of scope.) Non-blocking notes
None block merge. |
|
Correct the followings:
|
|
Starting fresh to test the v2 flow end-to-end. |
…api (BerriAI#33110) * feat(guardrails): support streaming text transformation in generic_guardrail_api * chore(guardrails): address PR review feedback * fix(guardrails): fail closed on tool-call and prefix-rewrite leaks in streaming transform * fix(guardrails): address Bugbot review on streaming transform correctness * fix(guardrails): coerce holdback in handler for in-process guardrails * fix(guardrails): harden streaming transform (holdback coercion, tool-call passthrough, n>1 finish_reason) * test(guardrails): targeted _mode_matches coverage for all guardrail_mode shapes * fix(guardrails): inspect streamed tool calls and harden incremental_diff edge cases * test: move ComplianceChecker mode tests to the compliance PR * fix(guardrails): strip content from tool-call passthrough so streamed text can't bypass the transform * fix(guardrails): four correctness fixes for incremental_diff streaming path Four bug fixes on top of the OSS PR's incremental_diff streaming text transformation, all inside the incremental_diff code paths only. No existing block_only, non-streaming, or pre_call behavior is touched. Fix #1 — Mixed content+tool_call finish_reason ordering _tool_call_passthrough_chunk now takes an optional finish_reason_per_choice map. For a choice carrying both delta.content and delta.tool_calls, finish_reason is stripped from the passthrough and recorded on the map so the final synthetic text chunk delivers it. Without this, SSE-compliant clients stopping at finish_reason drop the guardrailed text — defeating the redaction the whole feature exists for. (Greptile P1 twice, Veria.) Fix #2 — Choice index sort in _process_streaming_transform indices/texts_to_check were derived from dict insertion order. For n>1 streams where choice 1 emits before choice 0, guardrail-returned texts aligned to the input order mapped back to the wrong choice indices on write-back — wrong text goes to wrong choice. Sort raw_by_index.keys() up front so realignment is deterministic. (Bugbot Medium.) Fix #3 — Cross-chunk pre-tool-call text flush With default streaming_sampling_rate=5, text chunks followed by a pure tool-call chunk carrying finish_reason='tool_calls' would emit the passthrough with finish_reason before any transformed text delta had fired. Same failure mode as fix #1 but cross-chunk. Now we flush any accumulated text via _round(is_final=False) BEFORE yielding the tool-call passthrough. (Greptile P1.) Fix #4 — Terminator chunk for deferred finish_reason on empty mutated_text _build_transform_chunk returned None early when mutated_text_per_choice was empty. If a mixed content+tool_call chunk had deferred its finish_reason (via fix #1) and the guardrail then suppressed the text (empty return), the deferred finish_reason was never delivered. Now on is_final=True with empty mutated_text_per_choice, we emit a terminator carrying finish_reason per choice from finish_reason_per_choice. (Bugbot High.) Also normalized Optional[X] → X | None across the OSS PR's added surface via ruff UP045 autofix to keep the strict-rule gate within budget. Pure mechanical typing style change, no semantic effect. Regression tests for all four fixes: - test_mixed_chunk_finish_reason_arrives_after_transformed_text (#1) - test_text_flush_precedes_tool_call_passthrough (#3) - test_final_finish_reason_flushed_when_guardrail_suppresses_text (#4) - test_transform_sends_texts_sorted_by_choice_index (#2) All fixes reachable only when streaming_transform_mode == 'incremental_diff' is configured (via _run_incremental_transform_stream) or when a StreamTransformSink is present (via _process_streaming_transform). Verified scope-clean: no changes to block_only, non-streaming, pre_call, moderation, or sibling guardrails. --------- Co-authored-by: Marton Schneider <marton@schneider.co.nl>
Summary
Adds VoyageAI's latest models and fills in missing pricing/context data.
New models
voyage/voyage-context-4— contextualized chunk embeddings (latest), $0.12/M, 120K max total tokensvoyage/voyage-4— $0.06/M, 32K contextvoyage/voyage-4-large— $0.12/M, 32K contextvoyage/voyage-4-lite— $0.02/M, 32K contextvoyage/voyage-4-nano— open-weight, 32K contextAdded to both
model_prices_and_context_window.jsonandlitellm/model_prices_and_context_window_backup.json.Contextual embeddings input
The Voyage contextualized embeddings API expects
inputsasList[str]orList[List[str]].transform_embedding_requestnow normalizes a barestrinto aList[str]so the API is always called with the shape it accepts;List[str]/List[List[str]]pass through unchanged.Docs & tests
docs/my-website/docs/providers/voyage.mdwith the new models and input formats.tests/llm_translation/test_voyage_ai.pyfor context-4 detection and string normalization.To double-check
voyage-4-nanois open-weight with no listed per-token price; included in model map without a price override.