Skip to content

fix(voyage): add voyage-4 embedding models to model cost map - #32987

Closed
devin-ai-integration[bot] wants to merge 2 commits into
litellm_internal_stagingfrom
litellm_voyage_4_model_prices
Closed

fix(voyage): add voyage-4 embedding models to model cost map#32987
devin-ai-integration[bot] wants to merge 2 commits into
litellm_internal_stagingfrom
litellm_voyage_4_model_prices

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Relevant issues

Fixes #32984

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

voyage/voyage-4-large was configured as an embedding model but had no entry in the cost map, so /model_group/info reported mode: null and no pricing, which made anything filtering on mode === "embedding" (e.g. the UI embedding-model picker) drop it silently

Repro was run against a live proxy with a config listing voyage/voyage-4-large, hitting GET /model_group/info (this endpoint reads the bundled cost map, so no Voyage key is needed to demonstrate the metadata regression)

Before (base c136797805, model missing from the cost map):

$ curl -s http://localhost:4000/model_group/info | jq '.data[] | select(.model_group=="voyage-4-large")'
{
  "model_group": "voyage-4-large",
  "providers": ["voyage"],
  "max_input_tokens": null,
  "input_cost_per_token": 0.0,
  "mode": null,
  ...
}

After (this branch 3ba74065b9):

$ curl -s http://localhost:4000/model_group/info | jq '.data[] | select(.model_group=="voyage-4-large")'
{
  "model_group": "voyage-4-large",
  "providers": ["voyage"],
  "max_input_tokens": 32000.0,
  "input_cost_per_token": 1.2e-07,
  "mode": "embedding",
  ...
}

Type

🐛 Bug Fix

Changes

Adds the voyage-4 text-embedding family to model_prices_and_context_window.json (and the bundled backup), each with mode: embedding, a 32,000 token context window, and per-token pricing from Voyage's pricing page: voyage-4-large at $0.12/1M, voyage-4 at $0.06/1M, and voyage-4-lite at $0.02/1M

The sibling models (voyage-4, voyage-4-lite) were also missing and would have hit the same mode: null bug, so they're included alongside voyage-4-large

Adds a regression test covering the cost-map entries and asserting litellm.get_model_info surfaces mode: embedding and the correct pricing for all three, which is what /model_group/info relies on

Link to Devin session: https://app.devin.ai/sessions/6164227c42d841c7813ddc2a533b679d

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds three missing Voyage AI voyage-4 embedding models (voyage-4, voyage-4-large, voyage-4-lite) to both the primary and backup cost maps, fixing a bug where these models reported mode: null and no pricing via /model_group/info. A regression test is included to verify the cost-map entries, get_model_info output, and consistency between the two JSON files.

  • Pricing (1.2e-07, 6e-08, 2e-08 per token) and 32k context window match the official Voyage AI pricing page.
  • The new test mutates litellm.model_cost as a plain assignment without a teardown fixture, which leaves global state dirty for subsequent tests in the same session.

Confidence Score: 4/5

The cost-map additions are correct and safe; the only concern is a test that mutates a module-level variable without cleanup

The JSON additions are accurate (pricing and context window verified against official Voyage AI docs) and the backup file is kept in sync. The test file contains a global-state mutation that is never restored, which can cause ordering-dependent failures in a full test run.

tests/test_litellm/test_voyage_4_model_metadata.py — global state mutation and fragile url="" pattern

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds voyage-4, voyage-4-large, voyage-4-lite to the cost map with correct pricing and 32k context window — verified against official Voyage AI docs
litellm/model_prices_and_context_window_backup.json Backup file updated identically to the primary cost map — entries are consistent with main
tests/test_litellm/test_voyage_4_model_metadata.py New test file; the get_model_info test mutates litellm.model_cost globally without teardown and uses an empty URL as a hack to force local fallback rather than a proper test fixture

Reviews (1): Last reviewed commit: "fix(voyage): keep backup diff minimal to..." | Re-trigger Greptile

Comment on lines +34 to +37
litellm.model_cost = litellm.get_model_cost_map(url="")
info = litellm.get_model_info(model=model)
assert info["mode"] == "embedding"
assert info["input_cost_per_token"] == input_cost

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Unguarded global state mutation

litellm.model_cost is a module-level singleton; assigning to it here and never restoring it means any test that runs after this one in the same pytest worker will see the modified value. If, for example, the module-level cost map was already populated from the remote URL before this test ran, subsequent tests that rely on the original value could produce wrong results or false-positive passes. Use a monkeypatch fixture (or @pytest.fixture(autouse=True)) to scope the override to this test only.

Comment on lines +32 to +37
@pytest.mark.parametrize("model,input_cost", VOYAGE_4_MODELS.items())
def test_voyage_4_get_model_info_surfaces_mode(model, input_cost):
litellm.model_cost = litellm.get_model_cost_map(url="")
info = litellm.get_model_info(model=model)
assert info["mode"] == "embedding"
assert info["input_cost_per_token"] == input_cost

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Passing url="" to get_model_cost_map relies on an empty-string URL causing httpx to raise an exception, which then triggers the local-fallback path. This is an indirect side-effect of error handling rather than an intentional API contract, and it could silently break if httpx changes its URL-validation behaviour. Setting LITELLM_LOCAL_MODEL_COST_MAP=True via monkeypatch.setenv is the documented, supported way to force the local backup without any network-call attempt.

Suggested change
@pytest.mark.parametrize("model,input_cost", VOYAGE_4_MODELS.items())
def test_voyage_4_get_model_info_surfaces_mode(model, input_cost):
litellm.model_cost = litellm.get_model_cost_map(url="")
info = litellm.get_model_info(model=model)
assert info["mode"] == "embedding"
assert info["input_cost_per_token"] == input_cost
@pytest.mark.parametrize("model,input_cost", VOYAGE_4_MODELS.items())
def test_voyage_4_get_model_info_surfaces_mode(model, input_cost, monkeypatch):
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
monkeypatch.setattr(litellm, "model_cost", litellm.get_model_cost_map(url=""))
info = litellm.get_model_info(model=model)
assert info["mode"] == "embedding"
assert info["input_cost_per_token"] == input_cost

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_voyage_4_model_prices (3ba7406) with litellm_internal_staging (c136797)

Open in CodSpeed

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

The base branch already contains this change (verified against the official provider docs); registry consolidation is tracked in the rolling PR #38990. Closing as superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

voyage-4-large missing from model_prices_and_context_window.json, so mode/pricing is unset

1 participant