fix(voyage): add voyage-4 embedding models to model cost map - #32987
fix(voyage): add voyage-4 embedding models to model cost map#32987devin-ai-integration[bot] wants to merge 2 commits into
Conversation
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
Greptile SummaryAdds three missing Voyage AI voyage-4 embedding models (
Confidence Score: 4/5The cost-map additions are correct and safe; the only concern is a test that mutates a module-level variable without cleanup The JSON additions are accurate (pricing and context window verified against official Voyage AI docs) and the backup file is kept in sync. The test file contains a global-state mutation that is never restored, which can cause ordering-dependent failures in a full test run. tests/test_litellm/test_voyage_4_model_metadata.py — global state mutation and fragile url="" pattern
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds voyage-4, voyage-4-large, voyage-4-lite to the cost map with correct pricing and 32k context window — verified against official Voyage AI docs |
| litellm/model_prices_and_context_window_backup.json | Backup file updated identically to the primary cost map — entries are consistent with main |
| tests/test_litellm/test_voyage_4_model_metadata.py | New test file; the get_model_info test mutates litellm.model_cost globally without teardown and uses an empty URL as a hack to force local fallback rather than a proper test fixture |
Reviews (1): Last reviewed commit: "fix(voyage): keep backup diff minimal to..." | Re-trigger Greptile
| litellm.model_cost = litellm.get_model_cost_map(url="") | ||
| info = litellm.get_model_info(model=model) | ||
| assert info["mode"] == "embedding" | ||
| assert info["input_cost_per_token"] == input_cost |
There was a problem hiding this comment.
Unguarded global state mutation
litellm.model_cost is a module-level singleton; assigning to it here and never restoring it means any test that runs after this one in the same pytest worker will see the modified value. If, for example, the module-level cost map was already populated from the remote URL before this test ran, subsequent tests that rely on the original value could produce wrong results or false-positive passes. Use a monkeypatch fixture (or @pytest.fixture(autouse=True)) to scope the override to this test only.
| @pytest.mark.parametrize("model,input_cost", VOYAGE_4_MODELS.items()) | ||
| def test_voyage_4_get_model_info_surfaces_mode(model, input_cost): | ||
| litellm.model_cost = litellm.get_model_cost_map(url="") | ||
| info = litellm.get_model_info(model=model) | ||
| assert info["mode"] == "embedding" | ||
| assert info["input_cost_per_token"] == input_cost |
There was a problem hiding this comment.
Passing
url="" to get_model_cost_map relies on an empty-string URL causing httpx to raise an exception, which then triggers the local-fallback path. This is an indirect side-effect of error handling rather than an intentional API contract, and it could silently break if httpx changes its URL-validation behaviour. Setting LITELLM_LOCAL_MODEL_COST_MAP=True via monkeypatch.setenv is the documented, supported way to force the local backup without any network-call attempt.
| @pytest.mark.parametrize("model,input_cost", VOYAGE_4_MODELS.items()) | |
| def test_voyage_4_get_model_info_surfaces_mode(model, input_cost): | |
| litellm.model_cost = litellm.get_model_cost_map(url="") | |
| info = litellm.get_model_info(model=model) | |
| assert info["mode"] == "embedding" | |
| assert info["input_cost_per_token"] == input_cost | |
| @pytest.mark.parametrize("model,input_cost", VOYAGE_4_MODELS.items()) | |
| def test_voyage_4_get_model_info_surfaces_mode(model, input_cost, monkeypatch): | |
| monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True") | |
| monkeypatch.setattr(litellm, "model_cost", litellm.get_model_cost_map(url="")) | |
| info = litellm.get_model_info(model=model) | |
| assert info["mode"] == "embedding" | |
| assert info["input_cost_per_token"] == input_cost |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
The base branch already contains this change (verified against the official provider docs); registry consolidation is tracked in the rolling PR #38990. Closing as superseded. |
Relevant issues
Fixes #32984
Linear ticket
Pre-Submission checklist
Screenshots / Proof of Fix
voyage/voyage-4-largewas configured as an embedding model but had no entry in the cost map, so/model_group/inforeportedmode: nulland no pricing, which made anything filtering onmode === "embedding"(e.g. the UI embedding-model picker) drop it silentlyRepro was run against a live proxy with a config listing
voyage/voyage-4-large, hittingGET /model_group/info(this endpoint reads the bundled cost map, so no Voyage key is needed to demonstrate the metadata regression)Before (base
c136797805, model missing from the cost map):After (this branch
3ba74065b9):Type
🐛 Bug Fix
Changes
Adds the
voyage-4text-embedding family tomodel_prices_and_context_window.json(and the bundled backup), each withmode: embedding, a 32,000 token context window, and per-token pricing from Voyage's pricing page:voyage-4-largeat $0.12/1M,voyage-4at $0.06/1M, andvoyage-4-liteat $0.02/1MThe sibling models (
voyage-4,voyage-4-lite) were also missing and would have hit the samemode: nullbug, so they're included alongsidevoyage-4-largeAdds a regression test covering the cost-map entries and asserting
litellm.get_model_infosurfacesmode: embeddingand the correct pricing for all three, which is what/model_group/inforelies onLink to Devin session: https://app.devin.ai/sessions/6164227c42d841c7813ddc2a533b679d