Repository navigation
feat(sambanova): add MiniMax-M3, update model limits, support extra chat params - #44651
jhpiedrahitao wants to merge 6 commits into
Conversation
|
| assert info["max_input_tokens"] == 1048576 | ||
| assert info["input_cost_per_token"] == 6e-07 | ||
| assert info["output_cost_per_token"] == 2.4e-06 |
There was a problem hiding this comment.
Tests pin outside facts These assertions hardcode vendor prices and limits; others require retired models to remain absent. Repository rules require owned invariants or dated sources before merging
Context Used: AGENTS.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
|
docs PR BerriAI/litellm-docs#2092 |
TLDR
Problem this solves:
sambanova/MiniMax-M3is missing from the model map, so it has no cost or limitsseed,n, penalties,reasoning_effort) get rejected or droppedX-Integration-Sourceheader is not sent, so SambaNova can't tell LiteLLM traffic apartHow it solves it:
sambanova/MiniMax-M3(1M context, tools, reasoning, response schema)/v1/modelsreasoning_effortonly on reasoning modelsX-Integration-Source(defaultlitellm), overridable per request or by env varIntentional product change:
sambanova/DeepSeek-R1andsambanova/Llama-4-Maverick-17B-128E-Instructare no longer in the model map, so cost and limit lookups for them stop working.User Flow
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Setup:
SAMBANOVA_API_KEYset; proxy started withpython litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debugand a
sambanova/MiniMax-M3entry in the config.Before
MiniMax-M3 chat completion with seed
curl http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"sambanova/MiniMax-M3","messages":[{"role":"user","content":"hi"}],"seed":7}'Integration header
--detailed_debug, inspect the outgoing request headersAfter
MiniMax-M3 chat completion with seed
Integration header
--detailed_debug, inspect the outgoing request headersSAMBANOVA_INTEGRATION_SOURCE=my-frameworksetType
🆕 New Feature
Caveats (if any)
QA runbook
Final Attestation