v1.61.19.dev1

What's Changed

• vertex ai anthropic thinking param support + cost calculation for vertex_ai/claude-3-7-sonnet by @krrishdholakia in #8853

Full Changelog: v1.61.19-nightly...v1.61.19.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.19.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	279.81049794793904	6.161760602979549	0.0	1844	0	217.23908899997468	4265.23701100001
Aggregated	Passed ✅	250.0	279.81049794793904	6.161760602979549	0.0	1844	0	217.23908899997468	4265.23701100001

v1.61.19-nightly

What's Changed

• fix(get_litellm_params.py): handle no-log being passed in via kwargs by @krrishdholakia in #8830
• fix(o_series_transformation.py): fix optional param check for o-serie… by @krrishdholakia in #8787
• chore: set ttlSecondsAfterFinished on the migration job in the litellm-helm chart by @ashwin153 in #8593
• Litellm dev bedrock anthropic 3 7 v2 by @krrishdholakia in #8843
• Mark Claude Haiku 3.5 as vision-capable by @minhduc0711 in #8840
• feat: enhance migrations job with additional configurable properties by @mknet3 in #8636
• (UI + Backend) Fix Adding Azure, Azure AI Studio models on LiteLLM by @ishaan-jaff in #8856
• fix caching on main branch by @krrishdholakia in #8858
• [Bug]: Deepseek error on proxy after upgrading to 1.61.13-stable by @ishaan-jaff in #8860

New Contributors

• @ashwin153 made their first contribution in #8593
• @mknet3 made their first contribution in #8636

Full Changelog: v1.61.17-nightly...v1.61.19-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.19-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	190.0	197.9691385267413	6.185555293319119	0.003341737057438746	1851	1	53.4934159999807	967.1408859999815
Aggregated	Passed ✅	190.0	197.9691385267413	6.185555293319119	0.003341737057438746	1851	1	53.4934159999807	967.1408859999815

v1.61.17.dev1

What's Changed

• fix(get_litellm_params.py): handle no-log being passed in via kwargs by @krrishdholakia in #8830
• fix(o_series_transformation.py): fix optional param check for o-serie… by @krrishdholakia in #8787
• chore: set ttlSecondsAfterFinished on the migration job in the litellm-helm chart by @ashwin153 in #8593

New Contributors

• @ashwin153 made their first contribution in #8593

Full Changelog: v1.61.17-nightly...v1.61.17.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	170.0	197.91492646891462	6.342832402753434	6.342832402753434	1898	1898	138.25161099998695	4268.093897000028
Aggregated	Failed ❌	170.0	197.91492646891462	6.342832402753434	6.342832402753434	1898	1898	138.25161099998695	4268.093897000028

v1.61.17-nightly

What's Changed

• (UI) Fixes for managing Internal Users by @ishaan-jaff in #8786
• Litellm contributor prs 02 24 2025 by @krrishdholakia in #8781
• (UI) Allow adding MSFT SSO on UI by @ishaan-jaff in #8779
• (UI) Minor fix, clear new team form after adding a new team by @ishaan-jaff in #8776
• [ui] Icons on navbar profile dropdown by @Mte90 in #8792
• fix(UI): model name overflow in model hub cards by @Aditya-A-G in #8749
• fix vertex_ai claude 3.7 naming by @emerzon in #8807
• (Router) - If allowed_fails or allowed_fail_policy set, use that for single deployment cooldown logic by @ishaan-jaff in #8668
• (Bug fix) - reading /parsing request body when on hypercorn by @ishaan-jaff in #8734
• (Bug fix) - running litellm proxy on wndows by @ishaan-jaff in #8735
• Anthropic model cost map updates by @krrishdholakia in #8816
• Adding Azure Phi-4 by @emerzon in #8808
• (Bug Fix) Using LiteLLM Python SDK with model=litellm_proxy/ for embedding, image_generation, transcription, speech, rerank by @ishaan-jaff in #8815
• (Bug fix) - allow using Assistants GET, DELETE on /openai pass through routes by @ishaan-jaff in #8818
• (Bug fix) dd-trace used by default on litellm proxy by @ishaan-jaff in #8817
• Adding openrouter claude-3.7-sonnet by @fengjiajie in #8826
• (UI) - Create Key flow for existing users by @ishaan-jaff in #8844

New Contributors

• @Mte90 made their first contribution in #8792
• @Aditya-A-G made their first contribution in #8749

Full Changelog: v1.61.16-nightly...v1.61.17-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	110.0	132.37492176765105	6.343856763236461	6.343856763236461	1898	1898	93.89094700003398	3315.9179240000185
Aggregated	Failed ❌	110.0	132.37492176765105	6.343856763236461	6.343856763236461	1898	1898	93.89094700003398	3315.9179240000185

v1.61.16-nightly

What's Changed

• fix: remove aws params from bedrock embedding request body (#8618) by @ishaan-jaff in #8696
• Add anthropic3-7-sonnet by @dragosMC91 in #8766
• fix incorrect variable name in reliability section of docs by @niinpatel in #8753
• Litellm contributor prs 02 24 2025 by @krrishdholakia in #8775
• Add anthropic thinking + reasoning content support by @krrishdholakia in #8778

New Contributors

• @niinpatel made their first contribution in #8753

Full Changelog: v1.61.15-nightly...v1.61.16-nightly

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.16-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	150.0	173.09079325414655	6.248259928705226	6.248259928705226	1869	1869	131.67032300003711	1529.8640780000028
Aggregated	Failed ❌	150.0	173.09079325414655	6.248259928705226	6.248259928705226	1869	1869	131.67032300003711	1529.8640780000028

v1.61.15-nightly

What's Changed

• Add cost tracking for rerank via bedrock + jina ai by @krrishdholakia in #8691
• add bedrock llama vision support + cohere / infinity rerank - 'return_documents' support by @krrishdholakia in #8684
• Correct spelling in user_management_heirarchy.md by @oaustegard in #8716
• (Feat) - UI, Allow sorting models by Created_At and all other columns on the UI by @ishaan-jaff in #8725
• (UI) Edit Model flow improvements by @ishaan-jaff in #8729
• Support arize phoenix on litellm proxy (#7756) by @krrishdholakia in #8715
• fix(amazon_deepseek_transformation.py): remove from stream o… by @krrishdholakia in #8717
• Add cohere v2/rerank support (#8421) by @krrishdholakia in #8605
• fix(proxy/_types.py): fixes issue where internal user able to escalat… by @krrishdholakia in #8740

New Contributors

• @oaustegard made their first contribution in #8716

Full Changelog: v1.61.13-nightly...v1.61.15-nightly

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.15-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	140.0	149.21232461729608	6.452882828983183	6.452882828983183	1931	1931	114.22628599996187	662.278525000005
Aggregated	Failed ❌	140.0	149.21232461729608	6.452882828983183	6.452882828983183	1931	1931	114.22628599996187	662.278525000005

v1.61.13-stable

Full Changelog: v1.61.13-nightly...v1.61.13-stable

🚨 Known Issue:

• DD Trace was enabled by default on litellm docker: #8788
• Expect a patched v1.61.13-stable with the fix

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.61.13-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	180.0	204.35323738803527	6.313934995711798	6.313934995711798	1889	1889	146.12962300003574	2180.2391240000247
Aggregated	Failed ❌	180.0	204.35323738803527	6.313934995711798	6.313934995711798	1889	1889	146.12962300003574	2180.2391240000247

v1.55.8-stable-patched

Full Changelog: v1.55.8-stable...v1.55.8-stable-patched

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm_stable_release_branch-v1.55.8-stable-patched

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	150.0	172.39167931961987	6.334948735889217	6.334948735889217	1896	1896	131.01931900001773	2316.6445349999663
Aggregated	Failed ❌	150.0	172.39167931961987	6.334948735889217	6.334948735889217	1896	1896	131.01931900001773	2316.6445349999663

v1.61.13.rc

What's Changed

• LiteLLM Contributor PRs (02/18/2025). by @krrishdholakia in #8643
• fix(utils.py): handle token counter error when invalid message passed in by @krrishdholakia in #8670
• (Bug fix) - Cache Health not working when configured with prometheus service logger by @ishaan-jaff in #8687
• (Redis fix) - use mget_non_atomic by @ishaan-jaff in #8682
• (Observability) - Add more detailed dd tracing on Proxy Auth, Bedrock Auth by @ishaan-jaff in #8693
• (Infra/DB) - Allow running older litellm version when out of sync with current state of DB by @ishaan-jaff in #8695

Full Changelog: v1.61.11-nightly...v1.61.13.rc

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.13.rc

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	150.0	182.1689693668056	6.422359014184563	6.422359014184563	1922	1922	131.7662689999679	3100.3508269999998
Aggregated	Failed ❌	150.0	182.1689693668056	6.422359014184563	6.422359014184563	1922	1922	131.7662689999679	3100.3508269999998

v1.61.13-nightly

What's Changed

• LiteLLM Contributor PRs (02/18/2025). by @krrishdholakia in #8643
• fix(utils.py): handle token counter error when invalid message passed in by @krrishdholakia in #8670
• (Bug fix) - Cache Health not working when configured with prometheus service logger by @ishaan-jaff in #8687
• (Redis fix) - use mget_non_atomic by @ishaan-jaff in #8682
• (Observability) - Add more detailed dd tracing on Proxy Auth, Bedrock Auth by @ishaan-jaff in #8693
• (Infra/DB) - Allow running older litellm version when out of sync with current state of DB by @ishaan-jaff in #8695

Full Changelog: v1.61.11-nightly...v1.61.13-nightly

## Docker Run LiteLLM Proxy

```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.13-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	150.0	177.70403054473582	6.34954513279557	6.34954513279557	1900	1900	131.07585400001653	3605.6919650000054
Aggregated	Failed ❌	150.0	177.70403054473582	6.34954513279557	6.34954513279557	1900	1900	131.07585400001653	3605.6919650000054