v1.63.0.dev5

What's Changed

• Return signature on bedrock converse thinking + Fix {} empty dictionary on streaming + thinking by @krrishdholakia in #9023
• (Refactor) /v1/messages to follow simpler logic for Anthropic API spec by @ishaan-jaff in #9013

Full Changelog: v1.63.0-nightly...v1.63.0.dev5

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.0.dev5

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	278.42101090109276	6.116149255066882	0.0	1830	0	214.94648899999902	4750.29671599998
Aggregated	Passed ✅	250.0	278.42101090109276	6.116149255066882	0.0	1830	0	214.94648899999902	4750.29671599998

v1.63.0.dev1

Full Changelog: v1.63.0-nightly...v1.63.0.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.0.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	190.0	209.86284151312142	6.250523763835477	0.0	1867	0	163.62763399996538	3461.6653150000047
Aggregated	Passed ✅	190.0	209.86284151312142	6.250523763835477	0.0	1867	0	163.62763399996538	3461.6653150000047

v1.63.0-nightly

What's Changed

• Fix #7629 - Add tzdata package to Dockerfile (#8915) by @krrishdholakia in #9009
• Return signature on anthropic streaming + migrate to signature field instead of signature_delta [MINOR bump] by @krrishdholakia in #9021
• Support format param for specifying image type by @krrishdholakia in #9019

v1.63.0 fixes Anthropic 'thinking' response on streaming to return the signature block. Github Issue

It also moves the response structure from signature_delta to signature to be the same as Anthropic. Anthropic Docs

Diff

"message": {
    ...
    "reasoning_content": "The capital of France is Paris.",
    "thinking_blocks": [
        {
            "type": "thinking",
            "thinking": "The capital of France is Paris.",
-            "signature_delta": "EqoBCkgIARABGAIiQL2UoU0b1OHYi+..." # 👈 OLD FORMAT
+            "signature": "EqoBCkgIARABGAIiQL2UoU0b1OHYi+..." # 👈 KEY CHANGE
        }
    ]
}

Full Changelog: v1.62.4-nightly...v1.63.0-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.0-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.0-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	272.1226933173393	6.127690671911355	0.0	1834	0	217.38513100001455	3752.371346000018
Aggregated	Passed ✅	250.0	272.1226933173393	6.127690671911355	0.0	1834	0	217.38513100001455	3752.371346000018

v1.62.4-nightly

What's Changed

• Fix deepseek 'reasoning_content' error by @krrishdholakia in #8963
• (UI) Fix session handling with cookies by @ishaan-jaff in #8969
• (UI) - Improvements to session handling logic by @ishaan-jaff in #8970
• fix(route_llm_request.py): move to using common router, for client-side credentials by @krrishdholakia in #8966
• Litellm dev 03 01 2025 p2 by @krrishdholakia in #8944
• Support caching on reasoning content + other fixes by @krrishdholakia in #8973
• fix(common_utils.py): handle $id in response schema when calling vert… by @krrishdholakia in #8991
• (bug fix) - Fix Cache Health Check for Redis when redis_version is float by @ishaan-jaff in #8979
• (UI) - Security Improvement, move to JWT Auth for Admin UI Sessions by @ishaan-jaff in #8995
• Litellm dev 03 04 2025 p3 by @krrishdholakia in #8997
• fix(base_aws_llm.py): remove region name before sending in args by @krrishdholakia in #8998

Full Changelog: v1.62.1-nightly...v1.62.4-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.62.4-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	230.0	255.24015655585677	6.161171624266898	0.0	1844	0	200.43409900000597	1911.432934000004
Aggregated	Passed ✅	230.0	255.24015655585677	6.161171624266898	0.0	1844	0	200.43409900000597	1911.432934000004

v1.62.1-nightly

What's Changed

• Allow team/org filters to be searchable on the Create Key Page + Show team alias on Keys Table by @krrishdholakia in #8881
• Add created_by and updated_by fields to Keys table by @krrishdholakia in #8885
• (Proxy improvement) - Raise BadRequestError when unknown model passed in request by @ishaan-jaff in #8886
• (Improvements) use /openai/ pass through with OpenAI Ruby for Assistants API by @ishaan-jaff in #8884
• Update model path and documentation for Cerebras API call by @marscod in #8862
• docs: update sambanova docs by @jhpiedrahitao in #8875
• Update model settings data by @yurchik11 in #8871
• (security fix) - Enforce model access restrictions on Azure OpenAI route by @krrishdholakia in #8888
• Show 'user_email' on key table on UI by @krrishdholakia in #8887
• fix: ollama chat async stream error propagation by @Tomas2D in #8870
• Litellm dev 02 27 2025 p6 by @krrishdholakia in #8891
• Fix calling claude via invoke route + response_format support for claude on invoke route by @krrishdholakia in #8908
• converse_transformation: pass 'description' if set in response_format by @krrishdholakia in #8907
• Fix bedrock passing response_format: {"type": "text"} by @krrishdholakia in #8900
• (Feat) - Show Error Logs on LiteLLM UI by @ishaan-jaff in #8904
• UI - Allow admin to control default model access for internal users by @krrishdholakia in #8912
• (bug fix - patch) - don't overwrite litellm.anthropic_models when running auth checks by @ishaan-jaff in #8924
• (patch) ui remove search button on internal users tab by @ishaan-jaff in #8926
• (bug fix) - don't log messages, prompt, input in model_parameters in StandardLoggingPayload by @ishaan-jaff in #8923
• Litellm stable release notes v1 61 20 by @krrishdholakia in #8929
• (bug fix) - dd tracer, only send traces when user opts into sending dd-trace by @ishaan-jaff in #8928
• docs(index.md): add demo instance to docs for easy testing by @krrishdholakia in #8931
• (Bug fix) - don't log messages in model_parameters in StandardLoggingPayload by @ishaan-jaff in #8932
• (UI) Error Logs improvements - Store Raw proxy server request for success and failure by @ishaan-jaff in #8917
• (UI) - Allow Internal Users to View their own logs by @ishaan-jaff in #8933
• Add supports_pdf_input: true for specific Bedrock Claude models by @iwamot in #8655
• Fix prometheus metrics w/ custom metrics + Handle sending image_url as str to openai by @krrishdholakia in #8935
• fix(proxy_server.py): fix setting router redis cache, if cache enable… by @krrishdholakia in #8859
• Fix relation "dailytagspend" does not exist error by @Schnitzel in #8947

New Contributors

• @marscod made their first contribution in #8862
• @jhpiedrahitao made their first contribution in #8875
• @Tomas2D made their first contribution in #8870
• @Schnitzel made their first contribution in #8947

Full Changelog: v1.61.20.rc...v1.62.1-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.62.1-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	264.65819933898456	6.116726934503129	0.0033443012217075608	1829	1	88.08753300002081	3128.6442510000256
Aggregated	Passed ✅	250.0	264.65819933898456	6.116726934503129	0.0033443012217075608	1829	1	88.08753300002081	3128.6442510000256

v1.61.20.rc.dev1

Full Changelog: v1.61.20.rc...v1.61.20.rc.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.20.rc.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	180.0	191.79059422653788	6.202260528600653	0.0	1854	0	161.13682500002824	1614.0894129999879
Aggregated	Passed ✅	180.0	191.79059422653788	6.202260528600653	0.0	1854	0	161.13682500002824	1614.0894129999879

v1.61.20-stable

Full Changelog: v1.61.20.rc.dev1...v1.61.20-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.61.20-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	180.0	196.04185046530006	6.374061708350503	0.0	1902	0	162.48068100003366	3938.7112709999883
Aggregated	Passed ✅	180.0	196.04185046530006	6.374061708350503	0.0	1902	0	162.48068100003366	3938.7112709999883

v1.61.20.dev1

What's Changed

• Allow team/org filters to be searchable on the Create Key Page + Show team alias on Keys Table by @krrishdholakia in #8881
• Add created_by and updated_by fields to Keys table by @krrishdholakia in #8885
• (Proxy improvement) - Raise BadRequestError when unknown model passed in request by @ishaan-jaff in #8886
• (Improvements) use /openai/ pass through with OpenAI Ruby for Assistants API by @ishaan-jaff in #8884
• Update model path and documentation for Cerebras API call by @marscod in #8862
• docs: update sambanova docs by @jhpiedrahitao in #8875
• Update model settings data by @yurchik11 in #8871
• (security fix) - Enforce model access restrictions on Azure OpenAI route by @krrishdholakia in #8888
• Show 'user_email' on key table on UI by @krrishdholakia in #8887
• fix: ollama chat async stream error propagation by @Tomas2D in #8870

New Contributors

• @marscod made their first contribution in #8862
• @jhpiedrahitao made their first contribution in #8875
• @Tomas2D made their first contribution in #8870

Full Changelog: v1.61.20.rc...v1.61.20.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.20.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	240.0	254.1835915962261	6.2026122151904675	0.0	1855	0	211.9885020000538	1218.0083719999857
Aggregated	Passed ✅	240.0	254.1835915962261	6.2026122151904675	0.0	1855	0	211.9885020000538	1218.0083719999857

v1.61.20.rc

What's Changed

• Add new gpt-4.5-preview model + other updates by @krrishdholakia in #8879
• (fix) Anthropic pass through cost tracking by @ishaan-jaff in #8874
• (Bug Fix) - Accurate token counting for /anthropic/ API Routes on LiteLLM Proxy by @ishaan-jaff in #8880
• (fix) Pass through spend tracking - ensure custom_llm_provider is tracked for Vertex, Google AI Studio, Anthropic by @ishaan-jaff in #8882

Full Changelog: v1.61.13-stable...v1.61.20.rc

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.20.rc

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	271.87043339059113	6.110384289826154	0.003342660990058071	1828	1	218.3162040000184	3401.295629999993
Aggregated	Passed ✅	250.0	271.87043339059113	6.110384289826154	0.003342660990058071	1828	1	218.3162040000184	3401.295629999993

v1.61.20-nightly

What's Changed

• vertex ai anthropic thinking param support + cost calculation for vertex_ai/claude-3-7-sonnet by @krrishdholakia in #8853

Full Changelog: v1.61.19-nightly...v1.61.20-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.20-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	274.35423830943404	6.203695089639108	0.0033443100213687914	1855	1	99.50650100000757	3744.788088000007
Aggregated	Passed ✅	250.0	274.35423830943404	6.203695089639108	0.0033443100213687914	1855	1	99.50650100000757	3744.788088000007