Skip to content

LiteLLM Minor Fixes & Improvements (10/28/2024) - #6475

Merged
23 commits merged into
mainfrom
litellm_dev_10_28_2024
Oct 30, 2024
Merged

LiteLLM Minor Fixes & Improvements (10/28/2024) #6475
23 commits merged into
mainfrom
litellm_dev_10_28_2024

Conversation

@ghost

@ghost ghost commented Oct 28, 2024 •
edited by ghost
Loading

Copy link
Copy Markdown

Title

Relevant issues

Fixes #6456
Closes #6427
Fixes #6465

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

@vercel

vercel Bot commented Oct 28, 2024 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 29, 2024 10:44pm

github-advanced-security[bot]
github-advanced-security AI found potential problems Oct 28, 2024
View reviewed changes
Comment thread tests/llm_translation/test_anthropic_completion.py Fixed
Krrish Dholakia
fix(vertex_ai/common_utils.py): parse out '$schema' when calling vert…
c4ee99c 
…ex ai

Fixes issue when trying to call vertex from vercel sdk
github-advanced-security[bot]
github-advanced-security AI found potential problems Oct 28, 2024
View reviewed changes
Comment thread tests/local_testing/test_amazing_vertex_completion.py Fixed
github-advanced-security[bot]
github-advanced-security AI found potential problems Oct 28, 2024
View reviewed changes
Comment thread tests/llm_translation/test_azure_openai.py
},
**input,
)
print(response)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

This expression logs [sensitive data (secret)](1) as clear text. This expression logs [sensitive data (secret)](2) as clear text. This expression logs [sensitive data (secret)](3) as clear text. This expression logs [sensitive data (secret)](4) as clear text. This expression logs [sensitive data (secret)](5) as clear text. This expression logs [sensitive data (secret)](6) as clear text. This expression logs [sensitive data (secret)](7) as clear text. This expression logs [sensitive data (secret)](8) as clear text. This expression logs [sensitive data (secret)](9) as clear text. This expression logs [sensitive data (secret)](10) as clear text. This expression logs [sensitive data (secret)](11) as clear text. This expression logs [sensitive data (secret)](12) as clear text. This expression logs [sensitive data (secret)](13) as clear text. This expression logs [sensitive data (secret)](14) as clear text. This expression logs [sensitive data (secret)](15) as clear text. This expression logs [sensitive data (secret)](16) as clear text. This expression logs [sensitive data (secret)](17) as clear text. This expression logs [sensitive data (secret)](18) as clear text. This expression logs [sensitive data (secret)](19) as clear text. This expression logs [sensitive data (secret)](20) as clear text. This expression logs [sensitive data (secret)](21) as clear text. This expression logs [sensitive data (secret)](22) as clear text. This expression logs [sensitive data (secret)](23) as clear text. This expression logs [sensitive data (secret)](24) as clear text. This expression logs [sensitive data (secret)](25) as clear text. This expression logs [sensitive data (secret)](26) as clear text. This expression logs [sensitive data (secret)](27) as clear text. This expression logs [sensitive data (secret)](28) as clear text. This expression logs [sensitive data (secret)](29) as clear text. This expression logs [sensitive data (secret)](30) as clear text. This expression logs [sensitive data (secret)](31) as clear text. This expression logs [sensitive data (secret)](32) as clear text. This expression logs [sensitive data (secret)](33) as clear text. This expression logs [sensitive data (secret)](34) as clear text. This expression logs [sensitive data (secret)](35) as clear text. This expression logs [sensitive data (secret)](36) as clear text. This expression logs [sensitive data (secret)](37) as clear text. This expression logs [sensitive data (secret)](38) as clear text. This expression logs [sensitive data (secret)](39) as clear text. This expression logs [sensitive data (secret)](40) as clear text. This expression logs [sensitive data (secret)](41) as clear text. This expression logs [sensitive data (secret)](42) as clear text. This expression logs [sensitive data (secret)](43) as clear text. This expression logs [sensitive data (secret)](44) as clear text. This expression logs [sensitive data (secret)](45) as clear text. This expression logs [sensitive data (secret)](46) as clear text. This expression logs [sensitive data (secret)](47) as clear text. This expression logs [sensitive data (secret)](48) as clear text. This expression logs [sensitive data (secret)](49) as clear text. This expression logs [sensitive data (secret)](50) as clear text. This expression logs [sensitive data (secret)](51) as clear text. This expression logs [sensitive data (secret)](52) as clear text. This expression logs [sensitive data (secret)](53) as clear text. This expression logs [sensitive data (secret)](54) as clear text. This expression logs [sensitive data (secret)](55) as clear text. This expression logs [sensitive data (secret)](56) as clear text. This expression logs [sensitive data (secret)](57) as clear text. This expression logs [sensitive data (secret)](58) as clear text. This expression logs [sensitive data (secret)](59) as clear text. This expression logs [sensitive data (secret)](60) as clear text. This expression logs [sensitive data (secret)](61) as clear text. This expression logs [sensitive data (secret)](62) as clear text. This expressi

Copilot Autofix

AI almost 2 years ago

To fix the problem, we need to ensure that sensitive information is not logged in clear text. Instead of printing the entire response object, we can log only the non-sensitive parts of the response or mask the sensitive information before logging.

  • Identify the lines where sensitive information is being logged.
  • Replace the direct logging of the response object with a sanitized version that masks or omits sensitive data.
  • Ensure that the fix does not alter the functionality of the tests.
Suggested changeset 1
tests/llm_translation/test_azure_openai.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/tests/llm_translation/test_azure_openai.py b/tests/llm_translation/test_azure_openai.py
--- a/tests/llm_translation/test_azure_openai.py
+++ b/tests/llm_translation/test_azure_openai.py
@@ -140,3 +140,8 @@
             )
-            print(response)
+            sanitized_response = response.copy()
+            if 'api_key' in sanitized_response:
+                sanitized_response['api_key'] = '***REDACTED***'
+            if 'Authorization' in sanitized_response:
+                sanitized_response['Authorization'] = '***REDACTED***'
+            print(sanitized_response)
         except Exception as e:
EOF
@@ -140,3 +140,8 @@
)
print(response)
sanitized_response = response.copy()
if 'api_key' in sanitized_response:
sanitized_response['api_key'] = '***REDACTED***'
if 'Authorization' in sanitized_response:
sanitized_response['Authorization'] = '***REDACTED***'
print(sanitized_response)
except Exception as e:
Copilot is powered by AI and may make mistakes. Always verify output.
github-advanced-security[bot]
github-advanced-security AI found potential problems Oct 29, 2024
View reviewed changes
Comment thread tests/llm_translation/test_anthropic_completion.py
# headers={"anthropic-beta": "computer-use-2024-10-22"},
)

print(resp)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

This expression logs [sensitive data (secret)](1) as clear text. This expression logs [sensitive data (secret)](2) as clear text. This expression logs [sensitive data (secret)](3) as clear text. This expression logs [sensitive data (secret)](4) as clear text. This expression logs [sensitive data (secret)](5) as clear text. This expression logs [sensitive data (secret)](6) as clear text. This expression logs [sensitive data (secret)](7) as clear text. This expression logs [sensitive data (secret)](8) as clear text. This expression logs [sensitive data (secret)](9) as clear text. This expression logs [sensitive data (secret)](10) as clear text. This expression logs [sensitive data (secret)](11) as clear text. This expression logs [sensitive data (secret)](12) as clear text. This expression logs [sensitive data (secret)](13) as clear text. This expression logs [sensitive data (secret)](14) as clear text. This expression logs [sensitive data (secret)](15) as clear text. This expression logs [sensitive data (secret)](16) as clear text. This expression logs [sensitive data (secret)](17) as clear text. This expression logs [sensitive data (secret)](18) as clear text. This expression logs [sensitive data (secret)](19) as clear text. This expression logs [sensitive data (secret)](20) as clear text. This expression logs [sensitive data (secret)](21) as clear text. This expression logs [sensitive data (secret)](22) as clear text. This expression logs [sensitive data (secret)](23) as clear text. This expression logs [sensitive data (secret)](24) as clear text. This expression logs [sensitive data (secret)](25) as clear text. This expression logs [sensitive data (secret)](26) as clear text. This expression logs [sensitive data (secret)](27) as clear text. This expression logs [sensitive data (secret)](28) as clear text. This expression logs [sensitive data (secret)](29) as clear text. This expression logs [sensitive data (secret)](30) as clear text. This expression logs [sensitive data (secret)](31) as clear text. This expression logs [sensitive data (secret)](32) as clear text. This expression logs [sensitive data (secret)](33) as clear text. This expression logs [sensitive data (secret)](34) as clear text. This expression logs [sensitive data (secret)](35) as clear text. This expression logs [sensitive data (secret)](36) as clear text. This expression logs [sensitive data (secret)](37) as clear text. This expression logs [sensitive data (secret)](38) as clear text. This expression logs [sensitive data (secret)](39) as clear text. This expression logs [sensitive data (secret)](40) as clear text. This expression logs [sensitive data (secret)](41) as clear text. This expression logs [sensitive data (secret)](42) as clear text. This expression logs [sensitive data (secret)](43) as clear text. This expression logs [sensitive data (secret)](44) as clear text. This expression logs [sensitive data (secret)](45) as clear text. This expression logs [sensitive data (secret)](46) as clear text. This expression logs [sensitive data (secret)](47) as clear text. This expression logs [sensitive data (secret)](48) as clear text. This expression logs [sensitive data (secret)](49) as clear text. This expression logs [sensitive data (secret)](50) as clear text. This expression logs [sensitive data (secret)](51) as clear text. This expression logs [sensitive data (secret)](52) as clear text. This expression logs [sensitive data (secret)](53) as clear text. This expression logs [sensitive data (secret)](54) as clear text. This expression logs [sensitive data (secret)](55) as clear text. This expression logs [sensitive data (secret)](56) as clear text. This expression logs [sensitive data (secret)](57) as clear text. This expression logs [sensitive data (secret)](58) as clear text. This expression logs [sensitive data (secret)](59) as clear text. This expression logs [sensitive data (secret)](60) as clear text. This expression logs [sensitive data (secret)](61) as clear text. This expression logs [sensitive data (secret)](62) as clear text. This expressi

Copilot Autofix

AI almost 2 years ago

To fix the problem, we should avoid logging the entire response object directly. Instead, we can log only non-sensitive parts of the response or use a placeholder message indicating that the response was received without exposing its contents.

  • Identify the specific lines in tests/llm_translation/test_anthropic_completion.py where sensitive information might be logged.
  • Replace the print(resp) statement with a safer logging approach that does not expose sensitive information.
  • Ensure that the fix does not alter the existing functionality of the tests.
Suggested changeset 1
tests/llm_translation/test_anthropic_completion.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/tests/llm_translation/test_anthropic_completion.py b/tests/llm_translation/test_anthropic_completion.py
--- a/tests/llm_translation/test_anthropic_completion.py
+++ b/tests/llm_translation/test_anthropic_completion.py
@@ -557,3 +557,3 @@
 
-    print(resp)
+    print("[INFO] Response received successfully.")
 
EOF
@@ -557,3 +557,3 @@

print(resp)
print("[INFO] Response received successfully.")

Copilot is powered by AI and may make mistakes. Always verify output.
Krish Dholakia
LiteLLM Minor Fixes & Improvements (10/24/2024) (#6441)
0cc883d 
* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes #6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes #6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes #6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes #6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes #6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits
github-advanced-security[bot]
github-advanced-security AI found potential problems Oct 29, 2024
View reviewed changes
Comment thread tests/local_testing/test_function_calling.py
]
for _ in range(2):
resp = completion(model=model, messages=messages, tools=tools)
print(resp)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information

This expression logs [sensitive data (secret)](1) as clear text. This expression logs [sensitive data (secret)](2) as clear text. This expression logs [sensitive data (secret)](3) as clear text. This expression logs [sensitive data (secret)](4) as clear text. This expression logs [sensitive data (secret)](5) as clear text. This expression logs [sensitive data (secret)](6) as clear text. This expression logs [sensitive data (secret)](7) as clear text. This expression logs [sensitive data (secret)](8) as clear text. This expression logs [sensitive data (secret)](9) as clear text. This expression logs [sensitive data (secret)](10) as clear text. This expression logs [sensitive data (secret)](11) as clear text. This expression logs [sensitive data (secret)](12) as clear text. This expression logs [sensitive data (secret)](13) as clear text. This expression logs [sensitive data (secret)](14) as clear text. This expression logs [sensitive data (secret)](15) as clear text. This expression logs [sensitive data (secret)](16) as clear text. This expression logs [sensitive data (secret)](17) as clear text. This expression logs [sensitive data (secret)](18) as clear text. This expression logs [sensitive data (secret)](19) as clear text. This expression logs [sensitive data (secret)](20) as clear text. This expression logs [sensitive data (secret)](21) as clear text. This expression logs [sensitive data (secret)](22) as clear text. This expression logs [sensitive data (secret)](23) as clear text. This expression logs [sensitive data (secret)](24) as clear text. This expression logs [sensitive data (secret)](25) as clear text. This expression logs [sensitive data (secret)](26) as clear text. This expression logs [sensitive data (secret)](27) as clear text. This expression logs [sensitive data (secret)](28) as clear text. This expression logs [sensitive data (secret)](29) as clear text. This expression logs [sensitive data (secret)](30) as clear text. This expression logs [sensitive data (secret)](31) as clear text. This expression logs [sensitive data (secret)](32) as clear text. This expression logs [sensitive data (secret)](33) as clear text. This expression logs [sensitive data (secret)](34) as clear text. This expression logs [sensitive data (secret)](35) as clear text. This expression logs [sensitive data (secret)](36) as clear text. This expression logs [sensitive data (secret)](37) as clear text. This expression logs [sensitive data (secret)](38) as clear text. This expression logs [sensitive data (secret)](39) as clear text. This expression logs [sensitive data (secret)](40) as clear text. This expression logs [sensitive data (secret)](41) as clear text. This expression logs [sensitive data (secret)](42) as clear text. This expression logs [sensitive data (secret)](43) as clear text. This expression logs [sensitive data (secret)](44) as clear text. This expression logs [sensitive data (secret)](45) as clear text. This expression logs [sensitive data (secret)](46) as clear text. This expression logs [sensitive data (secret)](47) as clear text. This expression logs [sensitive data (secret)](48) as clear text. This expression logs [sensitive data (secret)](49) as clear text. This expression logs [sensitive data (secret)](50) as clear text. This expression logs [sensitive data (secret)](51) as clear text. This expression logs [sensitive data (secret)](52) as clear text. This expression logs [sensitive data (secret)](53) as clear text. This expression logs [sensitive data (secret)](54) as clear text. This expression logs [sensitive data (secret)](55) as clear text. This expression logs [sensitive data (secret)](56) as clear text. This expression logs [sensitive data (secret)](57) as clear text. This expression logs [sensitive data (secret)](58) as clear text. This expression logs [sensitive data (secret)](59) as clear text. This expression logs [sensitive data (secret)](60) as clear text. This expression logs [sensitive data (secret)](61) as clear text. This expression logs [sensitive data (secret)](62) as clear text. This expressi

Copilot Autofix

AI almost 2 years ago

To fix the problem, we should avoid logging the entire resp object directly. Instead, we can log only non-sensitive parts of the response or use a logging mechanism that redacts sensitive information.

  1. Identify the non-sensitive parts of the resp object that can be safely logged.
  2. Modify the print statement to log only the non-sensitive parts.
  3. Ensure that any sensitive information is either redacted or not logged at all.
Suggested changeset 1
tests/local_testing/test_function_calling.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/tests/local_testing/test_function_calling.py b/tests/local_testing/test_function_calling.py
--- a/tests/local_testing/test_function_calling.py
+++ b/tests/local_testing/test_function_calling.py
@@ -611,3 +611,3 @@
         resp = completion(model=model, messages=messages, tools=tools)
-        print(resp)
+        print(f"Completion response: {resp.choices}")  # Log only non-sensitive parts
 
EOF
@@ -611,3 +611,3 @@
resp = completion(model=model, messages=messages, tools=tools)
print(resp)
print(f"Completion response: {resp.choices}") # Log only non-sensitive parts

Copilot is powered by AI and may make mistakes. Always verify output.
@ghost ghost changed the title fix(anthropic/chat/transformation.py): support anthropic disable_para… LiteLLM Minor Fixes & Improvements (10/28/2024) Oct 29, 2024
@ghost
ghost merged commit 6b9be50 into main Oct 30, 2024
ishaan-jaff added a commit that referenced this pull request Oct 30, 2024
Krish Dholakia @ishaan-jaff
LiteLLM Minor Fixes & Improvements (10/28/2024) (#6475)
e716220 
* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes #6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes #6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes #6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes #6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes #6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes #6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes #6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes #6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
ishaan-jaff added a commit that referenced this pull request Oct 30, 2024
Krish Dholakia @ishaan-jaff
LiteLLM Minor Fixes & Improvements (10/28/2024) (#6475)
f765e79 
* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes #6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes #6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes #6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes #6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes #6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes #6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes #6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes #6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
ishaan-jaff added a commit that referenced this pull request Oct 30, 2024
@ishaan-jaff Krish Dholakia
(UI) Fix viewing members, keys in a team + added testing (#6514)
cf877c8 
* fix listing teams on ui

* LiteLLM Minor Fixes & Improvements (10/28/2024)  (#6475)

* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes #6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes #6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes #6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes #6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes #6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes #6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes #6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes #6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
ishaan-jaff added a commit that referenced this pull request Oct 30, 2024
@ishaan-jaff Krish Dholakia
(UI) Fix viewing members, keys in a team + added testing (#6514)
cd56705 
* fix listing teams on ui

* LiteLLM Minor Fixes & Improvements (10/28/2024)  (#6475)

* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes #6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes #6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes #6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes #6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes #6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes #6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes #6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes #6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
@ghost ghost mentioned this pull request Nov 1, 2024
Closed
ghost pushed a commit that referenced this pull request Nov 1, 2024
@ishaan-jaff Krish Dholakia
(UI) Fix viewing members, keys in a team + added testing (#6514)
0477a7e 
* fix listing teams on ui

* LiteLLM Minor Fixes & Improvements (10/28/2024)  (#6475)

* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes #6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes #6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes #6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes #6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes #6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes #6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes #6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes #6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
ghost pushed a commit that referenced this pull request Nov 1, 2024
Krish Dholakia @ishaan-jaff @dbczumar
LiteLLM Minor Fixes & Improvements (10/30/2024) (#6519)
f79365d 
* refactor: move gemini translation logic inside the transformation.py file

easier to isolate the gemini translation logic

* fix(gemini-transformation): support multiple tool calls in message body

Merges https://github.com/BerriAI/litellm/pull/6487/files

* test(test_vertex.py): add remaining tests from #6487

* fix(gemini-transformation): return tool calls for multiple tool calls

* fix: support passing logprobs param for vertex + gemini

* feat(vertex_ai): add logprobs support for gemini calls

* fix(anthropic/chat/transformation.py): fix disable parallel tool use flag

* fix: fix linting error

* fix(_logging.py): log stacktrace information in json logs

Closes #6497

* fix(utils.py): fix mem leak for async stream + completion

Uses a global executor pool instead of creating a new thread on each request

Fixes #6404

* fix(factory.py): handle tool call + content in assistant message for bedrock

* fix: fix import

* fix(factory.py): maintain support for content as a str in assistant response

* fix: fix import

* test: cleanup test

* fix(vertex_and_google_ai_studio/): return none for content if no str value

* test: retry flaky tests

* (UI) Fix viewing members, keys in a team + added testing  (#6514)

* fix listing teams on ui

* LiteLLM Minor Fixes & Improvements (10/28/2024)  (#6475)

* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes #6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes #6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes #6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes #6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes #6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes #6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes #6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes #6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* ui new build

* Add retry strat (#6520)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* (fix) slack alerting - don't spam the failed cost tracking alert for the same model  (#6543)

* fix use failing_model as cache key for failed_tracking_alert

* fix use standard logging payload for getting response cost

* fix  kwargs.get("response_cost")

* fix getting response cost

* (feat) add XAI ChatCompletion Support  (#6373)

* init commit for XAI

* add full logic for xai chat completion

* test_completion_xai

* docs xAI

* add xai/grok-beta

* test_xai_chat_config_get_openai_compatible_provider_info

* test_xai_chat_config_map_openai_params

* add xai streaming test

---------

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
@ishaan-berri
ishaan-berri deleted the litellm_dev_10_28_2024 branch March 26, 2026 21:52
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
Krish Dholakia @ishaan-jaff
LiteLLM Minor Fixes & Improvements (10/28/2024) (BerriAI#6475)
9972014 
* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes BerriAI#6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes BerriAI#6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes BerriAI#6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (BerriAI#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes BerriAI#6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes BerriAI#6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes BerriAI#6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes BerriAI#6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes BerriAI#6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (BerriAI#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (BerriAI#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (BerriAI#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
@ishaan-jaff Krish Dholakia
(UI) Fix viewing members, keys in a team + added testing (BerriAI#6514)
04e188e 
* fix listing teams on ui

* LiteLLM Minor Fixes & Improvements (10/28/2024)  (BerriAI#6475)

* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes BerriAI#6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes BerriAI#6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes BerriAI#6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (BerriAI#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes BerriAI#6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes BerriAI#6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes BerriAI#6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes BerriAI#6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes BerriAI#6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (BerriAI#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (BerriAI#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (BerriAI#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
Krish Dholakia @ishaan-jaff @dbczumar
LiteLLM Minor Fixes & Improvements (10/30/2024) (BerriAI#6519)
fbfb95a 
* refactor: move gemini translation logic inside the transformation.py file

easier to isolate the gemini translation logic

* fix(gemini-transformation): support multiple tool calls in message body

Merges https://github.com/BerriAI/litellm/pull/6487/files

* test(test_vertex.py): add remaining tests from BerriAI#6487

* fix(gemini-transformation): return tool calls for multiple tool calls

* fix: support passing logprobs param for vertex + gemini

* feat(vertex_ai): add logprobs support for gemini calls

* fix(anthropic/chat/transformation.py): fix disable parallel tool use flag

* fix: fix linting error

* fix(_logging.py): log stacktrace information in json logs

Closes BerriAI#6497

* fix(utils.py): fix mem leak for async stream + completion

Uses a global executor pool instead of creating a new thread on each request

Fixes BerriAI#6404

* fix(factory.py): handle tool call + content in assistant message for bedrock

* fix: fix import

* fix(factory.py): maintain support for content as a str in assistant response

* fix: fix import

* test: cleanup test

* fix(vertex_and_google_ai_studio/): return none for content if no str value

* test: retry flaky tests

* (UI) Fix viewing members, keys in a team + added testing  (BerriAI#6514)

* fix listing teams on ui

* LiteLLM Minor Fixes & Improvements (10/28/2024)  (BerriAI#6475)

* fix(anthropic/chat/transformation.py): support anthropic disable_parallel_tool_use param

Fixes BerriAI#6456

* feat(anthropic/chat/transformation.py): support anthropic computer tool use

Closes BerriAI#6427

* fix(vertex_ai/common_utils.py): parse out '$schema' when calling vertex ai

Fixes issue when trying to call vertex from vercel sdk

* fix(main.py): add 'extra_headers' support for azure on all translation endpoints

Fixes BerriAI#6465

* fix: fix linting errors

* fix(transformation.py): handle no beta headers for anthropic

* test: cleanup test

* fix: fix linting error

* fix: fix linting errors

* fix: fix linting errors

* fix(transformation.py): handle dummy tool call

* fix(main.py): fix linting error

* fix(azure.py): pass required param

* LiteLLM Minor Fixes & Improvements (10/24/2024) (BerriAI#6441)

* fix(azure.py): handle /openai/deployment in azure api base

* fix(factory.py): fix faulty anthropic tool result translation check

Fixes BerriAI#6422

* fix(gpt_transformation.py): add support for parallel_tool_calls to azure

Fixes BerriAI#6440

* fix(factory.py): support anthropic prompt caching for tool results

* fix(vertex_ai/common_utils): don't pop non-null required field

Fixes BerriAI#6426

* feat(vertex_ai.py): support code_execution tool call for vertex ai + gemini

Closes BerriAI#6434

* build(model_prices_and_context_window.json): Add 'supports_assistant_prefill' for bedrock claude-3-5-sonnet v2 models

Closes BerriAI#6437

* fix(types/utils.py): fix linting

* test: update test to include required fields

* test: fix test

* test: handle flaky test

* test: remove e2e test - hitting gemini rate limits

* Litellm dev 10 26 2024 (BerriAI#6472)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected  (BerriAI#6471)

* test test_dual_cache_get_set

* unit testing for dual cache

* fix async_set_cache_sadd

* test_dual_cache_local_only

* redis otel tracing + async support for latency routing (BerriAI#6452)

* docs(exception_mapping.md): add missing exception types

Fixes Aider-AI/aider#2120 (comment)

* fix(main.py): register custom model pricing with specific key

Ensure custom model pricing is registered to the specific model+provider key combination

* test: make testing more robust for custom pricing

* fix(redis_cache.py): instrument otel logging for sync redis calls

ensures complete coverage for all redis cache calls

* refactor: pass parent_otel_span for redis caching calls in router

allows for more observability into what calls are causing latency issues

* test: update tests with new params

* refactor: ensure e2e otel tracing for router

* refactor(router.py): add more otel tracing acrosss router

catch all latency issues for router requests

* fix: fix linting error

* fix(router.py): fix linting error

* fix: fix test

* test: fix tests

* fix(dual_cache.py): pass ttl to redis cache

* fix: fix param

* fix(dual_cache.py): set default value for parent_otel_span

* fix(transformation.py): support 'response_format' for anthropic calls

* fix(transformation.py): check for cache_control inside 'function' block

* fix: fix linting error

* fix: fix linting errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* ui new build

* Add retry strat (BerriAI#6520)

Signed-off-by: dbczumar <corey.zumar@databricks.com>

* (fix) slack alerting - don't spam the failed cost tracking alert for the same model  (BerriAI#6543)

* fix use failing_model as cache key for failed_tracking_alert

* fix use standard logging payload for getting response cost

* fix  kwargs.get("response_cost")

* fix getting response cost

* (feat) add XAI ChatCompletion Support  (BerriAI#6373)

* init commit for XAI

* add full logic for xai chat completion

* test_completion_xai

* docs xAI

* add xai/grok-beta

* test_xai_chat_config_get_openai_compatible_provider_info

* test_xai_chat_config_map_openai_params

* add xai streaming test

---------

Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

2 participants

@github-advanced-security @ishaan-jaff