Litellm litellm gemini penalty params not supported #15503
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
4 tasks
Sameerlite
added a commit
that referenced
this pull request
Oct 17, 2025
ishaan-jaff
added a commit
that referenced
this pull request
Oct 18, 2025
* Implement fix for thinking_blocks and converse API calls This fixes Claude's models via the Converse API, which should also fix Claude Code. * Add thinking literal * Fix mypy issues * Type fix for redacted thinking * Add voyage model integration in sagemaker * Add config file logic * Use already exiting voyage transformation * refactor code as per comments * fix merge error * refactor code as per comments * refactor code as per comments * UI new build * [Fix] router - regression when adding/removing models (#15451) * fix(router): update model_name_to_deployment_indices on deployment removal When a deployment is deleted, the model_name_to_deployment_indices map was not being updated, causing stale index references. This could lead to incorrect routing behavior when deployments with the same model_name were dynamically removed. Changes: - Update _update_deployment_indices_after_removal to maintain model_name_to_deployment_indices mapping - Remove deleted indices and decrement indices greater than removed index - Clean up empty entries when no deployments remain for a model name - Update test to verify proper index shifting and cleanup behavior * fix(router): remove redundant index building during initialization Remove duplicate index building operations that were causing unnecessary work during router initialization: 1. Removed redundant `_build_model_id_to_deployment_index_map` call in __init__ - `set_model_list` already builds all indices from scratch 2. Removed redundant `_build_model_name_index` call at end of `set_model_list` - the index is already built incrementally via `_create_deployment` -> `_add_model_to_list_and_index_map` Both indices (model_id_to_deployment_index_map and model_name_to_deployment_indices) are properly maintained as lookup indexes through existing helper methods. This change eliminates O(N) duplicate work during initialization without any behavioral changes. The indices continue to be correctly synchronized with model_list on all operations (add/remove/upsert). * fix(prometheus): Fix Prometheus metric collection in a multi-workers environment (#14929) Co-authored-by: sotazhang <[email protected]> * Add tiered pricing and cost calculation for xai * Use generic cost calculator * Resolve conflicts in generated HTML files * Remove penalty params as supported params for gemini preview model (#15503) * fix conversion of thinking block * add application level encryption in SQS (#15512) * docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * add application level encryption in SQS * add application level encryption in SQS --------- Co-authored-by: Krrish Dholakia <[email protected]> Co-authored-by: Ishaan Jaff <[email protected]> Co-authored-by: deepanshu <[email protected]> * [Feat] Bedrock Knowledgebase - return search_response when using /chat/completions API with LiteLLM (#15509) * docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * add AnthropicCitation * fix async_post_call_success_deployment_hook * fix add vector_store_custom_logger to global callbacks * test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call * async_post_call_success_deployment_hook * add async_post_call_streaming_deployment_hook * async def test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call_streaming(setup_vector_store_registry): * fix _call_post_streaming_deployment_hook * fix async_post_call_streaming_deployment_hook * test update * docs: Accessing Search Results * docs KB * fix chatUI * fix searchResults * fix onSearchResults * fix kb --------- Co-authored-by: Krrish Dholakia <[email protected]> * [Feat] Add dynamic rate limits on LiteLLM Gateway (#15518) * docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * fix: KeyRequestBase * fix rpm_limit_type * fix dynamic rate limits * fix use dynamic limits here * fix _should_enforce_rate_limit * fix _should_enforce_rate_limit * fix counter * test_dynamic_rate_limiting_v3 * use _create_rate_limit_descriptors --------- Co-authored-by: Krrish Dholakia <[email protected]> * Add google rerank endpoint * Add docs * fix mypy error * fix mypy and lint errors * Add haiku 4.5 integration * Add haiku 4.5 integration for other regions as well * Handle citation field correctly * Fix filtering headers for signature calcs * Add haiku 4.5 integration (#15650) --------- Co-authored-by: Leslie Cheng <[email protected]> Co-authored-by: Sameer Kankute <[email protected]> Co-authored-by: Alexsander Hamir <[email protected]> Co-authored-by: Lucas <[email protected]> Co-authored-by: sotazhang <[email protected]> Co-authored-by: Deepanshu Lulla <[email protected]> Co-authored-by: Krrish Dholakia <[email protected]> Co-authored-by: deepanshu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Remove penalty params as supported params for gemini preview model
Relevant issues
Fixes LIT-1239
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🐛 Bug Fix
Changes
Added a method which checks if the model supports penalty params or not.