Fix json_tool_call filtering: streaming support, pop->get, behavioral parity#2
Open
jquinter wants to merge 365 commits intohaggai-backline:mainfrom
Open
Fix json_tool_call filtering: streaming support, pop->get, behavioral parity#2jquinter wants to merge 365 commits intohaggai-backline:mainfrom
jquinter wants to merge 365 commits intohaggai-backline:mainfrom
Conversation
…mantic_tool_filter.py tests
This reverts commit 1e8848c.
…lease Litellm tuesday cicd release
…esday_cicd_release Revert "Litellm tuesday cicd release"
) (BerriAI#19220)" This reverts commit 72e5193.
…mantic_tool_filter.py tests
This reverts commit 1e8848c.
BerriAI#20071)" This reverts commit ef73f33.
…lease_final Litellm tuesday cicd release final
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…81.7 bump litellm 1.81.7
update 02 staging PR
There is no reason to merge user messages.
adding together ai models to litellm models json
[Infra] UI - Testing: Adding Unit Testing Coverage
…uth_mh [Feature] UI - Admin Settings: Add option for Authentication for public AI Hub
[Fix] UI - Model Info Page: Fix Input and Output Labels
…resize [Fix] UI - Model Page: Column Resizing on Smaller Screens
…ation refactor: migrate Ant Design notifications to use `App.useApp()` cont…
…rvers (BerriAI#20602) - process_mcp_request() now falls back to OAuth2 passthrough when Authorization header contains a non-LiteLLM token (catches HTTPException and ProxyException 401/403) - MCPClient._get_auth_headers() adds missing MCPAuth.oauth2 case
…ternal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (BerriAI#20604) * v1 card resolver fix * fix: is_localhost_or_internal_url * fix code * test_fix_agent_card_url_replaces_localhost * test restruct * test_a2a_non_streaming * test agnts * add exception handling * init errors * add localhost retry * add agent_testing * test_a2a_non_streaming * _build_streaming_logging_obj * code qa fixes * test_card_resolver_fallback_from_new_to_old_path * fix linting
[Re-issue: Fix] Keys and Teams Router Setting + Allow Override of Router Settings
…tibility (BerriAI#20591) Keycloak (and similar OIDC providers) include role claims in the JWT access token but not in the UserInfo endpoint response. Previously, roles were only extracted from UserInfo, causing all SSO users to default to internal_user_view_only regardless of their actual role. Changes: - Extract user roles from JWT access token in process_sso_jwt_access_token() when UserInfo doesn't provide them (tries role_mappings first, then GENERIC_USER_ROLE_ATTRIBUTE) - Handle list-type role values in get_litellm_user_role() since Keycloak returns roles as arrays (e.g. ["proxy_admin"] instead of "proxy_admin") - Add 9 new unit tests covering role extraction and list handling - Update 3 existing tests for new JWT decode behavior Closes BerriAI#20407
…#20566) Notes: General support for Opus 4.6 was added in BerriAI#20506 however it omitted the AU (australian) specific instance profile used in Bedrock. This change only adds the the au id. It is copied from the US model settings which is consistent with past additions of this regional model profile.
…CP + Agent guardrail support (BerriAI#20619) * fix: fix styling * fix(custom_code_guardrail.py): add http support for custom code guardrails allows users to call external guardrails on litellm with minimal code changes (no custom handlers) Test guardrail integrations more easily * feat(a2a/): add guardrails for agent interactions allows the same guardrails for llm's to be applied to agents as well * fix(a2a/): support passing guardrails to a2a from the UI * style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails * feat(mcp/): support custom code guardrails for mcp calls allows custom code guardrails to work on mcp input * feat(chatui.tsx): support guardrails on mcp tool calls on playground
…erriAI#20618) * fix(mypy): resolve missing return statements and type casting issues * fix(pangea): use elif to prevent UnboundLocalError and handle None messages Address Greptile review feedback: - Make branches mutually exclusive using elif to prevent input_messages from being overwritten - Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API --------- Co-authored-by: Shin <shin@openclaw.ai>
…lable on Internet (BerriAI#20607) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers
* update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * test fix * fix linting * inint ui types * add ui for managing MCP private/public * add ui * fixes * add to schema * add types * fix endpoint * add endpoint * update manager * test mcp * dont use external party for ip address
[Fix] /key/list user_id Empty String Edge Case
- a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment - caching/redis_cache.py: Add type ignore for async ping return type - caching/redis_cluster_cache.py: Add type ignore for async ping return type - llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text - proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument All changes add appropriate type: ignore comments to handle library typing inconsistencies.
Replace text-embedding-004 with gemini-embedding-001. The old model was deprecated and returns 404: 'models/text-embedding-004 is not found for API version v1beta' Co-authored-by: Shin <shin@openclaw.ai>
…settings [Feature] UI - Team Settings: Soft Budget + Alerting Emails
Update opus 4.6 blog with adaptive thinking
… tools (BerriAI#18384) When both `tools` and `response_format` are used with Bedrock Converse, LiteLLM adds an internal `json_tool_call` tool for structured output. Bedrock may return both this internal tool and real user tools, which breaks consumers like the OpenAI Agents SDK. Changes: - Extract filtering logic into `_filter_json_mode_tools()` method - Filter out `json_tool_call` when mixed with real tools in responses - Add streaming support: `AWSEventStreamDecoder` now accepts `json_mode` and suppresses `json_tool_call` chunks, converting arguments to text - Fix `optional_params.pop("json_mode")` -> `.get()` to avoid mutating the caller's dict (affects logging/retries/metrics downstream) - Preserve original behavior of setting `tool_calls=[]` for empty lists
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses the 3 findings from the review of PR BerriAI#18384 (BerriAI#18384):
AWSEventStreamDecodernow acceptsjson_modeand suppressesjson_tool_callchunks in streaming, converting its arguments to text content. Updated all callers (make_call,make_sync_callin bothinvoke_handler.pyandconverse_handler.py) to passjson_modeto the decoder.optional_params.pop("json_mode")->.get()(flagged by Greptile): Avoids mutating the caller's dict, which could break downstream logging/retries/metrics that accessoptional_paramsafter response transformation.tool_callsbehavior:_filter_json_mode_tools()returnsNoneonly whenjson_tool_callwas the sole tool (converted to content). Returns the list (including[]) otherwise, preserving the original behavior of always settingtool_calls.Also extracts the filtering logic into
_filter_json_mode_tools()and movesimport jsonto module level per project conventions.Test plan
test_transform_response_with_both_json_tool_call_and_real_tool— non-streaming mixed toolstest_transform_response_does_not_mutate_optional_params— verifies.get()fixtest_streaming_decoder_filters_json_tool_call— streaming with mixed json_tool_call + real tooltest_streaming_decoder_without_json_mode_passes_all_tools— streaming passthrough when json_mode=Falsetest_converse_transformation.pypass