Skip to content

Fix json_tool_call filtering: streaming support, pop->get, behavioral parity#2

Open
jquinter wants to merge 365 commits intohaggai-backline:mainfrom
jquinter:fix/bedrock-json-tool-call-filtering
Open

Fix json_tool_call filtering: streaming support, pop->get, behavioral parity#2
jquinter wants to merge 365 commits intohaggai-backline:mainfrom
jquinter:fix/bedrock-json-tool-call-filtering

Conversation

@jquinter
Copy link

@jquinter jquinter commented Feb 9, 2026

Summary

Addresses the 3 findings from the review of PR BerriAI#18384 (BerriAI#18384):

  • Streaming support (requested by @krrishdholakia): AWSEventStreamDecoder now accepts json_mode and suppresses json_tool_call chunks in streaming, converting its arguments to text content. Updated all callers (make_call, make_sync_call in both invoke_handler.py and converse_handler.py) to pass json_mode to the decoder.
  • optional_params.pop("json_mode") -> .get() (flagged by Greptile): Avoids mutating the caller's dict, which could break downstream logging/retries/metrics that access optional_params after response transformation.
  • Preserve original tool_calls behavior: _filter_json_mode_tools() returns None only when json_tool_call was the sole tool (converted to content). Returns the list (including []) otherwise, preserving the original behavior of always setting tool_calls.

Also extracts the filtering logic into _filter_json_mode_tools() and moves import json to module level per project conventions.

Test plan

  • test_transform_response_with_both_json_tool_call_and_real_tool — non-streaming mixed tools
  • test_transform_response_does_not_mutate_optional_params — verifies .get() fix
  • test_streaming_decoder_filters_json_tool_call — streaming with mixed json_tool_call + real tool
  • test_streaming_decoder_without_json_mode_passes_all_tools — streaming passthrough when json_mode=False
  • All 61 existing tests in test_converse_transformation.py pass

Sameerlite and others added 30 commits February 3, 2026 12:08
…esday_cicd_release

Revert "Litellm tuesday cicd release"
…lease_final

Litellm tuesday cicd release final
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
There is no reason to merge user messages.
adding together ai models to litellm models json
yuneng-jiang and others added 30 commits February 6, 2026 12:33
[Infra] UI - Testing: Adding Unit Testing Coverage
…uth_mh

[Feature] UI - Admin Settings: Add option for Authentication for public AI Hub
[Fix] UI - Model Info Page: Fix Input and Output Labels
…resize

[Fix] UI - Model Page: Column Resizing on Smaller Screens
…ation

refactor: migrate Ant Design notifications to use `App.useApp()` cont…
…rvers (BerriAI#20602)

- process_mcp_request() now falls back to OAuth2 passthrough when Authorization header contains a non-LiteLLM token (catches HTTPException and ProxyException 401/403)
- MCPClient._get_auth_headers() adds missing MCPAuth.oauth2 case
…ternal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (BerriAI#20604)

* v1 card resolver fix

* fix: is_localhost_or_internal_url

* fix code

* test_fix_agent_card_url_replaces_localhost

* test restruct

* test_a2a_non_streaming

* test agnts

* add exception handling

* init errors

* add localhost retry

* add agent_testing

* test_a2a_non_streaming

* _build_streaming_logging_obj

* code qa fixes

* test_card_resolver_fallback_from_new_to_old_path

* fix linting
[Re-issue: Fix] Keys and Teams Router Setting + Allow Override of Router Settings
…tibility (BerriAI#20591)

Keycloak (and similar OIDC providers) include role claims in the JWT
access token but not in the UserInfo endpoint response. Previously,
roles were only extracted from UserInfo, causing all SSO users to
default to internal_user_view_only regardless of their actual role.

Changes:
- Extract user roles from JWT access token in process_sso_jwt_access_token()
  when UserInfo doesn't provide them (tries role_mappings first, then
  GENERIC_USER_ROLE_ATTRIBUTE)
- Handle list-type role values in get_litellm_user_role() since Keycloak
  returns roles as arrays (e.g. ["proxy_admin"] instead of "proxy_admin")
- Add 9 new unit tests covering role extraction and list handling
- Update 3 existing tests for new JWT decode behavior

Closes BerriAI#20407
…#20566)

Notes: General support for Opus 4.6 was added in BerriAI#20506 however
it omitted the AU (australian) specific instance profile used
in Bedrock. This change only adds the the au id. It is copied
from the US model settings which is consistent with past
additions of this regional model profile.
…CP + Agent guardrail support (BerriAI#20619)

* fix: fix styling

* fix(custom_code_guardrail.py): add http support for custom code guardrails

allows users to call external guardrails on litellm with minimal code changes (no custom handlers)

Test guardrail integrations more easily

* feat(a2a/): add guardrails for agent interactions

allows the same guardrails for llm's to be applied to agents as well

* fix(a2a/): support passing guardrails to a2a from the UI

* style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails

* feat(mcp/): support custom code guardrails for mcp calls

allows custom code guardrails to work on mcp input

* feat(chatui.tsx): support guardrails on mcp tool calls on playground
…erriAI#20618)

* fix(mypy): resolve missing return statements and type casting issues

* fix(pangea): use elif to prevent UnboundLocalError and handle None messages

Address Greptile review feedback:
- Make branches mutually exclusive using elif to prevent input_messages from being overwritten
- Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API

---------

Co-authored-by: Shin <shin@openclaw.ai>
…lable on Internet (BerriAI#20607)

* update MCPAuthenticatedUser

* add available_on_public_internet for MCPs

* update claude.md

* init IPAddressUtils

* init available_on_public_internet

* add on REST endpoints

* filter with IP

* TestIsInternalIp

* _extract_mcp_headers_from_request

* init get_mcp_client_ip

* _get_general_settings

* allowed_server_ids

* address PR comments

* get_mcp_server_by_name fix

* fix server

* fix review comments

* get_public_mcp_servers

* address _get_allowed_mcp_servers
* update MCPAuthenticatedUser

* add available_on_public_internet for MCPs

* update claude.md

* init IPAddressUtils

* init available_on_public_internet

* add on REST endpoints

* filter with IP

* TestIsInternalIp

* _extract_mcp_headers_from_request

* init get_mcp_client_ip

* _get_general_settings

* allowed_server_ids

* address PR comments

* get_mcp_server_by_name fix

* fix server

* fix review comments

* get_public_mcp_servers

* address _get_allowed_mcp_servers

* test fix

* fix linting

* inint ui types

* add ui for managing MCP private/public

* add ui

* fixes

* add to schema

* add types

* fix endpoint

* add endpoint

* update manager

* test mcp

* dont use external party for ip address
[Fix] /key/list user_id Empty String Edge Case
- a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment
- caching/redis_cache.py: Add type ignore for async ping return type
- caching/redis_cluster_cache.py: Add type ignore for async ping return type
- llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text
- proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument

All changes add appropriate type: ignore comments to handle library typing inconsistencies.
Replace text-embedding-004 with gemini-embedding-001.

The old model was deprecated and returns 404:
'models/text-embedding-004 is not found for API version v1beta'

Co-authored-by: Shin <shin@openclaw.ai>
…settings

[Feature] UI - Team Settings: Soft Budget + Alerting Emails
Update opus 4.6 blog with adaptive thinking
… tools (BerriAI#18384)

When both `tools` and `response_format` are used with Bedrock Converse,
LiteLLM adds an internal `json_tool_call` tool for structured output.
Bedrock may return both this internal tool and real user tools, which
breaks consumers like the OpenAI Agents SDK.

Changes:
- Extract filtering logic into `_filter_json_mode_tools()` method
- Filter out `json_tool_call` when mixed with real tools in responses
- Add streaming support: `AWSEventStreamDecoder` now accepts `json_mode`
  and suppresses `json_tool_call` chunks, converting arguments to text
- Fix `optional_params.pop("json_mode")` -> `.get()` to avoid mutating
  the caller's dict (affects logging/retries/metrics downstream)
- Preserve original behavior of setting `tool_calls=[]` for empty lists
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.