Fix/nova grounding

[jquinter]

Relevant issues

Fixes #17482 (leveraging PR #19474)

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all scoped unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature - Added support for web_search_options call for Amazon nova model
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

Changes the approach for enabling Nova grounding (web search) to follow the LiteLLM standard pattern used by other providers (Vertex AI, Anthropic).

Instead of:

tools=[{
  "type": "system_tool", 
  "system_tool": {
    "name":"nova_grounding"
  }
}]

Users now use:

web_search_options={}                                                        

Example:

response = completion(                                                                                                                                              
    model="bedrock/us.amazon.nova-pro-v1:0",                                                                                                                        
    messages=[{
       "role": "user", 
       "content": "What's the latest news?"
    }],
    web_search_options={},  # Enables Nova grounding                                                                                                                
)
#Citations will be available in:                                                                                                                                           
#response
#  .choices[0]
#  .message
#  .provider_specific_fields["citationsContent"]                                                                                          

Note: Nova doesn't support search_context_size or user_location parameters (unlike Anthropic), so those will be silently ignored - similar to how Vertex AI handles it.

Implementation changes:

• Add _map_web_search_options() in AmazonConverseConfig to transform
  web_search_options to Nova's systemTool format
• Add web_search_options to supported params for Nova models
• Add BedrockToolBlock TypedDict with systemTool support
• Remove system_tool handling from _bedrock_tools_pt() as it's no longer needed (web_search_options is the entry point)

Test changes:

• Convert integration tests to mocked unit tests using HTTPHandler
• Fix Pylance type warnings by using .get() for optional TypedDict keys
• Follow existing test patterns (api_base + client mocking)

Ref: https://docs.aws.amazon.com/nova/latest/userguide/grounding.html"

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
litellm	Ready	Preview, Comment	Jan 23, 2026 0:17am

[CLAassistant]

All committers have signed the CLA.

[juhiechandra]

@jquinter Thank you for refactoring my PR per @krrishdholakia comments.

please provide details on how you verified this update for nova grounding on your end. I am trying to replicate the same and test but need some support there.

Could you share same sample scripts or the payload you ran to test for nova grounding output?

I tested my earlier PR and received the response body for citations, however facing some issues after you've refactored it a bit.

[jquinter]

Verification details for Nova grounding update

@juhiechandra Here's how to verify the update, plus two bugs I found and fixed while testing.

Verification script (mocked — no AWS credentials needed)

import json
from unittest.mock import patch, MagicMock
from litellm import completion
from litellm.llms.custom_httpx.http_handler import HTTPHandler

client = HTTPHandler()

with patch.object(client, "post") as mock_post:
    mock_post.return_value = MagicMock(
        status_code=200,
        json=lambda: {
            "output": {
                "message": {
                    "role": "assistant",
                    "content": [
                        {"text": "The population of Tokyo is approximately 14 million."},
                        {
                            "citationsContent": {
                                "citations": [
                                    {
                                        "location": {
                                            "web": {
                                                "url": "https://example.com/tokyo",
                                                "domain": "example.com",
                                            }
                                        }
                                    }
                                ]
                            }
                        },
                    ],
                }
            },
            "stopReason": "end_turn",
            "usage": {"inputTokens": 20, "outputTokens": 30, "totalTokens": 50},
        },
    )

    response = completion(
        model="bedrock/us.amazon.nova-pro-v1:0",
        messages=[{"role": "user", "content": "What is the population of Tokyo?"}],
        web_search_options={},
        max_tokens=500,
        client=client,
    )

    # Verify request payload
    assert mock_post.called, "HTTP POST was not called"
    request_body = json.loads(mock_post.call_args.kwargs["data"])

    assert "toolConfig" in request_body, "toolConfig missing from request"
    tool_config = request_body["toolConfig"]
    system_tools = [t for t in tool_config["tools"] if "systemTool" in t]
    assert len(system_tools) == 1, f"Expected 1 systemTool, got {len(system_tools)}"
    assert system_tools[0]["systemTool"]["name"] == "nova_grounding"

    print("Request payload (toolConfig):")
    print(json.dumps(tool_config, indent=2))

    # Verify response parsing
    print("Response text:", response.choices[0].message.content)

    provider_fields = response.choices[0].message.provider_specific_fields
    assert provider_fields is not None, "provider_specific_fields is None"
    assert "citationsContent" in provider_fields, "citationsContent missing from response"

    citations = provider_fields["citationsContent"]
    assert len(citations) == 1
    assert citations[0]["citations"][0]["location"]["web"]["domain"] == "example.com"

    print("Citations:", json.dumps(citations, indent=2))
    print("PASSED")

Live test (requires AWS credentials + Nova quota)

from litellm import completion

response = completion(
    model="bedrock/us.amazon.nova-pro-v1:0",
    messages=[{"role": "user", "content": "What are the latest developments in quantum computing?"}],
    web_search_options={},
    max_tokens=500,
    aws_region_name="us-east-1",
)

print("Response:", response.choices[0].message.content)

provider_fields = getattr(response.choices[0].message, "provider_specific_fields", None)
if provider_fields and "citationsContent" in provider_fields:
    for block in provider_fields["citationsContent"]:
        for citation in block.get("citations", []):
            web = citation.get("location", {}).get("web", {})
            print(f"  Source: {web.get('url')} ({web.get('domain')})")
else:
    print("No citations returned (model may not have used grounding for this query)")

What to look for

• The outgoing request body should contain toolConfig.tools with {"systemTool": {"name": "nova_grounding"}}
• For live calls, the response content should include web-sourced information, and citationsContent should appear in provider_specific_fields when the model uses grounding
• Passing web_search_options={} to a non-Nova model (e.g., bedrock/anthropic.claude-3-sonnet...) should silently ignore it (no error)

---

⚠️ Two bugs found during testing

While running the verification script against the merged code, I found two bugs that prevent web_search_options={} from working. I'll open a follow-up PR with fixes.

Bug 1: Empty dict falsy check — converse_transformation.py:770

# BEFORE (broken): {} is falsy in Python, so web_search_options={} never triggers the mapping
if param == "web_search_options" and value and isinstance(value, dict):

# AFTER (fixed): matches Anthropic's pattern at anthropic/chat/transformation.py:796
if param == "web_search_options" and isinstance(value, dict):

web_search_options={} is the documented API (empty dict to enable grounding), but {} is falsy, so value and isinstance(value, dict) short-circuits to False. The Anthropic implementation correctly uses just isinstance(value, dict) without the truthy check.

Bug 2: Pre-formatted tools mangled by _bedrock_tools_pt — converse_transformation.py in _process_tools_and_beta

_map_web_search_options returns a BedrockToolBlock(systemTool={"name": "nova_grounding"}) which is already in Bedrock format. This gets added to optional_params["tools"] via _add_tools_to_optional_params. But later, _process_tools_and_beta passes all tools through _bedrock_tools_pt() — which expects OpenAI-format tools (with function.name, function.parameters, etc.). This corrupts the systemTool into an empty toolSpec:

{"toolSpec": {"inputSchema": {"json": {"type": "object", "properties": {}, "required": []}}, "name": "", "description": ""}}

Fix: Separate systemTool blocks from OpenAI-format tools before _bedrock_tools_pt processes them, then append them after:

# In _process_tools_and_beta, before the filtering loop:
pre_formatted_tools: List[ToolBlock] = []
if original_tools:
    for tool in original_tools:
        if "systemTool" in tool:
            pre_formatted_tools.append(tool)
            continue
        # ... existing filtering logic ...

# After _bedrock_tools_pt processing:
bedrock_tools.extend(pre_formatted_tools)

[jquinter]

Live test (requires AWS credentials + Nova quota)

from litellm import completion

response = completion(
    model="bedrock/us.amazon.nova-pro-v1:0",
    messages=[{"role": "user", "content": "What are the latest developments in quantum computing?"}],
    web_search_options={},
    max_tokens=500,
    aws_region_name="us-east-1",
)

print("Response:", response.choices[0].message.content)

provider_fields = getattr(response.choices[0].message, "provider_specific_fields", None)
if provider_fields and "citationsContent" in provider_fields:
    for block in provider_fields["citationsContent"]:
        for citation in block.get("citations", []):
            web = citation.get("location", {}).get("web", {})
            print(f"  Source: {web.get('url')} ({web.get('domain')})")
else:
    print("No citations returned (model may not have used grounding for this query)")

After looking for some minutes, I could successfully run my code in us-west-2 region.

Part of the output I got:

> poetry run python nova_grounding_test.py)                                                                                                             
  ⎿ Response:                                                                                                                                                
     Quantum computing is a rapidly evolving field, and several exciting developments have taken place in recent years. Here are some of the latest          
    advancements:                                                                                                                                            
                                                                                                                                                             
    1. **Quantum Supremacy**:                                                                                                                                
       - **Google's Sycamore Processor**: In 2019, Google announced achieving quantum supremacy with its Sycamore processor, completing a specific           
    computation in 200 seconds that would take the world's fastest supercomputer approximately 10,000 years. This marked a significant milestone in          
    demonstrating the potential of quantum computers to outperform classical computers for certain tasks.                                                    
                                                                                                                                                             
    2. **Error Correction and Fault Tolerance**:                                                                                                             
       - **Surface Codes**: Researchers have made strides in developing more efficient quantum error-correction codes, such as surface codes, which are      
    essential for building fault-tolerant quantum computers. These advancements are crucial for scaling quantum computers to handle more complex             
    computations.                                                                                                                                            
...