fix(anthropic, fireworks): inline legacy $ref defs in tool schemas

[milan-berri]

PR description (paste below the title on GitHub)

Relevant issues

Fixes #26692 — MCP tool schemas with $ref definitions fail on Anthropic and Fireworks providers (PointerToNowhere / Invalid tool schema, $ref is not supported).

Linear ticket

Resolves LIT-3141

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Root cause

Anthropic and Fireworks tool-schema resolvers only recognise $defs (JSON Schema 2020-12). Tools coming from MCP servers and OpenAPI-derived gateways frequently back their $refs with one of two other JSON-Schema dialects instead:

Source	Where the defs live
Legacy MCP servers (e.g. DevRev)	"definitions": { ... } (JSON Schema draft-04)
OpenAPI-derived gateways (e.g. AWS AgentCore + Google Workspace)	"components": {"schemas": { ... }}

LiteLLM's tool-schema filters silently strip those def blocks (they aren't in Anthropic's AnthropicInputSchema allow-list, and Fireworks doesn't recognise them either) but leave any $ref pointers in properties intact. The upstream therefore receives dangling pointers and 400s:

• Anthropic: tools.0.input_schema: Invalid tool schema, $ref is not supported
• Fireworks: Error resolving schema reference '#/definitions/...' (PointerToNowhere)

What changes

A new helper unpack_legacy_defs(schema, *, copy=False) is added next to the existing unpack_defs in litellm/litellm_core_utils/prompt_templates/common_utils.py. It pops draft-04 definitions and OpenAPI components.schemas and feeds them through unpack_defs in place; $defs is intentionally untouched so providers continue to resolve it natively.

Both provider transforms now call into the helper instead of forwarding broken schemas:

• AnthropicConfig._map_tool_helper calls unpack_legacy_defs(_input_schema, copy=True) so the caller's tool dict is preserved.
• FireworksAIConfig._transform_tools calls unpack_legacy_defs(params) in place (matching the existing pop("strict", None) contract for the same loop).

How to reproduce (transform-level, no API key required)

Drop the snippet below into a file and run it twice — once against main and once against this branch. It exercises the exact transformation function that builds the Anthropic / Fireworks tool payload, so the output is what would land in the outbound HTTP body. No API call, no proxy, no network.

# repro.py
import copy, json
from litellm.llms.anthropic.chat.transformation import AnthropicConfig
from litellm.llms.fireworks_ai.chat.transformation import FireworksAIConfig

# OpenAPI-derived tool (AgentCore + Google Workspace shape)
tool = {
    "type": "function",
    "function": {
        "name": "slides_presentations_create",
        "description": "Create a Google Slides presentation",
        "parameters": {
            "type": "object",
            "properties": {"body": {"$ref": "#/components/schemas/Presentation"}},
            "required": ["body"],
            "components": {
                "schemas": {
                    "Presentation": {
                        "type": "object",
                        "properties": {
                            "title": {"type": "string"},
                            "presentationId": {"type": "string"},
                        },
                    }
                }
            },
        },
    },
}

a, _ = AnthropicConfig()._map_tool_helper(copy.deepcopy(tool))
print("ANTHROPIC input_schema:")
print(json.dumps(a["input_schema"], indent=2))
print("contains $ref:", "$ref" in json.dumps(a))

f = FireworksAIConfig()._transform_tools([copy.deepcopy(tool)])
print("\nFIREWORKS parameters:")
print(json.dumps(f[0]["function"]["parameters"], indent=2))
print("contains $ref:", "$ref" in json.dumps(f))

Before this PR (current main)

{
  "type": "object",
  "properties": {
    "body": {"$ref": "#/components/schemas/Presentation"}
  },
  "required": ["body"]
}

contains $ref: True

The components block has been stripped by the allow-list filter, but the $ref pointing into it is still in the body. The upstream therefore rejects the request:

• Anthropic: HTTP 400 tools.0.input_schema: Invalid tool schema, $ref is not supported
• Fireworks: Error resolving schema reference '#/components/schemas/Presentation' (PointerToNowhere)

The same failure mode reproduces with legacy draft-04 definitions (e.g. tools coming from MCP servers like DevRev) — replace components.schemas with a top-level definitions: {Thing: {...}} block and $ref: "#/definitions/Thing".

After this PR

{
  "type": "object",
  "properties": {
    "body": {
      "type": "object",
      "properties": {
        "title": {"type": "string"},
        "presentationId": {"type": "string"}
      }
    }
  },
  "required": ["body"]
}

contains $ref: False

The pointer is resolved against its def block, the def block itself is dropped (it isn't part of Anthropic's allow-list anyway), and the schema sent upstream is now self-contained. Anthropic and Fireworks accept the tool. $defs (JSON Schema 2020-12) is not touched — providers continue to resolve it natively, so existing $defs-based tool calls behave exactly as before.

Test invariants (covered in tests/test_litellm/)

For both AnthropicConfig._map_tool_helper and FireworksAIConfig._transform_tools:

Invariant	Why it matters
No $ref substring remains in the serialised transformed schema (json.dumps(schema).find("$ref") == -1) when only legacy / OpenAPI defs are present.	Guarantees we never forward a dangling pointer to the upstream.
The inlined value equals the original def body (not just deleted).	Pointer is resolved, not silently dropped — the tool stays semantically equivalent.
components and definitions are not present in the transformed schema.	Anthropic's allow-list rejects unknown keys; we must clean up after ourselves.
$defs and any $ref pointing into $defs survive untouched.	Native JSON Schema 2020-12 path is unaffected; pre-existing test_anthropic_map_openai_params_tools_with_defs continues to hold.
(Anthropic only) The caller's tool dict is byte-for-byte unchanged after the call.	_map_tool_helper is called per-request; mutating caller state would cause cross-request leakage and confuse downstream code paths.

The Anthropic suite adds 4 tests (one per invariant). The Fireworks suite adds 3 (it mutates in place by design, so the no-mutation invariant doesn't apply).

Type

🐛 Bug Fix

Changes

• litellm/litellm_core_utils/prompt_templates/common_utils.py — add unpack_legacy_defs(schema, *, copy=False) -> dict (and a private _has_legacy_defs predicate) next to the existing unpack_defs. Pops draft-04 definitions and OpenAPI components.schemas and resolves their $refs in place; $defs is preserved. copy=True deep-copies first, but only when there is actually work to do.
• litellm/llms/anthropic/chat/transformation.py — _map_tool_helper now calls unpack_legacy_defs(_input_schema, copy=True) before the AnthropicInputSchema allow-list filter.
• litellm/llms/fireworks_ai/chat/transformation.py — _transform_tools now calls unpack_legacy_defs(params) in place inside the existing per-tool loop.
• tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py — 4 regression tests (one per invariant above).
• tests/test_litellm/llms/fireworks_ai/chat/test_fireworks_ai_chat_transformation.py — 3 regression tests.

[greptile-apps]

Greptile Summary

Fixes dangling $ref pointers in tool schemas for Anthropic and Fireworks by adding unpack_legacy_defs, which inlines draft-04 definitions and OpenAPI components.schemas before provider-specific filters strip those def blocks. A max_inlined_bytes byte budget is also added to unpack_defs to guard against schema-bomb amplification.

• unpack_legacy_defs is wired into AnthropicConfig._map_tool_helper (with copy=True to preserve the caller's dict) and FireworksAIConfig._transform_tools (in-place, matching the existing pop("strict") contract).
• The byte-budget check (_estimate_json_bytes) runs before each deepcopy, correctly bounding fan-out, node-count, and scalar-byte amplification vectors, with a 10 MB default two orders of magnitude above any realistic MCP/OpenAPI tool schema.
• Tests cover all three schema-bomb vectors, the legitimate-large-schema baseline, caller-dict immutability for Anthropic, native $defs pass-through, and the documented name-collision preference.

Confidence Score: 5/5

Safe to merge; the core fix is well-scoped, tested, and backward-compatible.

The change is narrowly scoped to tool-schema transformation: the happy path (no legacy defs) is a fast predicate check with no side effects, and the inlining logic is well-covered by unit tests. The one nit — pop("components") discarding non-schemas sub-keys — has no observable impact on the Anthropic path and is an unusual scenario for Fireworks tool schemas.

common_utils.py — the components pop behaviour is worth a follow-up if tools with mixed components.* sub-keys ever appear in the wild.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/prompt_templates/common_utils.py	Adds _estimate_json_bytes, _has_legacy_defs, unpack_legacy_defs, and extends unpack_defs with a max_inlined_bytes budget. Core logic is sound; one issue: schema.pop("components", {}) drops non-schemas sub-keys of components silently.
litellm/llms/anthropic/chat/transformation.py	Calls unpack_legacy_defs(_input_schema, copy=True) before the allow-list filter; placement is correct and copy=True correctly prevents mutation of the caller's dict.
litellm/llms/fireworks_ai/chat/transformation.py	Refactors loop to use continue for non-function tools and calls unpack_legacy_defs(params) in-place, matching the existing pop("strict") in-place contract.
tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_common_utils.py	Adds TestUnpackLegacyDefs covering no-op branches, all three schema-bomb amplification vectors, a legitimate-large-schema baseline, and the components no-schemas guard.
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py	Adds 5 regression tests: components.schemas inlining, draft-04 definitions inlining, native $defs pass-through, caller-dict immutability, and name-collision preference documented as a known limitation.
tests/test_litellm/llms/fireworks_ai/chat/test_fireworks_ai_chat_transformation.py	Adds 4 regression tests mirroring the Anthropic suite, plus a test for the new continue-based non-function tool path.

_{Reviews (3): Last reviewed commit: "fix(anthropic, fireworks): inline legacy..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 82.14286% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...itellm_core_utils/prompt_templates/common_utils.py	78.26%	10 Missing ⚠️

📢 Thoughts on this report? Let us know!

[milan-berri]

@greptileai Concern was valid; full disambiguation would require changing unpack_defs to key by full ref path, which is out of scope. Took the minimal fix: swapped merge order so definitions wins on collision (more semantically correct default since unpack_defs already treats definitions as a native def-source via JSON-Schema scoping). Documented the residual limitation in the code. Regression test added.

[veria-ai]

PR overview

Legacy tool-schema references are inlined for Anthropic and Fireworks

This PR adds bounded expansion for legacy definitions and OpenAPI components.schemas references before provider-specific schema filtering. I checked the new inlining helper, byte budget enforcement before deep-copy expansion, and the Anthropic/Fireworks call sites for request-controlled schema amplification or cross-request effects.

Security review

• No new security issues were flagged in the latest review.
• 1 previously flagged issue(s) appear fixed in the latest changes.
• No review issues remain open on this pull request.

Risk: 2/10

[milan-berri]

@greptileai check again

[milan-berri]

@veria-ai check again — added a complementary max_inlined_nodes (default 100k) that bounds the cumulative size of inlined targets, checked before each deepcopy so an oversized expansion is rejected without first materialising it. Closes the "thousands of refs to one large definition" amplification path that max_refs alone didn't bound. Tests cover both rejection and the legitimate-large-but-bounded case.

[milan-berri]

@veria-ai @greptileai check again on 0ea6547493.

I've replaced the two prior budgets (max_refs + max_inlined_nodes) with a single max_inlined_bytes cap that closes all three amplification vectors flagged so far in one bound:

• fan-out chain — exponential ref count -> caught (every ref produces `>= a few output bytes)
• node-count amplification (many refs to one large dict / list target) -> caught (every node is `>= a few bytes)
• scalar-byte amplification (many refs to a target with a long description / const / enum) -> caught directly (a 100KB string costs 100KB in the running total, not 1 "node")

Bytes are the wire-size measure -- by definition any amplification vector reduces to bytes, so a single byte cap is mathematically complete. Estimated before each deepcopy so an oversized expansion is rejected without first materialising it; default 10 MB sits two orders of magnitude above any legitimate MCP / OpenAPI tool schema (Anthropic's per-tool input cap is itself ~200 KB).

Tests cover all three vectors plus the legitimate-large-schema baseline. 302 passing.

[mateo-berri]

LGTM; thanks!