fix: strip OpenAI 'annotations' field from outbound Databricks chat messages - #110
Merged
josemaria-vilaplana merged 3 commits intoJun 5, 2026
Conversation
…essages OpenAI added an 'annotations' field to chat-completion content blocks for citation parity with the Responses API. LiteLLM normalizes every provider response to that spec, so even when the upstream is Anthropic-via-Databricks the chat-completion response surfaced to the caller may include annotations on text content blocks. The OpenAI Agents SDK persists this OpenAI-shaped history verbatim and replays it on every follow-up turn. Databricks Model Serving rejects the unknown field on input with: 400 BAD_REQUEST - messages.<n>.content.<m>.text.annotations: Extra inputs are not permitted Strip annotations from each content block in Databricks's outbound _transform_messages, next to _sanitize_empty_content. Adds a helper and four regression tests (unit + transformer). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
josemaria-vilaplana
marked this pull request as ready for review
June 3, 2026 10:11
Companion to the chunk_parser deletion in #109. Databricks's streaming protocol emits parameterless tool_call arguments as one or two empty-string deltas (no "{}"), which accumulate to "" in the consumer. The OpenAI Agents SDK persists that empty string in conversation history and replays it on every follow-up turn. Databricks Model Serving rejects the unknown shape with: INVALID_PARAMETER_VALUE: Param 'arguments' in the tool_calls function specification is not a valid JSON string. No content to map due to end-of-input Validated empirically by calling Databricks Sonnet 4-6 directly: the non-streaming response returns arguments='{}' but the streamed response emits two delta chunks both with arguments=''. #109's deletion only prevented LiteLLM from corrupting a valid '{}' to ''; it did not produce '{}' when the model natively streamed ''. Coerce empty/missing tool_call arguments to '{}' in the outbound _transform_messages so subsequent turns send valid JSON, alongside the existing _sanitize_empty_content and _strip_openai_annotations helpers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…chunk
Pair to the outbound _normalize_empty_tool_call_arguments fix. The
outbound coercion in _transform_messages prevents Databricks from
rejecting a replay with INVALID_PARAMETER_VALUE, but the SAME-TURN
consumer (OpenAI Agents SDK runtime, frontend tool runners) still calls
JSON.parse on whatever the stream yielded — empty string from
Databricks's parameterless arguments protocol — and surfaces:
An error occurred while parsing tool arguments.
Default arguments to "{}" on the chunk that introduces the tool_call
(i.e. the chunk carrying function.name) so the consumer's first observed
value is valid JSON. Subsequent args-only delta chunks stay empty so
their concatenation doesn't double up ("{}" + "" + "" = "{}").
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
josemaria-vilaplana
merged commit Jun 5, 2026
de7b5de
into
upstream-sync/v1.83.14-stable
5 checks passed
3 tasks
3 tasks
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion to #109. #109 fixed the streaming-side corruption that broke turn-1 tool calls; this one fixes the request-side corruption that breaks turn 2+ of any conversation.
OpenAI added an
annotationsfield to chat-completion content blocks for citation parity with the Responses API. LiteLLM normalizes every provider response to that spec, so even when the upstream is Anthropic-via-Databricks (e.g.databricks-claude-sonnet-4-6) the chat-completion response surfaced to the caller may includeannotationson text content blocks.The OpenAI Agents SDK persists this OpenAI-shaped history verbatim and replays it on every follow-up turn. Databricks Model Serving rejects the unknown field on input with:
Symptom in CARTO: turn 1 of a UI chat against
databricks-claude-*succeeds; turn 2 always fails with the above.Changes
litellm/llms/databricks/chat/transformation.py— new_strip_openai_annotations()helper next to_sanitize_empty_content, called from_transform_messagesso every outbound message hasannotationsstripped from its content blocks.tests/test_litellm/llms/databricks/chat/test_databricks_chat_transformation.py— four regression tests:annotations, ignores string content, no-op when absent.annotationssurvives_transform_messagescleanly.Test plan
test_databricks_chat_transformation.pysuite: 16/16 pass.Notes
_strip_openai_annotations) refers to the schema origin (OpenAI chat-completion spec), not the underlying model. The field leaks regardless of which model is behind LiteLLM.upstream-sync/v1.83.14-stable, so the build image will carry v1.83.14 + fix: stop Databricks streaming from rewriting tool_call arguments '{}' to '' #109's chunk-parser fix + this fix.