Skip to content

feat(anthropic): preserve thinking display and URL image sources - #484

Merged
furionw merged 3 commits into
ai-dynamo:mainfrom
maheshrbapatu:fix/anthropic-display-url-sources
Oct 30, 2026
Merged

furionw merged 3 commits into
ai-dynamo:mainfrom
maheshrbapatu:fix/anthropic-display-url-sources

Conversation

@maheshrbapatu

@maheshrbapatu maheshrbapatu commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Anthropic URL image sources currently fail deserialization, and thinking.display disappears while parsing. Add typed Base64/URL source variants and preserve the summarized/omitted display values and beta updates variant, including ordinary images and image-bearing tool results. Reject non-null, non-boolean tool_result.is_error values. Propagate malformed nested tool-content errors instead of silently treating them as missing content; omission, null, empty arrays, and valid text retain their behavior.

This is the protocol prerequisite for ai-dynamo/dynamo#17376 and ai-dynamo/dynamo#17377, consumed by draft PRs ai-dynamo/dynamo#17381 and ai-dynamo/dynamo#17380. Fetching, URL security policy, display enforcement, and semantic validation of thinking modes remain in the serving layer. Dynamo explicitly rejects the beta updates mode until it can distinguish progress updates from reasoning; representing the value is not an implementation of that feature.

Rust API migration: callers must replace AnthropicImageSource struct literals/field access with enum variants/matches and add display: None to existing ThinkingConfig literals. The README documents this migration.

Draft release gate: agree on the crate release/version migration before marking ready. Downstream draft PRs pin the identical changes cherry-picked onto protocol 8.2.0 at bc547588eac8890bfeda2e206bc559959b273f72 on maheshrbapatu/frontend-crates:fix/anthropic-display-url-sources-8.2. This avoids unrelated 8.4 API changes that do not compile with Dynamo's pinned parser. All three downstream workspaces (root, Python bindings, KVBM bindings) pin the backport independently. They must switch to the agreed published release before readiness. This PR does not claim compatibility with existing Rust struct literals.

Validation:

  • cargo test --locked -p dynamo-protocols
  • cargo clippy --locked -p dynamo-protocols --all-targets -- -D warnings
  • cargo fmt -p dynamo-protocols --check
  • Serialization round trips, omitted fields, malformed source combinations through whole requests, tool-result content omission/null/empty semantics, and invalid display values are covered. Both the main-branch protocol suite and the 8.2 backport suite pass. Removing strict source-field validation makes the new test fail (mutation restored).
  • Independent review completed. The is_error regression fails when the old permissive parser is restored, then passes with strict parsing. Main and backport protocol tests pass; main clippy with warnings denied and formatting pass.

Contract: https://platform.claude.com/docs/en/build-with-claude/vision and https://platform.claude.com/docs/en/build-with-claude/thinking

Signed-off-by: maheshrbapatu <maheshbapatu@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added external-contribution PR originates from a fork feat labels Oct 27, 2026
Signed-off-by: maheshrbapatu <maheshbapatu@gmail.com>
Signed-off-by: maheshrbapatu <maheshbapatu@gmail.com>
@rmccorm4
rmccorm4 marked this pull request as ready for review October 29, 2026 17:29
@furionw furionw self-assigned this Oct 30, 2026

@furionw furionw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

furionw commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

/ok to test 893b8ef

furionw commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

[P2: Low-risk, non-blocking issue such as a narrow edge case, defensive-hardening opportunity, or minor maintainability/test gap]
Compatibility note: deny_unknown_fields tightens previously accepted image-source JSON

@maheshrbapatu Recording a compatibility caveat alongside the documented Rust API migration. Adding the URL variant preserves the standard base64 wire shape, but deny_unknown_fields independently rejects extra source fields that were previously ignored. For example, this image block inside messages[].content[] (image data abbreviated):

{
  "type": "image",
  "source": {
    "type": "base64",
    "media_type": "image/jpeg",
    "data": "<base64 JPEG bytes>",
    "url": null
  }
}

Before this PR, the source struct reads type, media_type, and data and ignores url. After this PR, Serde selects Base64 and rejects url as an unknown field, even though its value is null. This can affect clients that serialize a common source object with unused fields set to null. Standard base64 sources without extra fields continue to parse, and standard URL sources gain support.

No affected client has been identified; this is a potential compatibility impact of the deliberate validation tightening, not an observed client regression or a request to change the behavior. It is useful context for consumers upgrading the server as well as those migrating Rust struct construction.

@furionw
furionw merged commit f468683 into ai-dynamo:main Oct 30, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contribution PR originates from a fork feat

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants