Skip to content

[Fix] Anthropic: accept thinking.display="updates" - #45129

Merged
JustinTong0323 merged 3 commits into
sgl-project:mainfrom
JustinTong0323:anthropic-thinking-display-updates
Nov 3, 2026
Merged

JustinTong0323 merged 3 commits into
sgl-project:mainfrom
JustinTong0323:anthropic-thinking-display-updates

Conversation

@JustinTong0323

@JustinTong0323 JustinTong0323 commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Claude Code sends thinking: {"type": "adaptive", "display": "updates"} on its /v1/messages?beta=true requests. AnthropicThinkingParam.display only accepts "summarized" and "omitted", so the first request of every Claude Code session against sglang fails with:

400 invalid_request_error: thinking.display: Input should be 'summarized' or 'omitted'

Claude Code then retries without the field, so sessions work, but each one starts with a rejected request.

"updates" is a documented Anthropic value, not a client quirk. The anthropic Python SDK defines it on the beta thinking configs:

  • BetaThinkingConfigAdaptiveParam.display: Literal["summarized", "omitted", "updates"]
  • BetaThinkingConfigEnabledParam.display: Literal["summarized", "omitted", "updates"]

The non-beta configs list only "summarized" and "omitted". The thinking docs describe "updates" (beta header thinking-display-updates-2026-08-18) as: reasoning blocks come back with an empty thinking field, as with "omitted", and only short progress updates between tool calls carry text.

Modifications

  • entrypoints/anthropic/protocol.py: AnthropicThinkingParam.display accepts "updates". Other values are still rejected, and display with type="disabled" is still a 400.
  • entrypoints/anthropic/serving.py: "updates" is served like "omitted". Reasoning stays on and its text is still emitted, because the OpenAI-side pipeline cannot blank reasoning blocks or produce progress-update blocks. A warning names the requested value.

Nothing else in Claude Code's opening request needs a change. Replaying the captured request through AnthropicMessagesRequest with only display adjusted passes validation: output_config.effort="high" is already accepted, and context_management (clear_thinking_20251015) is ignored as an undeclared field.

Accuracy Tests

Request validation only; generation is unchanged.

Unit tests (test/registered/unit/entrypoints/anthropic/test_serving.py):

  • test_request_claude_code_display_updates_is_accepted uses Claude Code's opening-request shape: adaptive + display="updates" + output_config + context_management + metadata. It fails before this change and passes after.
  • test_request_thinking_unknown_display_is_rejected checks that a value outside the SDK set still fails validation.
  • 70 passed.

Live: Claude Code 2.1.296 (claude -p with ANTHROPIC_BASE_URL pointing at an sglang server built from this branch, serving thinkingmachines/Inkling-NVFP4 on 8x H200):

  • Both /v1/messages?beta=true requests carried thinking={"type": "adaptive", "display": "updates"}, and both returned 200. The server log shows 2x 200 and no 400.
  • The session completed its tool call and ended with end_turn.
  • Before this change, the same CLI's first request returned the 400 above.

Speed Tests and Profiling

Not performance-related.

Checklist


CI States

Latest PR Test (Base): ✅ Run #40349105163
Latest PR Test (Extra): ❌ Run #40349105055
Latest PR Test (AMD ROCm 10): ✅ Run #40349104895

The Anthropic beta thinking config defines display values "summarized",
"omitted" and "updates" (anthropic SDK BetaThinkingConfig*Param; beta
header thinking-display-updates-2026-08-18). Claude Code sends
{"type": "adaptive", "display": "updates"} on every /v1/messages request
and received a 400 until it retried without the field. Accept "updates"
and serve it like "omitted": reasoning stays on and its text is still
emitted, with a warning. Unknown display values are still rejected.
@JustinTong0323 JustinTong0323 added the run-ci CI: run the baseline test suite on this PR label Oct 19, 2026
@JustinTong0323

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@JustinTong0323
JustinTong0323 merged commit 96301ec into sgl-project:main Nov 3, 2026
193 of 199 checks passed
Yuxingwang-intel pushed a commit to Yuxingwang-intel/sglang that referenced this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants