[vMaroon]

Overview

The DeepSeek V4.1 encoder rejects tool_reference content parts with a 400, and /v1/messages turns a tool_result with tool_reference items into two tool messages for one call. This renders each reference on V4.1 as the tool's name and emits one tool message per tool_result, so conversations that use Claude Code's tool search work on DeepSeek V4.1 and render one result per tool call.

Claims

• Requests with tool_reference parts render on DeepSeek V4.1. On main they fail with DeepSeek V4.1 supports text and image content only; got 'tool_reference'. This covers tool_result blocks with tool_reference items on /v1/messages, and tool_reference parts on /v1/chat/completions.
• On V4.1, each reference renders as the tool's name inside its call's <tool_result>. Several references are joined with \n\n, the separator the encoder already uses between parts.
• /v1/messages emits one tool message per tool_result: the tool_reference parts first, then the text. On main, a tool search result renders an extra empty result before the references, on GLM-5.3 as well as V4.1, so after parallel tool calls the results no longer line up with the calls.
• Templates that expand references keep working: GLM-5.3 renders the referenced tool definitions as before.
• A tool_result without tool_reference items converts as before, and other unsupported part types still return 400 on V4.1.
• The change is in _convert_user_tool_result (vllm/entrypoints/anthropic/serving.py) and _normalize_messages (vllm/tokenizers/deepseek_v41.py): 15 lines added and 11 removed, plus three tests.

Validation

Check	Command	main 5d55ef1590	This PR
New tests	pytest tests/entrypoints/anthropic/test_anthropic_messages_conversion.py::TestToolResultContent::test_tool_result_with_tool_references tests/entrypoints/scale_out/render/test_render.py::test_messages_render_tool_search_result_for_deepseek_v41 tests/tokenizers_/test_deepseek_v41.py::test_tool_references_render_as_tool_names	3 failed (below)	3 passed
Changed test files	pytest tests/entrypoints/anthropic/test_anthropic_messages_conversion.py tests/entrypoints/scale_out/render/test_render.py tests/tokenizers_/test_deepseek_v41.py tests/tokenizers_/test_deepseek_v4.py	271 passed, 4 failed (the 3 new tests and 1 below)	274 passed, 1 failed (below)
vllm launch render with the DeepSeek-V4.1-Flash config and tokenizer, 4 requests (below)		4 of 4 HTTP 400	4 of 4 HTTP 200, one <tool_result> per call
vllm launch render with the GLM-5.3 config, tokenizer and chat template, 3 requests (below)		3 of 3 HTTP 200, one extra <tool_response> each	3 of 3 HTTP 200, one <tool_response> per call

The conversion test sends a tool_result with text, two tool_reference items and an image, and checks for one tool message with the references first, followed by the image's user message. The render test starts vllm launch render deepseek-ai/DeepSeek-V4.1-Flash through RemoteLaunchRenderServer, which downloads only the config and tokenizer, like the existing GPT-OSS render test. It sends a /v1/messages turn where a tool search and a file read ran in parallel, and checks the rendered user turn through /detokenize. On main the new tests fail like this:

E       AssertionError: assert ['assistant',...user', 'tool'] == ['assistant', 'tool', 'user']
E           AssertionError: {"error":{"message":"DeepSeek V4.1 supports text and image content only; got 'tool_reference'","type":"BadRequestError","param":null,"code":400}}
E                       ValueError: DeepSeek V4.1 supports text and image content only; got 'tool_reference'
3 failed, 3 warnings in 18.48s

With only the V4.1 encoder change and main's converter, the render test still fails: the parallel turn renders three <tool_result>s for two calls.

The one other failure in the changed files is test_tokenize_matches_chat_render_for_gpt_oss, which fails the same way on main here because the container cannot download the Harmony vocab file. Tests ran on CPU in vllm-ci-postmerge-repo:074c83a09a0b872cec92e244d91adf2b3c55e41b-arm64-cpu with the branch tree on PYTHONPATH. The repo's pre-commit hooks pass, including ruff, mypy and the test tethering check. No model evaluation is included: the change only affects prompts with tool_reference items.

Render server checks

DeepSeek V4.1. The server runs vllm launch render deepseek-ai/DeepSeek-V4.1-Flash --tokenizer-mode deepseek_v41 --reasoning-parser deepseek_v41 --tool-call-parser deepseek_v41 --enable-auto-tool-choice --language-model-only, with config.json and the tokenizer from revision 314113f8 and no weights. Every request declares a ToolSearch tool, plus get_weather and get_time with defer_loading: true.

1. /v1/messages/render, shaped like a Claude Code tool search: the last user turn is a tool_result whose content is [{"type": "tool_reference", "tool_name": "get_weather"}].
2. The same conversation one turn later, after get_weather was called and returned.
3. /v1/chat/completions/render with a tool message whose content is [{"type": "tool_reference", "name": "get_weather"}].
4. /v1/messages/render where one assistant turn calls ToolSearch and Read in parallel, and the user turn returns the reference result and a text result.

On main all four return the 400 above. On this branch, requests 1 and 3 end with <tool_result>get_weather</tool_result> before the generation prompt, request 2 has one result for each of its two calls, request 4 ends with <tool_result>get_weather</tool_result>\n\n<tool_result>NOTES-FILE-BODY</tool_result>, and the system prompt of every request lists all tools.

GLM-5.3. The server runs vllm launch render with the GLM-5.3 config.json and tokenizer, the chat template from Hub revision ba32f8c19c, --tool-call-parser glm47 --reasoning-parser glm45 --enable-auto-tool-choice, and requests 1 and 4 above plus a third /v1/messages request whose tool_result holds a text block and a tool_reference.

Request	main	This PR
Tool search	<tool_response></tool_response><tool_response><tools>{get_weather}</tools></tool_response>	<tool_response><tools>{get_weather}</tools></tool_response>
Parallel ToolSearch + Read	3 <tool_response>s for 2 calls, the first one empty	<tool_response><tools>{get_weather}</tools></tool_response><tool_response>NOTES-FILE-BODY</tool_response>
Text and reference in one result	<tool_response>Found 1 tool</tool_response><tool_response><tools>{get_weather}</tools></tool_response>	<tool_response><tools>{get_weather}</tools></tool_response>

{get_weather} stands for the tool's full JSON definition, which both arms render identically.

Details

Root cause. #40190 added tool_reference parts. _convert_user_tool_result emits a tool message with the result's text (empty for a plain tool search), then a second tool message with the tool_reference items, both for the same tool_use_id. An image in the result goes into a user message between the two. Templates and encoders that render one result per tool message render one result too many. Separately, V4.1 renders with its own encoder, and _normalize_messages raises ValueError for any part type other than text and image. The tool_result stays in the conversation history, so on V4.1 every later request in that conversation fails too, and an agent session that runs a tool search cannot continue.

Why one message, references first. Templates that expand references look for a tool message whose first part is a tool_reference (GLM-5.3's does), so the references lead and the text follows. Images stay in the user message after the tool message. Limitation: GLM-5.3's template expands such a message and skips its other parts, so text that sits beside references in one tool_result no longer renders there; on main it rendered as an extra result. The tool search docs show results that hold only tool_reference blocks.

Why the name. apply_chat_template puts every request tool into the V4.1 system prompt, deferred ones included, so the definitions a reference points at are already in context. The name tells the model which tools the search returned. The Cohere renderer renders these parts the same way (vllm/renderers/cohere.py). The model's Jinja template on the Hub drops unknown block types, which would render the search result empty.

Related. #52498 and #52610 (open, for #52489) rewrite tool_reference items as text in the Anthropic converter for every model. That would also stop the V4.1 400 on /v1/messages, but /v1/chat/completions requests with these parts would still fail, and the structured parts that templates expanding references rely on would be gone.

AI assistance. Claude Code was used for the root-cause analysis, the fix, the tests and this description.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[sfeng33]

Thanks for digging into this! I think the root cause is a bit upstream though. The Anthropic converter (_convert_user_tool_result) turns one tool_result with tool_reference parts into two tool messages: an empty one, then one with the references. So every renderer that pairs results with calls by position sees one extra result, not just V4.1.

I checked with vllm launch render. DeepSeek V4 and GLM-5.1 (the template tool_reference support was first built for) both render an extra empty result for a plain tool search result on main. The V4.1 merge also misses two cases: a result with an image (the image's user message sits between the two tool messages) and a result with no tool_use_id.

What do you think about fixing it in the converter instead? It would emit a single tool message per tool_result, with the tool_reference parts first and then any text. Then the merge pass in the V4.1 encoder isn't needed, and you keep your tool_reference-to-name rendering.

[vMaroon]

Agreed, the split in _convert_user_tool_result was the root cause, and both misses were real. Pushed 3b32554: one tool message per tool_result, tool_reference parts first, then text, and the V4.1 merge pass is gone. With vllm launch render, GLM-5.3 on main renders an extra empty <tool_response> (three for two parallel calls), and the branch renders one per call with the expansion intact. One tradeoff: GLM-5.3's template expands a references-first message and skips its other parts, so text next to references no longer renders there. The tool search docs show references-only results, so I think that can be a template follow-up.

[sfeng33]

Thank you, LGTM!

[sfeng33]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99040 for commit 2ece821140fa.