[Bugfix] Fix tool_calls Iterable consumed when debug logging is enabled

[wojciech-wais]

Fixes #34792.

When VLLM_LOGGING_LEVEL=debug is set, tool calling with Mistral models fails with ValueError: Unexpected tool call id ....

Root cause

The OpenAI Python library types tool_calls as Iterable[...] in ChatCompletionAssistantMessageParam. When Pydantic v2 validates a ChatCompletionRequest from Python objects (not from a raw JSON body, as happens in the Anthropic → OpenAI conversion path inside _convert_anthropic_to_openai_request), it wraps Iterable fields in a one-shot lazy iterator. Debug logging then calls chat_req.model_dump_json(), which serialises the model and exhausts the iterator. Subsequent readers — notably the Mistral tokenizer's apply_chat_template — see an empty tool_calls sequence and raise the "Unexpected tool call id" error.

Fix

1. vllm/entrypoints/chat_utils.py — change tool_calls from Iterable[...] to list[...] in both vLLM-owned TypedDicts (CustomChatCompletionMessageParam and ConversationMessage). This corrects the type annotation and prevents Pydantic from creating lazy iterators for these vLLM-specific message types.

2. vllm/entrypoints/openai/chat_completion/protocol.py — add a field_validator on ChatCompletionRequest.messages that eagerly materialises any non-list tool_calls to list. This covers the external OpenAI library types (e.g. ChatCompletionAssistantMessageParam) that still annotate tool_calls as Iterable[...] and therefore can produce lazy iterators during Python-object validation.

Purpose

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

The pull request effectively resolves the issue where tool_calls iterators were being prematurely exhausted when debug logging was enabled. By changing the type annotation to list in vLLM's internal message types and adding a Pydantic field_validator to eagerly materialize tool_calls in ChatCompletionRequest, the fix ensures that these calls can be accessed multiple times without data loss. The implementation is correct, follows Pydantic v2 best practices for field normalization, and includes comprehensive regression tests. No high or critical severity issues were identified during the review.

[DarkLight1337]

Thanks for fixing!

[NickLucche]

cc @dtrifiro

[DarkLight1337]

Please fix https://buildkite.com/vllm/ci/builds/52197/steps/canvas?sid=019c73be-e177-4ae9-b62b-82e88e15cf99&tab=output

[wojciech-wais]

I've rebased changes and fixed failures in test_tool_calls_serialization.py test.

[bbrowning]

These pydantic lazy iterators have bit me a few times in the past in other places, so solving this generally would be reasonable. There's already some code in both mistral and gpt-oss model paths that do some of this materialization, but only for those models and perhaps not in all cases.

However, some of the added unit tests are still failing. Those will need to be investigated and fixed for this to merge. If you need help root-causing those, let me know and I'm happy to help out.

Thanks!

[wojciech-wais]

Thanks @bbrowning!

I've investigated and fixed the failing unit tests.

The initial mode="after" model validator wasn't sufficient because Pydantic v2 consumes one-shot generators/iterators during union type validation of ChatCompletionMessageParam (before the "after" validator runs). Switching to mode="before" alone also didn't work - Pydantic re-wraps even plain lists in a ValidatorIterator when validating against Iterable[...] typed fields in the union. The fix uses both a mode="before" validator (catches generators before Pydantic consumes them) and a mode="after" validator (unwraps Pydantic's ValidatorIterator back to lists).

Fixed tests:

• test_tool_calls_list_preserved_after_model_dump
• test_tool_calls_from_generator_are_materialised
• test_tool_calls_list_passthrough
• test_messages_without_tool_calls_unaffected
• test_multiple_tool_calls_materialised[1] / test_multiple_tool_calls_materialised[3]

Remaining CI failures:
The only failing job is async-engine-inputs-utils-worker-config-cpu. The failures there are test_extract_tool_calls_anyof_type_conversion and test_extract_tool_calls_anyof_type_conversion_streaming in tests/tool_parsers/test_qwen3coder_tool_parser.py. IMO these are pre-existing issues with the qwen3coder tool parser's anyOf type conversion. This PR does not touch qwen3coder_tool_parser.py or any related code.

[bbrowning]

I agree the CI failures you hit here were caused by the merging of #37831 and it looks like it was reverted by #38751. So, if you merge latest main into this you should get that revert and end up with green tests here.