Add filtering for chat template kwargs

[russellb]

• GHSA-6fvq-23cw-5628
• https://github.com/vllm-project/vllm/security/advisories/GHSA-4hw3-g54f-xfrj

[gemini-code-assist]

Code Review

This pull request addresses a critical security vulnerability (GHSA-6fvq-23cw-5628) related to arbitrary code execution via malicious chat templates. The core of the fix is the new resolve_chat_template_kwargs function, which intelligently filters keyword arguments passed to the Jinja2 template renderer. It ensures that only arguments both supported by the apply_chat_template function and explicitly declared as variables within the template are passed, effectively blocking the injection of a malicious chat_template. The changes also introduce a --trust-request-chat_template flag for defense-in-depth, preventing user-supplied templates from being used unless explicitly allowed. The implementation is robust, and the accompanying tests correctly validate the fix. Overall, this is a solid and important security patch.

[DarkLight1337]

@Isotr0py PTAL at the V1 test failure

[DarkLight1337]

Let's also reject the request if it passes an untrusted chat template, to avoid silent regressions

[youkaichao]

is it called by each request? do we need to cache it?

[Isotr0py]

Yea, it's called by each request. This function won't compile the chat template, so I think it won't introduces too much overhead.

Given that kwargs can be various depending on the request's extra body, it's not really applicable to cache it.

[Isotr0py]

Add cache in #26227

[Eviannn]

Yea, it's called by each request. This function won't compile the chat template, so I think it won't introduces too much overhead.

Given that kwargs can be various depending on the request's extra body, it's not really applicable to cache it.

using npu(ascend 910c) it costs about 10-20ms, it is great to cache it

[wuwenbin]

It seems this change might be linked to a ~10x latency jump we saw in our benchmarks on 0.11.0 (where CPU hit 100%).
The caching in 0.11.1 appears to solve it, but upgrading is a bit tricky since it requires CUDA ≥ 12.9.

[DarkLight1337]

You can still use older CUDA versions if you install vLLM from source.

[youkaichao]

in which case do we ever use chat template from users? I thought we should just disallow it.

[bbrowning]

We may also want a similar trust_request_chat_template around

vllm/vllm/entrypoints/openai/serving_tokenization.py

Line 83 in 2a6dc67

chat_template=request.chat_template or self.chat_template,