Skip to content

[CI] Include chat template fallbacks in package_data - #53762

Merged
DarkLight1337 merged 4 commits into
vllm-project:mainfrom
tarukumar:fix/chat-templates-package-data
Sep 2, 2026
Merged

DarkLight1337 merged 4 commits into
vllm-project:mainfrom
tarukumar:fix/chat-templates-package-data

Conversation

@tarukumar

@tarukumar tarukumar commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem

vllm/transformers_utils/chat_templates/registry.py resolves built-in multimodal chat template fallbacks from files such as template_chatml.jinja in the same directory. After installing vLLM from a wheel built from a source archive (without a git checkout), only the Python modules are present:

chat_templates/
  __init__.py
  registry.py

The *.jinja files are missing, which breaks models that rely on these fallbacks when no Hugging Face chat template is available.

Root cause

  • __init__.py and registry.py are installed automatically as Python package modules.
  • *.jinja files are package data and are not listed in package_data or MANIFEST.in.
  • Builds from a git checkout may still include the templates because setuptools discovers them during egg_info.
  • Builds from a source archive regenerate SOURCES.txt without the .jinja files, so they are omitted from sdists and wheels.

Fix

Declare the templates explicitly in setup.py:

"transformers_utils/chat_templates/*.jinja",

Verification

git archive --format=tar HEAD | tar x -C /tmp/vllm-src
cd /tmp/vllm-src
VLLM_TARGET_DEVICE=empty BUILD_VERSION=0.0.0 python3 -m pip wheel . --no-deps -w /tmp/wheels
unzip -l /tmp/wheels/vllm-*.whl | grep transformers_utils/chat_templates

Before this change, the wheel contains only __init__.py and registry.py. After this change, all template_*.jinja files are included.

LOG:
If people do not copy the template it fails like this if you are creating a wheel and installing it

(APIServer pid=1)   File "/opt/app-root/lib64/python3.12/site-packages/vllm/renderers/base.py", line 977, in render_chat
(APIServer pid=1)     self.render_messages(conversation, chat_params)
(APIServer pid=1)   File "/opt/app-root/lib64/python3.12/site-packages/vllm/renderers/hf.py", line 829, in render_messages
(APIServer pid=1)     content_format=resolve_chat_template_content_format(
(APIServer pid=1)                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/opt/app-root/lib64/python3.12/site-packages/vllm/renderers/hf.py", line 512, in resolve_chat_template_content_format
(APIServer pid=1)     detected_format = _resolve_chat_template_content_format(
(APIServer pid=1)                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/opt/app-root/lib64/python3.12/site-packages/vllm/renderers/hf.py", line 455, in _resolve_chat_template_content_format
(APIServer pid=1)     resolved_chat_template = resolve_chat_template(
(APIServer pid=1)                              ^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/opt/app-root/lib64/python3.12/site-packages/vllm/renderers/hf.py", line 308, in resolve_chat_template
(APIServer pid=1)     chat_template = load_chat_template(path)
(APIServer pid=1)                     ^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/opt/app-root/lib64/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 1354, in load_chat_template
(APIServer pid=1)     return _cached_load_chat_template(chat_template, is_literal=is_literal)
(APIServer pid=1)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/opt/app-root/lib64/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 1315, in _load_chat_template
(APIServer pid=1)     with open(chat_template) as f:
(APIServer pid=1)          ^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) FileNotFoundError: [Errno 2] No such file or directory: '/opt/app-root/lib64/python3.12/site-packages/vllm/transformers_utils/chat_templates/template_basic.jinja'

@mergify mergify Bot added the ci/build label Aug 25, 2026
@tarukumar
tarukumar marked this pull request as ready for review August 25, 2026 16:25
@tarukumar
tarukumar requested a review from khluu as a code owner August 25, 2026 16:25

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Ship transformers_utils/chat_templates/*.jinja in sdists and wheels so
built-in multimodal fallback templates referenced by registry.py are
available after pip install, including builds from source archives
without a git checkout.

Signed-off-by: Tarun Kumar <takumar@redhat.com>
@tarukumar
tarukumar force-pushed the fix/chat-templates-package-data branch from 5a962ee to 900b125 Compare August 25, 2026 16:32
@DarkLight1337
DarkLight1337 enabled auto-merge (squash) September 2, 2026 03:56
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 2, 2026
@tarukumar tarukumar changed the title Include chat template fallbacks in package_data [CI] Include chat template fallbacks in package_data Sep 2, 2026
@DarkLight1337

Copy link
Copy Markdown
Member

/ci run

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #86792 for commit e6f96bf5bcff.

@DarkLight1337
DarkLight1337 merged commit 3140531 into vllm-project:main Sep 2, 2026
293 of 294 checks passed
mylibrar pushed a commit to tanyuqian/vllm that referenced this pull request Sep 3, 2026
Signed-off-by: Tarun Kumar <takumar@redhat.com>
michael-gpu pushed a commit to practical-inference/vllm-ampere-extended that referenced this pull request Oct 20, 2026
Signed-off-by: Tarun Kumar <takumar@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants