From e9b854ab99d8da0626948e1598dd35b414cf892a Mon Sep 17 00:00:00 2001 From: C2 Wang Date: Wed, 22 Apr 2026 13:08:01 -0700 Subject: [PATCH 1/3] build(vllm-tensorizer): Bump vllm to v0.19.1, pin transformers 5.5.4 + nixl 1.0.0 vLLM bump Upstream released v0.19.1 (2026-04-22, commit b1388b1fb) cherry-picking the transformers-v5 refactor (vllm#30566) on top of the v0.19.0 tree. Its common.txt declares `transformers >= 4.56.0, != 5.5.0` and its test.txt pins the tested combo at transformers 5.5.3 / hf-hub 1.10.2 / tokenizers 0.22.2 / hf-xet 1.4.3. compressed-tensors bumps 0.14.0.1 -> 0.15.0.1. Transformers pin Replace the old `transformers >= 5.5.0` floor with an exact pin to 5.5.4. 5.5.4 is the first release containing huggingface/transformers PR #45359, which removes `kimi_k25` from MODELS_WITH_INCORRECT_HUB_TOKENIZER_CLASS and restores the custom TikTokenTokenizer path with sparse added_tokens_decoder IDs. Earlier 5.x versions dense-pack those IDs and scramble Kimi K2.5 tool-call output (`<|tool_call_begin|>` encoded as 163595 instead of 163597, shifting every tool-call marker by -2 or -3). 5.5.4 also satisfies the Gemma 4 > 5.5.0 requirement mentioned in the previous comment. Nixl pin nixl-cu12 1.0.1 wheel (released after the previous ml-containers build) ships a different bundled libucs.so than 1.0.0. Its ucs_topo_release_devices() has a static-destructor ordering bug that SIGSEGVs during Python interpreter shutdown (atexit). vLLM's model-registry subprocess calls check_returncode() before reading its pickled output file, so a shutdown-time SIGSEGV surfaces as `pydantic ValidationError: Model architectures [...] failed to be inspected`, even though the inspection work itself completed. Pinning nixl-cu12 and nixl to 1.0.0 swaps in the known-good bundled libucs.so. Drop this pin once nixl ships a fixed release. Evidence: full gdb backtrace at projects/auto-debug-v0191-segv/logs/research-02/ gdb-backtrace-v2.txt in the k25-bug workspace. Validation 2x2 factorial tested on GB200 in the ace-inference namespace (dev-us-e-01a): R1A (v0.19.1 no nixl pin) -> CrashLoop, 6+ restarts, UCX atexit SIGSEGV R1B (v0.19.1 nixl 1.0.0) -> 1/1 Running, 15/15 functional tests PASS R2C (v0.19.0 no nixl pin) -> 1/1 Running, 15/15 PASS (v0.19.0 base was built before nixl 1.0.1 released) R2D (v0.19.0 nixl 1.0.0) -> 1/1 Running, 15/15 PASS (pin is no-op on v0.19.0 base) See projects/k25-bug/result.md for the full test matrix. --- .github/configurations/vllm-tensorizer.yml | 2 +- vllm-tensorizer/Dockerfile | 19 ++++++++++++++++--- 2 files changed, 17 insertions(+), 4 deletions(-) diff --git a/.github/configurations/vllm-tensorizer.yml b/.github/configurations/vllm-tensorizer.yml index a50ccd32..da1fedf0 100644 --- a/.github/configurations/vllm-tensorizer.yml +++ b/.github/configurations/vllm-tensorizer.yml @@ -1,5 +1,5 @@ vllm-commit: - - 'v0.19.0' + - 'v0.19.1' flashinfer-commit: - 'v0.6.6' lmcache-commit: diff --git a/vllm-tensorizer/Dockerfile b/vllm-tensorizer/Dockerfile index f8669b34..80fe0087 100644 --- a/vllm-tensorizer/Dockerfile +++ b/vllm-tensorizer/Dockerfile @@ -293,8 +293,19 @@ ARG TARGETPLATFORM # Ray was removed as a default dependancy by vllm in v18, but it is still required # for the multi-node setup. -# Gemma 4 needs transformers > 5.5.0, but is packaged with 4 in vllm 19. -# We must explicitly update it until vLLM updates their dependency. +# Pin transformers to 5.5.4: satisfies both Gemma 4 (needs > 5.5.0) and +# Kimi K2.5 (HF PR #45359 landed in 5.5.4 restores the custom slow +# TikTokenTokenizer path with sparse added_tokens_decoder IDs — earlier +# 5.x versions dense-pack them and scramble tool-call output). vLLM +# 0.19.1's common.txt declares `transformers >= 4.56.0, != 5.5.0`, so +# 5.5.4 is in-spec. +# +# Pin nixl and nixl-cu12 to 1.0.0: nixl-cu12 1.0.1 bundles a different +# libucs.so with a static-destructor ordering bug in ucs_topo_release_devices +# that SIGSEGVs during Python interpreter shutdown (atexit), causing +# vLLM's model-registry subprocess to return non-zero even though the +# inspection work completes. 1.0.0's bundled libucs.so does not have +# this bug. Drop this pin once nixl ships a fixed release. RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \ BITSANDBYTES_VER='0.42.0'; \ @@ -303,7 +314,9 @@ RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \ fi && \ python3 -m pip install --no-cache-dir \ accelerate hf_transfer 'modelscope!=1.15.0' "bitsandbytes>=${BITSANDBYTES_VER:?}" 'timm>=1.0.17' \ - 'runai-model-streamer[s3,gcs]>=0.15.3' "ray[cgraph]>=2.48.0" "transformers>=5.5.0" -c /tmp/constraints.txt && \ + 'runai-model-streamer[s3,gcs]>=0.15.3' "ray[cgraph]>=2.48.0" \ + 'transformers==5.5.4' 'nixl==1.0.0' 'nixl-cu12==1.0.0' \ + -c /tmp/constraints.txt && \ rm /tmp/constraints.txt EXPOSE 8080 From 24b0ba03e2719c6916db5c7599acabe8b120a04d Mon Sep 17 00:00:00 2001 From: C2 Wang Date: Wed, 22 Apr 2026 13:18:38 -0700 Subject: [PATCH 2/3] chore(vllm-tensorizer): Shorten pin-rationale comments --- vllm-tensorizer/Dockerfile | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/vllm-tensorizer/Dockerfile b/vllm-tensorizer/Dockerfile index 80fe0087..e5a08f3c 100644 --- a/vllm-tensorizer/Dockerfile +++ b/vllm-tensorizer/Dockerfile @@ -293,19 +293,9 @@ ARG TARGETPLATFORM # Ray was removed as a default dependancy by vllm in v18, but it is still required # for the multi-node setup. -# Pin transformers to 5.5.4: satisfies both Gemma 4 (needs > 5.5.0) and -# Kimi K2.5 (HF PR #45359 landed in 5.5.4 restores the custom slow -# TikTokenTokenizer path with sparse added_tokens_decoder IDs — earlier -# 5.x versions dense-pack them and scramble tool-call output). vLLM -# 0.19.1's common.txt declares `transformers >= 4.56.0, != 5.5.0`, so -# 5.5.4 is in-spec. -# -# Pin nixl and nixl-cu12 to 1.0.0: nixl-cu12 1.0.1 bundles a different -# libucs.so with a static-destructor ordering bug in ucs_topo_release_devices -# that SIGSEGVs during Python interpreter shutdown (atexit), causing -# vLLM's model-registry subprocess to return non-zero even though the -# inspection work completes. 1.0.0's bundled libucs.so does not have -# this bug. Drop this pin once nixl ships a fixed release. +# * Gemma 4 needs transformers > 5.5.0 +# * Kimi K2.5 tool-call scramble fixed in transformers 5.5.4 (huggingface/transformers#45359) +# * Kimi K2.5 registry-subprocess SIGSEGV on GB200 with nixl-cu12 1.0.1 — pin both to 1.0.0 RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \ BITSANDBYTES_VER='0.42.0'; \ From b4d9159505f5bfb058073d1444571699d66bf192 Mon Sep 17 00:00:00 2001 From: C2 Wang Date: Wed, 22 Apr 2026 13:19:29 -0700 Subject: [PATCH 3/3] docs(vllm-tensorizer): Link nixl pin to vllm-project/vllm#40642 --- vllm-tensorizer/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vllm-tensorizer/Dockerfile b/vllm-tensorizer/Dockerfile index e5a08f3c..ccb036bc 100644 --- a/vllm-tensorizer/Dockerfile +++ b/vllm-tensorizer/Dockerfile @@ -295,7 +295,7 @@ ARG TARGETPLATFORM # * Gemma 4 needs transformers > 5.5.0 # * Kimi K2.5 tool-call scramble fixed in transformers 5.5.4 (huggingface/transformers#45359) -# * Kimi K2.5 registry-subprocess SIGSEGV on GB200 with nixl-cu12 1.0.1 — pin both to 1.0.0 +# * Kimi K2.5 registry-subprocess SIGSEGV on GB200 with nixl-cu12 1.0.1 (vllm-project/vllm#40642) — pin both to 1.0.0 RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \ BITSANDBYTES_VER='0.42.0'; \