Repository navigation
fix(plugin): fix Kimi-K3's KDA autotune key in the plugin, not the vLLM image - #2835
Merged
Merged
Conversation
…LM image #2818 fixed Kimi-K3's import failure under Triton 3.8 ValueError: key contains names that are not kernel arguments: NC. by applying vllm-project/vllm#60446 as a patch on the vLLM build. That only reaches a deployment once its vLLM image is rebuilt: the nightly vllm-latest built before #2818 merged still fails every PR's Kimi-K3 cell. Fix it where the plugin fixes vLLM. register_model wraps triton.autotune: for a kernel defined under vllm.models.kimi_k3 it drops the key names that are not kernel arguments, which is what #60446 does by hand, and passes every other kernel through unchanged. The decorator runs at import, so the wrapper has to precede any import of vllm.models.kimi_k3; vLLM's model-inspection subprocess, where the failure surfaced, runs load_general_plugins and so this hook first. A vLLM that carries #60446 names no such key and nothing is dropped; an INFO line names each dropped key. The image patch and its dockerfile steps are removed; the vLLM recipe is back to its state before #2818. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
valarLip
approved these changes
Oct 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows up #2818. Ready for review.
#2818 fixed Kimi-K3's Triton 3.8 import failure (
key contains names that are not kernel arguments: NC) as a patch on the vLLM image build, which only reaches CI oncevllm-latestis rebuilt. Until then every PR's Kimi-K3 cell still fails, as in the runs of #2785 and #2816 after #2818 merged. This moves the fix into the plugin, where it takes effect with the ATOM install.Dev record
@triton.autotuneat import time, so the kernel cannot be patched after import.register_modelwrapstriton.autotuneand drops non-argument keys only for kernels defined undervllm.models.kimi_k3, the same edit as vllm#60446. Check first: vLLM's model-inspection subprocess runsload_general_pluginsbefore importing the model class, so the wrapper is in place there too.docker/matches main before fix(plugin): green the vLLM 0.31 plugin CI cells; V4-Flash and GLM-5.2 on Model Runner V2 #2818. The PR job's rebuild-on-recipe-change rule from fix(plugin): green the vLLM 0.31 plugin CI cells; V4-Flash and GLM-5.2 on Model Runner V2 #2818 stays, so this PR's job builds an image without the patch.Named result
On the unpatched
vllm-latestimage (Triton 3.8.0):import vllm.models.kimi_k3raisesNC. Afterload_general_plugins(), ATOM's K3 model module andkda_backendimport, andchunk_kda_fwd_kernel_intra_sub_chunkkeys on['BK', 'BT', 'HV'].This PR's CI (40996045263) built a vLLM image without the patch; all six cells pass on V2. Kimi-K3 0.956 (threshold 0.94) with the plugin's INFO line for the dropped
NCand noNCerror; V4-Flash 0.9575, GLM-5.2 0.9272, M3 0.9424, V4.1 0.9287, Qwen3.5 0.8453.Gates
tests/test_vllm_kimi_k3_kda_autotune_patch.py: 2 passed (real Triton, no vLLM or GPU).🤖 Generated with Claude Code