Skip to content

fix(plugin): fix Kimi-K3's KDA autotune key in the plugin, not the vLLM image - #2835

Merged
valarLip merged 1 commit into
mainfrom
perzhang/k3-kda-autotune-plugin-patch
Oct 31, 2026
Merged

valarLip merged 1 commit into
mainfrom
perzhang/k3-kda-autotune-plugin-patch

Conversation

@PerryZhang01

@PerryZhang01 PerryZhang01 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Follows up #2818. Ready for review.

#2818 fixed Kimi-K3's Triton 3.8 import failure (key contains names that are not kernel arguments: NC) as a patch on the vLLM image build, which only reaches CI once vllm-latest is rebuilt. Until then every PR's Kimi-K3 cell still fails, as in the runs of #2785 and #2816 after #2818 merged. This moves the fix into the plugin, where it takes effect with the ATOM install.

Dev record

Named result

On the unpatched vllm-latest image (Triton 3.8.0): import vllm.models.kimi_k3 raises NC. After load_general_plugins(), ATOM's K3 model module and kda_backend import, and chunk_kda_fwd_kernel_intra_sub_chunk keys on ['BK', 'BT', 'HV'].

This PR's CI (40996045263) built a vLLM image without the patch; all six cells pass on V2. Kimi-K3 0.956 (threshold 0.94) with the plugin's INFO line for the dropped NC and no NC error; V4-Flash 0.9575, GLM-5.2 0.9272, M3 0.9424, V4.1 0.9287, Qwen3.5 0.8453.

Gates

  • tests/test_vllm_kimi_k3_kda_autotune_patch.py: 2 passed (real Triton, no vLLM or GPU).
  • black clean; ruff clean on the new files.

🤖 Generated with Claude Code

…LM image

#2818 fixed Kimi-K3's import failure under Triton 3.8

    ValueError: key contains names that are not kernel arguments: NC.

by applying vllm-project/vllm#60446 as a patch on the vLLM build. That
only reaches a deployment once its vLLM image is rebuilt: the nightly
vllm-latest built before #2818 merged still fails every PR's Kimi-K3 cell.

Fix it where the plugin fixes vLLM. register_model wraps
triton.autotune: for a kernel defined under vllm.models.kimi_k3 it drops
the key names that are not kernel arguments, which is what #60446 does
by hand, and passes every other kernel through unchanged. The decorator
runs at import, so the wrapper has to precede any import of
vllm.models.kimi_k3; vLLM's model-inspection subprocess, where the
failure surfaced, runs load_general_plugins and so this hook first. A
vLLM that carries #60446 names no such key and nothing is dropped; an
INFO line names each dropped key.

The image patch and its dockerfile steps are removed; the vLLM recipe
is back to its state before #2818.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2835 --add-label <label>

@valarLip
valarLip merged commit e281d58 into main Oct 31, 2026
28 of 45 checks passed
@valarLip
valarLip deleted the perzhang/k3-kda-autotune-plugin-patch branch October 31, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants