Skip to content

[Bugfix] precompile Triton KV-zero and slot-mapping kernels (#43009) - #43879

Draft
zhougit86 wants to merge 1 commit into
vllm-project:mainfrom
zhougit86:fix/43009
Draft

zhougit86 wants to merge 1 commit into
vllm-project:mainfrom
zhougit86:fix/43009

Conversation

@zhougit86

Copy link
Copy Markdown
Contributor

Purpose

Draft fix for Triton JIT happening in inference hot paths.

This change adds minimal warmup calls for:

  • KV block zeroing
  • slot mapping

The intent is only to trigger these Triton paths once during warmup through the existing wrapper paths, so their first compilation does not happen during inference.

Related issue:

Test Plan

todo

Test Result

todo


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

@zhougit86
zhougit86 marked this pull request as draft May 28, 2026 13:28
@mergify mergify Bot added the bug Something isn't working label May 28, 2026
@Nightwing-77

Nightwing-77 commented Jun 3, 2026

Copy link
Copy Markdown

not sure if the bug fix for #43009 should be focused on adding another warmup rather can we debug why the existing warmup stage is being cut short

@mergify

mergify Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @zhougit86.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working needs-rebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Triton kernel JIT compilation during inference

2 participants