[Core] Profile persistent FlashInfer workspace before KV sizing - #102

Open

lesj0610 wants to merge 24 commits into

mainfrom

lesj/use-flashinfer-workspace-sizing-20260626

Commits on Sep 15, 2026

[Core] Profile FlashInfer's persistent attention workspace before KV sizing
lesj0610
committed
8e7795f
[Core] Profile FlashInfer's persistent attention workspace before KV sizing
lesj0610
committed
d90f87a
[Core] Do not close the profiling lifecycle on a model flag alone
lesj0610
committed
2c83c48

Commits on Sep 16, 2026

[Bugfix][Responses] Clean up MCP tool sessions once before closing them (#56988)

shaohuaxi
and
codex
authored
e0301af
[Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014)

authored
0f8fa53
[Kernel] Use Murmur3 RNG for Gumbel sampling (#51367)

authored
18553c5
[Bugfix] Fix incorrect Mamba block allocation estimate that prevents request admission (#57050)
wzhao18
authored
f30a195
[ROCm][AITER] Skip AITER norm kernels when flattening to 2D would copy (#55991)

authored
ab35354
[Bugfix][KV Offload] Track cache recency once per request (#51787)

mindungil
and
orozery
authored
f12fe10
[Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985)
bigPYJ1151
authored
b684080
fix(multimodal): tolerate malformed EXIF metadata during hashing (#56527) (#56576)
amasen02
authored
9ca6dbb
[Parser] Fix: correct parser frontend handling of reasoning end and boundary tokens (#56635)
devtyagi3909
authored
8be5205
[Bugfix][Frontend] Lazy-import model_hosting_container_standards to prevent log suppression (#51366)
wenjinhust
authored
0d8173d
[Feature][Frontend] Expose effective attention block size for DCP (#56538)

authored
f37c550
[XPU][Bugfix] Fix Qwen2-Audio ValueError on audio clips longer than 30s (#56912)

authored
35f6047
[UX] Add thinking support to `vllm chat` (#57045)
mgoin
authored
0384e72
[Perf][DSV4.1] Pad shared experts for native MegaMoE fusion (#56568)

gcanlin
and
codex
authored
03f67b3
[Bugfix][CPU] Fall back to CpuPlatform when zentorch fails to import (#54923)

ganeshr10
and
cursoragent
authored
8273193
[Docs] Split slash-combined docstring parameters (#57141)

hmellor
and
claude
authored
5f203ba
[Misc] Remove no-op self-assignments across vLLM (#55988)

AdaAibaby
and
adababys
authored
ceb87de
[Model][DSv4.1] FlashMLA mega attention and the NVFP4 compressed KV cache (#56935)

authored
d6a1677
[Platform] Move env check function to platform interface (#48599)

wangxiyuan
and
mergify[bot]
authored
118e17f
Merge upstream main into lesj/use-flashinfer-workspace-sizing-20260626
lesj0610
committed
f8bbb71
[Bugfix][Test] Keep the CUDA graph profiler out of the workspace warmup test
lesj0610
committed
da9e219

Navigation Menu

[Core] Profile persistent FlashInfer workspace before KV sizing - #102

[Core] Profile persistent FlashInfer workspace before KV sizing#102
lesj0610 wants to merge 24 commits into
mainfrom
lesj/use-flashinfer-workspace-sizing-20260626

Commits on Sep 15, 2026

[Core] Profile FlashInfer's persistent attention workspace before KV sizing

[Core] Profile FlashInfer's persistent attention workspace before KV sizing

[Core] Do not close the profiling lifecycle on a model flag alone

Commits on Sep 16, 2026

[Bugfix][Responses] Clean up MCP tool sessions once before closing them (#56988)

[Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014)

[Kernel] Use Murmur3 RNG for Gumbel sampling (#51367)

[Bugfix] Fix incorrect Mamba block allocation estimate that prevents request admission (#57050)

[ROCm][AITER] Skip AITER norm kernels when flattening to 2D would copy (#55991)

[Bugfix][KV Offload] Track cache recency once per request (#51787)

[Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985)

fix(multimodal): tolerate malformed EXIF metadata during hashing (#56527) (#56576)

[Parser] Fix: correct parser frontend handling of reasoning end and boundary tokens (#56635)

[Bugfix][Frontend] Lazy-import model_hosting_container_standards to prevent log suppression (#51366)

[Feature][Frontend] Expose effective attention block size for DCP (#56538)

[XPU][Bugfix] Fix Qwen2-Audio ValueError on audio clips longer than 30s (#56912)

[UX] Add thinking support to `vllm chat` (#57045)

[Perf][DSV4.1] Pad shared experts for native MegaMoE fusion (#56568)

[Bugfix][CPU] Fall back to CpuPlatform when zentorch fails to import (#54923)

[Docs] Split slash-combined docstring parameters (#57141)

[Misc] Remove no-op self-assignments across vLLM (#55988)

[Model][DSv4.1] FlashMLA mega attention and the NVFP4 compressed KV cache (#56935)

[Platform] Move env check function to platform interface (#48599)

Merge upstream main into lesj/use-flashinfer-workspace-sizing-20260626

[Bugfix][Test] Keep the CUDA graph profiler out of the workspace warmup test