[WIP] Run eagle with full cudagraph#20190

Closed

zixi-qi wants to merge 64 commits intovllm-project:mainfrom

zixi-qi:eagle-full-cudagraph

Commits on Jul 14, 2025

run eagle with full cudagraph
zixi-qi
committed

Commits on Jul 23, 2025

run eagle with full cudagraph
zixi-qi
committed
Merge branch 'eagle-full-cudagraph' of github.com:zixi-qi/vllm into eagle-full-cudagraph
zixi-qi
committed
rebase and add unit test
zixi-qi
committed
Fix bad lm-eval fork (#21318)

mgoin
authored and
zixi-qi
committed
[perf] Speed up align sum kernels (#21079)

hj-mistral
authored and
zixi-qi
committed
[v1][sampler] Inplace logprobs comparison to get the token rank (#21283)

houseroad
authored and
zixi-qi
committed
[XPU] Enable external_launcher to serve as an executor via torchrun (#21021)

chaojun-zhang
authored and
zixi-qi
committed
[Doc] Fix CPU doc format (#21316)

bigPYJ1151
authored and
zixi-qi
committed
[Intel GPU] Ray Compiled Graph avoid NCCL for Intel GPU (#21338)

ratnampa
authored and
zixi-qi
committed
Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334)

minosfuture
authored and
zixi-qi
committed
[Core] Minimize number of dict lookup in _maybe_evict_cached_block (#21281)

Jialin
authored and
zixi-qi
committed
[V1] [Hybrid] Add new test to verify that hybrid views into KVCacheTensor are compatible (#21300)

tdoublep
authored and
zixi-qi
committed
[Refactor] Fix Compile Warning #1444-D (#21208)

yewentao256
authored and
zixi-qi
committed
Fix kv_cache_dtype handling for out-of-tree HPU plugin (#21302)

authored and
zixi-qi
committed
[Misc] DeepEPHighThroughtput - Enable Inductor pass (#21311)

authored and
zixi-qi
committed
[Bug] DeepGemm: Fix Cuda Init Error (#21312)

yewentao256
authored and
zixi-qi
committed
Update fp4 quantize API (#21327)

wenscarl
authored and
zixi-qi
committed
[Feature][eplb] add verify ep or tp or dp (#21102)

lengrongfu
authored and
zixi-qi
committed
Add arcee model (#21296)

authored and
zixi-qi
committed
[Bugfix] Fix eviction cached blocked logic (#21357)

simon-mo
authored and
zixi-qi
committed
[Misc] Remove deprecated args in v0.10 (#21349)

kebe7jun
authored and
zixi-qi
committed
[Core] Optimize update checks in LogitsProcessor (#21245)

Jialin
authored and
zixi-qi
committed
[benchmark] Port benchmark request sent optimization to benchmark_serving (#21209)

Jialin
authored and
zixi-qi
committed
[Core] Introduce popleft_n and append_n in FreeKVCacheBlockQueue to further optimize block_pool (#21222)

Jialin
authored and
zixi-qi
committed
[Misc] unify variable for LLM instance v2 (#21356)

andyxning
authored and
zixi-qi
committed
[perf] Add fused MLA QKV + strided layernorm (#21116)

authored and
zixi-qi
committed
[feat]: add SM100 support for cutlass FP8 groupGEMM (#20447)

authored and
zixi-qi
committed
[Perf] Cuda Kernel for Per Token Group Quant (#21083)

yewentao256
authored and
zixi-qi
committed
Adds parallel model weight loading for runai_streamer (#21330)

authored and
zixi-qi
committed
[feat] Enable mm caching for transformers backend (#21358)

zucchini-nlp
authored and
zixi-qi
committed
Revert "[Refactor] Fix Compile Warning #1444-D (#21208)" (#21384)

yewentao256
authored and
zixi-qi
committed
Add tokenization_kwargs to encode for embedding model truncation (#21033)

Receiling
authored and
zixi-qi
committed
[Bugfix] Decode Tokenized IDs to Strings for `hf_processor` in `llm.chat()` with `model_impl=transformers` (#21353)

ariG23498
authored and
zixi-qi
committed
[CI/Build] Fix test failure due to updated model repo (#21375)

DarkLight1337
authored and
zixi-qi
committed
Fix Flashinfer Allreduce+Norm enable disable calculation based on `fi_allreduce_fusion_max_token_num` (#21325)

xinli-sw
authored and
zixi-qi
committed
[Model] Add Qwen3CoderToolParser (#21396)

authored and
zixi-qi
committed
[Misc] Copy HF_TOKEN env var to Ray workers (#21406)

ruisearch42
authored and
zixi-qi
committed
[BugFix] Fix ray import error mem cleanup bug (#21381)

authored and
zixi-qi
committed
[CI/Build] Fix model executor tests (#21387)

DarkLight1337
authored and
zixi-qi
committed
[Bugfix][ROCm][Build] Fix build regression on ROCm (#21393)

gshtras
authored and
zixi-qi
committed
Simplify weight loading in Transformers backend (#21382)

hmellor
authored and
zixi-qi
committed
[BugFix] Update python to python3 calls for image; fix prefix & input calculations. (#21391)

authored and
zixi-qi
committed
[BUGFIX] deepseek-v2-lite failed due to fused_qkv_a_proj name update (#21414)

xuechendi
authored and
zixi-qi
committed
[Bugfix][CUDA] fixes CUDA FP8 kv cache dtype supported (#21420)

elvischenv
authored and
zixi-qi
committed
Changing "amdproduction" allocation. (#21409)

Alexei-V-Ivanov-AMD
authored and
zixi-qi
committed
[Bugfix] Fix nightly transformers CI failure (#21427)

Isotr0py
authored and
zixi-qi
committed
[Core] Add basic unit test for maybe_evict_cached_block (#21400)

Jialin
authored and
zixi-qi
committed
[Cleanup] Only log MoE DP setup warning if DP is enabled (#21315)

mgoin
authored and
zixi-qi
committed
add clear messages for deprecated models (#21424)

youkaichao
authored and
zixi-qi
committed
[Bugfix] ensure tool_choice is popped when `tool_choice:null` is passed in json payload (#19679)

gcalmettes
authored and
zixi-qi
committed
Fixed typo in profiling logs (#21441)

sergiopaniego
authored and
zixi-qi
committed
[Docs] Fix bullets and grammars in tool_calling.md (#21440)

windsonsea
authored and
zixi-qi
committed
[Sampler] Introduce logprobs mode for logging (#21398)

houseroad
authored and
zixi-qi
committed
Mamba V2 Test not Asserting Failures. (#21379)

fabianlim
authored and
zixi-qi
committed
[Misc] fixed nvfp4_moe test failures due to invalid kwargs (#21246)

chenyang78
authored and
zixi-qi
committed
[Docs] Clean up v1/metrics.md (#21449)

windsonsea
authored and
zixi-qi
committed
[Model] add Hunyuan V1 Dense Model support. (#21368)

Asher
authored and
zixi-qi
committed
[V1] Check all pooling tasks during profiling (#21299)

DarkLight1337
authored and
zixi-qi
committed
[Bugfix][Qwen][DCA] fixes bug in dual-chunk-flash-attn backend for qwen 1m models. (#21364)

sighingnow
authored and
zixi-qi
committed
[Tests] Add tests for headless internal DP LB (#21450)

njhill
authored and
zixi-qi
committed
[Core][Model] PrithviMAE Enablement on vLLM v1 engine (#20577)

christian-pinto
authored and
zixi-qi
committed
Add test case for compiling multiple graphs (#21044)

sarckk
authored and
zixi-qi
committed
[TPU][TEST] Fix the downloading issue in TPU v1 test 11. (#21418)

QiliangCui
authored and
zixi-qi
committed