Olmo3 tool parser and tests#26126

Closed

pdasigi wants to merge 167 commits intovllm-project:mainfrom

pdasigi:olmo3_parser

Commits on Oct 2, 2025

[Olmo 3] olmo3 tool parser and tests
pdasigi
committed
[CI/Build] fix doc build warning: Failed to get 'name: description' pair (#25733)

yitingdc
authored and
pdasigi
committed
fix: revert cast to cpu in `MsgpackEncoder._encode_tensor` to avoid hidden performance regressions (#25738)

qthequartermasterman
authored and
pdasigi
committed
perf: Avoid copying inputs_embeds tensors to GPU unless prompt_embeds is enabled (#25739)

qthequartermasterman
authored and
pdasigi
committed
[Harware][AMD][Model] Triton MoE tuning configs for GLM-4.5 for MI300X (#25703)

xaguilar-amd
authored and
pdasigi
committed
fix: print outputt offline_inference/base/chat.py example (#25744)

Iceber
authored and
pdasigi
committed
[Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride bug in causal_conv_1d. (#25743)

sighingnow
authored and
pdasigi
committed
Remove cuda hard-code in compute_causal_conv1d_metadata (#25555)

wxsIcey
authored and
pdasigi
committed
[misc] refactor speculative config (#25657)

yyzxw
authored and
pdasigi
committed
[Bugfix] Fix Shared Expert/Zero expert code in FusedMoE.process_chunk (#25698)

authored and
pdasigi
committed
Support LongCat-Flash-Chat tool call (#24083)

Xu-Wenqing
authored and
pdasigi
committed
[Doc] Update Batch-level DP docs (#25757)

DarkLight1337
authored and
pdasigi
committed
[Model] Mamba2 varlen refactor (#21467)

authored and
pdasigi
committed
[CI] Fix test_shared_storage_connector_hashes (#25748)

chaunceyjiang
authored and
pdasigi
committed
[Bugfix] Properly abort pooling request. (#25734)

authored and
pdasigi
committed
[CI/Build] Split up Distributed Tests (#25572)

DarkLight1337
authored and
pdasigi
committed
[CI/Build] Fix some V1 tests not being run (#25569)

DarkLight1337
authored and
pdasigi
committed
[Quantization] Add field to skip unquantized modules for GPTQ config (#25455)

Isotr0py
authored and
pdasigi
committed
[BugFix] Fix using `dbo_decode_token_threshold` always (and ignoring `dbo_prefill_token_threshold`) (#25622)

LucasWilkinson
authored and
pdasigi
committed
[ray][metrics] Replace ':' with '_' for OpenTelemetry compatibility in Ray (#25439)

authored and
pdasigi
committed
[Misc] fix unique_filepath (#25732)

authored and
pdasigi
committed
Eagle3 that supports the Minicpm3 model (#24243)

authored and
pdasigi
committed
[Doc]: improve CPU(x86) build-wheel-from-source section (#25617)

brokedba
authored and
pdasigi
committed
[Bugfix] Improve GLM4 MoE Reasoning Parser's is_reasoning_end Condition (#25355)

authored and
pdasigi
committed
[Docs] Add Toronto Meetup (#25773)

mgoin
authored and
pdasigi
committed
[CI] Add E2E Blackwell Quantized MoE Test (#25723)

mgoin
authored and
pdasigi
committed
[V1] address post issues related to #20059 (part 1) (#23046)

authored and
pdasigi
committed
[CI] Fix FlashInfer AOT in release docker image (#25730)

mgoin
authored and
pdasigi
committed
[spec decode] Consolidate speculative decode method name for MTP (#25232)

zixi-qi
authored and
pdasigi
committed
Reduce the Cuda Graph memory footprint when running with DBO (#25779)

SageMoore
authored and
pdasigi
committed
Kernel-override Determinism [1/n] (#25603)

bwasti
authored and
pdasigi
committed
[Bugfix] Optimize CpuGpuBuffer initialization (#25447)

namanlalitnyu
authored and
pdasigi
committed
[Spec decode] automatically disable mm for text-only draft models (#25667)

jmkuebler
authored and
pdasigi
committed
[Core] Don't count preempted tokens in prefix cache hit rate (#25787)

zhuohan123
authored and
pdasigi
committed
Add option to restrict media domains (#25783)

authored and
pdasigi
committed
Add flashinfer-build.sh and register precompiled cu128 wheel in Dockerfile (#25782)

mgoin
authored and
pdasigi
committed
[Multimodal][Speculative Decoding]Eagle Eagle3 mm support, enablement on qwen2.5vl (#22872)

authored and
pdasigi
committed
[Bugfix] Allow Only SDPA Backend for ViT on B200 for Qwen3-VL (#25788)

yewentao256
authored and
pdasigi
committed
[CI/Build] Consolidate model loader tests and requirements (#25765)

DarkLight1337
authored and
pdasigi
committed
[CI/Build] Add timing to Model Executor Test (#25799)

22quinn
authored and
pdasigi
committed
[CI/Build] Reorganize root-level V1 tests (#25767)

DarkLight1337
authored and
pdasigi
committed
[Misc] Fix codeowners override for v1 sample and attention (#25037)

22quinn
authored and
pdasigi
committed
[Misc] Update openai client example file for multimodal (#25795)

authored and
pdasigi
committed
[Bugfix] Add missing `image_size` for phi4_multimodal (#25796)

Renovamen
authored and
pdasigi
committed
[Bugfix] Merge MM embeddings by index instead of token IDs (#16229)

authored and
pdasigi
committed
Validate API tokens in constant time (#25781)

authored and
pdasigi
committed
Add filtering for chat template kwargs (#25794)

authored and
pdasigi
committed
Fix GPTQ model loading in Transformers backend (#25770)

authored and
pdasigi
committed
[Bugfix] Fix triton import precommit failure (#25803)

tlrmchlsmth
authored and
pdasigi
committed
[Bugfix][WideEP] Apply TP Attn + EP MoE fix to other models (#24982)

tlrmchlsmth
authored and
pdasigi
committed
[docs] transcriptions API audio upload (#25446)

yyzxw
authored and
pdasigi
committed
[env] default nixl side port conflicts with kv-event zmq port (#25056)

panpan0000
authored and
pdasigi
committed
[Core] Refactor self.model() to call a helper for subclassing. (#25084)

patrick-toulme
authored and
pdasigi
committed
[torch.compile]: Add VLLM_DEBUG_DUMP_PATH environment variable (#25651)

authored and
pdasigi
committed
[Bug]: Set LD_LIBRARY_PATH to include the 'standard' CUDA location (#25766)

smarterclayton
authored and
pdasigi
committed
[Core] GC Debug callback (#24829)

Jialin
authored and
pdasigi
committed
[Bugfix][NIXL] Fix Async Scheduler timeout issue (#25808)

NickLucche
authored and
pdasigi
committed
[MM] Optimize memory profiling for scattered multimodal embeddings (#25810)

ywang96
authored and
pdasigi
committed
[Bugfix] Fix Qwen3-VL regression from #24982 (#25814)

ywang96
authored and
pdasigi
committed
[VLM] Update Qwen3-VL max_num_video_tokens calculation for configurable video profiling (#25557)

authored and
pdasigi
committed
Fix random dataset mismatched token length with config. (#24937)

authored and
pdasigi
committed
Update GLM-4.5 Doc transformers version (#25830)

zRzRzRzRzRzRzR
authored and
pdasigi
committed
[Bugfix] fix Qwen3VLMoe load when pp > 1 (#25838)

authored and
pdasigi
committed
Remove redundant cudagraph dispatcher warning (#25841)

mgoin
authored and
pdasigi
committed
[Misc] fix tests failure by using current_platform (#25825)

kingsmad
authored and
pdasigi
committed
[P/D] NIXL Updates (#25844)

authored and
pdasigi
committed
Add Phi4FlashForCausalLM to _PREVIOUSLY_SUPPORTED_MODELS (#25832)

tdoublep
authored and
pdasigi
committed
[XPU]Fix xpu spec decoding UTs, avoid using cuda graph (#25847)

jikunshang
authored and
pdasigi
committed
[Bugfix] Fallback ViT attn backend to SDPA for blackwell (#25851)

ywang96
authored and
pdasigi
committed
[V0 Deprecation][Models] Remove all V0 condition for mm embeddings merge (#25331)

Isotr0py
authored and
pdasigi
committed
[Misc] Remove more `get_input_embeddings_v0` (#25857)

DarkLight1337
authored and
pdasigi
committed
update to latest deepgemm for dsv3.2 (#25871)

youkaichao
authored and
pdasigi
committed
[Bugfix] Fix requirements paths in install instructions (#25827)

yingjun-mou
authored and
pdasigi
committed
[Model][Bugfix] Fix issues in MiDashengLM implementation for quantized models (#25854)

zhoukezi
authored and
pdasigi
committed
[torch.compile] serialize cudagraph_mode as its enum name instead of value (#25868)

ZJY0516
authored and
pdasigi
committed
[Nixl][P/D] Add cuda2cpu support (HD->DH transfer) (#24690)

authored and
pdasigi
committed
[Bugfix][Speculative Decoding] Fix Eagle3 quantization config issue (#25883)

rahul-tuli
authored and
pdasigi
committed
[CI/Build] Include Transformers backend test in nightly transformers test (#25885)

Isotr0py
authored and
pdasigi
committed
[Model] Remove MotifForCausalLM (#25866)

jeejeelee
authored and
pdasigi
committed
[Bugfix] Use correct key "ignore" for config.json non-quantized layers (#25706)

leejnau
authored and
pdasigi
committed
[BugFix][torch.compile] KV scale calculation issues with FP8 quantization (#25513)

adabeyta
authored and
pdasigi
committed
[Doc] Add documentation for vLLM continuous benchmarking and profiling (#25819)

namanlalitnyu
authored and
pdasigi
committed
[Bugfix][ROCm] Fixing trying to import non-existent symbols from libnccl.so (#25605)

gshtras
authored and
pdasigi
committed
[Kernel] Chunk-aligned mamba2 (#24683)

tdoublep
authored and
pdasigi
committed
[Doc] Polish example for torchrun dp (#25899)

zhuohan123
authored and
pdasigi
committed
[NIXL] Increase default KV block eviction timeout on P (#25897)

NickLucche
authored and
pdasigi
committed
[V0 Deprecation] Remove `vllm.worker` and update according imports (#25901)

aarnphm
authored and
pdasigi
committed
Test Prompt Embeds/LoRA compatibility and Enable LoRA Support for OPT Models (#25717)

qthequartermasterman
authored and
pdasigi
committed
[Bug] Fix Weight Loading for Block FP8 Cutlass SM90 (#25909)

authored and
pdasigi
committed
[Benchmark] Support benchmark throughput for external launcher DP (#25913)

zhuohan123
authored and
pdasigi
committed
Move`VllmConfig` from `config/__init__.py` to `config/vllm.py` (#25271)

hmellor
authored and
pdasigi
committed
[BugFix] Fix DP/EP hang (#25906)

LucasWilkinson
authored and
pdasigi
committed
[BugFix] Pass config_format via try_get_generation_config (#25912)

acisseJZhong
authored and
pdasigi
committed
[Model][Bugfix] Fix MiDashengLM audio encoder mask by removing incorrect `logical_not` (#25925)

zhoukezi
authored and
pdasigi
committed
[Bugfix]: Clean up chunked prefill logging when using whisper (#25075)

simondanielsson
authored and
pdasigi
committed
[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896)

authored and
pdasigi
committed
[Doc] Add Cambricon MLU support (#25942)

a120092009
authored and
pdasigi
committed
Updated TRL integration docs (#25684)

authored and
pdasigi
committed
[Bugfix][Model]fix ernie45 moe gate&bias dtype to float32 (#25936)

CSWYF3634076
authored and
pdasigi
committed
[Model] Move `vision_feature_select_strategy` into `resolve_visual_encoder_outputs` (#25938)

DarkLight1337
authored and
pdasigi
committed
[perf] Use CPU tensor to reduce GPU->CPU sync (#25884)

lhtin
authored and
pdasigi
committed
[NIXL] Add support for MLA caches with different latent dim (#25902)

authored and
pdasigi
committed
[CI] Move applicable tests to CPU (#24080)

authored and
pdasigi
committed
[Fix] Improve CPU backend compatibility for RISC-V (#25816)

ihb2032
authored and
pdasigi
committed
[Kernel][Moe Configs] Add more tuned triton configs for ExpertsInt8 and FP8 (#25858)

Josephasafg
authored and
pdasigi
committed
Add Hugging Face Inference Endpoints guide to Deployment docs (#25886)

authored and
pdasigi
committed
[Bugfix][Model] Fix inference for Hunyuan dense models (#25354)

Anionex
authored and
pdasigi
committed
[Bugfix] Fix accuracy issue of TRTLLM FP8 MOE and improve logging (#25895)

pavanimajety
authored and
pdasigi
committed
[Bugfix] Token type and position embeddings fail to be applied to `inputs_embeds` (#25922)

DarkLight1337
authored and
pdasigi
committed
[bugfix][deepseek] fix flashmla kernel selection (#25956)

youkaichao
authored and
pdasigi
committed
[Bug] Fix AttributeError: 'QKVParallelLinear' object has no attribute 'orig_dtype' (#25958)

yewentao256
authored and
pdasigi
committed
[Doc] Improve MM Pooling model documentation (#25966)

DarkLight1337
authored and
pdasigi
committed
[Docs] Add moe kernel features doc (#25297)

authored and
pdasigi
committed
OffloadingConnector: Fix GPU block tracking bug (#25856)

orozery
authored and
pdasigi
committed
[Llama4] [multimodal] Fix misplaced dtype cast of `cos_sin_cache` in `Llama4VisionRotaryEmbedding` (#25889)

cjackal
authored and
pdasigi
committed
[Bench] Add DeepSeekV32 to MoE benchmark (#25962)

jeejeelee
authored and
pdasigi
committed
[V1] [P/D] Add Support for KV Load Failure Recovery (#19330)

authored and
pdasigi
committed
Add explicit pooling classes for the Transformers backend (#25322)

authored and
pdasigi
committed
[Docs] Remove API Reference from search index (#25949)

hmellor
authored and
pdasigi
committed
[gpt-oss] use vLLM instead of openai types for streaming (#25186)

authored and
pdasigi
committed
[Misc] Make EP kernels install script support uv (#25785)

LucasWilkinson
authored and
pdasigi
committed
[Model] MTP fallback to eager for DeepSeek v32 (#25982)

luccafong
authored and
pdasigi
committed
Update launch_bounds_utils.h for correct compile on Multiple Cuda Arch - PTXAS out of range Warning (#25843)

authored and
pdasigi
committed
[Log] Optimize Log for FP8MOE (#25709)

yewentao256
authored and
pdasigi
committed
Fix INT8 quantization error on Blackwell GPUs (SM100+) (#25935)

certainly-param
authored and
pdasigi
committed
[MM] Add text-only mode for Qwen3-VL (#26000)

ywang96
authored and
pdasigi
committed
[Bugfix] Fix `__syncwarp` on ROCM (#25996)

zhewenl
authored and
pdasigi
committed
[BugFix] Fix default kv-cache-dtype default for DeepseekV3.2 (#25988)

LucasWilkinson
authored and
pdasigi
committed
Update to Transformers `v4.56.2` (#24638)

authored and
pdasigi
committed
[Misc]allow disable pynccl (#25421)

authored and
pdasigi
committed
[Doc] updating torch.compile doc link (#25989)

authored and
pdasigi
committed
[BugFix][MM] Fix Nonetype error when video is cache in qwen2.5-omni-thinker (#26004)

wwl2755
authored and
pdasigi
committed
[Misc] Factor out common `_apply_feature_select_strategy` (#26003)

DarkLight1337
authored and
pdasigi
committed
[CI] Only capture a single CUDA graph size in CI by default (#25951)

hmellor
authored and
pdasigi
committed
[MISC] Fix misleading batch_size_capture_list when cuda_graph_sizes < 4 (#25829)

authored and
pdasigi
committed
[Benchmark] Finish documented v0.11.0 deprecation of --endpoint-type (#26007)

natoscott
authored and
pdasigi
committed
[Bugfix] Apply same sampling parameters for both `n=1` and `n>1` (#26005)

kmaehashi
authored and
pdasigi
committed
[NVIDIA] Blackwell Family (#24673)

authored and
pdasigi
committed
Fix test_mamba_ssm_ssd.py due to missing _query_start_loc_to_chunk_indices_offsets (#25995)

hl475
authored and
pdasigi
committed
[CI] Tweaks to GPT-OSS Eval (Blackwell) for stability (#26030)

mgoin
authored and
pdasigi
committed
[BugFix][DP/EP] Fix CUTLASS MLA hang under load (#26026)

authored and
pdasigi
committed
[ROCm][Build] Add support for AMD Ryzen AI MAX / AI 300 Series (#25908)

hyoon1
authored and
pdasigi
committed
[Bug] Fix Negative Cuda Memory Usage (#25683)

yewentao256
authored and
pdasigi
committed
[BugFix] ChunkedLocalAttention is currently not CG compatible (#26034)

LucasWilkinson
authored and
pdasigi
committed
Support RL online quantization with torchao (#23014)

jerryzh168
authored and
pdasigi
committed
[ROCm][Bugfix] Add missing parameter to ROCm backend (#26029)

gshtras
authored and
pdasigi
committed
[Misc] Make handling of SamplingParams clearer in n>1 case (#26032)

njhill
authored and
pdasigi
committed
Run:ai model streamer add GCS package support (#24909)

pwschuurman
authored and
pdasigi
committed
Update base image to 22.04 (jammy) (#26065)

huydhn
authored and
pdasigi
committed
Change size of single CUDA graph for CI to 4 (#26089)

tdoublep
authored and
pdasigi
committed
[FA/Chore] Bump vllm-flash-attention (#25537)

LucasWilkinson
authored and
pdasigi
committed
[Model] Use `merge_by_field_config` for MM models (A-C) (#26073)

DarkLight1337
authored and
pdasigi
committed
[Model] Use `merge_by_field_config` for MM models (D-F) (#26076)

DarkLight1337
authored and
pdasigi
committed
[Platform][CI] Added OOT platform interface e2e test that running on Ascend NPU (#25470)

leo-pony
authored and
pdasigi
committed
[Qwen][ROCm] Flash Attention Rotary Embeddings (#24642)

vllmellm
authored and
pdasigi
committed
[CI] Add Blackwell DeepSeek FP8 FlashInfer MoE tests (#26040)

mgoin
authored and
pdasigi
committed
[CI/Build] Replace `vllm.entrypoints.openai.api_server` entrypoint with `vllm serve` command (#25967)

DarkLight1337
authored and
pdasigi
committed
[BugFix] Fix FI accuracy issue when used for MLA prefill (#26063)

authored and
pdasigi
committed
[Small] Prevent bypassing media domain restriction via HTTP redirects (#26035)

huachenheli
authored and
pdasigi
committed
[Deepseek v3.2] Support indexer prefill chunking (#25999)

heheda12345
authored and
pdasigi
committed
EAGLE 3: Fix preamble so that measured speedup over Eagle 1 becomes 32% instead of 5% on MTBench (#25916)

ekagra-ranjan
authored and
pdasigi
committed
[Mamba][KVCacheManager] Simplify kv cache manage logic for mamba + MTP (#25119)

heheda12345
authored and
pdasigi
committed
[Olmo 3] pre-commit fixes
pdasigi
committed
[Olmo 3] safer xml tag removal in tool parser
pdasigi
committed
[Perf] Fix and reapply move apply w8a8 block fp8 linear to class (#25696)

authored and
pdasigi
committed
Fix MTP with deepep_low_latency (#25904)

MatthewBonanni
authored and
pdasigi
committed
Merge branch 'main' into olmo3_parser
pdasigi
authored