Add factories for logits_processors#38

Closed

maxdebayser wants to merge 616 commits intomainfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on May 6, 2024

[Misc] Exclude the `tests` directory from being packaged (#4552)

itechbear
authored and
joerunde
committed
[BugFix] Include target-device specific requirements.txt in sdist (#4559)

markmc
authored and
joerunde
committed
[Misc] centralize all usage of environment variables (#4548)

youkaichao
authored and
joerunde
committed
[kernel] fix sliding window in prefix prefill Triton kernel (#4405)

authored and
joerunde
committed
[CI/Build] AMD CI pipeline with extended set of tests. (#4267)

authored and
joerunde
committed
[Core] Ignore infeasible swap requests. (#4557)

rkooo567
authored and
joerunde
committed
[Core][Distributed] enable allreduce for multiple tp groups (#4566)

youkaichao
authored and
joerunde
committed
[BugFix] Prevent the task of `_force_log` from being garbage collected (#4567)

Atry
authored and
joerunde
committed
[Misc] remove chunk detected debug logs (#4571)

DefTruth
authored and
joerunde
committed
[Doc] add env vars to the doc (#4572)

youkaichao
authored and
joerunde
committed
[Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518)

rkooo567
authored and
joerunde
committed
[Bugfix] Allow "None" or "" to be passed to CLI for string args that default to None (#4586)

mgoin
authored and
joerunde
committed
Fix/async chat serving (#2727)

schoennenbeck
authored and
joerunde
committed
[Kernel] Use flashinfer for decoding (#4353)

authored and
joerunde
committed
[Speculative decoding] Support target-model logprobs (#4378)

cadedaniel
authored and
joerunde
committed
[Misc] add installation time env vars (#4574)

youkaichao
authored and
joerunde
committed
[Misc][Refactor] Introduce ExecuteModelData (#4540)

comaniac
authored and
joerunde
committed
[Doc] Chunked Prefill Documentation (#4580)

rkooo567
authored and
joerunde
committed
[Kernel] Support MoE Fp8 Checkpoints for Mixtral (Static Weights with Dynamic/Static Activations) (#4527)

mgoin
authored and
joerunde
committed
[CI] check size of the wheels (#4319)

simon-mo
authored and
joerunde
committed
[Bugfix] Fix inappropriate content of model_name tag in Prometheus metrics (#3937)

DearPlanet
authored and
joerunde
committed
bump version to v0.4.2 (#4600)

simon-mo
authored and
joerunde
committed
[CI] Reduce wheel size by not shipping debug symbols (#4602)

simon-mo
authored and
joerunde
committed
Disable cuda version check in vllm-openai image (#4530)

zhaoyang-star
authored and
joerunde
committed
[Bugfix] Fix `asyncio.Task` not being subscriptable (#4623)

DarkLight1337
authored and
joerunde
committed
Update vLLM to 323f27b9
joerunde
committed

Commits on May 8, 2024

Commits on May 13, 2024

fix: use vllm_nccl installed nccl version (#26)
tjohnson31415
authored

Commits on May 14, 2024

:bug: fix prometheus metric labels (#27)
joerunde
authored

Commits on May 16, 2024

[CI] use ccache actions properly in release workflow (#4629)

simon-mo
authored and
tjohnson31415
committed
[CI] Add retry for agent lost (#4633)

cadedaniel
authored and
tjohnson31415
committed
Update lm-format-enforcer to 0.10.1 (#4631)

noamgat
authored and
tjohnson31415
committed
[Kernel] Make static FP8 scaling more robust (#4570)

pcmoritz
authored and
tjohnson31415
committed
[Core][Optimization] change python dict to pytorch tensor (#4607)

youkaichao
authored and
tjohnson31415
committed
[Build/CI] Fixing 'docker run' to re-enable AMD CI tests. (#4642)

Alexei-V-Ivanov-AMD
authored and
tjohnson31415
committed
[Bugfix] Fixed error in slice_lora_b for MergedQKVParallelLinearWithLora (#4609)

FurtherAI
authored and
tjohnson31415
committed
[Core][Optimization] change copy-on-write from dict[int, list] to list (#4648)

youkaichao
authored and
tjohnson31415
committed
[Bug fix][Core] fixup ngram not setup correctly (#4551)

authored and
tjohnson31415
committed
[Core][Distributed] support cpu&device in broadcast tensor dict (#4660)

youkaichao
authored and
tjohnson31415
committed
[Core] Optimize sampler get_logprobs (#4594)

rkooo567
authored and
tjohnson31415
committed
[CI] Make mistral tests pass (#4596)

rkooo567
authored and
tjohnson31415
committed
[Bugfix][Kernel] allow non-power-of-2 for prefix prefill with alibi (#4573)

DefTruth
authored and
tjohnson31415
committed
[Misc] Add `get_name` method to attention backends (#4685)

WoosukKwon
authored and
tjohnson31415
committed
[Core] Faster startup for LoRA enabled models (#4634)

Yard1
authored and
tjohnson31415
committed
[Core][Optimization] change python dict to pytorch tensor for blocks to swap (#4659)

youkaichao
authored and
tjohnson31415
committed
[CI/Test] fix swap test for multi gpu (#4689)

youkaichao
authored and
tjohnson31415
committed
[Misc] Use vllm-flash-attn instead of flash-attn (#4686)

WoosukKwon
authored and
tjohnson31415
committed
[Dynamic Spec Decoding] Auto-disable by the running queue size (#4592)

authored and
tjohnson31415
committed
[Speculative decoding] [Bugfix] Fix overallocation in ngram + spec logprobs (#4672)

cadedaniel
authored and
tjohnson31415
committed
[Bugfix] Fine-tune gptq_marlin configs to be more similar to marlin (#4626)

alexm-redhat
authored and
tjohnson31415
committed
[Frontend] add tok/s speed metric to llm class when using tqdm (#4400)

authored and
tjohnson31415
committed
[Frontend] Move async logic outside of constructor (#4674)

DarkLight1337
authored and
tjohnson31415
committed
[Misc] Remove unnecessary ModelRunner imports (#4703)

WoosukKwon
authored and
tjohnson31415
committed
[Misc] Set block size at initialization & Fix test_model_runner (#4705)

WoosukKwon
authored and
tjohnson31415
committed
[ROCm] Add support for Punica kernels on AMD GPUs (#3140)

authored and
tjohnson31415
committed
[Bugfix] Fix CLI arguments in OpenAI server docs (#4709)

DarkLight1337
authored and
tjohnson31415
committed
[Bugfix] Update grafana.json (#4711)

robertgshaw2-redhat
authored and
tjohnson31415
committed
[Bugfix] Add logs for all model dtype casting (#4717)

mgoin
authored and
tjohnson31415
committed
[Model] Snowflake arctic model implementation (#4652)

authored and
tjohnson31415
committed
[Kernel] [FP8] Improve FP8 linear layer performance (#4691)

pcmoritz
authored and
tjohnson31415
committed
[Kernel] Refactor FP8 kv-cache with NVIDIA float8_e4m3 support (#4535)

comaniac
authored and
tjohnson31415
committed
[Core][Distributed] refactor pynccl (#4591)

youkaichao
authored and
tjohnson31415
committed
[Misc] Keep only one implementation of the create_dummy_prompt function. (#4716)

AllenDou
authored and
tjohnson31415
committed
chunked-prefill-doc-syntax (#4603)

authored and
tjohnson31415
committed
[Core]fix type annotation for `swap_blocks` (#4726)

jikunshang
authored and
tjohnson31415
committed
[Misc] Apply a couple g++ cleanups (#4719)

stevegrubb
authored and
tjohnson31415
committed
[Core] Fix circular reference which leaked llm instance in local dev env (#4737)

rkooo567
authored and
tjohnson31415
committed
[Bugfix] Fix CLI arguments in OpenAI server docs (#4729)

AllenDou
authored and
tjohnson31415
committed
[Speculative decoding] CUDA graph support (#4295)

authored and
tjohnson31415
committed
[CI] Nits for bad initialization of SeqGroup in testing (#4748)

robertgshaw2-redhat
authored and
tjohnson31415
committed
[Core][Test] fix function name typo in custom allreduce (#4750)

youkaichao
authored and
tjohnson31415
committed
[Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734)

CatherineSue
authored and
tjohnson31415
committed
[Model] Add support for IBM Granite Code models (#4636)

yikangshen
authored and
tjohnson31415
committed
[CI/Build] Tweak Marlin Nondeterminism Issues (#4713)

robertgshaw2-redhat
authored and
tjohnson31415
committed
[CORE] Improvement in ranks code (#4718)

SwapnilDreams100
authored and
tjohnson31415
committed
[Core][Distributed] refactor custom allreduce to support multiple tp groups (#4754)

youkaichao
authored and
tjohnson31415
committed
[CI/Build] Move `test_utils.py` to `tests/utils.py` (#4425)

DarkLight1337
authored and
tjohnson31415
committed
[Scheduler] Warning upon preemption and Swapping (#4647)

authored and
tjohnson31415
committed
[Misc] Enhance attention selector (#4751)

WoosukKwon
authored and
tjohnson31415
committed
[Frontend] [Core] perf: Automatically detect vLLM-tensorized model, update `tensorizer` to version 2.9.0 (#4208)

sangstar
authored and
tjohnson31415
committed
[Speculative decoding] Improve n-gram efficiency (#4724)

comaniac
authored and
tjohnson31415
committed
[Kernel] Use flash-attn for decoding (#3648)

authored and
tjohnson31415
committed
[Bugfix] Fix dynamic FP8 quantization for Mixtral (#4793)

pcmoritz
authored and
tjohnson31415
committed
[Doc] Shorten README by removing supported model list (#4796)

zhuohan123
authored and
tjohnson31415
committed
[Doc] Add API reference for offline inference (#4710)

DarkLight1337
authored and
tjohnson31415
committed
[Doc] Add meetups to the doc (#4798)

zhuohan123
authored and
tjohnson31415
committed
[Core][Hash][Automatic Prefix caching] Accelerating the hashing function by avoiding deep copies (#4696)

KuntaiDu
authored and
tjohnson31415
committed
[Bugfix][Doc] Fix CI failure in docs (#4804)

DarkLight1337
authored and
tjohnson31415
committed
[Core] Add MultiprocessingGPUExecutor (#4539)

authored and
tjohnson31415
committed
Add 4th meetup announcement to readme (#4817)

simon-mo
authored and
tjohnson31415
committed
Revert "[Kernel] Use flash-attn for decoding (#3648)" (#4820)

rkooo567
authored and
tjohnson31415
committed
[Core][2/N] Model runner refactoring part 2. Combine prepare prefill / decode to a single API (#4681)

rkooo567
authored and
tjohnson31415
committed
[CI/Build] Further decouple HuggingFace implementation from ours during tests (#4166)

DarkLight1337
authored and
tjohnson31415
committed
[Bugfix] Properly set distributed_executor_backend in ParallelConfig (#4816)

zifeitong
authored and
tjohnson31415
committed
[Doc] Highlight the fourth meetup in the README (#4842)

zhuohan123
authored and
tjohnson31415
committed
[Frontend] Re-enable custom roles in Chat Completions API (#4758)

DarkLight1337
authored and
tjohnson31415
committed
[Frontend] Support OpenAI batch file format (#4794)

authored and
tjohnson31415
committed
[Core] Implement sharded state loader (#4690)

authored and
tjohnson31415
committed
[Speculative decoding][Re-take] Enable TP>1 speculative decoding (#4840)

authored and
tjohnson31415
committed
Add marlin unit tests and marlin benchmark script (#4815)

alexm-redhat
authored and
tjohnson31415
committed
[Kernel] add bfloat16 support for gptq marlin kernel (#4788)

jinzhen-lin
authored and
tjohnson31415
committed
[docs] Fix typo in examples filename openi -> openai (#4864)

wuisawesome
authored and
tjohnson31415
committed
[Frontend] Separate OpenAI Batch Runner usage from API Server (#4851)

wuisawesome
authored and
tjohnson31415
committed
[Bugfix] Bypass authorization API token for preflight requests (#4862)

dulacp
authored and
tjohnson31415
committed
Add GPTQ Marlin 2:4 sparse structured support (#4790)

authored and
tjohnson31415
committed
Add JSON output support for benchmark_latency and benchmark_throughput (#4848)

simon-mo
authored and
tjohnson31415
committed
[ROCm][AMD][Bugfix] adding a missing triton autotune config (#4845)

hongxiayang
authored and
tjohnson31415
committed
[Core][Distributed] remove graph mode function (#4818)

youkaichao
authored and
tjohnson31415
committed
[Misc] remove old comments (#4866)

youkaichao
authored and
tjohnson31415
committed
[Kernel] Add punica dimension for Qwen1.5-32B LoRA (#4850)

authored and
tjohnson31415
committed
Update vLLM to 8435b207
tjohnson31415
committed
:bug: fixup merge conflicts
joerunde
committed
:fire: remove flash attention
joerunde
committed
Use fork for worker multiprocessing method (#29)
njhill
authored

Commits on May 20, 2024

[Kernel] Add w8a8 CUTLASS kernels (#4749)

tlrmchlsmth
authored and
joerunde
committed
[Bugfix] Fix FP8 KV cache support (#4869)

WoosukKwon
authored and
joerunde
committed
Support to serve vLLM on Kubernetes with LWS (#4829)

kerthcet
authored and
joerunde
committed
[Frontend] OpenAI API server: Do not add bos token by default when encoding (#4688)

bofenghuang
authored and
joerunde
committed
[Build/CI] Extending the set of AMD tests with Regression, Basic Correctness, Distributed, Engine, Llava Tests (#4797)

Alexei-V-Ivanov-AMD
authored and
joerunde
committed
[Bugfix] fix rope error when load models with different dtypes (#4835)

jinzhen-lin
authored and
joerunde
committed
Sync huggingface modifications of qwen Moe model (#4774)

eigen2017
authored and
joerunde
committed
[Doc] Update Ray Data distributed offline inference example (#4871)

Yard1
authored and
joerunde
committed
[Bugfix] Relax tiktoken to >= 0.6.0 (#4890)

mgoin
authored and
joerunde
committed
[ROCm][Hardware][AMD] Adding Navi21 to fallback to naive attention if Triton is not used (#4658)

akondrat-amd
authored and
joerunde
committed
[Lora] Support long context lora (#4787)

rkooo567
authored and
joerunde
committed
[Bugfix][Model] Add base class for vision-language models (#4809)

DarkLight1337
authored and
joerunde
committed
[Kernel] Add marlin_24 unit tests (#4901)

alexm-redhat
authored and
joerunde
committed
[Kernel] Add flash-attn back (#4907)

WoosukKwon
authored and
joerunde
committed
[Model] LLaVA model refactor (#4910)

DarkLight1337
authored and
joerunde
committed
Remove marlin warning (#4918)

alexm-redhat
authored and
joerunde
committed
Update vLLM to da5a0b53
joerunde
committed
:sparkles: log all errored requests (#30)
joerunde
authored

Commits on May 28, 2024

Apply temp. patch to Triton code to resolve conflicting cache dirs in TP case (#34)
tdoublep
authored

Commits on May 30, 2024

Add guided decoding to TGIS gRPC API (#31)
njhill
authored

Commits on Jun 3, 2024

feat: install tensorizer in the UBI image (#36)
tjohnson31415
authored
Small changes to support releases (#37)

joerunde
and
tjohnson31415
authored
[Misc]: allow user to specify port in distributed setting (#4914)

ZwwWayne
authored and
joerunde
committed
[Build/CI] Enabling AMD Entrypoints Test (#4834)

authored and
joerunde
committed
[Bugfix] Fix dummy weight for fp8 (#4916)

authored and
joerunde
committed
[Core] Sharded State Loader download from HF (#4889)

aurickq
authored and
joerunde
committed
[Doc]Add documentation to benchmarking script when running TGI (#4920)

KuntaiDu
authored and
joerunde
committed
[Core] Fix scheduler considering "no LoRA" as "LoRA" (#4897)

Yard1
authored and
joerunde
committed
[Model] add rope_scaling support for qwen2 (#4930)

hzhwcmhf
authored and
joerunde
committed
[Model] Add Phi-2 LoRA support (#4886)

Isotr0py
authored and
joerunde
committed
[Docs] Add acknowledgment for sponsors (#4925)

simon-mo
authored and
joerunde
committed
[CI/Build] Codespell ignore `build/` directory (#4945)

mgoin
authored and
joerunde
committed
[Bugfix] Fix flag name for `max_seq_len_to_capture` (#4935)

kerthcet
authored and
joerunde
committed
[Bugfix][Kernel] Add head size check for attention backend selection (#4944)

Isotr0py
authored and
joerunde
committed
[Frontend] Dynamic RoPE scaling (#4638)

sasha0552
authored and
joerunde
committed
[CI/Build] Enforce style for C++ and CUDA code with `clang-format` (#4722)

mgoin
authored and
joerunde
committed
[misc] remove comments that were supposed to be removed (#4977)

rkooo567
authored and
joerunde
committed
[Kernel] Fixup for CUTLASS kernels in CUDA graphs (#4954)

tlrmchlsmth
authored and
joerunde
committed
[Misc] Load FP8 kv-cache scaling factors from checkpoints (#4893)

comaniac
authored and
joerunde
committed
[Model] LoRA gptbigcode implementation (#3949)

raywanb
authored and
joerunde
committed
[Core] Eliminate parallel worker per-step task scheduling overhead (#4894)

njhill
authored and
joerunde
committed
[Minor] Fix small typo in llama.py: QKVParallelLinear -> QuantizationConfig (#4991)

pcmoritz
authored and
joerunde
committed
[Misc] Take user preference in attention selector (#4960)

comaniac
authored and
joerunde
committed
Marlin 24 prefill performance improvement (about 25% better on average) (#4983)

alexm-redhat
authored and
joerunde
committed
[Bugfix] Update Dockerfile.cpu to fix NameError: name 'vllm_ops' is not defined (#5009)

LetianLee
authored and
joerunde
committed
[Core][1/N] Support send/recv in PyNCCL Groups (#4988)

andoorve
authored and
joerunde
committed
[Kernel] Initial Activation Quantization Support (#4525)

authored and
joerunde
committed
[Core]: Option To Use Prompt Token Ids Inside Logits Processor (#4985)

kezouke
authored and
joerunde
committed
[Doc] add ccache guide in doc (#5012)

authored and
joerunde
committed
[Bugfix] Fix Mistral v0.3 Weight Loading (#5005)

authored and
joerunde
committed
[Core][Bugfix]: fix prefix caching for blockv2 (#4764)

authored and
joerunde
committed
[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799)

authored and
joerunde
committed
[Misc] add logging level env var (#5045)

youkaichao
authored and
joerunde
committed
[Dynamic Spec Decoding] Minor fix for disabling speculative decoding (#5000)

LiuXiaoxuanPKU
authored and
joerunde
committed
[Misc] Make Serving Benchmark More User-friendly (#5044)

ywang96
authored and
joerunde
committed
[Bugfix / Core] Prefix Caching Guards (merged with main) (#4846)

authored and
joerunde
committed
[Core] Allow AQLM on Pascal (#5058)

sasha0552
authored and
joerunde
committed
[Model] Add support for falcon-11B (#5069)

Isotr0py
authored and
joerunde
committed
[Core] Sliding window for block manager v2 (#4545)

authored and
joerunde
committed
[BugFix] Fix Embedding Models with TP>1 (#5075)

robertgshaw2-redhat
authored and
joerunde
committed
[Kernel][ROCm][AMD] Add fused_moe Triton configs for MI300X (#4951)

divakar-amd
authored and
joerunde
committed
[Docs] Add Dropbox as sponsors (#5089)

simon-mo
authored and
joerunde
committed
[Core] Consolidate prompt arguments to LLM engines (#4328)

authored and
joerunde
committed
[Bugfix] Remove the last EOS token unless explicitly specified (#5077)

jsato8094
authored and
joerunde
committed
[Misc] add gpu_memory_utilization arg (#5079)

pandyamarut
authored and
joerunde
committed
[Core][Optimization] remove vllm-nccl (#5091)

youkaichao
authored and
joerunde
committed
[Bugfix] Fix arguments passed to `Sequence` in stop checker test (#5092)

DarkLight1337
authored and
joerunde
committed
[Core][Distributed] improve p2p access check (#4992)

youkaichao
authored and
joerunde
committed
[Core] Cross-attention KV caching and memory-management (towards eventual encoder/decoder model support) (#4837)

afeldman-nm
authored and
joerunde
committed
[Doc]Replace deprecated flag in readme (#4526)

ronensc
authored and
joerunde
committed
[Bugfix][CI/Build] Fix test and improve code for `merge_async_iterators` (#5096)

DarkLight1337
authored and
joerunde
committed
[Bugfix][CI/Build] Fix codespell failing to skip files in `git diff` (#5097)

DarkLight1337
authored and
joerunde
committed
[Core] Avoid the need to pass `None` values to `Sequence.inputs` (#5099)

DarkLight1337
authored and
joerunde
committed
[Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031)

Etelis
authored and
joerunde
committed
[Doc][Build] update after removing vllm-nccl (#5103)

authored and
joerunde
committed
[Bugfix] gptq_marlin: Ensure g_idx_sort_indices is not a Parameter (#5108)

alexm-redhat
authored and
joerunde
committed
[CI/Build] Docker cleanup functionality for amd servers (#5112)

authored and
joerunde
committed
[BUGFIX] [FRONTEND] Correct chat logprobs (#5029)

br3no
authored and
joerunde
committed
[Bugfix] Automatically Detect SparseML models (#5119)

robertgshaw2-redhat
authored and
joerunde
committed
[CI/Build] increase wheel size limit to 200 MB (#5130)

youkaichao
authored and
joerunde
committed
[Misc] remove duplicate definition of `seq_lens_tensor` in model_runner.py (#5129)

ita9naiwa
authored and
joerunde
committed
[Doc] Use intersphinx and update entrypoints docs (#5125)

DarkLight1337
authored and
joerunde
committed
add doc about serving option on dstack (#3074)

authored and
joerunde
committed
Bump version to v0.4.3 (#5046)

simon-mo
authored and
joerunde
committed
[Build] Disable sm_90a in cu11 (#5141)

simon-mo
authored and
joerunde
committed
[Bugfix] Avoid Warnings in SparseML Activation Quantization (#5120)

robertgshaw2-redhat
authored and
joerunde
committed
[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5) (#5136)

alexm-redhat
authored and
joerunde
committed
Fix cutlass sm_90a vesrion in CMakeList

simon-mo
authored and
joerunde
committed
[Model] Support MAP-NEO model (#5081)

authored and
joerunde
committed
Revert "[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5)" (#5149)

simon-mo
authored and
joerunde
committed
[Misc]: optimize eager mode host time (#4196)

authored and
joerunde
committed
[Model] Enable FP8 QKV in MoE and refine kernel tuning script (#5039)

comaniac
authored and
joerunde
committed
[Doc] Add checkmark for GPTBigCodeForCausalLM LoRA support (#5171)

njhill
authored and
joerunde
committed
[Build] Guard against older CUDA versions when building CUTLASS 3.x kernels (#5168)

tlrmchlsmth
authored and
joerunde
committed
Update vLLM to 1197e021
joerunde
committed
Revert previous attempt at Triton patch; use CustomCacheManger approach instead. (#35)

authored
Add factories for logits_processors
maxdebayser
committed

Commits on Jun 4, 2024

Address review commments and simplify code
maxdebayser
committed
Revert to original formatting
maxdebayser
committed
043 release fixes (#40)
joerunde
authored
[CI/Build] CMakeLists: build all extensions' cmake targets at the same time (#5034)

dtrifiro
authored and
joerunde
committed
[Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137)

tlrmchlsmth
authored and
joerunde
committed
[Kernel] Update Cutlass fp8 configs (#5144)

authored and
joerunde
committed
[Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151)

dashanji
authored and
joerunde
committed
[Bugfix] Fix call to init_logger in openai server (#4765)

NadavShmayo
authored and
joerunde
committed
[Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776)

chenqianfzh
authored and
joerunde
committed
[Bugfix] Remove deprecated @abstractproperty (#5174)

zhuohan123
authored and
joerunde
committed
[Bugfix]: Fix issues related to prefix caching example (#5177) (#5180)

Delviet
authored and
joerunde
committed
[BugFix] Prevent `LLM.encode` for non-generation Models (#5184)

authored and
joerunde
committed
Update test_ignore_eos (#4898)

simon-mo
authored and
joerunde
committed
[Frontend][OpenAI] Support for returning max_model_len on /v1/models response (#4643)

Avinash-Raj
authored and
joerunde
committed
[Kernel][ROCm][AMD] enable fused topk_softmax kernel for moe layer (#4927)

divakar-amd
authored and
joerunde
committed
[Misc] Simplify code and fix type annotations in `conftest.py` (#5118)

DarkLight1337
authored and
joerunde
committed
[Core] Support image processor (#4197)

DarkLight1337
authored and
joerunde
committed
[Core] Remove unnecessary copies in flash attn backend (#5138)

Yard1
authored and
joerunde
committed
[Kernel] Pass a device pointer into the quantize kernel for the scales (#5159)

tlrmchlsmth
authored and
joerunde
committed
[CI/BUILD] enable intel queue for longer CPU tests (#4113)

zhouyuan
authored and
joerunde
committed
[Misc]: Implement CPU/GPU swapping in BlockManagerV2 (#3834)

Kaiyang-Chen
authored and
joerunde
committed
New CI template on AWS stack (#5110)

khluu
authored and
joerunde
committed
[FRONTEND] OpenAI `tools` support named functions (#5032)

br3no
authored and
joerunde
committed
[Bugfix] Support `prompt_logprobs==0` (#5217)

toslunar
authored and
joerunde
committed
[Bugfix] Add warmup for prefix caching example (#5235)

zhuohan123
authored and
joerunde
committed
[Kernel] Enhance MoE benchmarking & tuning script (#4921)

WoosukKwon
authored and
joerunde
committed
[Bugfix]: During testing, use pytest monkeypatch for safely overriding the env var that indicates the vLLM backend (#5210)

afeldman-nm
authored and
joerunde
committed
[Bugfix] Fix torch.compile() error when using MultiprocessingGPUExecutor (#5229)

zifeitong
authored and
joerunde
committed
[CI/Build] Add inputs tests (#5215)

DarkLight1337
authored and
joerunde
committed
[Bugfix] Fix a bug caused by pip install setuptools>=49.4.0 for CPU backend (#5249)

DamonFool
authored and
joerunde
committed
[Kernel] Add back batch size 1536 and 3072 to MoE tuning (#5242)

WoosukKwon
authored and
joerunde
committed
[CI/Build] Simplify model loading for `HfRunner` (#5251)

DarkLight1337
authored and
joerunde
committed
[CI/Build] Reducing CPU CI execution time (#5241)

bigPYJ1151
authored and
joerunde
committed
[CI] mark AMD test as softfail to prevent blockage (#5256)

simon-mo
authored and
joerunde
committed
[Misc] Add transformers version to collect_env.py (#5259)

mgoin
authored and
joerunde
committed

Commits on Jun 6, 2024