Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upstream merge 24 09 27 0.6.2 #213

Merged
merged 51 commits into from
Oct 2, 2024
Merged

Upstream merge 24 09 27 0.6.2 #213

merged 51 commits into from
Oct 2, 2024

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Sep 27, 2024

Merge in vllm 0.6.2 with LLama 3.2 support
For llama3.2 a cross-attention implementation is still required. Either through ROCm implementation, or using (building) xformers >= 0.28

Due to versioning changes in setup.py, a tag of a form v0.6.2+rocm will be required for this to build after merge

dtrifiro and others added 30 commits September 23, 2024 09:44
@dtrifiro @youkaichao
[CI/Build] use setuptools-scm to set __version__ (vllm-project#4738)
ee5f34b 
Co-authored-by: youkaichao <[email protected]>
@LucasWilkinson @mgoin
@divakar-amd @tlrmchlsmth
[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GP…
86e9c8d 
…TQMarlin (vllm-project#7701)

Co-authored-by: mgoin <[email protected]>
Co-authored-by: Divakar Verma <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
@jeejeelee
[Kernel][LoRA] Add assertion for punica sgmv kernels (vllm-project#7585)
9b0e3ec
@russellb
[Core] Allow IPv6 in VLLM_HOST_IP with zmq (vllm-project#8575)
b05f5c9 
Signed-off-by: Russell Bryant <[email protected]>
@jiqing-feng
Fix typical acceptance sampler with correct recovered token ids (vllm…
5f7bb58 
…-project#8562)
@alexm-redhat
Add output streaming support to multi-step + async while ensuring Req…
1a2aef3 
…uestOutput obj reuse (vllm-project#8335)
@hongxiayang
[Hardware][AMD] ROCm6.2 upgrade (vllm-project#8674)
530821d
@sroy745
Fix tests in test_scheduler.py that fail with BlockManager V2 (vllm-p…
88577ac 
…roject#8728)
@youkaichao @LunrEclipse
re-implement beam search on top of vllm core (vllm-project#8726)
0250dd6 
Co-authored-by: Brendan Wong <[email protected]>
@simon-mo
Revert "[Core] Rename PromptInputs to PromptType, and inputs to…
3185fb0 
… `prompt`" (vllm-project#8750)
@comaniac
[MISC] Skip dumping inputs when unpicklable (vllm-project#8744)
b8747e8
@petersalas
[Core][Model] Support loading weights by ID within models (vllm-proje…
3f06bae 
…ct#7931)
@alex-jw-brooks @DarkLight1337
[Model] Expose Phi3v num_crops as a mm_processor_kwarg (vllm-project#…
8ff7ced 
…8658)

Signed-off-by: Alex-Brooks <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@hanzhi713
[Bugfix] Fix potentially unsafe custom allreduce synchronization (vll…
cc4325b 
…m-project#8558)
@ElizaWszola @mgoin
[Kernel] Split Marlin MoE kernels into multiple files (vllm-project#8661
a928ded 
)

Co-authored-by: mgoin <[email protected]>
@aandyw @DarkLight1337 @ywang96
[Frontend] Batch inference for llm.chat() API (vllm-project#8648)
2529d09 
Co-authored-by: Cyrus Leung <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@LucasWilkinson
[Bugfix] Fix torch dynamo fixes caused by replace_parameters (vllm-…
72fc97a 
…project#8748)
@dtrifiro
[CI/Build] fix setuptools-scm usage (vllm-project#8771)
2467b64
@youkaichao
[misc] soft drop beam search (vllm-project#8763)
1e7d5c0
@jeejeelee
[[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (vllm-projec…
13f9f7a 
…t#8768)
@tjohnson31415
[Core][Bugfix] Support prompt_logprobs returned with speculative deco…
01b6f9e 
…ding (vllm-project#8047)

Signed-off-by: Travis Johnson <[email protected]>
@apatke
[Core] Adding Priority Scheduling (vllm-project#5958)
6da1ab6
@joerunde
[Bugfix] Use heartbeats instead of health checks (vllm-project#8583)
6e0c9d6
@sroy745
Fix test_schedule_swapped_simple in test_scheduler.py (vllm-project#8780
ee777d9 
)
@sasha0552
[Bugfix][Kernel] Implement acquire/release polyfill for Pascal (vllm-…
b452247 
…project#8776)
@sroy745
Fix tests in test_chunked_prefill_scheduler which fail with BlockMana…
fc3afc2 
…ger V2 (vllm-project#8752)
@zifeitong
[BugFix] Propagate 'trust_remote_code' setting in internvl and minicp…
e3dd069 
…mv (vllm-project#8250)
@Isotr0py
[Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (vll…
c239536 
…m-project#8770)
@sohamparikh
[Bugfix] load fc bias from config for eagle (vllm-project#8790)
3e073e6
@agt
[Frontend] OpenAI server: propagate usage accounting to FastAPI middl…
1ac3de0 
…eware layer (vllm-project#8672)
darthhexx and others added 17 commits September 25, 2024 00:52
@darthhexx
[Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (vllm-…
3368c3a 
…project#8767)

Signed-off-by: darthhexx <[email protected]>
@wooyeonlee0
[Misc] Fix minor typo in scheduler (vllm-project#8765)
8fae5ed
@hongxiayang
[CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 up…
1c04644 
…grade (vllm-project#8777)
@bnellnm
[Kernel] Fullgraph and opcheck tests (vllm-project#8479)
300da09
@jeejeelee
[[Misc]] Add extra deps for openai server image (vllm-project#8792)
c6f2485
@DefTruth
[VLM][Bugfix] internvl with num_scheduler_steps > 1 (vllm-project#8614)
0c4d2ad
@DarkLight1337
rename PromptInputs and inputs with backward compatibility (vllm-proj…
28e1299 
…ect#8760)
@Abatom
[Frontend] MQLLMEngine supports profiling. (vllm-project#8761)
64840df
@mgoin
[Misc] Support FP8 MoE for compressed-tensors (vllm-project#8588)
873edda
@simon-mo
Revert "rename PromptInputs and inputs with backward compatibility (v…
4f1ba08 
…llm-project#8760) (vllm-project#8810)
@heheda12345 @simon-mo
@CatherineSue @ywang96
[Model] Add support for the multi-modal Llama 3.2 model (vllm-project…
770ec60 
…#8811)

Co-authored-by: simon-mo <[email protected]>
Co-authored-by: Chang Su <[email protected]>
Co-authored-by: Simon Mo <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@ywang96
[Doc] Update doc for Transformers 4.45 (vllm-project#8817)
e2c6e0a
@mgoin
[Misc] Support quantization of MllamaForCausalLM (vllm-project#8822)
7193774
@ywang96
[Misc] Update config loading for Qwen2-VL and remove Granite (vllm-pr…
4bb98f2 
…oject#8837)
@gshtras
Merge remote-tracking branch 'upstream/main' into main
c5b1012
@gshtras
Add setuptools-scm requirement to requirements-rocm since we don't us…
1adaa9a 
…e requirements-build

Changing back PromptType to PromptInputs following refactoring revert
@gshtras
Merge branch 'main' into upstream_merge_24_09_27_0.6.2
aac2e0b
shajrawi
shajrawi approved these changes Oct 2, 2024
View reviewed changes
Copy link
Collaborator

@shajrawi shajrawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ship it

@gshtras gshtras merged commit 030374b into main Oct 2, 2024
16 of 17 checks passed
@gshtras gshtras deleted the upstream_merge_24_09_27_0.6.2 branch October 2, 2024 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

@maleksan85 maleksan85 Awaiting requested review from maleksan85

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.