V1 embeddings full #2

maxdebayser · 2025-06-16T13:27:36Z

No description provided.

github-actions · 2025-06-16T13:27:44Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

…llm-project#21281) Signed-off-by: Jialin Ouyang <[email protected]>

…nsor are compatible (vllm-project#21300) Signed-off-by: Thomas Parnell <[email protected]>

Signed-off-by: yewentao256 <[email protected]>

…21302) Signed-off-by: Konrad Zawora <[email protected]> Signed-off-by: Chendi.Xue <[email protected]> Co-authored-by: Chendi.Xue <[email protected]>

Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

Signed-off-by: yewentao256 <[email protected]>

Signed-off-by: Shu Wang <[email protected]>

Signed-off-by: rongfu.leng <[email protected]>

Signed-off-by: alyosha-swamy <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Co-authored-by: Jee Jee Li <[email protected]>

Signed-off-by: simon-mo <[email protected]>

Signed-off-by: Kebe <[email protected]>

Signed-off-by: Jialin Ouyang <[email protected]>

…ving (vllm-project#21209) Signed-off-by: Jialin Ouyang <[email protected]>

…urther optimize block_pool (vllm-project#21222) Signed-off-by: Jialin Ouyang <[email protected]>

Signed-off-by: Andy Xie <[email protected]>

Signed-off-by: Mickael Seznec <[email protected]> Co-authored-by: mgoin <[email protected]>

Signed-off-by: Duncan Moss <[email protected]> Signed-off-by: jiahanc <[email protected]> Co-authored-by: jiahanc <[email protected]> Co-authored-by: mgoin <[email protected]>

Signed-off-by: yewentao256 <[email protected]>

…1330) Signed-off-by: bbartels <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

Signed-off-by: raushan <[email protected]>

…ect#21208)" (vllm-project#21384) Signed-off-by: yewentao256 <[email protected]>

…m-project#21033)

…hat()` with `model_impl=transformers` (vllm-project#21353) Signed-off-by: ariG23498 <[email protected]>

…1375) Signed-off-by: DarkLight1337 <[email protected]>

…_allreduce_fusion_max_token_num` (vllm-project#21325) Signed-off-by: XIn Li <[email protected]>

Signed-off-by: simon-mo <[email protected]> Co-authored-by: simon-mo <[email protected]>

Signed-off-by: Rui Qiao <[email protected]>

Signed-off-by: Travis Johnson <[email protected]> Signed-off-by: Joe Runde <[email protected]> Co-authored-by: Travis Johnson <[email protected]>

…roject#21761) Signed-off-by: Lucas Wilkinson <[email protected]>

…ng (vllm-project#20396) Signed-off-by: Faqin Zhong <[email protected]> Co-authored-by: Duncan Moss <[email protected]>

…#21779)

…thod` and `CompressedTensorsW8A8Fp8MoECutlassMethod` (vllm-project#21775) Signed-off-by: yewentao256 <[email protected]>

Signed-off-by: mgoin <[email protected]>

Signed-off-by: calvin chen <[email protected]>

…CM (vllm-project#21766) Signed-off-by: Gregory Shtrasberg <[email protected]>

…llm-project#21684) Signed-off-by: Benji Beck <[email protected]>

Signed-off-by: Nick Hill <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: isotr0py <[email protected]> Signed-off-by: Isotr0py <[email protected]> Signed-off-by: Isotr0py <[email protected]>

Signed-off-by: Reza Barazesh <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]>

…roject#21523) Signed-off-by: Kay Yan <[email protected]> Co-authored-by: Harry Mellor <[email protected]>

Signed-off-by: Chen Zhang <[email protected]>

…llm-project#21831) Signed-off-by: DarkLight1337 <[email protected]>

…ect#19340) Signed-off-by: Richard Zou <[email protected]>

Signed-off-by: Harry Mellor <[email protected]>

Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Harry Mellor <[email protected]>

…Round (vllm-project#21802) Signed-off-by: Wenhua Cheng <[email protected]>

vllm-project#21525) Signed-off-by: elvischenv <[email protected]>

Signed-off-by: David Xia <[email protected]>

Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

Signed-off-by: David Xia <[email protected]>

Signed-off-by: mgoin <[email protected]>

Signed-off-by: Max de Bayser <[email protected]>

…ect#26445) Signed-off-by: Nick Hill <[email protected]>

github-actions · 2025-10-28T02:05:41Z

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!

maxdebayser force-pushed the v1_embeddings_full branch from ddd968e to 5dee54d Compare June 23, 2025 14:35

Jialin and others added 28 commits July 21, 2025 22:37

[Core] Minimize number of dict lookup in _maybe_evict_cached_block (v…

af376ca

…llm-project#21281) Signed-off-by: Jialin Ouyang <[email protected]>

[V1] [Hybrid] Add new test to verify that hybrid views into KVCacheTe…

488d8a9

…nsor are compatible (vllm-project#21300) Signed-off-by: Thomas Parnell <[email protected]>

[Refactor] Fix Compile Warning vllm-project#1444-D (vllm-project#21208)

6e5b5ca

Signed-off-by: yewentao256 <[email protected]>

Fix kv_cache_dtype handling for out-of-tree HPU plugin (vllm-project#…

c17231e

…21302) Signed-off-by: Konrad Zawora <[email protected]> Signed-off-by: Chendi.Xue <[email protected]> Co-authored-by: Chendi.Xue <[email protected]>

[Misc] DeepEPHighThroughtput - Enable Inductor pass (vllm-project#21311)

8425f78

Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

[Bug] DeepGemm: Fix Cuda Init Error (vllm-project#21312)

e69a92a

Signed-off-by: yewentao256 <[email protected]>

Update fp4 quantize API (vllm-project#21327)

9e23ad9

Signed-off-by: Shu Wang <[email protected]>

[Feature][eplb] add verify ep or tp or dp (vllm-project#21102)

3779eb8

Signed-off-by: rongfu.leng <[email protected]>

Add arcee model (vllm-project#21296)

82b8027

Signed-off-by: alyosha-swamy <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Co-authored-by: Jee Jee Li <[email protected]>

[Bugfix] Fix eviction cached blocked logic (vllm-project#21357)

32142b3

Signed-off-by: simon-mo <[email protected]>

[Misc] Remove deprecated args in v0.10 (vllm-project#21349)

bc8a8ce

Signed-off-by: Kebe <[email protected]>

[Core] Optimize update checks in LogitsProcessor (vllm-project#21245)

a322376

Signed-off-by: Jialin Ouyang <[email protected]>

[benchmark] Port benchmark request sent optimization to benchmark_ser…

10904e6

…ving (vllm-project#21209) Signed-off-by: Jialin Ouyang <[email protected]>

[Core] Introduce popleft_n and append_n in FreeKVCacheBlockQueue to f…

ed25054

…urther optimize block_pool (vllm-project#21222) Signed-off-by: Jialin Ouyang <[email protected]>

[Misc] unify variable for LLM instance v2 (vllm-project#21356)

0df4d9b

Signed-off-by: Andy Xie <[email protected]>

[perf] Add fused MLA QKV + strided layernorm (vllm-project#21116)

4fb5691

Signed-off-by: Mickael Seznec <[email protected]> Co-authored-by: mgoin <[email protected]>

[feat]: add SM100 support for cutlass FP8 groupGEMM (vllm-project#20447)

2c8db17

Signed-off-by: Duncan Moss <[email protected]> Signed-off-by: jiahanc <[email protected]> Co-authored-by: jiahanc <[email protected]> Co-authored-by: mgoin <[email protected]>

[Perf] Cuda Kernel for Per Token Group Quant (vllm-project#21083)

774d0c0

Signed-off-by: yewentao256 <[email protected]>

Adds parallel model weight loading for runai_streamer (vllm-project#2…

b194557

…1330) Signed-off-by: bbartels <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

[feat] Enable mm caching for transformers backend (vllm-project#21358)

f38ee34

Signed-off-by: raushan <[email protected]>

Revert "[Refactor] Fix Compile Warning vllm-project#1444-D (vllm-proj…

226b452

…ect#21208)" (vllm-project#21384) Signed-off-by: yewentao256 <[email protected]>

Add tokenization_kwargs to encode for embedding model truncation (vll…

44554a0

…m-project#21033)

[Bugfix] Decode Tokenized IDs to Strings for hf_processor in `llm.c…

2226d5b

…hat()` with `model_impl=transformers` (vllm-project#21353) Signed-off-by: ariG23498 <[email protected]>

[CI/Build] Fix test failure due to updated model repo (vllm-project#2…

35366ae

…1375) Signed-off-by: DarkLight1337 <[email protected]>

Fix Flashinfer Allreduce+Norm enable disable calculation based on `fi…

ae268b6

…_allreduce_fusion_max_token_num` (vllm-project#21325) Signed-off-by: XIn Li <[email protected]>

[Model] Add Qwen3CoderToolParser (vllm-project#21396)

4594fc3

Signed-off-by: simon-mo <[email protected]> Co-authored-by: simon-mo <[email protected]>

[Misc] Copy HF_TOKEN env var to Ray workers (vllm-project#21406)

35bc8bd

Signed-off-by: Rui Qiao <[email protected]>

[BugFix] Fix ray import error mem cleanup bug (vllm-project#21381)

b77c7d3

Signed-off-by: Travis Johnson <[email protected]> Signed-off-by: Joe Runde <[email protected]> Co-authored-by: Travis Johnson <[email protected]>

LucasWilkinson and others added 27 commits July 28, 2025 18:49

[Perf] Disable chunked local attention by default with llama4 (vllm-p…

8aa1485

…roject#21761) Signed-off-by: Lucas Wilkinson <[email protected]>

[Kernel] SM90 CUTLASS FP8 GEMM: add support for swap AB + kernel tuni…

c6c9122

…ng (vllm-project#20396) Signed-off-by: Faqin Zhong <[email protected]> Co-authored-by: Duncan Moss <[email protected]>

[Docs] Minimize spacing for supported_hardware.md table (vllm-project…

947e982

…#21779)

[Refactor] Merge Compressed Tensor FP8 `CompressedTensorsW8A8Fp8MoEMe…

48b763d

…thod` and `CompressedTensorsW8A8Fp8MoECutlassMethod` (vllm-project#21775) Signed-off-by: yewentao256 <[email protected]>

[CI] Parallelize Kernels MoE Test (vllm-project#21764)

afa2607

Signed-off-by: mgoin <[email protected]>

skip fusedmoe layer for start_load_kv (vllm-project#21378)

e18f085

Signed-off-by: calvin chen <[email protected]>

[AMD][CI/Build][Bugfix] Guarding CUDA specific functions by ifndef RO…

12a223e

…CM (vllm-project#21766) Signed-off-by: Gregory Shtrasberg <[email protected]>

Migrate InternVLImageInputs and InternVLVideoInputs to TensorSchema (v…

f1e2c09

…llm-project#21684) Signed-off-by: Benji Beck <[email protected]>

[Misc] Rework process titles (vllm-project#21780)

7234fe2

Signed-off-by: Nick Hill <[email protected]>

[Doc] Link to RFC for pooling optimizations (vllm-project#21806)

a248025

Signed-off-by: DarkLight1337 <[email protected]>

[Model]: Fused MoE for nomic-embed-text-v2-moe (vllm-project#18321)

a4528f0

Signed-off-by: isotr0py <[email protected]> Signed-off-by: Isotr0py <[email protected]> Signed-off-by: Isotr0py <[email protected]>

[V0 deprecation] Guided decoding (vllm-project#21347)

37efc63

Signed-off-by: Reza Barazesh <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]>

[Model] Refactor JambaForCausalLM (vllm-project#21394)

61a6905

Signed-off-by: Jee Jee Li <[email protected]>

[Docs] Fix the outdated URL for installing from vLLM binaries (vllm-p…

2470419

…roject#21523) Signed-off-by: Kay Yan <[email protected]> Co-authored-by: Harry Mellor <[email protected]>

[KVCache] Make KVCacheSpec hashable (vllm-project#21791)

755fa8b

Signed-off-by: Chen Zhang <[email protected]>

[Doc] Update compatibility matrix for pooling and multimodal models (v…

ab71413

…llm-project#21831) Signed-off-by: DarkLight1337 <[email protected]>

[Bugfix] VLLM_V1 supports passing other compilation levels (vllm-proj…

04e3850

…ect#19340) Signed-off-by: Richard Zou <[email protected]>

[Docs] Merge design docs for a V1 only future (vllm-project#21832)

f693b06

Signed-off-by: Harry Mellor <[email protected]>

[TPU] Add an optimization doc on TPU (vllm-project#21155)

759b87e

Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Harry Mellor <[email protected]>

[Bugfix]fix mixed bits and visual language model quantization in Auto…

ad341c5

…Round (vllm-project#21802) Signed-off-by: Wenhua Cheng <[email protected]>

[Bugfix] Fix workspace buffer None issue for Flashinfer TRTLLM Backend (

58b11b2

vllm-project#21525) Signed-off-by: elvischenv <[email protected]>

[Docs] use uv in GPU installation docs (vllm-project#20277)

37f86d9

Signed-off-by: David Xia <[email protected]>

[Doc] Add FusedMoE Modular Kernel Documentation (vllm-project#21623)

f03e9cf

Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

[Doc] update Contributing page's testing section (vllm-project#18272)

7b49cb1

Signed-off-by: David Xia <[email protected]>

Add flashinfer_python to CUDA wheel requirements (vllm-project#21389)

a33ea28

Signed-off-by: mgoin <[email protected]>

Merge branch 'upstream_main' into v1_embeddings_full

b62a51a

Signed-off-by: Max de Bayser <[email protected]>

simplify score code

cc970ab

Signed-off-by: Max de Bayser <[email protected]>

maxdebayser pushed a commit that referenced this pull request Oct 9, 2025

[Bugfix] Catch and log invalid token ids in detokenizer #2 (vllm-proj…

bb6d8c2

…ect#26445) Signed-off-by: Nick Hill <[email protected]>

github-actions bot added the stale label Oct 28, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

V1 embeddings full #2

V1 embeddings full #2

Uh oh!

maxdebayser commented Jun 16, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Jun 16, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

123 participants

V1 embeddings full #2

Are you sure you want to change the base?

V1 embeddings full #2

Uh oh!

Conversation

maxdebayser commented Jun 16, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jun 16, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

123 participants

maxdebayser commented Jun 16, 2025 •

edited by github-actions bot

Loading