[Cherry-pick][releases/v0.23.0][Doc][Misc] Add limitation for reduce sample (from #13468) - #13469
Conversation
Cherry-pick of PR vllm-project#13468 by @zouzy5137
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request cherry-picks documentation improvements from the main branch to the v0.23.0 release. It clarifies the constraints of the Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the documentation for the enable_reduce_sample configuration option in additional_config.md to outline its limitations regarding PD disaggregation and logprob sampling. The reviewer noted that the PR title and summary do not comply with the repository's style guide and provided a formatted suggestion to correct this.
| | `rejection_sampler_config` | dict | `{}` | Configuration options for rejection sampler (block verify and entropy verify). | | ||
| | `multistream_dsv4_dsa_overlap` | bool | `True` | Whether to enable dsa multi-stream overlap for DeepSeek V4. | | ||
| | `enable_reduce_sample` | bool | `False` | Whether to enable reduce sample optimization to reduce communication and computation overheads in the tensor parallelism scenario. When enabled, logits are kept partitioned across TP ranks and only the small set of top-k candidate values/indices is communicated, instead of performing a full-vocabulary all-to-all/all-gather. | | ||
| | `enable_reduce_sample` | bool | `False` | Whether to enable reduce sample optimization to reduce communication and computation overheads in the tensor parallelism scenario. When enabled, logits are kept partitioned across TP ranks and only the small set of top-k candidate values/indices is communicated, instead of performing a full-vocabulary all-to-all/all-gather. **Limitations**: (1) PD disaggregation scenarios are not supported. (2) Must be disabled when sampling logprobs are requested. When reduce sample is enabled, logprobs are silently computed over partitioned logits instead of the full vocabulary, producing incorrect logprob values and top-k rankings.| |
There was a problem hiding this comment.
The current Pull Request Title does not fully adhere to the Repository Style Guide. Specifically, it includes an extra [Cherry-pick] prefix and does not strictly follow the [Branch][Module][Action] Pull Request Title format.
According to the Repository Style Guide, please update the PR Title and Summary as suggested below:
Suggested PR Title:
[releases/v0.23.0][Doc][Misc] Add limitation for reduce sample (from #13468)Suggested PR Summary:
### What this PR does / why we need it?
This PR updates the documentation for the `enable_reduce_sample` configuration option in vLLM Ascend. It adds explicit limitations for this optimization, specifically noting that:
1. PD (Prefill-Decode) disaggregation scenarios are not supported.
2. It must be disabled when sampling logprobs are requested, as logprobs would otherwise be silently computed over partitioned logits instead of the full vocabulary, leading to incorrect logprob values and top-k rankings.
Fixes #13468
### Does this PR introduce _any_ user-facing change?
No, this is a documentation-only update.
### How was this patch tested?
- vLLM version: v0.26.0
- vLLM main: https://github.com/vllm-project/vllm/commit/0351e9aa1fdf1a51329d1906881528dfe61fc88eReferences
- The PR Title must follow the format
[Branch][Module][Action] Pull Request Titleas specified in the Repository Style Guide. (link)
github-actions
Bot
added
the
documentation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
* releases/v0.23.0: (104 commits) [Doc][BugFix] Update proxy script name in DeepSeek-V3.2 tutorial (vllm-project#13537) [Doc] Fix link errors and update documentation structure (vllm-project#13483) [BugFix][releases/v0.23.0] fix fiaV2 contiguous err in GQA (vllm-project#13458) [v0.23.0][BugFix] Isolate layerwise GVA keys by parallel rank (vllm-project#13513) [Doc][Feature] Add model support of Ascend 950 (vllm-project#13525) [Doc] fix DeepSeek V4 Flash&Pro model tutorial docs link error (vllm-project#13497) [Cherry-pick][releases/v0.23.0][Doc][Misc] Add limitation for reduce sample (from vllm-project#13468) (vllm-project#13469) [BugFix][v0.23.0][KV Pool] Include MTP KV in layerwise AscendStore transfer (vllm-project#13454) [Doc][Misc] Standardize TorchNPU capitalization and update Ascend 950 product terminology (vllm-project#13089) [v0.23.0][Doc] Translated Doc files 2026-08-04 (vllm-project#13437) [Misc][v0.23.0] Fix translation extraction for tables nested in tabs (vllm-project#13413) [Doc] Fix translation and formatting in documentation (vllm-project#13390) [releases/v0.23.0][Doc][Misc] Backport Kimi-K2-Thinking tuning docs to v0.23.0 (vllm-project#13361) [Doc] Deployment key parameter supplement- vllm-project#13297 (vllm-project#13299) [v0.23.0][Doc] Translated Doc files 2026-07-31 (vllm-project#13283) [Doc][Misc] Update max-num-seqs configurations in GLM5 tutorial (vllm-project#13203) [Cherry-pick][releases/v0.23.0][Doc][Misc] Add deployment reference notice for GLM-5 (from vllm-project#12958) (vllm-project#12960) [BugFix][v0.23.0][KV Pool] Guard batch_get_key_info before memcache backend init (vllm-project#13307) [DOC]Modify the scope of scenarios supported by CP (vllm-project#13303) Revert "[cherry-pick][v0.23.0][Performance] remove D2H sync in QLIMetadata builder for DSA_CP" (vllm-project#13289) ...
3 participants
Cherry-pick of PR #13468 onto
releases/v0.23.0.Original PR: #13468
Original author: @zouzy5137
What this PR does / why we need it?
This PR updates the documentation for the
enable_reduce_sampleconfiguration option in vLLM Ascend. It adds explicit limitations for this optimization, specifically noting that:Does this PR introduce any user-facing change?
No, this is a documentation-only update.
How was this patch tested?