Skip to content

[Cherry-pick][releases/v0.23.0][Doc][Misc] Add limitation for reduce sample (from #13468) #13469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zzzzwwjj merged 1 commit into from
Aug 4, 2026
Merged

[Cherry-pick][releases/v0.23.0][Doc][Misc] Add limitation for reduce sample (from #13468) #13469

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/user_guide/configuration/additional_config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ The following table lists additional configuration options available in vLLM Asc
| `enable_dsa_cp` | bool | `False` | Whether to enable dsa_cp for DeepSeek V3.2, DeepSeek V4, and other models with the same architecture. This feature depends on FlashComm1. Please ensure that FlashComm1 is enabled before enabling this feature.|
| `rejection_sampler_config` | dict | `{}` | Configuration options for rejection sampler (block verify and entropy verify). |
| `multistream_dsv4_dsa_overlap` | bool | `True` | Whether to enable dsa multi-stream overlap for DeepSeek V4. |
| `enable_reduce_sample` | bool | `False` | Whether to enable reduce sample optimization to reduce communication and computation overheads in the tensor parallelism scenario. When enabled, logits are kept partitioned across TP ranks and only the small set of top-k candidate values/indices is communicated, instead of performing a full-vocabulary all-to-all/all-gather. |
| `enable_reduce_sample` | bool | `False` | Whether to enable reduce sample optimization to reduce communication and computation overheads in the tensor parallelism scenario. When enabled, logits are kept partitioned across TP ranks and only the small set of top-k candidate values/indices is communicated, instead of performing a full-vocabulary all-to-all/all-gather. **Limitations**: (1) PD disaggregation scenarios are not supported. (2) Must be disabled when sampling logprobs are requested. When reduce sample is enabled, logprobs are silently computed over partitioned logits instead of the full vocabulary, producing incorrect logprob values and top-k rankings.|

@gemini-code-assist gemini-code-assist Bot Aug 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current Pull Request Title does not fully adhere to the Repository Style Guide. Specifically, it includes an extra [Cherry-pick] prefix and does not strictly follow the [Branch][Module][Action] Pull Request Title format.

According to the Repository Style Guide, please update the PR Title and Summary as suggested below:

Suggested PR Title:

[releases/v0.23.0][Doc][Misc] Add limitation for reduce sample (from #13468)

Suggested PR Summary:

### What this PR does / why we need it?

This PR updates the documentation for the `enable_reduce_sample` configuration option in vLLM Ascend. It adds explicit limitations for this optimization, specifically noting that:
1. PD (Prefill-Decode) disaggregation scenarios are not supported.
2. It must be disabled when sampling logprobs are requested, as logprobs would otherwise be silently computed over partitioned logits instead of the full vocabulary, leading to incorrect logprob values and top-k rankings.

Fixes #13468

### Does this PR introduce _any_ user-facing change?

No, this is a documentation-only update.

### How was this patch tested?

- vLLM version: v0.26.0
- vLLM main: https://github.com/vllm-project/vllm/commit/0351e9aa1fdf1a51329d1906881528dfe61fc88e
References
  1. The PR Title must follow the format [Branch][Module][Action] Pull Request Title as specified in the Repository Style Guide. (link)


The details of each configuration option are as follows:

Expand Down