From 8612c600cad0ad5f738d1bb746716a9a2b24f92c Mon Sep 17 00:00:00 2001 From: holegots Date: Sun, 24 May 2026 03:21:21 +0800 Subject: [PATCH] [None][docs] fix incorrect auto sampler behavior description for beam search The sampling.md documentation incorrectly stated that the `auto` sampler mode uses TRTLLMSampler when beam search is enabled. In practice, the `auto` mode always selects TorchSampler, as reflected in the field description in llm_args.py and the actual sampler instantiation logic in _torch/pyexecutor/_util.py. TRTLLMSampler is only used when explicitly set via sampler_type="TRTLLMSampler", which is deprecated and scheduled for removal in release 1.4. Signed-off-by: holegots --- docs/source/features/sampling.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/source/features/sampling.md b/docs/source/features/sampling.md index 13b24f533243..fc3cb3ed9324 100644 --- a/docs/source/features/sampling.md +++ b/docs/source/features/sampling.md @@ -36,10 +36,7 @@ llm = LLM(model='nvidia/Llama-3.1-8B-Instruct-FP8', sampler_type="TRTLLMSampler") ``` -By default, the sampling backend is chosen to be `auto`. This will use: - -* TRTLLM Sampler when using Beam Search. -* Torch Sampler otherwise. +By default, the sampling backend is chosen to be `auto`. This will use Torch Sampler for all requests. Here is an example to run a model with basic usage of sampling parameters. This example prepares two identical prompts which will give different results due to the sampling parameters chosen: