Skip to content

[None][docs] fix incorrect auto sampler behavior description for beam search - #14487

Merged
chang-l merged 1 commit into
NVIDIA:mainfrom
fuergaosi233:fix/sampling-doc-auto-sampler-beam-search
May 28, 2026
Merged

[None][docs] fix incorrect auto sampler behavior description for beam search#14487
chang-l merged 1 commit into
NVIDIA:mainfrom
fuergaosi233:fix/sampling-doc-auto-sampler-beam-search

Conversation

@fuergaosi233

@fuergaosi233 fuergaosi233 commented May 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • docs/source/features/sampling.md incorrectly stated that the auto sampler mode uses TRTLLMSampler when beam search is enabled
  • In practice, auto always selects TorchSampler for all requests, including beam search
  • This contradicts both the field description in llm_args.py:3810 ("Defaults to auto, which will use TorchSampler") and the actual sampler instantiation logic in _torch/pyexecutor/_util.py:1821

Root Cause

The auto mode was historically documented as routing to TRTLLMSampler for beam search, but this behavior does not exist in the code. TRTLLMSampler is only used when explicitly set via sampler_type="TRTLLMSampler", which is deprecated and scheduled for removal in release 1.4.

Code evidence:

  • tensorrt_llm/_torch/pyexecutor/_util.py:1821: Only checks if llm_args.sampler_type == SamplerType.TRTLLMSampler — no auto-to-TRTLLM path for beam search
  • tensorrt_llm/llmapi/llm_args.py:3810: Field description says "Defaults to auto, which will use TorchSampler"
  • tensorrt_llm/_torch/auto_deploy/shim/ad_executor.py:843-844: if sampler_type == SamplerType.auto: sampler_type = SamplerType.TorchSampler

Test plan

  • Verify the updated description is consistent with llm_args.py field description
  • Verify no code path exists where auto + beam search selects TRTLLMSampler

Summary by CodeRabbit

  • Documentation
    • Updated sampling backend documentation to clarify that the auto mode defaults to using Torch Sampler for all requests, providing consistent behavior across sampling configurations.

Review Change Stack

… search

The sampling.md documentation incorrectly stated that the `auto` sampler
mode uses TRTLLMSampler when beam search is enabled. In practice, the
`auto` mode always selects TorchSampler, as reflected in the field
description in llm_args.py and the actual sampler instantiation logic in
_torch/pyexecutor/_util.py.

TRTLLMSampler is only used when explicitly set via sampler_type="TRTLLMSampler",
which is deprecated and scheduled for removal in release 1.4.

Signed-off-by: holegots <fuergaosi@gmail.com>
@fuergaosi233
fuergaosi233 requested a review from a team as a code owner May 23, 2026 19:23
@coderabbitai

coderabbitai Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR updates the sampling backend documentation to clarify that the default auto selection uses Torch Sampler for all requests, removing the prior conditional behavior that selected TRTLLM Sampler for Beam Search operations.

Changes

Sampling Backend Documentation

Layer / File(s) Summary
Sampling backend default behavior documentation
docs/source/features/sampling.md
The default auto sampling backend documentation now explicitly states that Torch Sampler is used for all requests, replacing the previous conditional description that selected TRTLLM Sampler when using Beam Search.

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: fixing incorrect documentation about auto sampler behavior for beam search.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The PR description comprehensively explains the issue, root cause, and provides specific code evidence with line references.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@Funatiq

Funatiq commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Good catch. Somehow we missed that.

@Funatiq

Funatiq commented May 28, 2026

Copy link
Copy Markdown
Collaborator

/bot skip --comment "Changed only docs"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #50727 [ skip ] triggered by Bot. Commit: 8612c60 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #50727 [ skip ] completed with state SUCCESS. Commit: 8612c60
Skipping testing for commit 8612c60

Link to invocation

@chang-l
chang-l merged commit 82f69a0 into NVIDIA:main May 28, 2026
14 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants