[cycancode]

Overview

With --use-fp64-gumbel, Model Runner V2 still sends unseeded top-k/top-p batches to FlashInfer's fused sampler, which draws fp32 noise, so the flag is silently ignored for those steps. This skips the FlashInfer sampler when fp64 Gumbel is requested, matching the V1 sampler and MRV2's own XPU path.

Claims

• With use_fp64_gumbel=True, MRV2 samples unseeded top-k/top-p requests with fp64 Gumbel noise instead of FlashInfer.
• Without the flag, MRV2 keeps using the FlashInfer sampler as before.
• With the flag, MRV2 no longer probes FlashInfer or logs "Using FlashInfer for top-p & top-k sampling".

Validation

End to end on 34ac4f96a3 (Qwen3-0.6B, RTX 4090, MRV2, --enforce-eager, FlashInfer kernels installed with vllm download-kernels). The engine ran in-process (VLLM_ENABLE_V1_MULTIPROCESSING=0) so I could count which sampling function each step called. Workload: 4 prompts, temperature=0.8, top_p=0.9, max_tokens=16, no seed, so 16 sampling steps per run.

Build	use_fp64_gumbel	Sampling calls
main	True	flashinfer_sample × 16
this PR	True	gumbel_sample(use_fp64=True) × 16
this PR	False	flashinfer_sample × 16 (unchanged)

Tests:

• New tests/v1/worker/test_gpu_sampler_flags.py::test_fp64_gumbel_bypasses_flashinfer_sampler runs Sampler.sample on an unseeded top-p request with FlashInfer reported as supported and flashinfer_sample patched to fail. It fails on main and passes with the fix.
• pytest tests/v1/worker/test_gpu_sampler_flags.py: 17 passed.
• pytest tests/v1/worker/test_gpu_logits_processors.py tests/v1/worker/test_gpu_thinking_budget.py tests/v1/sample/test_topk_topp_sampler.py: 168 passed, 1 skipped.
• pre-commit run --files <changed files>: all hooks pass, including mypy.

Details

Root cause: Sampler.__init__ in vllm/v1/worker/gpu/sample/sampler.py set use_flashinfer from flashinfer_sampler_supported() alone. sample() then routes a batch to flashinfer_sample whenever it has top-k/top-p and no greedy, seeded or processed-logprobs requests, so use_fp64_gumbel only took effect for steps that happened to fall back to gumbel_sample.

Other paths already handle this: V1's TopKTopPSampler.forward_cuda and forward_hip fall back to native sampling when use_fp64_gumbel is set (#43150), and MRV2 already excludes its XPU sampler for the same reason (#57277). MRV2 picked up the FlashInfer sampler in #42472, two days before #43150, which only touched V1.

Not a duplicate: I found no issue or PR for this (searched "fp64 gumbel", "use_fp64_gumbel", "fp64 flashinfer", "MRV2 flashinfer sampler"). #54843 and #54952 cover the CPU spec-decode recovered-token path. Of the open PRs that touch this sampler, #60993 edits the FlashInfer call site but does not change when FlashInfer is used.

AI assistance was used to find this bug, write the fix and the test, and run the validation shown above.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.