[Bugfix][Mamba] Reject stochastic rounding with Triton speculation

[milesial]

Overview

Reject Triton Mamba stochastic rounding with speculative decoding at startup. The speculative kernel silently ignores the rounding flag, including on supported Blackwell GPUs.

Claims

The shared VllmConfig validation raises an actionable error for this combination. The error suggests --mamba-backend flashinfer or disabling stochastic rounding.

FlashInfer with rounding, Triton without rounding, and Triton rounding without speculation remain valid.

Validation

The four configuration cases pass. With the base implementation, the rejection case fails because startup accepts the combination. The other three cases pass.

.venv/bin/python3 -m pytest --noconftest tests/test_config.py -q -k mamba_stochastic_rounding_with_speculation
pre-commit run --files tests/test_config.py vllm/config/vllm.py

The local run uses macOS ARM64 and PyTorch 2.13.0. The test mocks CUDA capability queries and calls the real VllmConfig constructor. --noconftest skips unrelated global fixtures. All applicable repository hooks pass.

No new model evaluation is needed for this startup rejection. Accepted configurations retain their existing inference behavior.

Details

The Triton speculative branch saves FP16 state with a plain cast. Stochastic conversion only runs in the non-speculative branch. This omission dates to #35753 and affects real and synthetic acceptance.

The guard applies to every speculative method. Existing validation already requires an explicit FP16 state cache when stochastic rounding is enabled.

The PR changes only startup validation and its regression test. It makes no kernel, performance, or model-quality claims. The existing Engine Core CI job includes tests/test_config.py.

Duplicate searches covered 35753 in:body, Mamba stochastic rounding, and speculative stochastic rounding Triton. #59729 changes FlashInfer seed scheduling. #40012 adds quantized state formats. Neither adds this startup rejection.

AI assistance was used for the implementation, tests, and description.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.