Reject repetition_penalty=0 in SamplingParams.verify()

[RulinJuice]

Motivation

SamplingParams.verify() currently accepts repetition_penalty=0.0 (range [0, 2]), but the kernel apply_scaling_penalties in python/sglang/srt/sampling/penaltylib/repetition_penalty.py divides logits by the penalty:

@torch.compile(dynamic=True, backend=get_compiler_backend())
def apply_scaling_penalties(logits, scaling_penalties):
    logits[:] = torch.where(
        logits < 0,
        logits * scaling_penalties,
        logits / scaling_penalties,   # <-- penalty=0 -> inf
    )

A penalty of 0 produces inf -> NaN in the probability tensor and crashes every TP rank with:

/pytorch/aten/src/ATen/native/cuda/TensorCompare.cu:109: _assert_async_cuda_kernel:
  Assertion `probability tensor contains either `inf`, `nan` or element < 0` failed.
torch.AcceleratorError: CUDA error: device-side assert triggered
  ...in scheduler.event_loop_overlap() -> process_batch_result_decode() -> result.copy_done.synchronize()

This brings down the entire engine and requires a full restart.

Encountered in the wild when an upstream wrapper used 0.0 as a fallback default for "the user didn't specify a value" (it should default to 1.0, but verify() should also reject the invalid value early instead of letting it reach the GPU kernel and crash all TP ranks).

Modifications

• python/sglang/srt/sampling/sampling_params.py: tighten the valid range from [0, 2] to (0, 2], matching the strict-positive convention already used for top_p (if not 0.0 < self.top_p <= 1.0). Update the error message to clarify the new range and the 1.0 = no penalty convention.
• test/registered/unit/sampling/test_sampling_params.py:
  • Add test_repetition_penalty_zero_raises covering the new rejection.
  • Split the previous test_repetition_penalty_boundaries_valid (which asserted both 0.0 and 2.0 were valid) into test_repetition_penalty_boundary_two_valid (only the 2.0 boundary remains valid).
  • Add test_repetition_penalty_small_positive_valid to confirm small but positive values like 1e-3 are still accepted.

Accuracy Tests

N/A — input validation only, no kernel/forward change. The kernel itself is unmodified.

Speed Tests and Profiling

N/A — verify() runs once per request at submission time, not in the hot path.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations — N/A, no user-facing API change.
• Provide accuracy and speed benchmark results — N/A, validation-only change.
• Follow the SGLang code style guidance.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[Ratish1]

Hey @RulinJuice , thanks for this fix. It makes sense to me. Can we also update the docs which documents the wrong range? . The file to update is docs/basic_usage/sampling_params.md

[RulinJuice]

Thanks @Ratish1! Updated docs/basic_usage/sampling_params.md to reflect the new range.

[Ratish1]

/tag-and-rerun-ci

[hzh0425]

LGTM, Let's move on the ci

[Ratish1]

/rerun-failed-ci