[Fix] Validate stop_token_ids range in SamplingParams.verify (OOB scatter DoS)

[v-code01]

Motivation

User-supplied stop_token_ids flow into the min_new_tokens penalizer, which builds a scatter_add_ index over a [vocab_size + 1] tensor (penaltylib/min_new_tokens.py:43-58). SamplingParams.verify() validates logit_bias keys against vocab_size but does not range-check stop_token_ids, so:

• an id >= vocab_size + 1 is an out-of-bounds scatter index. On CPU it's a RuntimeError; on CUDA it's a device-side assert / illegal memory access that poisons the context and takes down the whole server process (all in-flight requests) from a single malformed request. Both min_new_tokens > 0 and the id are user-controlled (OpenAI stop_token_ids), so this is a network-reachable DoS.
• an id == vocab_size does not crash but is silently dropped by the trailing [:, :vocab_size] slice, so that stop token never fires.

Repro (CPU, no model): SamplingParams(min_new_tokens=1, stop_token_ids=[100000]).verify(32000) is accepted today, and driving the real penalizer with that request raises index 100000 is out of bounds for dimension 1 with size 32001.

Modifications

Reject out-of-range stop_token_ids in verify(), mirroring the existing logit_bias key validation, so a malformed request fails cleanly at the validation boundary instead of at the scatter.

Tests

Four cases added to test_sampling_params.py::TestSamplingParamsVerify (CPU): id >= vocab_size + 1, id == vocab_size, negative id all raise; in-range ids pass. All fail on the pre-fix code.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.

---

CI States

Latest PR Test (Base): ❌ Run #29597096447
Latest PR Test (Extra): ❌ Run #29597096626

[gemini-code-assist]

Code Review

This pull request adds validation for stop_token_ids in SamplingParams.verify to ensure they fall within the valid range [0, vocab_size - 1], preventing potential out-of-bounds CUDA errors or silent drops. It also adds corresponding unit tests. The reviewer suggested removing the redundant int() conversion on token_id during validation, as stop_token_ids are already guaranteed to be integers.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Since self.stop_token_ids is already processed in __post_init__ (where all elements are filtered and converted to integers via int(t)), each token_id is guaranteed to be an int. Calling int(token_id) here is redundant and can be safely omitted to clean up the code and avoid unnecessary overhead in the request validation hot path.

Suggested change

	if not 0 <= int(token_id) < vocab_size:
	if not 0 <= token_id < vocab_size:

[v-code01]

cc @hnyls2002 (you have touched this penalizer) and @merrymercy as sampling/managers owner. User stop_token_ids reach the min_new_tokens penalizer's scatter_add_ over a [vocab_size+1] tensor with no range check; an id >= vocab_size+1 is an out-of-bounds scatter (a CUDA device-side assert that takes the server down), and == vocab_size is silently dropped. This mirrors the logit_bias range check already in verify(). Reproduced the real crash on CPU.

[v-code01]

Good point, dropped the redundant int() since stop_token_ids are already normalized to a set of ints in __post_init__ (the logit_bias check above keeps int() because its keys are strings).

[v-code01]

Part of #31597 (validates one of the unbounded request fields described there).

[github-actions]

Thanks @v-code01. Closing this because you have 15 PRs open and none updated in 7 days, over our soft cap of 5 for idle PRs.

Reopen the two or three you most want reviewed - a short active set moves faster than a long idle one.

Some directories moved recently, so an older branch may need retargeting:
sgl-kernel/ -> python/sglang/kernels/aot/, python/sglang/jit_kernel/
-> python/sglang/kernels/jit/, docs/ -> docs/docs/ (.mdx),
bench_serving.py -> benchmark/serving.py, test/srt/ -> test/registered/.