feat: parse vLLM's native sampling_mask field

[mikasenghaas]

Summary

• Surface vLLM's native sampling_mask field on the generate result.
• Preserve one list of surviving token ids for each completion token.
• Remove the custom base64 kept_tokens response splice.

Consumed by PrimeIntellect-ai/verifiers#2460 and prime-rl's native sampling-replay PR.

Breaking

• Replace the renderer result key kept_tokens with sampling_mask.
• Update consumers to read vLLM's native list-of-lists payload.

Verification

• uv run pytest tests/test_client.py -q: 21 passed, 5 skipped.
• Ruff, format, and diff checks pass.

🤖 Generated with Claude Code

Note

Replace kept_tokens with vLLM's native sampling_mask field

• Updates build_trajectory_step in renderers/base.py to expose sampling_mask (set to None) instead of kept_tokens in the returned trajectory step dict.
• Removes KEPT_TOKENS_IDS_PREFIX and the stripping/decoding of kept_tokens.ids in parse_generate_response in renderers/client.py.
• Updates generate async function in renderers/client.py to read sampling_mask from the engine response instead of kept_tokens, matching vLLM's kept-set sampling mask.
• Behavioral Change: all downstream consumers that read kept_tokens from trajectory steps or generation results must switch to sampling_mask; the old key is no longer present.

^{Macroscope summarized a9dccfb.}

---

Note

Medium Risk
Breaking change to the generate/trajectory result schema; downstream verifiers and prime-rl must switch field names and payload shape, but logic is limited to optional metadata parsing.

Overview
Breaking: Renderer generate results and trajectory steps now expose sampling_mask instead of kept_tokens. Consumers must read vLLM’s native list-of-lists (one surviving vocab-id list per completion token when --return-sampling-mask is enabled).

In renderers/client.py, the custom kept_tokens.ids base64 splice (KEPT_TOKENS_IDS_PREFIX in parse_generate_response) is removed. generate takes choice.get("sampling_mask") and returns it on the result dict.

In renderers/base.py, build_trajectory_step initializes the step payload with sampling_mask: None instead of kept_tokens.

^{Reviewed by Cursor Bugbot for commit a9dccfb. Bugbot is set up for automated code reviews on this repo. Configure here.}