Repository navigation
[Refactor] scheduler: Drop the dead Req.decoded_text and BatchTokenIDOutput.decoded_texts - #47249
Conversation
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
|
/tag-and-rerun-ci |
|
/rerun-test test_flat_raw_top_logprobs.py test_multi_tokenizer_mixin.py test_output_streamer_customized_info.py test_output_streamer_logprobs.py test_stop_str_speculative.py test_trim_matched_stop.py test_skip_tokenizer_init.py test_multi_tokenizer.py test_basic_sanity.py test_srt_endpoint.py registered/openai_server/validation/test_matched_stop.py |
|
🚀 🚀 🚀 |
…t that only measured it
…Output.decoded_texts (sgl-project#47249)
Req.decoded_textis set to""at construction and never assigned again, so every batch the scheduler sends to the detokenizer carries a column of empty strings.Req.decoded_text, thedecoded_textsfield ofBatchTokenIDOutput, and the code that fills, copies, and reads it. The detokenizer starts each request's text at""directly._check_str_based_finish,stop_str in self.decoded_textwas always False, so drop it together with the inner condition that was then always True.No behavior change: the removed values were constant, and the rewritten stop-string loop was checked against the old one on every stop list of up to two strings and every tail drawn from a small alphabet (including empty stop strings). Tests that built or read the field are updated; the test that set
decoded_textby hand to reach the unreachable branch is removed.Verification
test/registered/unit/managers/plusbeam_search/test_output_decode.pyandsampling/test_sampling_params.py, one process per file: 1050 passed before, 1049 after (the removed test); the one failure is the same on both heads and unrelated.CI States
Latest PR Test (Base): 🚫 Run #40955057028
Latest PR Test (Extra): ❌ Run #40955056883
Latest PR Test (AMD ROCm 10): 🚫 Run #40955056894