Skip to content

[Refactor] scheduler: Drop the dead Req.decoded_text and BatchTokenIDOutput.decoded_texts - #47249

Merged
hnyls2002 merged 2 commits into
mainfrom
lsyin/drop-dead-decoded-text
Oct 29, 2026
Merged

hnyls2002 merged 2 commits into
mainfrom
lsyin/drop-dead-decoded-text

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Req.decoded_text is set to "" at construction and never assigned again, so every batch the scheduler sends to the detokenizer carries a column of empty strings.

  • Remove Req.decoded_text, the decoded_texts field of BatchTokenIDOutput, and the code that fills, copies, and reads it. The detokenizer starts each request's text at "" directly.
  • In _check_str_based_finish, stop_str in self.decoded_text was always False, so drop it together with the inner condition that was then always True.

No behavior change: the removed values were constant, and the rewritten stop-string loop was checked against the old one on every stop list of up to two strings and every tail drawn from a small alphabet (including empty stop strings). Tests that built or read the field are updated; the test that set decoded_text by hand to reach the unreachable branch is removed.

Verification

  • CPU unit tests under test/registered/unit/managers/ plus beam_search/test_output_decode.py and sampling/test_sampling_params.py, one process per file: 1050 passed before, 1049 after (the removed test); the one failure is the same on both heads and unrelated.

CI States

Latest PR Test (Base): 🚫 Run #40955057028
Latest PR Test (Extra): ❌ Run #40955056883
Latest PR Test (AMD ROCm 10): 🚫 Run #40955056894

@hnyls2002
hnyls2002 requested a review from merrymercy as a code owner October 29, 2026 10:06

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_flat_raw_top_logprobs.py test_multi_tokenizer_mixin.py test_output_streamer_customized_info.py test_output_streamer_logprobs.py test_stop_str_speculative.py test_trim_matched_stop.py test_skip_tokenizer_init.py test_multi_tokenizer.py test_basic_sanity.py test_srt_endpoint.py registered/openai_server/validation/test_matched_stop.py

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

🚀 ubuntu-latest (6 tests): ✅ View workflow run

cd test/ && python3 registered/unit/managers/test_flat_raw_top_logprobs.py
cd test/ && python3 registered/unit/managers/test_multi_tokenizer_mixin.py
cd test/ && python3 registered/unit/managers/test_output_streamer_customized_info.py
cd test/ && python3 registered/unit/managers/test_output_streamer_logprobs.py
cd test/ && python3 registered/unit/managers/test_stop_str_speculative.py
cd test/ && python3 registered/unit/managers/test_trim_matched_stop.py

🚀 1-gpu-5090 (4 tests): ✅ View workflow run

cd test/ && python3 registered/tokenizer/test_skip_tokenizer_init.py
cd test/ && python3 registered/core/test_basic_sanity.py
cd test/ && python3 registered/core/test_srt_endpoint.py
cd test/ && python3 registered/openai_server/validation/test_matched_stop.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/tokenizer/test_multi_tokenizer.py

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Oct 29, 2026
@hnyls2002
hnyls2002 merged commit dfdee25 into main Oct 29, 2026
82 of 119 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/drop-dead-decoded-text branch October 29, 2026 21:09
Yuxingwang-intel pushed a commit to Yuxingwang-intel/sglang that referenced this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR speculative-decoding

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant