Reasoning parser by xihuai18 · Pull Request #4000 · sgl-project/sglang

xihuai18 · 2025-03-02T17:26:16Z

Based on #3202 and #3859, I fixed some bugs, simplified the implementations and added new features.

Only support --reasoning-parser server argument now. If users want to separate reasoning content, they must specify this option. The parser is chosen only based on this option, but not the model name.
separate_reasoning and stream_reasoning are set as True by default.
Add support for SGLang native api and offline engine.
Add and .ipynb docs.
Refactor and fix some bugs.

Motivation

Modifications

Checklist

Format your code according to the Code Formatting with Pre-Commit.
Add unit tests as outlined in the Running Unit Tests.
Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

…esponse behavior.

…model, also handle first response while separating reasoning.

…ive just in case.

… unittests.

…ust model references

python/sglang/srt/server_args.py

shuaills

Great Job!

python/sglang/srt/entrypoints/http_server.py

docs/backend/separate_reasoning.ipynb

python/sglang/srt/openai_api/adapter.py

python/sglang/srt/reasoning_parser.py

test/srt/test_reasoning_content.py

tot0 · 2025-03-02T23:59:53Z

So is the consensus to not include this change? be03171

I'd like to understand the reason why if so.

zhaochenyang20 · 2025-03-03T01:40:27Z

So is the consensus to not include this change? be03171

I'd like to understand the reason why if so.

cc @shuaills @XiaotongJiang

xihuai18 · 2025-03-03T06:38:33Z

Do we need more things to merge this PR ?

zhaochenyang20 · 2025-03-03T08:24:18Z

cc @zhyncs could you take a look?

Co-authored-by: Lucas Pickup <lupickup@microsoft.com>

Lucas Pickup and others added 30 commits February 6, 2025 16:08

Support reasoning_content in ChatCompletion choices like DeepSeek api.

ad623a0

Fix up silly mistakes in non-streaming path

1fe6654

Flip accumulate_reasoning to stream_reasoning to match the api changes.

0a0eaad

Ensure finish_reason is null by default to match OpenAI streaming r…

9cc7b76

…esponse behavior.

fix silly python tuple mistake.

6400800

Don't send streaming chunks for empty content.

d856124

Merge branch 'main' into lupickup/deepseek/reasoning_content

414d467

Adapt reasoning_parser to handle <think> token not being produced by …

99f2583

…model, also handle first response while separating reasoning.

Fix silly typo

f39a256

Fix up <think> token stripping.

132e5d6

use split correctly

4a60111

Remove unused reasoning_regex.

9ec06b1

wow i really can't read, or it's late, or both.

22a0b61

Fix another case.

cde50fc

parse_result.normal_text _shouldn't_ ever be None, but lets be defens…

7330b0b

…ive just in case.

Merge branch 'main' into lupickup/deepseek/reasoning_content

aa63f5d

Make content=None if iparse_results.normal_text returns an empty string

3fdce0d

Merge branch 'main' into lupickup/deepseek/reasoning_content

cf9d440

Merge branch 'main' into lupickup/deepseek/reasoning_content

de7618b

Run pre-commit hook to format changes.

1f7daae

Merge branch 'main' into lupickup/deepseek/reasoning_content

287be31

Merge branch 'main' into lupickup/deepseek/reasoning_content

210fbdc

Merge in awesome docs from sgl-project#3859 by @ShaoZhang0115 and add…

e165bf7

… unittests.

Adding missing format string.

5a89225

Remove local testing hacks.

fa85c96

Merge branch 'main' into lupickup/deepseek/reasoning_content

5309df7

Move reasoning_parser.md to docs/references

55acaaa

Fixup incorrect handling of request: list

2ae4fa4

[Refactor] Update reasoning handling in ChatCompletionRequest and adj…

94bee72

…ust model references

revert dockerfile changes

411473b

fix(reasoning content): 🐛 convert keys() to list

0cc54df

tot0 reviewed Mar 2, 2025

View reviewed changes

python/sglang/srt/server_args.py Show resolved Hide resolved

Merge branch 'main' into reasoning-parser

0d49daf

shuaills suggested changes Mar 2, 2025

View reviewed changes

python/sglang/srt/entrypoints/http_server.py Show resolved Hide resolved

docs/backend/separate_reasoning.ipynb Outdated Show resolved Hide resolved

XiaotongJiang reviewed Mar 2, 2025

View reviewed changes

python/sglang/srt/openai_api/adapter.py Outdated Show resolved Hide resolved

python/sglang/srt/reasoning_parser.py Show resolved Hide resolved

test/srt/test_reasoning_content.py Show resolved Hide resolved

zhaochenyang20 and others added 2 commits March 2, 2025 17:40

Merge branch 'main' into reasoning-parser

89630c5

Merge branch 'main' into reasoning-parser

8caf280

xihuai18 mentioned this pull request Mar 3, 2025

Support for reasoning_content in API #3202

Closed

4 tasks

xihuai18 added 3 commits March 3, 2025 13:03

chore(reasoning content): ✏️ fix some comments

7d38608

chore(reasoning content): change default value in reasoning_parser

258ad98

Merge branch 'main' into reasoning-parser

b28ae29

xihuai18 added 2 commits March 3, 2025 15:50

Merge branch 'main' into reasoning-parser

a661a00

Merge branch 'main' into reasoning-parser

1765302

xihuai18 and others added 2 commits March 3, 2025 19:29

Merge branch 'main' into reasoning-parser

3ed1223

Merge branch 'main' into reasoning-parser

1b0e0cf

tot0 approved these changes Mar 3, 2025

View reviewed changes

This was referenced Mar 3, 2025

Feat/support code completion #3612

Merged

[Feature] Refactor all parser features #4036

Closed

zhyncs mentioned this pull request Mar 3, 2025

chore: bump v0.4.4 #4041

Merged

12 tasks

zhyncs and others added 3 commits March 3, 2025 14:21

Merge branch 'main' into reasoning-parser

e8433df

Merge branch 'sgl-project:main' into reasoning-parser

894f096

Merge branch 'main' into reasoning-parser

0e5e33b

zhaochenyang20 merged commit 95575aa into sgl-project:main Mar 4, 2025

lambert0312 mentioned this pull request Mar 9, 2025

How to return reasoning_content from sglang server response? #3428

Closed

aoshen524 pushed a commit to aoshen524/sglang that referenced this pull request Mar 10, 2025

Reasoning parser (sgl-project#4000)

01eb2fa

Co-authored-by: Lucas Pickup <lupickup@microsoft.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reasoning parser#4000

Reasoning parser#4000
zhaochenyang20 merged 63 commits intosgl-project:mainfrom
xihuai18:reasoning-parser

xihuai18 commented Mar 2, 2025 •

edited

Loading

Uh oh!

Uh oh!

shuaills left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tot0 commented Mar 2, 2025

Uh oh!

zhaochenyang20 commented Mar 3, 2025

Uh oh!

xihuai18 commented Mar 3, 2025

Uh oh!

zhaochenyang20 commented Mar 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

xihuai18 commented Mar 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Checklist

Uh oh!

Uh oh!

shuaills left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tot0 commented Mar 2, 2025

Uh oh!

zhaochenyang20 commented Mar 3, 2025

Uh oh!

xihuai18 commented Mar 3, 2025

Uh oh!

zhaochenyang20 commented Mar 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

xihuai18 commented Mar 2, 2025 •

edited

Loading