Skip to content

fix: handle abort finish reason from SGLang - #5566

Closed
vladnosiv wants to merge 2 commits into
ai-dynamo:mainfrom
vladnosiv:handle-abort-from-sglang
Closed

fix: handle abort finish reason from SGLang#5566
vladnosiv wants to merge 2 commits into
ai-dynamo:mainfrom
vladnosiv:handle-abort-from-sglang

Conversation

@vladnosiv

@vladnosiv vladnosiv commented Jan 22, 2026

Copy link
Copy Markdown
Contributor

Overview:

SGLang is able to throw errors from its insides through the Abort status:
https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/managers/schedule_batch.py#L157

Dynamo is currently unaware of this status and is sending it as-is, leading to error:

�[2m2026-01-22T10:26:21.018066Z�[0m �[33m WARN�[0m �[1mhttp-request�[0m: �[2mdynamo_runtime::pipeline::network::egress::addressed_router�[0m�[2m:�[0m 
Failed deserializing JSON to response �[3merr�[0m�[2m=�[0m
unknown variant `abort`, expected one of `eos`, `length`, `stop`, `error`, `cancelled`, `content_filter` at line 1 column
 40 �[3mjson_str�[0m�[2m=�[0m
{"data":{"data":{"finish_reason":"abort","token_ids":[],"completion_usage":{"prompt_tokens":4,"completion_tokens":0,"total_tokens":4,"prompt_tokens_details":null}}},"complete_final":false} 
�[2m�[3mmethod�[0m�[2m=�[0mPOST �[3muri�[0m�[2m=�[0m/v1/chat/completions �[3mversion�[0m�[2m=�[0mHTTP/1.1 �[3mx_request_id�[0m�[2m=�[0m"4926d6f4-2e32-42ae-8ca7-f1c094d493c2"�[0m

This PR handles this status correctly and passes the error message on.

Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
@vladnosiv
vladnosiv requested a review from a team as a code owner January 22, 2026 11:18
@vladnosiv
vladnosiv requested a review from a team January 22, 2026 11:18
@copy-pr-bot

copy-pr-bot Bot commented Jan 22, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi vladnosiv! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

@github-actions github-actions Bot added fix external-contribution Pull request is from an external contributor backend::sglang Relates to the sglang backend labels Jan 22, 2026
@vladnosiv

Copy link
Copy Markdown
Contributor Author

cc @ishandhanani

@vladnosiv

Copy link
Copy Markdown
Contributor Author

Fixed in #5659

@vladnosiv vladnosiv closed this Feb 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::sglang Relates to the sglang backend external-contribution Pull request is from an external contributor fix size/S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant