Repository navigation
[PD] Preserve bootstrap metadata in native Messages requests - #39711
Merged
hnyls2002 merged 4 commits intoSep 29, 2026
Merged
Conversation
chromecast56
force-pushed
the
codex/native-messages-pd-bootstrap
branch
from
September 22, 2026 20:58
8ab894a to
98baf0d
Compare
This was referenced Sep 22, 2026
chromecast56
marked this pull request as ready for review
September 22, 2026 21:37
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chromecast56
force-pushed
the
codex/native-messages-pd-bootstrap
branch
from
September 23, 2026 21:57
98baf0d to
3283ea2
Compare
Collaborator
|
/tag-and-rerun-ci |
Collaborator
|
/rerun-test test_anthropic_tool_use.py test_openai_server.py::TestOpenAIServer |
Contributor
|
Results for 🚀 🚀 |
gilfordting
added a commit
to gilfordting/sglang
that referenced
this pull request
Oct 14, 2026
AnthropicMessagesRequest already inherits routed_dp_rank from PDRoutingFields and forwards it to the converted ChatCompletionRequest (sgl-project#39711, sgl-project#41756). Add the deprecated data_parallel_rank alias with the shared migration so a body pin on /v1/messages accepts the same fields as /v1/chat/completions, and cover the conversion in the unit tests. The header already reached this route through OpenAIServingChat and keeps precedence. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related entrypoints PRs: #40077, #39716. Each is independent and they can merge in any order.
Motivation
Native
/v1/messagesrequests drop the PD rendezvous fields when they are converted toChatCompletionRequest, so a decode worker never receives the bootstrap metadata a PD router sends. smg now routes/v1/messagesthrough PD dual dispatch and injects those fields (smg-project/smg#2250); this is the worker-side change that makes that path work.Modifications
bootstrap_host,bootstrap_port,bootstrap_room,routed_dp_rank, anddisagg_prefill_dp_ranktoAnthropicMessagesRequest, with the same types asChatCompletionRequest._convert_to_chat_completion_request. The deprecateddata_parallel_rankalias is not added to the new API surface.Tests
test/registered/unit/entrypoints/anthropic/test_serving.py::test_messages_preserves_pd_rendezvous_through_native_conversionfails on main (None != 'prefill.internal') and passes with this change. The file passes (62 tests).CI States
Latest PR Test (Base): ⏳ Run #36625259014
Latest PR Test (Extra): 🚫 Run #36625258672
Latest PR Test (AMD ROCm 10): 🚫 Run #36625259130