Skip to content

[Refactor] Share PD routing fields across request models - #41756

Merged
hnyls2002 merged 1 commit into
mainfrom
lsyin/pd-routing-fields
Sep 29, 2026
Merged

[Refactor] Share PD routing fields across request models#41756
hnyls2002 merged 1 commit into
mainfrom
lsyin/pd-routing-fields

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Sep 29, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Collaborator

Define the PD/DP routing fields (bootstrap_host/port/room, routed_dp_rank, disagg_prefill_dp_rank) once in a PDRoutingFields base shared by the Completions, Chat Completions, Responses, and Anthropic Messages requests, and forward them in the Messages conversion with pd_routing_kwargs(), so a new entrypoint cannot drop them (follows #39711).

CI States

Latest PR Test (Base): 🚫 Run #36628579787
Latest PR Test (Extra): ❌ Run #36628579736
Latest PR Test (AMD ROCm 10): 🚫 Run #36628579712

@hnyls2002

hnyls2002 commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@hnyls2002

hnyls2002 commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test registered/unit/entrypoints/anthropic/test_serving.py test_serving_chat.py test_serving_responses.py test_protocol.py test_openai_server.py::TestOpenAIServer test_openai_server.py::TestOpenAIServerv1Responses test_anthropic_tool_use.py registered/disaggregation/test_disaggregation_basic.py::TestDisaggregationAccuracy

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test registered/unit/entrypoints/anthropic/test_serving.py test_serving_chat.py test_serving_responses.py test_protocol.py test_openai_server.py::TestOpenAIServer test_openai_server.py::TestOpenAIServerv1Responses test_anthropic_tool_use.py registered/disaggregation/test_disaggregation_basic.py::TestDisaggregationAccuracy:

🚀 ubuntu-latest (4 tests): ✅ View workflow run 

cd test/ && python3 registered/unit/entrypoints/anthropic/test_serving.py
cd test/ && python3 registered/unit/entrypoints/openai/test_serving_chat.py
cd test/ && python3 registered/unit/entrypoints/openai/test_serving_responses.py
cd test/ && python3 registered/unit/entrypoints/openai/test_protocol.py

🚀 1-gpu-5090 (2 tests): ✅ View workflow run 

cd test/ && python3 registered/openai_server/basic/test_openai_server.py TestOpenAIServer
cd test/ && python3 registered/openai_server/basic/test_openai_server.py TestOpenAIServerv1Responses

🚀 1-gpu-h100 (1 test): ✅ View workflow run 

cd test/ && python3 registered/openai_server/function_call/test_anthropic_tool_use.py

🚀 2-gpu-h100 (1 test): ✅ View workflow run 

cd test/ && python3 registered/disaggregation/test_disaggregation_basic.py TestDisaggregationAccuracy

@hnyls2002
hnyls2002 merged commit 211b1d9 into main Sep 29, 2026
170 of 222 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/pd-routing-fields branch September 29, 2026 22:11
@Kaif10 Kaif10 mentioned this pull request Oct 8, 2026
Open
3 of 5 tasks
gilfordting added a commit to gilfordting/sglang that referenced this pull request Oct 14, 2026
@gilfordting @claude
Accept the deprecated data_parallel_rank alias on /v1/messages
708f00b 
AnthropicMessagesRequest already inherits routed_dp_rank from
PDRoutingFields and forwards it to the converted ChatCompletionRequest
(sgl-project#39711, sgl-project#41756). Add the deprecated data_parallel_rank alias with the
shared migration so a body pin on /v1/messages accepts the same fields
as /v1/chat/completions, and cover the conversion in the unit tests.
The header already reached this route through OpenAIServingChat and
keeps precedence.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JustinTong0323 JustinTong0323 Awaiting requested review from JustinTong0323 JustinTong0323 is a code owner

Assignees

No one assigned

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hnyls2002