Skip to content

[RL] update weight when pp_size > 1 - #24655

Open
xiuhu17 wants to merge 2 commits into
sgl-project:mainfrom
xiuhu17:1/2--upstream_from_sglang_miles
Open

[RL] update weight when pp_size > 1 #24655
xiuhu17 wants to merge 2 commits into
sgl-project:mainfrom
xiuhu17:1/2--upstream_from_sglang_miles

Conversation

@xiuhu17

@xiuhu17 xiuhu17 commented May 8, 2026

Copy link
Copy Markdown
Contributor

This PR fixes SGLang online weight update when pp_size > 1.

Problem

Weight update previously indexed payloads and distributed ranks only by tp_rank.
With pipeline parallelism, different PP stages could use the wrong tensor payload or join update collectives with incorrect ranks.

Fix

Use PP-aware rank mapping:

rank = tp_size * pp_rank + tp_rank

Changes

  • Serialize tensor payloads for tp_size * pp_size ranks.
  • Read update payloads with tp_size * pp_rank + tp_rank.
  • Initialize weight update process groups with PP-aware rank.
  • Forward weight update requests across PP stages before processing them locally.

Covered Requests

  UpdateWeightsFromDistributedReqInput
  UpdateWeightsFromTensorReqInput
  UpdateWeightsFromIPCReqInput

CI States

Latest PR Test (Base): ❌ Run #26910932808
Latest PR Test (Extra): ❌ Run #26910931808

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@xiuhu17 xiuhu17 changed the title [wip] update weight for pp > 1 [wip] update weight when pp_size > 1 May 8, 2026
@xiuhu17 xiuhu17 changed the title [wip] update weight when pp_size > 1 [WIP] update weight when pp_size > 1 May 8, 2026
@xiuhu17 xiuhu17 changed the title [WIP] update weight when pp_size > 1 update weight when pp_size > 1 May 12, 2026
@xiuhu17
xiuhu17 marked this pull request as ready for review May 12, 2026 19:12
@xiuhu17
xiuhu17 requested a review from xiezhq-hermann as a code owner May 12, 2026 19:32
@xiuhu17 xiuhu17 changed the title update weight when pp_size > 1 [RL] update weight when pp_size > 1 May 24, 2026
@ShangmingCai ShangmingCai self-assigned this May 26, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will test this change tomorrow. Others look good.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing!

Signed-off-by: zhihaow6 <zhihaow6@illinois.edu>
@xiuhu17
xiuhu17 force-pushed the 1/2--upstream_from_sglang_miles branch from 47b83e7 to 1eb6cf1 Compare June 3, 2026 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants