Repository navigation
Retire the per-runner parallel record - #40343
Merged
Merged
Conversation
ch-wan
requested review from
BBuf,
ByronHsu,
Duyi-Wang,
Edwardf0t1,
Fridge003,
HaiShaw,
Qiaolin-Yu,
ShangmingCai,
Ying1123,
hebiao064,
hnyls2002,
iforgetmyname,
ispobock,
kpham-sgl,
liusy58,
merrymercy,
ping1jing2,
pyc96,
sogalin,
whybeyoung,
xiezhq-hermann,
yeahdongcn and
yizhang2077
as code owners
September 19, 2026 11:06
ch-wan
force-pushed
the
cheng/refactor/deprecate-parallel-getters
branch
from
September 19, 2026 20:30
3c2d771 to
4465d49
Compare
ch-wan
force-pushed
the
cheng/refactor/retire-parallel-record
branch
from
September 21, 2026 19:26
6f43792 to
df01ee3
Compare
4 of 5 tasks
This was referenced Sep 22, 2026
This was referenced Sep 24, 2026
4 of 5 tasks
This was referenced Sep 30, 2026
5 tasks done
This was referenced Oct 2, 2026
8 tasks
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
ParallelStatewas a frozen per-process record carrying every parallel widthand rank. It existed because a rank could not be read before the process groups
were built, so scheduler components were handed a plain dataclass instead of
asking. Publishing the placement removed that reason, and the reads have been
moving to the context or onto the object that owns a placement ever since.
What was left was the record itself, threaded through twenty constructors so it
could reach readers that no longer exist.
Modifications
The last topology reads leave it. A speculative worker asks the target
runner it wraps, which froze its own placement at construction; the workers
around a runner ask that runner;
bootstrapreads the context directly,because the build it drives runs only on the target worker and no scope is open
over it. Sixty-three of these were reached through another object --
model_runner.ps.tp_size,self.scheduler.ps.gpu_id-- rather than offself.gpu_idis declared on the namespace that owns devices. It was the onlyfield of the record that was never a topology: the parent picks the device --
reindexing narrows what the child can see before the spawn, and Ray allocates
from its own pool -- so the process cannot work it out. Declared on
deviceasa name the configuration implies rather than one it carries: an
Argis theoperator's input and reaches
ServerArgs, and nobody types this one, so itbelongs with the ranks rather than sitting in the record as a flag with no CLI
that always reads
None. It arrives the way the spawn ranks do, in the bundlehanded to
publish, and is stated whatever it is --Noneincluded, becausemost roles run on no device and that is an answer rather than a name nobody
wrote. The record is the startup input and is never written to.
Four
Schedulerconstructor parameters go with it. The Ray actor and theforked entry stop passing them in the same change that stops reading them, so
neither launch path is left handing over arguments the constructor no longer
takes.
The record retires.
ParallelStateand its module are gone, along with thepsparameter in every signature that forwarded it and the doctored copy sixspeculative workers made to tell the draft it has one pipeline stage -- the
scope that states the draft's topology has said that since it started stating
the rest.
A name in this module now has one address.
One read does not become an instance attribute. A draft's cuda-graph metadata
is sized by how many attention-DP replicas it gathers with -- the target's --
and the draft runner is built inside a scope that correctly reports one replica
of its own. Reading that off the runner would have shaped the buffers for a
deployment that does not exist, so those three graph runners ask for the
deployment's width by name. The NPU draft runner's graph/eager consensus reads
the same thing: all ranks have to reach the same decision, and skipping the
reduction because the draft has one replica is how they stop agreeing.
A census pins that: nothing under
srt/or the benchmark package assigns adeclared field of the record. A second one pins the other side -- no test
stands a parallel record into the object under test any more, because nothing
reads one. The resolution pipeline is exempt, because
building the record is its job.
Accuracy Tests
Not run for this revision. Each read now answers from the object that owns the
placement it described, or from the configuration the record was built from.
Speed Tests and Profiling
Not applicable: attribute reads replace attribute reads.
Checklist
Test doubles that stood a record into something follow: they state the topology
on the context instead, whole rather than a leaf at a time, because a width
that does not factor describes no layout.
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): 🚫 Run #35644792162
Latest PR Test (Extra): 🚫 Run #35644791967
Latest PR Test (AMD ROCm 10): 🚫 Run #35644792129