Skip to content

Retire the per-runner parallel record - #40343

Merged
ch-wan merged 7 commits into
mainfrom
cheng/refactor/retire-parallel-record
Sep 21, 2026
Merged

ch-wan merged 7 commits into
mainfrom
cheng/refactor/retire-parallel-record

Conversation

@ch-wan

@ch-wan ch-wan commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

ParallelState was a frozen per-process record carrying every parallel width
and rank. It existed because a rank could not be read before the process groups
were built, so scheduler components were handed a plain dataclass instead of
asking. Publishing the placement removed that reason, and the reads have been
moving to the context or onto the object that owns a placement ever since.

What was left was the record itself, threaded through twenty constructors so it
could reach readers that no longer exist.

Modifications

The last topology reads leave it. A speculative worker asks the target
runner it wraps, which froze its own placement at construction; the workers
around a runner ask that runner; bootstrap reads the context directly,
because the build it drives runs only on the target worker and no scope is open
over it. Sixty-three of these were reached through another object --
model_runner.ps.tp_size, self.scheduler.ps.gpu_id -- rather than off self.

gpu_id is declared on the namespace that owns devices. It was the only
field of the record that was never a topology: the parent picks the device --
reindexing narrows what the child can see before the spawn, and Ray allocates
from its own pool -- so the process cannot work it out. Declared on device as
a name the configuration implies rather than one it carries: an Arg is the
operator's input and reaches ServerArgs, and nobody types this one, so it
belongs with the ranks rather than sitting in the record as a flag with no CLI
that always reads None. It arrives the way the spawn ranks do, in the bundle
handed to publish, and is stated whatever it is -- None included, because
most roles run on no device and that is an answer rather than a name nobody
wrote. The record is the startup input and is never written to.

Four Scheduler constructor parameters go with it. The Ray actor and the
forked entry stop passing them in the same change that stops reading them, so
neither launch path is left handing over arguments the constructor no longer
takes.

The record retires. ParallelState and its module are gone, along with the
ps parameter in every signature that forwarded it and the doctored copy six
speculative workers made to tell the draft it has one pipeline stage -- the
scope that states the draft's topology has said that since it started stating
the rest.

A name in this module now has one address.

One read does not become an instance attribute. A draft's cuda-graph metadata
is sized by how many attention-DP replicas it gathers with -- the target's --
and the draft runner is built inside a scope that correctly reports one replica
of its own. Reading that off the runner would have shaped the buffers for a
deployment that does not exist, so those three graph runners ask for the
deployment's width by name. The NPU draft runner's graph/eager consensus reads
the same thing: all ranks have to reach the same decision, and skipping the
reduction because the draft has one replica is how they stop agreeing.

A census pins that: nothing under srt/ or the benchmark package assigns a
declared field of the record. A second one pins the other side -- no test
stands a parallel record into the object under test any more, because nothing
reads one. The resolution pipeline is exempt, because
building the record is its job.

Accuracy Tests

Not run for this revision. Each read now answers from the object that owns the
placement it described, or from the configuration the record was built from.

Speed Tests and Profiling

Not applicable: attribute reads replace attribute reads.

Checklist

Test doubles that stood a record into something follow: they state the topology
on the context instead, whole rather than a leaf at a time, because a width
that does not factor describes no layout.

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): 🚫 Run #35644792162
Latest PR Test (Extra): 🚫 Run #35644791967
Latest PR Test (AMD ROCm 10): 🚫 Run #35644792129

@ch-wan
ch-wan force-pushed the cheng/refactor/retire-parallel-record branch from 6f43792 to df01ee3 Compare September 21, 2026 19:26
@ch-wan
ch-wan merged commit 970e946 into main Sep 21, 2026
45 of 59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant