Skip to content

[Refactor] Drop placement values nothing reads - #41811

Merged
ch-wan merged 1 commit into
mainfrom
cheng/refactor/drop-unread-placement
Sep 30, 2026
Merged

ch-wan merged 1 commit into
mainfrom
cheng/refactor/drop-unread-placement

Conversation

@ch-wan

@ch-wan ch-wan commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

This PR is part of a stack (oldest at bottom):

Motivation

A number of classes store or pass parallel ranks, sizes and groups that nothing reads. Each one reads as if it mattered, and each is one more copy of the placement to keep consistent.

Modifications

Remove the unread values:

  • linear layers passed tp_size to load_merged_column_weight, which no parameter class reads; the W8A8 INT8 MoE method, the humming linear weight, the FlashMLA backend (dcp_rank), the DSA indexer (cp_size) and the Triton / Flash3 vision attention backends (tp_size) kept values with no reader;
  • VocabParallelEmbedding.get_sharded_to_full_mapping has no caller;
  • BaseRunner stored tp_size and attn_tp_rank; the decode and prefill graph runners re-stored attn_tp_size, attn_tp_rank and dp_size right after BaseRunner.__init__ set them; four draft graph runners copied model_runner.tp_size. None of these is read;
  • the n-gram worker kept an unread tp_rank;
  • SchedulerRequestReceiver carried attn_tp_group and attn_cp_group that it never reads (their CPU groups stay);
  • PrefillBootstrapQueue took and stored gloo_group and tp_size, and DecodePreallocQueue stored tp_size and dp_size, without reading them.

No read value changes.

Accuracy Tests

Not applicable: nothing that is read changes.

  • All 19 modified modules import.
  • test/registered/unit at this PR's head, compared with main: no new failures.

Speed Tests and Profiling

Not applicable.

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): 🚫 Run #36772686757
Latest PR Test (Extra): 🚫 Run #36772686777
Latest PR Test (AMD ROCm 10): 🚫 Run #36772686680

@ch-wan
ch-wan force-pushed the cheng/hot-fix/solar-model-construction branch from 3280536 to 4f20ec7 Compare September 30, 2026 04:24
@ch-wan
ch-wan force-pushed the cheng/refactor/drop-unread-placement branch from 52fc839 to 02fdd93 Compare September 30, 2026 04:24
@ch-wan
ch-wan force-pushed the cheng/hot-fix/solar-model-construction branch from 4f20ec7 to 117af72 Compare September 30, 2026 05:05
@ch-wan
ch-wan force-pushed the cheng/refactor/drop-unread-placement branch from 02fdd93 to 87b938b Compare September 30, 2026 05:05
@ch-wan
ch-wan force-pushed the cheng/hot-fix/solar-model-construction branch from 117af72 to 15cf773 Compare September 30, 2026 20:26
Base automatically changed from cheng/hot-fix/solar-model-construction to main September 30, 2026 20:26
Remove parallel ranks, sizes and groups that are stored or passed but
never read:

- linear layers passed `tp_size` to `load_merged_column_weight`, which no
  parameter class reads; the W8A8 INT8 MoE method, the humming linear
  weight, the FlashMLA backend (`dcp_rank`), the DSA indexer (`cp_size`)
  and the Triton / Flash3 vision attention backends (`tp_size`) kept
  values with no reader;
- `VocabParallelEmbedding.get_sharded_to_full_mapping` has no caller;
- `BaseRunner` stored `tp_size` and `attn_tp_rank`, the decode and
  prefill graph runners re-stored `attn_tp_size`, `attn_tp_rank` and
  `dp_size` right after `BaseRunner.__init__` set them, and four draft
  graph runners copied `model_runner.tp_size`; none of these is read;
- the n-gram worker kept an unread `tp_rank`;
- `SchedulerRequestReceiver` carried `attn_tp_group` and `attn_cp_group`
  that it never reads (their CPU groups stay);
- `PrefillBootstrapQueue` took and stored `gloo_group` and `tp_size`, and
  `DecodePreallocQueue` stored `tp_size` and `dp_size`, without reading
  them.

No read value changes.
@ch-wan
ch-wan force-pushed the cheng/refactor/drop-unread-placement branch from 87b938b to 80e6449 Compare September 30, 2026 20:26
@ch-wan
ch-wan merged commit b0cc246 into main Sep 30, 2026
@ch-wan
ch-wan deleted the cheng/refactor/drop-unread-placement branch September 30, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Multi-modal multi-modal language model

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant