Skip to content

Add docs for TP LMHead All2All optimization - #35280

Closed
SYChen123 wants to merge 23 commits into
sgl-project:mainfrom
SYChen123:lm-head-docs
Closed

SYChen123 wants to merge 23 commits into
sgl-project:mainfrom
SYChen123:lm-head-docs

Conversation

@SYChen123

@SYChen123 SYChen123 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Motivation

Stack on #32313. Add docs to explain the optimization and show the usecase.

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ⏳ Run #32110674468
Latest PR Test (Extra): ⏳ Run #32110674232

SYChen123 and others added 23 commits July 24, 2026 15:13
…ccl contract and add cudagraph smoke test to ensure that capture doesn't fail.
…ed across DP ranks when all2all path is enabeld
… memory to avoid OOM at runtime when tense mem-fraction-static is set
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 18, 2026
@SYChen123 SYChen123 closed this Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants