Repository navigation
Worker: MSTAR_TP_ASYNC_SCHED on by default - #1075
Merged
Merged
Conversation
MSTAR_TP_ASYNC_SCHED and the serving config's tp_async_sched now default to on. It was off only because the path was new when #224 added it; the TP numbers quoted since have been taken with it set by hand. 0 / false still runs the parallel nodes serially. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 1, 2026
This was referenced Nov 1, 2026
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Turns async scheduling on by default for lockstep-parallel (TP / SP) nodes:
MSTAR_TP_ASYNC_SCHEDand the serving config'stp_async_schednow default to on.MSTAR_TP_ASYNC_SCHED=0(ortp_async_sched: false) still runs those nodes serially, and a node list still enables it for just those nodes.The default was
0only because the path was new when #224 added it (2026-09-11). Since then the TP benchmark numbers quoted in PRs have been taken with it set by hand, so the default is what has gone unmeasured, and a defaulttp2run doesn't exercise the path those numbers describe.The code change is one line, the fallback in
_tp_async_sched_setting(mstar/worker/worker.py). The rest updates the test that pins the default, plus the four places that documented0:docs/environment_variables.rst,docs/serving.rst, AGENTS.md (invariant 7's note) and the async-worker skill.Evidence
Qwen3.5-9B TP2 on 2 x H200, chunked prefill (#367's branch), two sessions per arm, interleaved, prefix cache off,
MSTAR_SHM_ARENA=1, Rust graph runtime. The 200- and 3000-token prompts are mixed 4:1, with 256 tokens out. Cells are session 1, session 2.MSTAR_TP_ASYNC_SCHED=0=1Greedy outputs on 24 prompts of 3k-30k tokens are identical between the two arms in one of the two sessions. In the other, the
0arm differs on one prompt from every other run, including its own second session. Neither arm had a hang or a failed request.How was it tested?
ruff check .: clean.python -m pytest test/modular test/distributedon CPU (OMP_NUM_THREADS=2), with and without the change: the same outcome per test, 218 failed, 8176 passed, 906 skipped, 18 errors on both. The failures come from the test machine's environment (torch 2.12 has no matching torchaudio build yet); none involve TP or the worker's scheduling.Checklist
ruff check .passesDeveloped with AI assistance (Claude).
🤖 Generated with Claude Code