Reduce docker size - #632
Merged
Merged
Conversation
hnyls2002
force-pushed
the
reduce-docker-size
branch
from
July 16, 2024 22:26
bdc15bd to
5c42f8b
Compare
Ying1123
approved these changes
Jul 16, 2024
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 8, 2026
… a2a — the mutual-deadlock root ROOT. The mesh and a2a kernels wait for an EQUALITY conjunction over all peers' flag lines — lines that are single per (topology,step,sender) and reused across collectives under a round counter that is GLOBAL across topologies. A rank entering its next same-topology collective overwrites a line a slow peer still awaits; the conjunction (short-circuit, must see ALL peers simultaneously) makes a lead of ONE round sufficient under per-path BAR1 propagation asymmetry. Flags only grow: the awaited equality never returns — permanent group deadlock, surfacing as the replay-abort-clean production class. EVIDENCE (farm 2026-08-08): sentinel FREEZE ring tails (all ranks frozen at one seq, source one op ahead); roundDev 612939 vs peer flag watermarks 612936; mid-wedge py-spy naming BOTH historical callchains — rank 0 at dcp/owner.py:566 (sgl-project#649) and ranks 1+2 at flashinfer_backend.py:7557 (sgl-project#622) — as prep-syncs stuck behind the spinning streams, which is why per-site fixes only relocated the wedge. Wedge MTTF 5-15 min amplified. FIX. Each rank acks its completed round (after its receive phase) into every peer's ack bank; a writer entering its next same-topology collective first waits — monotonic >=, deadline-guarded, distinct abort status 2 for attribution — until all peers acked its previous round. All device-side, capture/replay-safe (watermark and acks are device-resident, re-evaluated per replay). Ack banks appended at the flag-region end so every pre-existing offset stays byte-identical; _round_dev grows to 3 words (round, mesh watermark, a2a watermark). Ring is untouched with a proof comment: its 2(R-1) single-peer waits chain through every rank, so no rank can lead by a collective. Rejected alternatives documented in the strand log: parity flag banks (defeated by the global round counter via interleaved ring rounds) and monotonic flag waits (turn the deadlock into silent data corruption). NEW-EDGE AUDIT (in the kernel comment): the awaited ack is produced by a receive phase that depends only on the PREVIOUS round's flags, never on anything the writer does after it — the new edge closes no cycle; a dead peer hits the deadline and status 2, never a hang. FALSIFIERS, red-first: - test_barlink_ack_protocol_622.py: host-side discrete-event automaton of the flag/round/ack state machine; the OLD protocol deadlocks via reader-visible overshoot in the farm-proven one-round-lead scenario, the NEW protocol completes it; peer death yields named deadline aborts. 4 test functions, 0.05 s, no GPU. - test_barlink_ack_layout_622.py: 16 test functions pinning region arithmetic (banks strictly after all pre-existing lines incl. pipe, non-overlapping, +2*world*256 exactly) and kernel-source invariants (ack fields, >= entry wait, status 2, ring body ack-free). - On-card A/B pending: pre-fix wedge MTTF 5-15 min under amplified load; acceptance = zero FREEZE wedges at >= 8x that, with capture-safety proven via capture census + spaced SIGUSR1 round/ack monotonicity probes under replay load. Battery: 122 test items green across the 8 protocol/instrument files (60 s); full distributed suite 2670 passed / 1 pre-existing unrelated failure (live-process pgrep test); nvcc compile smoke of the CUDA source rc=0 (sm_86); ruff clean.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 8, 2026
…-async wedge family — both roots fixed Defect 1 (bcbe31d): base sampler never synced first sampled token ids across TP ranks; under mixed archs / uneven-TP reduction order the ranks read different tokens and batch membership diverges at EOS edges. Sync is now default-on for every tp>1 group (opt-out SGLANG_SYNC_SAMPLED_TOKENS=0). Defect 2 (b42405c, sgl-project#632): barlink bar1 mesh/a2a peer barrier was an equality spin on flag lines reused across collectives with one global round counter — a one-round lead overwrites a line a slow peer still awaits, the conjunction never assembles, mutual deadlock inside graph replay (the production replay-abort-clean wedge). Replaced by a consumption-ack barrier: each rank acks after its receive phase; writers wait monotonically for all peers' previous-round acks before re-entering the same topology. Device-side, capture/replay-safe; ring path untouched (2(R-1) single-peer waits, proof comment in-tree). Test results (documented in /spinning/622-farm/staging/MERGE_BLOCK_FINAL.md): - 22/22 falsifier tests green (sampler sync red-first 2, ack protocol automaton red-first 4, ack layout 16), re-run by the operator on the merge candidate. - 2h26m+ amplified-load soak on f2154ba: zero divergences, zero FREEZE, zero aborts, zero OOB (>=9.7x pre-fix wedge MTTF); load continuity proven per-minute; 9.2M ack-barrier rounds with strict cross-rank watermark growth through graph replay (capture-safety probes). - Cost: sampler sync +2.0% on 2048-token prefill chunks (wait-share); ack barrier within measurement noise; zero added collectives in spec decode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reduce the docker size and add the docker usage in readme