fix: add release tag workflow by zhyncs · Pull Request #754 · sgl-project/sglang

zhyncs · 2024-07-26T19:48:18Z

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
Modify documentation as needed, such as docstrings or example tutorials.

* Update pr-test-npu-debug.yml * Update run_suite.py * Update test_ascend_gpu_id.py * Update run_suite.py * Update test_ascend_gpu_id.py

…ded in (same resolution seam) Item 2. sgl-project#735 delivered the desk half -- crossing_schedule says which crossings exist, route_schedule binds each to a transport -- and moved no tensor. This is the forward wire that consumes both. WHY IT IS A MODULE AND NOT INLINE. The model's loop is boot-only territory: a wire written inside it can only be tested by booting, and F4-r5 owns the cards. pp_crossing_wire.py is the same logic as a small object with an injectable link, so the crossing algebra, the ordering, the counting and the byte-identity of the contiguous path are all hermetic. qwen3_5.py gains two call sites and no logic. BYTE-IDENTITY IS A NULL OBJECT, NOT A BRANCH. Without a layer set, or with the wire flag off, build_wire_for_model returns NoCrossingWire whose hooks return their arguments unchanged. The loop calls the same two methods either way, so the default path cannot drift from the wired one by acquiring a branch only one of them takes. ONE MECHANISM, NOT TWO INTERLEAVED. My first test asserted a contiguous set yields the null wire; that was wrong and the module was right. A contiguous 3-stage split still has 2 crossings -- the stage boundaries the old pp_proxy_tensors return carried. The wire takes ALL crossings rather than splitting the job with the old path: two mechanisms exchanging activations in the same loop is how they drift apart, and the boundary case is the one the old path already got right, which makes it the cheapest place to prove the wire equals it. Test corrected, module unchanged. THE OBSERVABLE. log_routing() had ZERO callers, so "31 crossings observed" was a claim no run could evidence. build_crossing_wire calls route_schedule + log_routing when given a peer map, and a test asserts the "31 crossings" line appears. HOST-STAGED BY THE RIG, NOT BY CHOICE. PpGroupLink rides the PP group's tensor-dict path; can_access_peer is false on all six directed pairs, so every crossing is GPU->host->GPU. That is the path sgl-project#735 priced at 24.68 ms/pass over 31 crossings, so wire and cost model agree by construction. sgl-project#754 FOLDED IN -- it is the SAME resolution seam, verified by reproducing it: get_pp_layer_set read the process-wide env regardless of pp_size, and the TP stack re-reads it during a phase flip with pp_size=1, where a 3-stage string is not merely inapplicable but invalid ("3 stage(s) given but pp_size is 1"). A flip with nothing to do with layer sets died on it. A single stage owns every layer, so the set form has nothing to express: pp_size <= 1 now answers None and hands the caller back to get_pp_indices -- the correct contiguous answer rather than a suppressed error. Pinned with a can-fail guard that the real pp_size still resolves AND that a genuinely wrong stage count (pp_size=2) is still refused, so the exit is not a blanket amnesty. TEST RESULTS test_pp_crossing_wire_753.py: 17 green. The load-bearing one is the BYTE GATE -- a full simulated forward driven through the wire must equal the same layers computed straight through, exactly, for BOTH the contiguous split and the user's gapped target layout. The layer function is order-sensitive on purpose (h = h*3 + id), so a skipped, duplicated or reordered crossing changes the number instead of looking close. A can-fail test drops one crossing and requires the result to diverge. Also pinned: crossing count == schedule length (31 for the target layout, the figure sgl-project#735 priced); per-rank sends/receives match the schedule; every failure path refuses BY NAME rather than continuing (a failed recv, a None payload, a failed send); the model entry point returns the null object without the flag. test_pp_layer_set_contiguity_753.py: 10 green (7 gate + 3 for sgl-project#754). Full distributed suite: 27 failed / 2983 passed, compared PER SUITE against the pre-753 baseline -- ZERO new failures. sgl-project#749 leak guard silent. One baseline failure absent (test_bar1_host_cleanup's pgrep self-match, environment sensitive) and NOT claimed as a fix. AST-verified that both hooks sit inside the owned_layer_ids loop rather than merely appearing in the file. Ruff: pp_crossing_wire.py clean. qwen3_5.py reports one F401 (triton), verified PRE-EXISTING against HEAD's blob. Codespell clean. Hermetic throughout, CUDA_VISIBLE_DEVICES=99; no cards touched, no boot.

…gated, resumable machine scripts/merge_train_pass3.py executes LEDGER_merge_train_0818 (c) mechanically, with the map's own disciplines enforced in code: - GATE: refuses without the comp4-green marker (/spinning/evidence-665-f1/COMP4_ACCEPTED) -- pass 3 stays boot-proof-gated per the ledger's rule; the marker path is a CLI parameter so the refusal is testable. - STANDING RULE first: re-derives the (a) lineage facts against the ACTUAL tip (contains review tip + composite, every step ref present, already-contained refs auto-skip) and aborts if the world moved. - Mapped resolutions ONLY: 'ours' (the sgl-project#754 fixture hunk drop), 'theirs' + load-bearing marker verification (the sgl-project#735 doc numbers 32607/4577), 'manual' (resumable stop with instructions, the sgl-project#727 gate hunks). An UNPREDICTED conflict aborts loudly WITHOUT resolving -- a stale map stops the train, it is never improvised around. - Stop-on-first-failure with per-step targeted suites and a resumable JSON state file; an all-ours pick that becomes empty is skipped as the mapped outcome. Nothing in the script pushes (pinned by test). SMOKED per the desk-written-never-executed rule, and the smoke EARNED ITS KEEP: the --dry-run scratch-clone execution of the real plan caught a ledger error -- d11b29d (sgl-project#740 s5a) is only a diff and conflicts without the original note commit 0480f4b underneath; the executor aborted on exactly that unpredicted conflict. Plan and ledger corrected to the sgl-project#740 PAIR; the re-run completes all 10 steps clean in the scratch. Bonus finding recorded in the ledger: the sgl-project#727 'manual' gate-hunk predictions did NOT fire -- those picks apply cleanly on the comp4 tip. Unit suite (hermetic, synthetic mini-repos): 12 passed -- gate refusal (helper + CLI end-to-end exit 2), unpredicted-conflict abort leaving the conflict for the operator, ours/theirs resolutions, marker-loss abort, manual resumable stop, plan-mirrors-ledger pins, state roundtrip. ruff and codespell clean. The real pass was NOT run; COMP4_ACCEPTED does not exist and no real branch moved.

…gl-project#754 retires into the 753 fold, 13-step plan re-smoked clean The first map froze before today's second wave; new authority is F4-r5's harvest composite 59ce2d8 (declared COMPLETE, review tip still an ancestor). Pass 3 redefined: harvest tip = train base, unabsorbed branches cherry-pick on top, feat/753 lands FIRST by its owner (10 in-flight commits; carries sgl-project#749; folds sgl-project#754 at the same seam -- distributed/utils.py:1709, its own pp_size=1 handling). Sweep results, same git-cherry/merge-base rigor as the first map, outputs quoted in the ledger REFRESH section: - ABSORBED by ancestry: comp4 and its whole lineage, 915ce1b (F4-r5's own sgl-project#757), 57b04b2 (sgl-project#540 fix). - ABSORBED as different commits (desk-sgl-project#752 hazard class, never merge): fix/748-armed-gate-scope, fix/759-arming-economy, feat/755-slot-reorder. The sgl-project#758 emitters need no branch -- the harvest TIP ITSELF is a sgl-project#758 commit. - SUPERSEDED: my own fix/754 -- semantic-not-patch folded by 753 (git cherry vs 082293f shows '+'); merging it after 753 lands guarantees a get_pp_layer_set conflict with zero gain. Retired from the plan without regret. - REVIEW-never-merge: 9e56477 (independent sgl-project#757), per its reviewer's own in-composite note naming 915ce1b as the baseline. - fix/706-remainder not yet visible; slot reserved. Executor updated: HARVEST constant joins the lineage check, DEFAULT_TIP moves to the harvest tip, the sgl-project#754 step is replaced by the sgl-project#745 reachability pick, and the second-wave picks join (727 head-chain, sgl-project#738 verdict, sgl-project#535 tickets). Dry-run scratch smoke against the REAL harvest tip: all 13 steps complete with ZERO conflicts (exit 0) -- cleaner than the first wave; the sgl-project#740 pair ordering from the previous smoke holds. Plan-mirror test updated (12 passed). Gate unchanged: COMP4_ACCEPTED still required, nothing pushes, nothing booted.

…ENT, loud review item -- and the harvest tip moved to da81871 The double-sgl-project#754 template does NOT apply. Both sgl-project#757 fixes attack the same root (corpse-S drain disabled, rank-local disarm routes) at DIFFERENT intervention points: 915ce1b (in harvest, regression- verified) drains the leftover at DISARM; 9e56477 (Slot-3) re-enables the ARMED drain with a pure 4-way demultiplex classifier and states a liveness property the disarm-time form lacks (the upstream's blocking commit waits on the wire being consumed AT ARRIVAL). Review question stated sharply in the ledger: does a long armed window stall the group under the disarm-time form? If yes, the two are COMPLEMENTARY halves, not duplicates. Slot-3's test suite (3-process gloo repro, 4 killed mutants, sgl-project#631-pin correction) binds to their classifier, so the tests ride the review verdict -- not cherry-pickable bare. 9e56477 stays OUT of the executor plan; the review joins feat/753 as a NAMED precondition of pass 3. Tip correction folded in: authority is now da81871 (contains the - sgl-project#758 phase-tag 59ce2d8 AND caca352 -- F4-r5 absorbed WINDOW_LADDER_0818 + its runner into the harvest). Executor HARVEST constant bumped; dry-run scratch smoke re-run against the REAL new tip: all 13 steps complete, zero conflicts, exit 0; plan-mirror tests 12 passed.

…- fix/753-on-harvest@8f2094f62b The blocker row flips from ABSENT to PREPARED: all three sgl-project#753 commits cherry-picked clean onto composite tip b7e6a41 (zero conflicts; the - sgl-project#754 seam had no competing resolution, exactly as the map adjudicated), plus a root-caused fix for the picked gloo suite's order-dependent flake -- a collection-time foreign-worktree sys.path prepend in test_prefill_graph_barlink.py poisoned every later spawn child; removed, and the gloo test hardened with a module-top __file__-derived pin. Distributed selection 105 passed / 0 failed vs the tip's 75/0 baseline; byte-gates, the 31-crossing pin, and the 3-process gloo suite green. F4-r5 merges one branch; nothing was merged into their composite from here.

…roup attribute current_stage_layer_set() read getattr(get_pp_group(), 'num_hidden_layers', None), but no site in the tree ever stamps that attribute onto the group object -- so on every live process it returned None and stage_owned_layer_ids degraded to the interval test. The v7 gapped boot (2026-08-18 17:30Z) showed the consequence: configurator cell_size=0 on PP0, pool still sized for the 15 span layers, KvVmmArena reserved=22.6 GiB, cuMemCreate OOM at 49s. The desk tests all passed owned= explicitly and never walked the lookup. Fix: when SGLANG_PP_LAYER_SET is present and the attribute is absent, derive the layer count from the set string itself -- parse_pp_layer_sets enforces full cover of [0, N), so N == max(layer) + 1 exactly. No env means None without touching the group (contiguous path byte-identical); pp world size 1 still answers None (sgl-project#754); a stamped attribute still wins. Tests: test_stage_layer_set_live_resolution_753.py, red-first proven (7 failed on the unfixed tree), plus the existing arena sizing suite -- 28 passed.

…at outlive their phase TWO HOLES, ONE WINDOW. The sgl-project#718 guard read the parallel_state routing global, which is toggled INSIDE the cutover -- one step among many -- so for the whole seam (waves moving KV rows, movers releasing the outgoing backing, the cutover rebuilding topology) the guard named a phase while pool bytes were in motion. And even a perfect predicate cannot recall a copy that is already riding the controller's private CUDA streams: write() and start_writing() run in the same Python instant (start_writing is called synchronously from write, its only caller), so the previous commit's consume-time re-check re-asks the question at the same moment it was first asked. The torn window is the STREAM's asynchrony, not the queue's: a device->host copy enqueued legitimately in PP outlives its Python call by seconds under load, and the seam releases the pool under it. Both crash specimens died exactly there -- 3 s after a pp_to_tp cutover, inside backup_from_device_all_layer, below the Python seam. THE FIX, in the two halves the window has: 1. AUTHORITY. PhaseFlipRuntime registers itself (weakly) as the phase guard's authority. Its _phase field is what the PHASE-FLIP DONE line reports -- truthful by the 3 s crash correlation -- and it alone knows the seam's extent: hicache_seam_active is raised at the no-return point (after the unanimous-abandon verdict, before the first wave) and cleared after the cutover installs the new phase, with a finally in the caller as insurance. During the seam the guard refuses device-tier I/O for EVERY binding. Outside it, the authority's phase wins over the routing global, and a disagreement logs the sgl-project#754-shape instrument line that settles whether that global was ever stale here. No authority registered (no flip runtime built) falls back to the routing global: non-flipping deployments stay byte-identical. 2. QUIESCE. At the same no-return point the runtime drains the controller's write_stream and load_stream while every pointer they hold still names live memory. Finishing those copies is correct (they become durable cache entries) and bounded (PCIe transfer of the backlog; this thread is the only device-tier producer, so nothing refills behind the drain; rank-local, so it cannot wedge the group). Ordered after the sgl-project#703 flip-writeback hook, whose staging copies are the largest legitimate producer of exactly such in-flight work. Tests: test_flip_seam_guard_760.py, 10 tests, red-first (all 10 fail on the parent commit: seam disarm and authority-wins are impossible there, quiesce/_quiesce_hicache do not exist). With the fix: 14/14 green including the sgl-project#760 stamp tests. Full mem_cache+managers sweep: 4057 passed, 43 failed -- the 43 are the pre-existing sgl-project#772 class (PhasePolicyConfig lacks idle_locked_settle_s), zero regressions, +10 passed vs the parent's 4047. ruff clean on the new code (the 13 E402 in cache_controller.py pre-exist on HEAD); codespell clean. Metal gate next: the 2-SIGSEGV repro arm must hold >=2 cutovers under 4-way load with 0 segfaults and logged seam refusals.

…ecated bridge The boot env was assembled by concatenating a captured shell environment, a heredoc and EXTRA_ENV. That is not a configuration mechanism, and it failed as one twice over: SGLANG_UNEVEN_TOKEN_VECTOR was set to 14,10,8 and then cleared by an empty override appended later. Uneven token sharding -- which must never be off -- was silently disabled for a day. SGLANG_PHASE_POLICY_TP_DECODE_FLOOR_S was 10 in the captured half and 8 in the heredoc. 8 won. The 10 had been dead for as long as both lines coexisted and nobody had noticed; it was found only because the first defect prompted a systematic check. Both are the same defect: a value someone deliberately set, silently overwritten by a value nobody compared it against. "Last one wins" is not a diagnostic any reader can see -- the unit line is one screen wide and the two entries sit forty keys apart. A flag cannot fail those ways. argparse rejects a typo that env silently ignores, there is one definition instead of a name matched by string in N places, the value is visible in `ps` and in the boot log, it cannot be written twice, and the planner can solve it. So the six phase-policy tuning knobs become ServerArgs fields, and config_from_env resolves each through _flag_or_env: the flag wins, the env is read only when the flag is unset and warns via the existing _warn_deprecated_env_to_cli_flag. Fields default to None -- "not specified" -- so every deployment that has not moved to flags is byte-identical today. ServerArgs is pickled whole into each scheduler subprocess, so the flag reaches every rank without an env round-trip. That also sidesteps the sgl-project#754 class of bug for these knobs: nothing has to re-parse a global env after the fork. Also declared here (fields only, consumers wired in the following commits): seam entry margin/delay, flip seam chunk, collective census interval, and phase_flip_corridor_floor_mib -- the last one lands on a hook that was already built and inert at phase_flip_spill.py, and replaces SGLANG_CORRIDOR_FLOOR_MIB, which had been silently overriding the corridor law in code (corridor_guard.CORRIDOR_LAW_MIB = 1024, band 819-1229) so that measurements were judged against a band the server did not use. Test is red-first and can-fail verified: with the flag lookup neutered so the resolver ignores ServerArgs, 11 of the assertions fail; restored, 6 pass with 15 subtests. The case that matters most pins a flag of 10 against an env of 8 -- the exact pair that was live in this repo.

…an internal detail Sixteen more SGLANG_* knobs get ServerArgs fields: barlink and its transport, the BAR1 aperture windows, the seam entry margin/delay and staging chunk, the collective census interval, mamba slot reorder, uneven DCP and its weighted form, KV backing relief, file-backed flip images, corridor rebalance, the TP memory imbalance check, and health-endpoint generation. The consumers for most of these live in modules that are never handed a ServerArgs -- the barlink transport, the weights arena, the KV relief ladder, the rebalance lender. Rather than rewrite thirty read sites in a live serving stack, this follows the mechanism already in the file: mamba_ssm_dtype publishes itself into the typed registry, and _publish_promoted_781_flags now does the same for the rest, from the process's own argv. What that buys, precisely: the FLAG is the single source of truth and the only operator-facing surface. argparse rejects a misspelling that the environment silently ignored -- verified, --seam-entry-margin-mibb is a hard parse error where SGLANG_SEAM_ENTRY_MARGIN_MIBB would have left the default riding. There is one definition instead of a name matched by string in N places. The value shows up in `ps` and in the boot log. It cannot be written twice. The planner can solve it. And the environment stops being something a human or a boot script writes, which is what lets the unit environment go identity-only so the boot gate can refuse any SGLANG_* it finds there. What it does not buy, stated plainly rather than left to be discovered: the consumers still read env. Threading each family through its own funnel per the sgl-project#754 template is follow-up work, not done here. This narrows the blast radius; it does not claim every consumer was rewired. phase_flip_corridor_floor_mib is different -- it lands on a hook that was already built and inert, with a comment explaining it was left there for exactly this flag. It replaces SGLANG_CORRIDOR_FLOOR_MIB=1536, which had been silently overriding the corridor law in code (corridor_guard.CORRIDOR_LAW_MIB = 1024, band 819-1229). The same file already described the consequence: "armed at 1536 and judged at 1024 ... the gap between the two numbers was exactly where five corridor breaches lived, unseen." Flag first now, env only as a deprecated bridge, and the log line names the flag instead of sending an operator after an env var they should not use. Defaults are None throughout: an unset flag publishes nothing and leaves the environment untouched, so nobody's running deployment moves today. Tests pin the parts that bite. False must publish "0" rather than be skipped as falsy, or a feature the operator turned off stays on. The TP-imbalance polarity is asserted in both directions because its retired predecessor SGL_DISABLE_TP_MEMORY_INBALANCE_CHECK was inverted. The BAR1 flag is asserted to expand into the per-group suffixed keys the consumer actually builds. 12 tests, 32 subtests, green.

… it is per phase --rank-kv-ratio looked like the flag this env var had already become. It is not, and a boot said so: an explicit --rank-kv-ratio vector is refused unless a non-uniform --rank-tp-ratio plan is installed, which a PP-prefill boot does not have. Promoting SGLANG_UNEVEN_TOKEN_VECTOR onto it killed the boot in 7 seconds. The deeper reason is that this instance runs TWO layouts. Under --enable-phase-flip it is PP in prefill and TP in decode, so an uneven split needs an entry per phase rather than one value that silently names only one of them. The prefill side is already carried by --pp-stage-ratio and --pp-attn-stage-ratio. This flag is the decode side, and its help says so. It is also not the weight shard split. The runtime prints the distinction itself: "KV token split [14, 10, 8] differs from the weight shard split [32, 16, 16] ... the weight shard follows compute; the token split follows each rank's memory left after its weights land." Two different questions, two different vectors, and --phase-flip-tp-vector answers the other one. Published into the environment rather than read from the field by the consumer, because the resolver runs inside the flip's SECOND stack build where the geometry is TP and not the boot-time PP -- the same scope split the sgl-project#754 fix addresses. Publishing keeps one value visible to both stacks. Worth recording how wrong the first reading was. Looking only at boot-time ServerArgs (tp_size=1, pp_size=3, dcp_size=1, rank_tp_ratio=None) the resolver bails at its first guard, and I concluded the vector had been inert all along. It is not inert; it is consumed in the decode layout, and the per-rank sizing lines show ratios 14/10/8 being applied. Reading one layout of a two-layout instance and generalising is the same mistake as reading one half of a concatenated env.

fix: add release tag workflow

b7eaf70

zhyncs merged commit 9a61182 into sgl-project:main Jul 26, 2024

zhyncs deleted the tmp branch July 26, 2024 19:48

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

fix: add release tag workflow (sgl-project#754)

5dcec12

amote-i pushed a commit to amote-i/sglang that referenced this pull request Dec 8, 2025

Upstream 4npu2 (sgl-project#754)

d268fe4

* Update pr-test-npu-debug.yml * Update run_suite.py * Update test_ascend_gpu_id.py * Update run_suite.py * Update test_ascend_gpu_id.py

Navigation Menu

Uh oh!

fix: add release tag workflow - #754

fix: add release tag workflow#754
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:tmp

zhyncs commented Jul 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

zhyncs commented Jul 26, 2024

Motivation

Modification

Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant