Make scripts under `/test/srt` as unit tests by Ying1123 · Pull Request #875 · sgl-project/sglang

Ying1123 · 2024-08-01T19:49:32Z

No description provided.

…d not carry 4390c13's message describes the sliced staging copy. Its diff is a FILE RENAME and nothing else -- zero product lines. This commit carries what that message promised. HOW. The `git add` before it listed a path that no longer existed (the pre-rename `test/registered/unit/managers/test_rotation_save_slices_875.py`). `git add` aborts the WHOLE invocation on a bad pathspec, and stderr was sent to /dev/null, so nothing was staged except the rename git had already recorded. `git status` said so plainly -- ` M` (worktree-only) on the two product files against `RM` on the rename -- and I read the line without reading the column. WHAT WAS AND WAS NOT AFFECTED, checked rather than assumed: * The MEASUREMENTS stand. pytest imports through PYTHONPATH from the working tree, not from the index, and the working tree had the full change. The after-arm's `813 passed` against the before-arm's `800` is +13, which is exactly this file's 13 cases -- i.e. the arm demonstrably ran the strengthened tests. The tier-2 gate arm likewise. * The COMMIT did not. It also froze the test file at its pre-strengthening content -- the version whose 4 KiB cases never reached the 1 MiB slicing threshold and let three of four mutants survive. NOT AMENDED. 4390c13 is pushed, and rewriting it needs a force-push, which needs the user's word. A follow-up that states the gap is worth more than a tidy history that hides it. Re-verified after this commit, from the INDEX and not the worktree: the sliced copy, the env knob and all 13 strengthened cases are present in the tree object.

…remise POSTEN 1 -- MEASURED FIRST, AND THE PREMISE DOES NOT SURVIVE THE MEASUREMENT. The instruction was: the guard is inert because there is no readable total, so cap `memory.current` against a constant 111.3 GiB from sgl-project#721. Measured on this box before touching anything: pinned_host_memory_bytes() -> total=118.05 GiB available=112.95 GiB so it does NOT land on the :245 "no honest number" branch. The guard is ACTIVE, a total IS readable, and `PINNED_HOST_RESERVE_BYTES` = 10 GiB is therefore NOT inert as a subtraction. The clamp binds too: MemAvailable 113.19 -> 112.95 GiB, reduced by the cgroup's own resident accounting exactly as rule 2 of `honest_host_memory_bytes` documents. `MemAvailable > MemTotal` is FALSE right now, so the pathological lxcfs reading is not present either. AND THE PROPOSED CAP WOULD NOT HAVE BOUND ANYTHING. Every level of the visible hierarchy is unlimited -- root, system.slice and system.slice/claude.service all read `memory.max = max`, and all three report `oom_kill 0` today (sgl-project#721's 17 kills predate this boot; `memory.peak` 101.98 GiB of 118.05). With no enforcing cgroup anywhere, the physical machine IS the ceiling, which is what MemTotal reports. A hand-carried 111.3 GiB constant would have been a third floor with no enforcer behind it -- the rule this ticket already established, applied to its own instruction. Not built. THE REAL DEFECT IS ONE BUCKET OVER, AND IT IS THE sgl-project#695 CLASS AGAIN. Prior art via the new index: commit c043235 measured that "CUDA pinned host memory is accounted in the cgroup's `file` bucket, not `anon` ... the offload ledger reported 20.78 + 14.44 + 14.44 = 49.66 GiB of pinned pool while `anon` sat steady at 14.6 GiB." Confirmed live on this box: anon 2.12 GiB, file 19.06 GiB, current 21.39 GiB -- the clamp charges ~2.4 GiB of a cgroup holding 21.4. `honest_host_memory_bytes` charges anon + kernel + unreclaimable shmem and NEVER charges `file`, correctly, because page cache is reclaimable. Pinned bytes there are not. sgl-project#695 already fixed one member of this class (shmem hiding in `file`); pinned host memory is the second and is still uncounted. IT ALSO PUTS A DOCUMENTED PREMISE IN DOUBT, and the doubt is precise: `pinned_host_budget.py:253` credits already-registered posts BACK to available because "their bytes are therefore already missing from it". For a post whose bytes land in `file`, they are NOT missing from it -- the clamp never subtracted them -- so the credit-back would count them as free twice. The 2026-08-17 measurement quoted there is sound for the weight images it was taken against; whether it holds for THIS pin depends on which bucket the pin's bytes land in. DELIBERATELY NOT FIXED HERE. That question is decidable only by watching `anon` and `file` across an actual pin allocation, which is a boot. Changing admission arithmetic on a shared path used by HiCache and kv-session-offload, on a guess about which bucket, would be the "capping on a fabricated figure" the module exists to prevent. So it is INSTRUMENTED instead -- the window script prints the before/after pair and names the consequence -- and the arithmetic is left alone until the boot answers it. POSTEN 2 -- `chunk_blocks_quiescence` (phase_flip_runtime.py:144): SAME CLOCK DIVERGENCE, and the determination is not a formality. It is the same class as the retention gap: a consumer (the cutover) and a writer (the chunk boundary that inserts into the tree) on different clocks. Under STRICT the clocks are SYNCHRONISED -- the cutover waits for the prefill, which is what "waiting for prefill to finish IS drain-and-flip" means in its docstring -- and the residual is exactly the NON-STRICT case where they are not. THE DIFFERENCE THAT MATTERS, and why this is one root and not two: the chunk case has a BOUNDED wait available (a prefill completes in finitely many chunks), while a mid-decode resident does not. That is why STRICT can fix one and nothing can fix the other by waiting. But the NON-STRICT chunk case cannot be fixed by waiting either -- an unconditional block re-creates sgl-project#631 defect O, the 32768-token prefill that ran in the slow layout and paid two cutovers for nothing. So BOTH halves need the same missing capability: persisting PARTIAL work at the seam. Neither is closable without it, and that capability is `seam_copy_state` / `restore_seam_state` (schedule_batch.py:2054/:2089) -- sgl-project#875's active territory. Reported, not touched, per Posten 4. Unlike the sgl-project#813/sgl-project#852/sgl-project#715 convergence I rejected last round, this one IS a shared root, and I am saying so because the evidence supports it, not because it tidies. POSTEN 3 -- `scripts/window_871a_verify.py`: one call, PASS/FAIL with numbers. Decides all three open claims from a boot log plus the cgroup. Exit 0 all passed, 1 a decided negative, 2 UNDECIDED -- and 2 is never a pass: "the evidence was not there" is a different fact from "the claim is false", and it sends the reader back to the boot. It boots nothing, claims no card, restores no serving. It also prints the sgl-project#721 HOST-LEDGER pair (posts + memory.current / peak / anon / file, before vs after) that Posten 1 needs. IT SHIPPED A FALSE PASS AND I CAUGHT IT BEFORE THE WINDOW, which is the whole argument for mock-smoking a window script. The first version summed `acked=` over the WHOLE log and returned PASS on the W40 boot -- the very boot in which all 21 fences reported `acked=0`. Seven lines there carry `acked=` from an unrelated subsystem, three of them `acked=24`, so it credited 72 acknowledgements no fence ever made. A window script that reports a false PASS is worse than none: it CLOSES an open claim. Now scoped to the fence's own lines, with that exact log shape as a permanent self-test case. The strongest desk check available is wired in: run against the real W40 log it must reproduce that boot's known split -- tier armed PASS, store delivery FAIL, exit 1 -- and it does. TESTS (hermetic; no boot; no card touched). CUDA_VISIBLE_DEVICES="" verified at the process and `nvidia-smi --query-compute-apps` empty throughout; GPUs 0/0/0. Runner reported its exclusions. * the script's own --self-test decides SIX cases in the intended direction, both polarities, including the false-PASS regression. * can-fail by mutation: un-scoping the acked regex reddens the real-boot reproduction and the self-test (2 red); restored green. * A SECOND SELF-INFLICTED DEFECT FOUND AND FIXED: the test located the script by counting `dirname` calls, landed on `test/` instead of the repo root, and failed all six cases -- a green arm red for a reason unrelated to the thing under test. It now walks up to the root. * partitioned tier-2 gate: 0 failing, 4421 passed (wide 3775 / narrow 292 / serial 354). Serial +6 over 348 is exactly the six tests added here. names=0 agrees with the summary on every lane. * test/registered/unit/mem_cache: 2 failed / 1802 passed -- the known pre-existing test_acceptance_emitters_758 RefillTiming pair, unchanged. POSTEN 4 -- boundary held. Nothing in schedule_batch.py, phase_flip_spill.py or seam_kv_recover was read into or written. Both Posten 2's root and A's remedy lead there; both are reported rather than resolved. NOT ESTABLISHED. Which cgroup bucket the phase-flip pin's bytes land in -- the question Posten 1 turns on -- needs the boot. Whether the admission ever refuses on this box, likewise. And sgl-project#721's 17 oom_kills are from a previous boot: today's counters are zero at every level, so nothing here re-measures that incident, it only records that the enforcement it implies is absent.

…the refusal's false leg User order: fix sgl-project#875, do not merely refuse it. The refusal in `restore_seam_state` is correct -- it is the only thing between the tree and the W40 IndexError one way and a silent wrong-layer write the other -- but its own counter comment states the cost: every flip in that direction loses its prefixes. This is the first axis of the answer, plus a correction that should have come first. THE REFUSAL'S JUSTIFICATION WAS PUT ON TRIAL AND ONE LEG DIED. `check_cpu_copy_layers` claimed "the KV head sharding also differs between the phases (PP holds all heads of its stage, TP a head shard of every layer), so even the overlapping entries are not interchangeable". FALSE on this rig, read in the code rather than assumed: under sgl-project#345 uneven-DCP replication the full-attention cache is TOKEN-sharded and every rank stores the FULL replicated kv-heads (model_runner_kv_cache_mixin.py:3155-3160; the boot line at :3228 prints `get_total_num_kv_heads()`), and the PP pool takes `get_num_kv_heads(attn_tp_size)` with `attn_tp_size == 1` -- the same total. Identical widths, interchangeable entries. That sentence was my own and it would have stopped the next reader from looking; it is removed rather than softened. THE SURVIVING LEG IS NARROWER THAN IT SOUNDED. A remap needs every layer the destination holds and rank-locally under PP it cannot have them -- 8 of 16 on this rig's `pp_attn_stage_ratio=[8,4,4]`. But the union over the three stages IS every layer and PP holds all tokens, so the data exists on a peer. The honest statement was never "a remap is impossible", it was "a RANK-LOCAL remap is impossible, and nobody asked whether a collective one was available". THREE AXES, NOT ONE -- the finding that decides the scope: LAYER PP stage-sharded vs TP complete. BUILT HERE, both directions proven. HEAD replicated in both phases. No remap needed. Settled above. TOKEN PP holds every token at allocator slots; TP holds an owner-rule SUBSET at compacted rows, `(L // cp_S) * cp_ratio + (L % cp_S - cp_lo)` (layers/dcp/owner.py:159). A SECOND collective-shaped remap, NOT solved. I had not named this axis at all before this round. SO THE CARRY IS NOT WIRED, AND THE REFUSAL STAYS. A layer-correct token-wrong carry produces matching row ids at mismatched widths -- the exact shape sgl-project#719 already walked into. `seam_layer_carry.py` is a pure module with the collective INJECTED by its caller: nothing in it communicates, because a collective on the per-request restore path is the sgl-project#630 wedge shape and the place a real carry belongs is the flip's own `pre_cutover_fns`, beside `gdn_state` and `weights_refill`, where the group is already synchronised. The refusal is annotated at both sites with what it is waiting for, so it reads as a staged non-answer rather than as a verdict. BOTH DIRECTIONS, SEPARATE FALSIFIERS, because they fail differently: PP -> TP the loud arm. Asserted as a refusal that NAMES the missing global layers. One stage alone refuses; three stages together assemble all 16 exactly; two of three refuses rather than restoring 12 right and 4 stale. TP -> PP the SILENT arm. No exception exists to assert on, so asserted on CONTENT: each destination slot must hold its own global layer, value by value, and all three stages together must tile the model exactly once. An exception-only test passes this arm by doing nothing. CAN-FAIL, and the first mutant is the one that matters: replacing the carry with the OLD naive positional prefix -- i.e. a no-op -- reds SEVEN tests across both arms. Filling missing layers instead of refusing reds three. Ignoring `start_layer` reds three. Dropping this rank's own copy from the union reds six. Same-layout is deliberately NOT special-cased: a PP->PP or TP->TP restore goes through the same path and comes out positionally identical, so the case that already works anchors the case that did not.

…NOT BUILD The three questions, in the binding order, answered against the source. Q1 -- DOES `req_to_token` IN THE TP PHASE HOLD ALL TOKENS OR ONLY THE OWNED ONES? ALL of them, and the values are GLOBAL cache slots. `dcp_weighted_read_slots` (layers/dcp/owner.py:440-465) states it in its own docstring: it takes "a flat list of GLOBAL cache slots (typically the ``req_to_token`` rows of a paged read)" and RETURNS `(compact, owned)`. The table is one row per request of width `max_context_len` (memory_pool.py:373-375) -- global positions, identical on every rank. The owner rule is applied DOWNSTREAM, in the attention backend. So owner.py:159 describes the ASSIGNMENT and says nothing about what the table materialises, exactly as the briefing suspected. The table materialises the global list. Q2 -- HOW DOES THE SOURCE'S POSITIONAL ROW LIST MAP ONTO THE DESTINATION'S COMPACTED ROWS? IT DOES NOT. No such function exists, and the absence is the finding. The only global->compact mapping in the tree is the owner rule, and an AST sweep of all of `srt/` finds its users are the attention backends and nothing else. CONSEQUENCE LARGER THAN THE CARRY, recorded but NOT claimed as a live defect. `Req.load_kv_cache` (schedule_batch.py:1806) hands `req_to_token[..., :seqlen-1]` -- global slots -- straight to `load_cpu_copy`, which indexes PHYSICAL rows. `HybridLinearKVPool` forwards them untouched (it translates only the mamba ids, and says so). Under PP `dcp_size == 1` and global == physical, so that path is correct. Under TP with a weighted vector they diverge. Whether the path runs there with uneven DCP active is a boot question and is not answered here; it is recorded with its address as the next check. The contrast that shows the tree knows this shape: `UnifiedSWAKVPool.get_cpu_copy` DOES translate, via `_virt_tokens_to_phys_tokens` (unified_memory_pool.py:1232). One pool family translates on this path and the other does not. Q3 -- CAN THE TWO FOLD INTO ONE COLLECTIVE? YES, an all-to-all. Rank r holds (its layers x ALL tokens); rank r' needs (ALL layers x its owned tokens); so r sends r' exactly the block (L_r x T_r') -- a distinct block per peer, which is the definition. One collective, not a gather followed by a scatter. VERDICT: DO NOT BUILD, taking the stop criterion that permits it. payload 16 layers x 13 rows x ~2 KiB per layer-token = ~416 KiB for the whole group at specimen scale. transport no P2P, no NVLink, all PHB. The tree's own number for this interconnect (sgl-project#656): 43.9 KiB per crossing, 166 us enqueue, 1777-9201 us receive. Latency-bound at this size. refusal recompute `extent` tokens of prefill; 13 tokens is sub-millisecond. A new all-to-all inside the cutover's no-return region would cost milliseconds to save microseconds, and would put a collective in the one region where a collective is the sgl-project#630 wedge shape. Unbounded risk against a benefit bounded by `extent`. FALSIFIABLE RATHER THAN FINAL: `extent` is `seqlen - 1`, so 13 is a property of the specimen's short request, not of the mechanism. Payload is linear in extent while the collective is latency-dominated, so a break-even exists somewhere in the hundreds-to-thousands of tokens. What would settle it is the DISTRIBUTION of `extent` over requests actually retracted at a flip. I do not have it and am not entitled to infer it from any record, in either direction. CONSEQUENTLY the pre_cutover_fns legality question is NOT answered here. It sits downstream of a decision that removed it, and inheriting "gdn_state and weights_refill already live there" was never a proof anyway. If the extent distribution ever reopens the carry, that check is the next gate and it must be done against the abandon and quorum paths, not by analogy. CAN-FAIL: giving the seam path a global->compact translation reds both sweep tests -- so the absence is observed, not merely stated. THE SWEEP HAD TO BE AST AND WAS CAUGHT BEING TEXT. `managers/prefetch_ballot.py` names `dcp_weighted_read_slots` in a docstring about an unrelated extraction and a substring match reported it as a caller. That is the THIRD time on this branch that a name in prose was mistaken for a use -- the 4 KiB threshold, then `outgoing_bytes=0` in its own docstring, now this -- so the sweep carries its own control asserting that a mention is not a use.

… and the sibling was never built I raised this in sgl-project#875 and called it "bigger than the carry". It is real and it closes as a pin. Q1 -- THE SIBLING ASYMMETRY IS VOID, AND THE OPERATOR'S REASON IS STRONGER THAN MINE. I had argued "one pool family translates on this path and the other does not". `UnifiedSWAKVPool` IS NEVER CONSTRUCTED ON THIS RIG: one construction site (unified_memory_pool.py:1325), behind `enable_unified_memory` defaulting False (server_args.py:1371) and False in this boot, behind `assert self.is_hybrid_swa` (model_runner_kv_cache_mixin.py:2809) which this GDN/Mamba-hybrid checkpoint does not satisfy, behind a help text excluding speculative decoding which this rig runs every boot. Verified independently here, not taken on report. A family nobody instantiates cannot be evidence that another family is missing something. My own reason -- the two translate DIFFERENT axes (`virtual_to_physical`, a lazily-bound page layer, versus the DCP owner rule) -- is true and is the weaker argument. Both point the same way and the asymmetry must never be the reason anyone touches this path: copying the unified side onto the hybrid side would install a virtual->physical mapping where no virtual layer exists. Q2 -- REACHABILITY, now the first question. `Req.offload_kv_cache` and `Req.load_kv_cache` have two callers each. `schedule_batch.py:2005` and `disaggregation/decode.py:736` are gated on `disaggregation_mode == "decode"`; this rig boots `'null'`, and the tree says so itself at phase_flip_runtime.py:1545 ("unreachable here"). The seam path is the only other one, and `copy_state=True` is passed at exactly ONE call site (phase_flip_runtime.py:1548) -- a copy is taken only at a flip, hence restored after the cutover, hence always cross-layout, hence declined by #861c's layout refusal BEFORE `load_cpu_copy` is reached. THE ONE SURVIVING WINDOW: a flip ABANDONED after `retract_all(copy_state=True)` has run. The restore then happens in the SAME phase, layouts match, the #861c guard passes, and the load runs with that phase's slots. Whether abandonment can occur after the copy is taken is the one thing I could NOT settle at the desk. Q3 -- DOES #783b's GUARD CATCH IT? Mostly, and the gap is bounded. `dcp_global_context_slots` (layers/dcp/owner.py:230-233) settles the weighted lane: "the allocator index space is C and each rank stores its ratio_r / S share". So `req_to_token` holds slots in [0, C) against a pool of about C * ratio_r / S rows. Most slots exceed it and `check_cpu_copy_rows` fires -- loud. Slots below the row count are in-range and map to the WRONG row. But the guard tests MIN and MAX of the whole vector, so one out-of-range entry raises for the call: only a request whose ENTIRE context sits in the low compact window is silent. Short contexts, inside an already narrow window. The guard is neither useless nor a proof of safety. VERDICT: PIN. No fix applied. The correct fix is the DCP owner rule on this path derived from the HYBRID pool's own addressing -- not copied from the unified pool -- and applying it while the path is unreachable would be a change nothing can observe, with a live chance of copying the wrong side. AND THE SAME QUESTION TURNED ON MY OWN sgl-project#877, which the operator asked me to check. On this rig NOT ONE of its six sites is constructed: `SGLANG_USE_HND_KVCACHE` False and absent from the boot; `enable_page_major_kv_layout` False; `enable_dsa_cache_layer_split` False; MLA is a DeepSeek-family pool and this is a GDN hybrid over `MHATokenToKVPool`; `SGLANG_NATIVE_MOVE_KV_CACHE` False and never set. The live path is `_move_kv_cache_impl`'s Triton branch, which was ALREADY chunked at 256 before sgl-project#877 touched anything. sgl-project#877 remains correct hardening and its class check stands, but my "pin because N <= 72" understated it: the stronger reason is that the sites are not built here at all. Recorded rather than left as a better-sounding claim. CAN-FAIL: removing the disagg gate, adding a second `copy_state=True` call site, and moving the load before the layout check each red their own test. The copy-state count had to be counted by AST -- written by text first, it counted a COMMENT quoting `copy_state=True` at line 1545 as a second site. Fourth instance on this branch of a name in prose read as a use, and the first one my own test caught rather than a reviewer.

… refusal, and it shipped In 25a667d I deleted the refusal's KV-head leg -- "PP holds all heads of its stage, TP a head shard of every layer" -- as FALSE. It is TRUE on this rig. The sentence is restored with the arithmetic attached, in all three places the wrong claim reached. THE ARITHMETIC. `_pool_kv_head_num` (model_runner_kv_cache_mixin.py:3176-3183) returns the replicated total only when `uneven_dcp_kv_replicated(dcp_size)`, and that predicate is `dcp_size > 1 AND get_tp_partition_ratios() is not None` (distributed/utils.py:479) -- it needs a `--rank-tp-ratio` base plan. This rig boots `rank_tp_ratio=None`, so the branch is never taken and the pool falls through to `get_num_kv_heads(attn_tp_size)` = `max(1, total_num_kv_heads // tp)` (configs/model_config.py:1370). With this checkpoint's `num_key_value_heads = 4`: PP (attn_tp_size 1) holds 4 heads per layer, TP (3) holds 1. A factor of four. The entries are NOT interchangeable. HOW I GOT IT WRONG, stated because the shape is the one this branch has been cataloguing all day. I read the replication BRANCH, treated its existence as reachability, and cited the boot line at :3228 ("%d replicated kv heads") as evidence. That line appears ZERO times in the boot I cited it from. I never checked the predicate's inputs. It is the same defect as the operator's UnifiedSWAKVPool support and as my own six sgl-project#877 sites -- existence read as reachability -- and this time it was not a briefing or an analysis but a change to shipped code. CONSEQUENCES: * Both legs of the original refusal stand. Only the LAYER leg's narrowing survives -- rank-locally impossible, collectively exact -- and that is what `seam_layer_carry.py` covers. * The seam carry needs THREE remaps, not two: layer, HEAD, token. Only the first is written. This makes the standing DO-NOT-BUILD verdict STRONGER. * The PP->TP head direction is not merely a remap but a lossy one (4 -> 1), so it is not even well defined without deciding which heads survive. NOT AMENDED: 25a667d is pushed, and rewriting it needs a force-push, which needs the user's word. The retraction is a follow-up that names the gap, as with the earlier empty-commit incident on this branch. PINNED so the retraction cannot be re-retracted silently: four cases asserting the PREDICATE'S INPUTS rather than the branch's existence, plus the head arithmetic itself. CAN-FAIL: dropping the plan requirement from the predicate reds two; making the head count undivided reds one. Found by the systematic sweep the operator asked for -- checking MY OWN cited evidence against the boot log, the same pass that found the sgl-project#877 sites unbuilt.

… my 18 commits: two claims withdrawn The sweep I named and had not run. Every citation of a log string or a borrowed measurement across all 18 commits on this branch, checked against THE source it cites -- not against any source. 31 citations, four states. BELEGT -- 27. Everything drawn from /spinning/evidence-665-f1/boot_w40_857strict_0826_0516.log verifies literally in that file: the three segment times, the 75 % worst-share, `save 4.342`, `gpu-span d2h 0.000s / h2d 0.000s` (21x), `checksum 0.319`, `gdn_state->refill_highwater` 0.2 and 7.0, `weights_refill->cutover` 113.5 and 972.1, the 498/268 aliasing counts, `REFILL pp_to_tp took 3.961 s for 8573.8 MiB`, `pp_attn_stage_ratio=[8, 4, 4]`, `disaggregation_mode='null'`, `rank_tp_ratio=None`, `uneven_token_vector='29,19,16'`, and the W40 specimen details I repeated from the briefing (05:20:50, the rid, the IndexError, `SEAM RESTORE ATTEMPT`, `FLIP EXTENT PROBE`, `extent=13`, `transfer_layer_num=32`). The log is present; source not missing. NIE-GEDRUCKT, CLAIM ALSO FELL -- 1. "the boot line at :3228 prints `get_total_num_kv_heads()`": zero occurrences. Already retracted in 19b8177; the sentence it supported was false and the true one is restored. NIE-GEDRUCKT, CLAIM SURVIVES -- 1. "24 s after health 200". The string `health 200` never printed. The log says `"GET /health HTTP/1.1" 200 OK` at 05:20:26, and the crash is at 05:20:50: exactly 24 s. The claim is right to the second; only my citation form was ungreppable, so anyone verifying it would have found nothing. The two cases are kept apart because their consequences differ. MISATTRIBUTED, AND MISREAD -- 1. I priced sgl-project#875's DO-NOT-BUILD verdict with "43.9 KiB per crossing, 166 us enqueue, 1777-9201 us receive", attributed to sgl-project#656. It is not in sgl-project#656. It is in 13c55d7 "[PP] sgl-project#201 slice 2: the stage boundary across two rigs" -- CROSS-RIG, two nodes, a 40G line, a different model (Qwen3.5-4B fp16), timing PP microbatch crossings and not an intra-node collective. And its own text refuses my reading: "`recv` is BLOCKING, i.e. bubble plus wire -- 9.2 ms on stage 1 is that stage waiting for stage 0, NOT the 40G line". I used a pipeline bubble as transport latency, from the wrong ticket, for the wrong link. WITHDRAWN, with no substitute: I have no measured local collective figure. QUELLE-FEHLT -- 1. "sgl-project#721's measured cgroup peak (111.3 of 118 GiB, oom_kill=17)", used in sgl-project#877 and in sgl-project#878's headroom. `oom_kill=17` has NO in-tree source, and the only `111.3` is `111.3%` in a benchmark table (docs/dev/631/PROD_BRINGUP_BENCH.md) -- an unrelated percentage. It came from a briefing and I repeated it as measured, twice. WITHDRAWN. This is the shape the operator hit this morning with the retracted "1833/1833 records". WHAT THE TWO WITHDRAWALS COST, stated rather than smoothed: * sgl-project#875's verdict loses its timing leg entirely. It now rests on two terms that need no transport number: the HEAD axis is LOSSY (PP 4 kv-heads per layer, TP 1), so PP->TP is not a remap until someone decides which heads survive and no such rule exists; and a new collective in the no-return region is the sgl-project#630 wedge shape, an unbounded risk. Point one alone is decisive. DO NOT BUILD stands, on better ground than it had. * sgl-project#878's headroom argument is re-grounded on a figure I took myself: this box reports 128711 MB total / 76971 MB free, so +96 MiB of pinned host image is negligible by four orders of magnitude, needing nobody's peak. * sgl-project#877's conclusion is untouched: N <= 72 rows of a per-LAYER temporary is kilobytes, and separately none of its six sites is constructed on this rig. THE CLASS: a citation is evidence only for the claim it is attached to, in the source it names. Three of the four defects here were true-sounding numbers from the right neighbourhood -- a real measurement of a different link, a real percentage read as gigabytes, a real code branch that never executes.

…ation of it was wrong Third judgement on this axis. I deleted the refusal's KV-head leg (25a667d), restored it as true (19b8177), and now delete it again. Both phases hold 4 kv-heads per layer. The original deletion was right. VERIFIED AT THE CODE, INDEPENDENTLY -- not taken on report, because two judgements on this axis were already wrong today: * `parse_flip_vector` (phase_flip_boot.py:254) reads `server_args.phase_flip_tp_vector`, NOT `--rank-tp-ratio`. This boot: '32,16,16'. * `set_tp_partition_ratios(list(vec))` runs at phase_flip_boot.py:1428; the TP `TpModelWorker` is built at :1473. Install precedes construction, and nothing between them clears it. * so at TP-pool build time `get_tp_partition_ratios()` is [32,16,16], dcp_size is 3, `uneven_dcp_kv_replicated` is TRUE, the sgl-project#345 exception is taken and `_pool_kv_head_num` returns `get_total_num_kv_heads()` = 4. * PP builds earlier, when the only installer that has run is scheduler.py:12223 with `rank_tp_ratio=None` -- ratios ARE None there, and `max(1, 4 // 1)` = 4. * ONE LAYER THE HANDED-OVER CHAIN DID NOT TRACE, checked here: `get_tp_partition_ratios` reads a context-local overlay BEFORE the installed plan (distributed/utils.py:168). It is written only by the sgl-project#274 dual-group lane's scoped manager (:207), and `dual_group_lane=False` on this boot -- so that layer is inert and cannot shadow the plan. The chain survives my check; I found nothing that breaks it. WHY I GOT IT WRONG THE SECOND TIME, and it is not carelessness: `uneven_dcp_kv_replicated`'s docstring says "a `--rank-tp-ratio` base plan is installed". It names a FLAG for a PROCESS state with TWO installers, and the flip is the other one. Reading the docstring is sufficient to reach my wrong conclusion. The root fix belongs on that docstring and is not mine. CONSEQUENCE: the head axis needs NO remap, the carry needs TWO (layer, token), and sgl-project#875's DO-NOT-BUILD verdict LOSES the argument I gave it last round. What still carries it: the token axis is unsolved and rank-locally unavailable, and a new collective in the cutover's no-return region is the sgl-project#630 wedge shape. The transport-timing leg was already withdrawn in 237189f as misattributed. So the verdict stands on the token axis and the no-return risk -- and it now rests on less than it did, which is worth saying plainly. TESTS: the four cases that pinned the WRONG arithmetic are replaced by five that assert the CHAIN -- the installer's source, the install-before-construction ORDER, the predicate's actual expression rather than its docstring's flag name, the head counts on both sides, and the overlay's inertness. Asserting the flag is what reproduced my error; asserting the order and the source is what makes the answer stable. CAN-FAIL: moving the install after the TP worker reds the order test; making the flip read `rank_tp_ratio` reds the installer test. The first mutant initially survived because I reinserted the install immediately BEFORE the worker line -- the order was never actually inverted. Fixed and re-run. NOT AMENDED: 19b8177 is pushed. Third correction-by-follow-up on this branch, same rule each time. RESIDUAL, inherited and not resolved: the chain is closed by CODE ORDER, not by a log line printing the ratios at pool-build time. A failed install between :1428 and :1473 would go undetected. No path to one is known.

…aging memcpy is structural; its single-core rate was not) into the flip train

…refusal becomes a carry in the direction that never needed a collective) into the flip train

… while the peers spin BOOT-PROVEN (boot 22, boot_855_1033b_0840f82601_0831_131955, 13:24-13:26). The first TP forward after a pp_to_tp cutover touches a shape specialisation never loaded in this process. Rank 0's crash-time stack (log line 53248 ff.): _dcp_write_scatter (flashinfer_backend.py:2574) run (triton/runtime/jit.py:743) -> _init_handles (compiler.py:466) loadBinary -> cuModuleLoadData (libcuda.so) + 11 libcuda frames NOT the compiler -- cuModuleLoadData. The module is built; LOADING it into the CUDA context blocks, because the load needs the device and the device is saturated by the peers' barlink BAR1 spin kernels, which are waiting in all_gather for this very rank. The cycle closes and the spin deadline fires: Bar1CollectiveAborted (ranks 1/3 and 2/3, group flip_dcp:0) -> SIGQUIT. WHY THE EXISTING MECHANISM MISSED IT, measured: 352 build-window lines in that boot, ZERO in 13:24-13:26, all 352 carrying 'full cuda-graph capture warmup'. cold_build_window had exactly three production callers (barlink BAR1 build, sampling warmup #603b, capture warmup) and none is on the path a cutover re-dispatches into. sgl-project#640 on a path sgl-project#615 never saw. The fix makes that path a caller rather than building a second mechanism beside it. A STALE COMMENT NEARLY REFUTED THIS FIX, and the correction ships with it. sampler_warmup.py said wrapping a lazy build in cold_build_window "does NOT work ... the window is PROCESS-LOCAL". True when written (8bddb93, 2026-08-06); falsified ONE DAY LATER by sgl-project#615 (38ec4fb, 2026-08-07), which hooked publication into cold_build_window so "every existing call site therefore becomes group-visible without moving". Never revised, both ancestors of this pin, verified with git log -S rather than assumed. The paragraph is annotated in place rather than deleted, because it is still right about ITS module (warm-at-boot + barrier REMOVES the race; a window only EXTENDS it). WHY A WINDOW OVER THE REAL FORWARDS, NOT AN ENUMERATED WARM SET. The warm set is not knowable by inspection -- it varies with direction, spec-decode, the sgl-project#887 one-chunk grant and any backend swap -- and missing one member reproduces the wedge exactly, at the next first-loader. Whatever loads, loads under the window. Residual stated in the code: this EXTENDS deadlines (900 s cap) rather than removing the race; boot 22's stall was ~150 s, so the cap is not binding. FIRST-LOADER CENSUS, as a standing table: devtools/CENSUS_1033c_first_loaders.md -- every first-loader site x triggering state change x covering window x collective proximity. Rows 1-6 verified at file:line (row 5 on metal); rows 7 (runtime recapture / drafter switch) and 8 (rung change sgl-project#704, resume-restore sgl-project#89) are written UNVERIFIED, so the table is a lower bound on the covered set and never a proof that nothing else is exposed. It also names the inverse column (armed windows that may cover paths that no longer first-load -- the 352 are a count of WARMUPS, not of BUILDS, and the window carries no modules-loaded counter, so live and dead coverage are indistinguishable from the log) and answers the upstream-minimal question: there are not four window mechanisms but two plus a front door, with one site (barlink_device.py:865) bypassing the front door. Named, not rebuilt. #1033d, same boot, INDEPENDENT DEFECT, minimal hardening only: the prefetch IO aux thread caught only Empty, so a page whose geometry did not match the incoming binding (mha.py:556 reshape, '[2,16,1,4,256]'=32768 against a 16384 page) ended the thread three seconds before the cutover -- killing no process, setting no exit code, appearing in no health probe, and leaving storage prefetch dead for the rest of the boot. It now fails the OPERATION (host slots released, so the requester gets a refusal instead of an unreachable completion), logs loudly with a counter, and stays alive. The broad except is defensible only because the alternative is silent thread death with no supervisor above the loop; the underlying two-geometry host pool across a flip (sgl-project#718/sgl-project#719/sgl-project#875 family, and the fork's own 'sgl-project#939 RE-HOME VIA RE-READ ... source page 16384 elems vs destination page 32768 elems' line names the same mismatch two lines earlier while handling it correctly) is NOT fixed here and is its own posten. DESK CHECKS, matched to each edit's failure class: * devtools/check_1033c_cutover_window.py -- 19 cases, hermetic, driving the real run_batch wrapper: RED-FIRST arm reproducing the boot-22 condition (counter unarmed -> no window -> nothing published), first forward covered AND published, window closes (the sgl-project#431 open-without-close shape), budget finite and steady state byte-identical, window closed and budget spent even when the forward raises, and an AST check that the arming sits under no `if` (the naive grep form of that assertion failed on a COMMENT -- fixed to ask the AST). * AST check on the aux thread: broad handler present, logs at ERROR, releases the failed operation's host slots, and continues. * Method-split check: wrapper delegates, window only on the armed path, lazy import off the fast path, inner body intact. ruff F401/F821/UP037 clean. DESK-PROVEN. Metal proof is boot 23.

…t identity the plan already publishes Operator ruling 2026-09-11 (build variant (a)) plus the operator direction of 2026-09-12 that re-cut the key. WHAT IT IS. A rank-local content digest over the weight seam: taken at the last instant before the weights family is paused, recomputed at the first instant after it has landed, same rank, same pieces. No peer, no collective, no byte moved between cards, both moments AWAKE and outside the no-return region (sgl-project#875 untouched). Cost is a hash, not an image, so the host-RAM law holds. WHY NOT A memcmp. TMS frees the pinned host image right after the H2D restore (core.cpp, both sites); a byte-faithful memcmp needs two rank-local copies and the user law guarantees exactly one. The round-trip digest is the stronger reading anyway: a memcmp against a second copy proves COPY fidelity, this proves ROUND-TRIP fidelity. THE KEY IS THE PLACEMENT, AND WHAT THAT REPLACED. The first build of this module walked model.named_parameters() itself and keyed on its own (name, shape, dtype) triple. That walk and that triple are GONE: they were a second inventory beside a fact the loader decides at boot and the tree already publishes, i.e. the W80/W84/W19 family. The walk is extracted instead -- weight_exchange_shadow.card_inventory, produced once and consumed by both derive_card_manifest and the grader, so the manifest the co-located pair agrees over and the pieces the grader hashes cannot be two different sets. The key is (manifest_entry(name, class, rows, cols, itemsize), tag, card): every field is the plan's own, and the four properties that move when the arena moves -- data_ptr, storage_offset, storage object, pitch -- are deliberately not in it. WHAT THE RE-KEYING BOUGHT. The verdict now NAMES the pieces whose content moved (moved=/pieces_moved=, gone=, arrived=), so the operator's condition (3) is narrowed rather than restated: the grader detects AND localises to the piece; it still does not localise within a piece, and a green digest is still never 'layout verified'. LIMIT_CLAUSE says exactly that, on the line. DEFAULT OFF. One named flag, --weg2-seam-digest, published as SGLANG_WEG2_SEAM_DIGEST and POPPED when absent so an inherited shell value cannot arm it; the env rung is resolved and reported through the sgl-project#901 knob authority. Armed it hashes the whole shard per leg, which on a serving boot would be a flip-cost regression caused by the instrument. A MISMATCH RAISES (W90 Weg2SeamDigestMismatch, enumerated from the sgl-project#1263 census and cross-checked by a word-bounded grep). The verdict line reaches the log before the raise. Call sites are exactly two, both in weight_updater: before the first weights pause of a sleep, and after family_complete + reload + shadow compare on the wake. There is no third at the cutover and there cannot be -- at flip time the sleeping group holds no weight bytes in VRAM.

add srt backend test

5898431

Ying1123 requested a review from zhyncs August 1, 2024 19:49

Ying1123 added 6 commits August 1, 2024 13:02

Update unit-test.yml

95ace1d

update srt backend

7c389cc

test openai api

a1d4374

update tests

217e54d

update oai test

4452058

fix

f700859

Ying1123 changed the title ~~Add srt backend test to CI~~ Make scripts under srt as unittests Aug 1, 2024

Ying1123 changed the title ~~Make scripts under srt as unittests~~ Make scripts under /test/srt as unit tests Aug 1, 2024

Ying1123 merged commit 72b6ea8 into main Aug 1, 2024

Ying1123 deleted the add-srt-backend-test branch August 1, 2024 21:34

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

Make scripts under /test/srt as unit tests (sgl-project#875)

91706bd

efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 26, 2026

Merge fix/875-arena-rotation-aliasing (sgl-project#875, the seam's st…

94f2806

…aging memcpy is structural; its single-core rate was not) into the flip train

efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 26, 2026

Merge fix/875-layout-neutral-seam (sgl-project#875, the seam's layer …

c543eae

…refusal becomes a carry in the direction that never needed a collective) into the flip train

Navigation Menu

Uh oh!

Make scripts under `/test/srt` as unit tests - #875

Make scripts under `/test/srt` as unit tests#875
Ying1123 merged 7 commits into
mainfrom
add-srt-backend-test

Ying1123 commented Aug 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Ying1123 commented Aug 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant