bump to 0.2.8 - #877
Merged
Merged
bump to 0.2.8#877
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
hhhh1252023
pushed a commit
to hhhh1252023/sglang_public
that referenced
this pull request
Jun 27, 2026
Update test_npu_json_mode.py, test_npu_openai_embedding.py, test_npu_input_embeddings.py
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
…l-project#876, which the register owns sgl-project#876 is the code-graph infrastructure (devindex/zoekt) in the register. I had taken it in-tree for two unrelated findings, so `grep 'sgl-project#876'` would have landed a future reader between two different subjects -- the anchor-drift class this register has been hit by before. sgl-project#877 the implicit gather temporary in the in-place row move (was sgl-project#876) sgl-project#878 the reversed copy-back precondition (was #876b) NINE anchor lines, not the three that were visible from the outside. Six were in places a spot-check would not reach: the INPLACE_MOVE_MAX_ROWS declaration, the runtime warning STRING (which would have printed a wrong ticket into a boot log, the worst copy of all), the per-layer hoist comment, and both test module docstrings. BOTH TEST FILENAMES RENAMED TOO. A filename carrying a ticket number is a greppable anchor like any other, and this tree's convention puts the number there; leaving `test_inplace_move_bound_876.py` would have reproduced the exact collision one directory over. Neither file is under the tier-2 gate's path (`unit/managers`), so the partition table is untouched. Ordered so the passes cannot collide: #876b -> sgl-project#878 FIRST, then sgl-project#876 -> sgl-project#877. The reverse order would have rewritten `#876b` into `#877b`. Commit messages on the pushed commits keep the old numbers -- rewriting them needs a force-push, which needs the user's word. The code is what gets grepped and the code is now right; the register carries the discrepancy. Verified: no `sgl-project#876` anchor remains anywhere in python/ or test/. The residual literal "876" matches are unrelated -- upstream PR numbers, float digits, bootstrap ports.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
…ing only The sgl-project#877 report named HND and PageMajor as the weakest part of that change: the chunking there was verified by READING, not by RUNNING. "No test exists" is not a closure, it is the desk-written-never-executed shape. Both turn out to be hermetically exercisable, so there was nothing to refuse. Neither needs a device: HND is plain 4-D advanced indexing, PageMajor routes to `move_kv_cache_native` with `page_size > 1` (memory_pool.py:4192), and `maybe_detect_oob` is inert unless SGLANG_ENABLE_ASYNC_ASSERT is set. Seven cases over the three shapes reading could not check: HND [num_pages, head, page_size, dim], indexed `kb[page, :, off, :]` -- a different axis PAIR from every other site, which is precisely why reading it was not enough. Small, large-enough-to-chunk, and rightward (the direction whose ascending chunking is measurably broken). PageMajor 4-D `(page_id, slot_in_page)`, small and large. The degenerate `page_size == 1` 4-D branch, which is separate source. The reference in each case is the UNCHUNKED statement's semantics, not a reimplementation of the new code -- a test that re-derives the implementation encodes its assumptions. CAN-FAIL: three mutants, each hitting exactly its own layout. Corrupting the HND page offset reds all three HND cases; corrupting the page-major slot index reds both PageMajor cases; corrupting the `page_size == 1` branch reds that case. So the new cases reach the code rather than sitting beside it -- which is the property the previous round of this file did NOT have.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
… and the sibling was never built I raised this in sgl-project#875 and called it "bigger than the carry". It is real and it closes as a pin. Q1 -- THE SIBLING ASYMMETRY IS VOID, AND THE OPERATOR'S REASON IS STRONGER THAN MINE. I had argued "one pool family translates on this path and the other does not". `UnifiedSWAKVPool` IS NEVER CONSTRUCTED ON THIS RIG: one construction site (unified_memory_pool.py:1325), behind `enable_unified_memory` defaulting False (server_args.py:1371) and False in this boot, behind `assert self.is_hybrid_swa` (model_runner_kv_cache_mixin.py:2809) which this GDN/Mamba-hybrid checkpoint does not satisfy, behind a help text excluding speculative decoding which this rig runs every boot. Verified independently here, not taken on report. A family nobody instantiates cannot be evidence that another family is missing something. My own reason -- the two translate DIFFERENT axes (`virtual_to_physical`, a lazily-bound page layer, versus the DCP owner rule) -- is true and is the weaker argument. Both point the same way and the asymmetry must never be the reason anyone touches this path: copying the unified side onto the hybrid side would install a virtual->physical mapping where no virtual layer exists. Q2 -- REACHABILITY, now the first question. `Req.offload_kv_cache` and `Req.load_kv_cache` have two callers each. `schedule_batch.py:2005` and `disaggregation/decode.py:736` are gated on `disaggregation_mode == "decode"`; this rig boots `'null'`, and the tree says so itself at phase_flip_runtime.py:1545 ("unreachable here"). The seam path is the only other one, and `copy_state=True` is passed at exactly ONE call site (phase_flip_runtime.py:1548) -- a copy is taken only at a flip, hence restored after the cutover, hence always cross-layout, hence declined by #861c's layout refusal BEFORE `load_cpu_copy` is reached. THE ONE SURVIVING WINDOW: a flip ABANDONED after `retract_all(copy_state=True)` has run. The restore then happens in the SAME phase, layouts match, the #861c guard passes, and the load runs with that phase's slots. Whether abandonment can occur after the copy is taken is the one thing I could NOT settle at the desk. Q3 -- DOES #783b's GUARD CATCH IT? Mostly, and the gap is bounded. `dcp_global_context_slots` (layers/dcp/owner.py:230-233) settles the weighted lane: "the allocator index space is C and each rank stores its ratio_r / S share". So `req_to_token` holds slots in [0, C) against a pool of about C * ratio_r / S rows. Most slots exceed it and `check_cpu_copy_rows` fires -- loud. Slots below the row count are in-range and map to the WRONG row. But the guard tests MIN and MAX of the whole vector, so one out-of-range entry raises for the call: only a request whose ENTIRE context sits in the low compact window is silent. Short contexts, inside an already narrow window. The guard is neither useless nor a proof of safety. VERDICT: PIN. No fix applied. The correct fix is the DCP owner rule on this path derived from the HYBRID pool's own addressing -- not copied from the unified pool -- and applying it while the path is unreachable would be a change nothing can observe, with a live chance of copying the wrong side. AND THE SAME QUESTION TURNED ON MY OWN sgl-project#877, which the operator asked me to check. On this rig NOT ONE of its six sites is constructed: `SGLANG_USE_HND_KVCACHE` False and absent from the boot; `enable_page_major_kv_layout` False; `enable_dsa_cache_layer_split` False; MLA is a DeepSeek-family pool and this is a GDN hybrid over `MHATokenToKVPool`; `SGLANG_NATIVE_MOVE_KV_CACHE` False and never set. The live path is `_move_kv_cache_impl`'s Triton branch, which was ALREADY chunked at 256 before sgl-project#877 touched anything. sgl-project#877 remains correct hardening and its class check stands, but my "pin because N <= 72" understated it: the stronger reason is that the sites are not built here at all. Recorded rather than left as a better-sounding claim. CAN-FAIL: removing the disagg gate, adding a second `copy_state=True` call site, and moving the load before the layout check each red their own test. The copy-state count had to be counted by AST -- written by text first, it counted a COMMENT quoting `copy_state=True` at line 1545 as a second site. Fourth instance on this branch of a name in prose read as a use, and the first one my own test caught rather than a reviewer.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
… refusal, and it shipped In 25a667d I deleted the refusal's KV-head leg -- "PP holds all heads of its stage, TP a head shard of every layer" -- as FALSE. It is TRUE on this rig. The sentence is restored with the arithmetic attached, in all three places the wrong claim reached. THE ARITHMETIC. `_pool_kv_head_num` (model_runner_kv_cache_mixin.py:3176-3183) returns the replicated total only when `uneven_dcp_kv_replicated(dcp_size)`, and that predicate is `dcp_size > 1 AND get_tp_partition_ratios() is not None` (distributed/utils.py:479) -- it needs a `--rank-tp-ratio` base plan. This rig boots `rank_tp_ratio=None`, so the branch is never taken and the pool falls through to `get_num_kv_heads(attn_tp_size)` = `max(1, total_num_kv_heads // tp)` (configs/model_config.py:1370). With this checkpoint's `num_key_value_heads = 4`: PP (attn_tp_size 1) holds 4 heads per layer, TP (3) holds 1. A factor of four. The entries are NOT interchangeable. HOW I GOT IT WRONG, stated because the shape is the one this branch has been cataloguing all day. I read the replication BRANCH, treated its existence as reachability, and cited the boot line at :3228 ("%d replicated kv heads") as evidence. That line appears ZERO times in the boot I cited it from. I never checked the predicate's inputs. It is the same defect as the operator's UnifiedSWAKVPool support and as my own six sgl-project#877 sites -- existence read as reachability -- and this time it was not a briefing or an analysis but a change to shipped code. CONSEQUENCES: * Both legs of the original refusal stand. Only the LAYER leg's narrowing survives -- rank-locally impossible, collectively exact -- and that is what `seam_layer_carry.py` covers. * The seam carry needs THREE remaps, not two: layer, HEAD, token. Only the first is written. This makes the standing DO-NOT-BUILD verdict STRONGER. * The PP->TP head direction is not merely a remap but a lossy one (4 -> 1), so it is not even well defined without deciding which heads survive. NOT AMENDED: 25a667d is pushed, and rewriting it needs a force-push, which needs the user's word. The retraction is a follow-up that names the gap, as with the earlier empty-commit incident on this branch. PINNED so the retraction cannot be re-retracted silently: four cases asserting the PREDICATE'S INPUTS rather than the branch's existence, plus the head arithmetic itself. CAN-FAIL: dropping the plan requirement from the predicate reds two; making the head count undivided reds one. Found by the systematic sweep the operator asked for -- checking MY OWN cited evidence against the boot log, the same pass that found the sgl-project#877 sites unbuilt.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
… my 18 commits: two claims withdrawn The sweep I named and had not run. Every citation of a log string or a borrowed measurement across all 18 commits on this branch, checked against THE source it cites -- not against any source. 31 citations, four states. BELEGT -- 27. Everything drawn from /spinning/evidence-665-f1/boot_w40_857strict_0826_0516.log verifies literally in that file: the three segment times, the 75 % worst-share, `save 4.342`, `gpu-span d2h 0.000s / h2d 0.000s` (21x), `checksum 0.319`, `gdn_state->refill_highwater` 0.2 and 7.0, `weights_refill->cutover` 113.5 and 972.1, the 498/268 aliasing counts, `REFILL pp_to_tp took 3.961 s for 8573.8 MiB`, `pp_attn_stage_ratio=[8, 4, 4]`, `disaggregation_mode='null'`, `rank_tp_ratio=None`, `uneven_token_vector='29,19,16'`, and the W40 specimen details I repeated from the briefing (05:20:50, the rid, the IndexError, `SEAM RESTORE ATTEMPT`, `FLIP EXTENT PROBE`, `extent=13`, `transfer_layer_num=32`). The log is present; source not missing. NIE-GEDRUCKT, CLAIM ALSO FELL -- 1. "the boot line at :3228 prints `get_total_num_kv_heads()`": zero occurrences. Already retracted in 19b8177; the sentence it supported was false and the true one is restored. NIE-GEDRUCKT, CLAIM SURVIVES -- 1. "24 s after health 200". The string `health 200` never printed. The log says `"GET /health HTTP/1.1" 200 OK` at 05:20:26, and the crash is at 05:20:50: exactly 24 s. The claim is right to the second; only my citation form was ungreppable, so anyone verifying it would have found nothing. The two cases are kept apart because their consequences differ. MISATTRIBUTED, AND MISREAD -- 1. I priced sgl-project#875's DO-NOT-BUILD verdict with "43.9 KiB per crossing, 166 us enqueue, 1777-9201 us receive", attributed to sgl-project#656. It is not in sgl-project#656. It is in 13c55d7 "[PP] sgl-project#201 slice 2: the stage boundary across two rigs" -- CROSS-RIG, two nodes, a 40G line, a different model (Qwen3.5-4B fp16), timing PP microbatch crossings and not an intra-node collective. And its own text refuses my reading: "`recv` is BLOCKING, i.e. bubble plus wire -- 9.2 ms on stage 1 is that stage waiting for stage 0, NOT the 40G line". I used a pipeline bubble as transport latency, from the wrong ticket, for the wrong link. WITHDRAWN, with no substitute: I have no measured local collective figure. QUELLE-FEHLT -- 1. "sgl-project#721's measured cgroup peak (111.3 of 118 GiB, oom_kill=17)", used in sgl-project#877 and in sgl-project#878's headroom. `oom_kill=17` has NO in-tree source, and the only `111.3` is `111.3%` in a benchmark table (docs/dev/631/PROD_BRINGUP_BENCH.md) -- an unrelated percentage. It came from a briefing and I repeated it as measured, twice. WITHDRAWN. This is the shape the operator hit this morning with the retracted "1833/1833 records". WHAT THE TWO WITHDRAWALS COST, stated rather than smoothed: * sgl-project#875's verdict loses its timing leg entirely. It now rests on two terms that need no transport number: the HEAD axis is LOSSY (PP 4 kv-heads per layer, TP 1), so PP->TP is not a remap until someone decides which heads survive and no such rule exists; and a new collective in the no-return region is the sgl-project#630 wedge shape, an unbounded risk. Point one alone is decisive. DO NOT BUILD stands, on better ground than it had. * sgl-project#878's headroom argument is re-grounded on a figure I took myself: this box reports 128711 MB total / 76971 MB free, so +96 MiB of pinned host image is negligible by four orders of magnitude, needing nobody's peak. * sgl-project#877's conclusion is untouched: N <= 72 rows of a per-LAYER temporary is kilobytes, and separately none of its six sites is constructed on this rig. THE CLASS: a citation is evidence only for the claim it is attached to, in the source it names. Three of the four defects here were true-sounding numbers from the right neighbourhood -- a real measurement of a different link, a real percentage read as gigabytes, a real code branch that never executes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.