chore: add copyright for srt by zhyncs · Pull Request #790 · sgl-project/sglang

zhyncs · 2024-07-28T13:06:07Z

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
Modify documentation as needed, such as docstrings or example tutorials.

…ving hot paths An isolation boot wedged for 25+ minutes with 120 ADMISSION-WEDGE markers. py-spy (two dumps 25 min apart, byte-identical, so hard-stuck) put PP0's MainThread INSIDE logging.emit while PP1 and PP2 starved in pp_chain_receiver.recv for a chain send PP0 never reached. ROOT CAUSE. PP0 admitted the first radix-carrying request and reached the sgl-project#767 instrument in HybridReqToTokenPool.alloc, which logged req.mamba_pool_idx -- a 1-element CUDA tensor -- as a %s argument. The format call runs Tensor.__repr__ -> _tensor_str -> a D2H copy -> a stream synchronize, inside logging.emit, on the admission path. The device was occupied by a spinning ncclDevKernel_SendRecv, so the sync never returned. The instrument produced ZERO output on that boot: the record died mid-format, which is why the log looks like the branch never ran. THE TRAP, NAMED SO IT IS NOT RE-INTRODUCED. Every obvious way to print the value synchronizes: %s/str()/repr(), .item(), .cpu(), .tolist(), float(), int(), and f-string interpolation. A fix that swaps one for another only relocates the sync. sync_free_tensor_repr returns host-resident metadata instead -- shape, dtype, device, and id() to correlate one tensor handle across lines -- and passes non-tensors through unchanged, so a value that is sometimes a tensor and sometimes a plain int is safe either way. SWEPT THE FAMILY (sgl-project#695: expensive work inside logging arguments on serving paths), fixing six more sites of the identical shape: mamba_component.py sgl-project#767-TRACE prefix-match model_runner.py sgl-project#767-TRACE cow_and_clear SKIP and body, per extend forward pass -- hotter than the original site mamba_radix_cache.py cache_finished / cache_unfinished / match dflash_solo_pool.py:_reclaim THE ONE WITH NO GATE The dflash site matters most: every other hit sits behind an opt-in debug flag, that one runs unconditionally on decode-time draft-slot allocation under real load. It computed self._slot_epoch[victims].max().item() purely for a diagnostic string. Victims are drawn from the ascending argsort's low end, so their epoch is bounded above by self._epoch, which is already a host int -- and the message text is REWORDED to match what is now reported ("at round X ... untouched since before this round") rather than quietly printing a different number under the old wording. LISTED, DELIBERATELY NOT FIXED, with reasons rather than silence: phase_flip_output_trace.trace_round and phase_flip_resident_carry's cutover falsifier (both bounded on purpose, the second says so in its own comment), ngram_corpus.debug_result (reachable only from a __main__ demo), and dspark_planner._log_verify_lens_decision -- mechanically the same shape and genuinely hot when armed, but its whole purpose is the exact per-request values, so an identity stand-in would gut the tool rather than trim it. It wants rate-limiting, not this treatment, and is left as a named follow-up. TEST. test_admission_log_no_device_sync_790.py, hermetic and CPU-only. A tripwire monkeypatches Tensor.__repr__/__str__/item/tolist/cpu/__float__/ __int__ to raise, and the test drives the REAL HybridReqToTokenPool.alloc through the sgl-project#767 branch rather than a re-implementation. Red-first was demonstrated by reverting just the one call-site argument: the tripwire fires through alloc -> logger.warning -> emit -> format -> getMessage -> msg % args, reproducing the incident's exact path. Green after: 5 passed. A can-fail case restores the identity formatting to prove the trap still fires from inside the real branch. Verified with the sgl-project#788 and sgl-project#787 suites alongside: 12 passed. Pre-existing and NOT caused by this change: 4 failures in test_mamba_anchor_seams_747.py (AttributeError: 'MambaComponent' object has no attribute 'cache' in _raw_token_pos), confirmed identical with the change reverted. Ruff findings in mamba_radix_cache.py and model_runner.py outside these hunks are pre-existing lint debt. This is the probable cure for the isolation boot's wedge: the log sync was the linchpin edge of that cycle. It does not touch sgl-project#789 -- one relay, two transports, no shared readiness contract -- which remains open debt.

…oor scope readable Two env-gated / one-shot diagnostics, both OFF or free by default, added to turn a mechanism proof into captured evidence on ONE instrumented boot. WHAT IS UNPROVEN. A TP=1/PP=3 boot deadlocks on the first radix-carrying request, measured twice. py-spy --locals showed the last rank holding a ScheduleBatch and blocking in _pp_recv_proxy_tensors for its slot, while both upstream ranks carried cur_batch=None and reported themselves idle -- so the proxy it waits for is never produced. The mb_ids were the CORRECT -1 stagger, so this is not a slot desync; the ranks diverge on BATCH PRESENCE. The mechanism is understood: PP ranks are N independent schedulers agreeing only by determinism (requests are chain-forwarded unconditionally, but each rank re-derives admission locally, and the proxy send is gated on that rank's own batch), and the #616g uniformity floors that would keep them aligned are scoped to tp_cpu_group -- which has ONE member on every rank of a TP=1/PP=3 boot, so all three floors switch off. What is missing is a captured value showing the ranks actually disagreeing. These diagnostics produce it, or falsify the theory honestly. 1. Scheduler._trace_pp_admission_verdict, called from get_new_batch_prefill where the verdict is known, behind SGLANG_PP_ADMISSION_TRACE (default False). Prints ADMIT/DECLINE, request count, up to four rids, per-request prefix lengths, available and evictable size, queue length, running batch size and the chunked flag. Truncated on purpose: this is a divergence signal, not a batch dump, and a log flood has cost a feature here a self-kill before. 2. A one-shot line at the #616g early return naming the tp_cpu_group world size together with pp_size and tp_size, and stating that the evict, host and mamba floors are off. The comment at that return reads "One rank: nothing to diverge from" -- true for TP, false for PP -- so the condition is now readable from the boot log instead of inferred from source. HOST-SIDE VALUES ONLY, and deliberately so. sgl-project#790 was a diagnostic passing a CUDA tensor as a logging argument: the formatting forced a D2H copy and a stream synchronize inside logging.emit and wedged the scheduler for 25 minutes. Prefix length is therefore taken with len(), which reads shape and does not synchronize; there is no .item(), .cpu(), .tolist() or float() on this path. The trace is wrapped so that a failing instrument degrades to one warning instead of killing the scheduler it is measuring. Default path is unchanged when the variable is unset. Verified: the module imports, the method resolves with its annotations, the env default reads False, and a source scan of the compiled method reports no sync-forcing call.

…n, instead of re-deriving it per rank This wires the decision model (sgl-project#791), its congruence guard (sgl-project#630) and the readiness contract (sgl-project#789) together. None of them changed behaviour until now; each was landed standalone and green. THE DEFECT. PP ranks were N independent schedulers agreeing only by determinism. The request is chain-forwarded to every stage unconditionally (scheduler_pp_mixin.py:1069-1074), but each stage re-derived admission from its OWN queue and radix state (_get_new_batch_prefill_raw, scheduler.py:6377, first gate :6414-6417), while the proxy send is gated on that rank's own cur_batch (:488-501). So a rank that declined forwarded the request and could never send the proxy its downstream was blocking on. Measured twice, deterministically, on the first radix-carrying request after health: the last rank held a ScheduleBatch on its slot while both upstreams sat idle with cur_batch=None. The #616g uniformity floors that would have kept them aligned are scoped to tp_cpu_group, which has ONE member on every rank of a TP=1/PP=3 boot (scheduler.py:4693-4703), so all three were off. WHAT NOW HAPPENS. Rank 0 -- already the sole tokenizer-receipt point (request_receiver.py:143,197-219) -- builds the decision once per admission pass, with the congruence guard clamping the told prefix length to any learned floor. The decision travels as its own typed-channel kind "admission_decision": an ordered list of (rid, prefix_len, extend_len, admitted). Downstream ranks consume it instead of re-deriving. LENGTHS TRAVEL, POINTERS NEVER DO. prefix_indices are slot pointers into the deciding rank's own pool and are meaningless off-rank, so each receiver resolves the told length against its OWN radix tree. A receiver that cannot honour it excludes that request, logs one warning naming rank/rid/told/local, and leaves its siblings untouched -- it does not raise. That path is the ordinary cache-hit shape on this rig, and raising there would have traded a silent wedge for a crash on every cache hit. THE RETURN TRIP IS PART OF CORRECTNESS, not an extra. The chain-reconciled decision flows back to rank 0 (scheduler_pp_mixin.py:721) so the guard learns the observed coverage and, on a clean pass, CLEARS the rid's floor. Wiring the decision without it would leak a floor per rid and poison that request's reuse for the life of the process. ORDERING: sgl-project#791's degrade runs before sgl-project#789's contract can fire. The contract is the backstop for a genuine protocol violation; the ordinary divergence must never reach it. On a healthy boot its raise path stays unfired, and that is a success criterion of the next instrumented boot rather than an assumption. CONSTRAINTS HELD, CHECKED AGAINST THE DIFF RATHER THAN INTENDED: - No collective on the admission path. scheduler.py:6405-6407 documents a 2026-08-17 deadlock of exactly that family and warns against it verbatim; the diff introduces no all_reduce, all_gather, barrier or broadcast. - No device tensor reaches a logging argument (sgl-project#790). - pp_size == 1 is byte-identical: the guard is None below pp_size 2 (scheduler.py:1577). NOT DONE, DELIBERATELY: the two disaggregation PP loops (:618, :765) are untouched and still carry the original chain-flush hazard; the readiness contract does not cover them either, since it lives in the proxy receive path they do not use. That remains its own open item. Verified in ONE run over ten files, 190 passed against a 187 baseline -- the delta is exactly the new integration cases. The suite list is deliberately wider than the change: a verification list scoped to the change at hand is what let a merge regression through earlier today.

…h a boolean context `x or []` asks `bool(x)`. `req.prefix_indices` is a tensor of KV-pool slot pointers, and torch refuses that question at both ends of the range this code actually sees: an empty tensor (a request with no cached prefix -- the common case) raises "Boolean value of Tensor with no values is ambiguous", and a tensor with several matched pages raises the "more than one element" variant. Only a single-element prefix would ever have passed through silently, so the spelling was broken for very nearly every request that could reach it. Two sites, both on the admission path, both reachable only once a request is actually being admitted -- which is why they survived: until the send-handle fix in 2323c92 the ring wedged at idle, and no boot had ever executed them. - pp_admission_congruence.py:352, in `build_pp_admission_decision`. This ABORTED PP0 on its first real prefill (boot instr5, evidence-665-f1/boot_instr5.log:6126-6155); PP1 and PP2 then died on the broken connection. - scheduler.py:6338, in `_trace_pp_admission_verdict`. This branch runs only on an ADMIT, so the failure was silent and precisely inverted from useful: every idle DECLINE pass logged cleanly while every admitting pass -- the only ones that can show the ranks agreeing or diverging on a real request -- threw into the instrument's own except-and-swallow. Boot instr6 spent a GPU window to produce three lines reading "trace unavailable: RuntimeError" at exactly the pass the first request arrived. The method's own docstring already stated len() is the correct spelling because it reads shape without synchronising (sgl-project#790); the `or []` slipped in regardless. Also: that except now logs the exception MESSAGE and not just its type. It stays swallowed -- an instrument must never kill the scheduler it measures -- but a bare type name is a diagnostic dead end, and it cost a boot. Tests: - New test_pp_admission_prefix_indices_tensor_796.py: 9 passed. Covers all four observed shapes (empty tensor, multi-element tensor, absent/None, plain list) for the builder, and the ADMIT-path trace for the instrument. - Can-fail measured for both halves against the old spelling: 3 failed / 3 passed for the builder, 2 failed / 7 passed for the trace. The cases that fail are exactly the empty and multi-element ones, as predicted; a test covering only None and a one-element tensor would have passed against the defect. - Full test/registered/unit/managers at the previous commit: 2745 passed, 18 skipped, 305 subtests passed, 47 failed -- all 47 in the same three files baselined at HEAD, which fail identically without any of these changes.

…what the tree handed over Boot instr10 died 4m55s after health, 4 seconds after a tp_to_pp flip: RuntimeError: Out of memory. Try to allocate 512 tokens. Available full tokens: 138089 (full_available_size=189 + full_evictable_size_=137900) Eviction reported delivering >= 512 tokens and the allocator then had 189 free. The proof is an absence: _eviction_shortfall_note returns "" if and only if evicted >= asked, and the note is missing from the specimen. The one diagnostic written to explain this failure silenced itself on it. MECHANISM. --kv-backing-relief is on. KvRowCap.engage subscribes to the allocator's free listener (kv_backing_relief.py:379) and KvRowCap._apply (:490-512) moves every freed id above the cap straight back out of free_pages into _withheld -- correctly, since those rows' pages are unmapped. available_size (allocator/token.py:52-54) counts neither. FullComponent. evict_component (full_component.py:115-119) takes its count the instant it hands the free over -- `self._free_full(cd.value); freed = len(cd.value)` -- and never checks receipt. So the tree's books moved by exactly the ask while the pool's did not move at all. Measured on the boot: the cap engaged at 01:53:40 backing 137135 rows instead of 161792, withholding 24243 ids; the flip 71 s later re-seeded 160822 live slots across the whole id space, putting the peel's frontier precisely on the rows the cap confiscates. Not an admission defect: admission's 138412 evictable was real, ~114k of it below the cap and payable throughout. The peel stopped after one round because it was told it had been paid. A tighter bound computed from the same wrong receipt would have admitted the batch too. CHANGES. payable_size() is the delivery measure -- available_size() plus what an open free group still owes (so sgl-project#681's staging still reads as the delivery it is) and never withheld ids. alloc_token_slots measures `delivered` as the payable_size delta across evict_from_tree_cache and feeds that to the note; evict_from_tree_cache's own return contract is untouched, so existing callers are unaffected. New rung _evict_past_confiscation re-peels past the cap after the sgl-project#681 flush and before the relief ladder, spending only recomputable prefix, escalating on rounds that pay nothing, bounded at 8 rounds. It REFUSES under an active uniform_avail_floor: the round count is rank-local and that is the #616g divergence exactly (precedent: uniform_host_floor_active). The note now names the confiscator and drops its "THIS LINE SHOULD BE UNREACHABLE" claim, which this specimen falsifies. TESTS. New test/registered/unit/mem_cache/test_residency_cap_eviction_790.py uses a real TokenToKVPoolAllocator on CPU and a real KvRowCap as the confiscator; the tree stand-in is deliberately as careless as FullComponent, so no double supplies the guarantee whose absence is the bug. fix reverted, test present -> 3 failed / 6 passed, reproducing the specimen (same "NO relief provider is registered" warning) with the fix -> 9 passed common.py's direct dependents (679, 681 x3, 694, 616g, 631 backing relief) -> 134 passed, 7 subtests passed ruff and codespell clean on all three files.

…instead of the group minimum Boots instr10, instr12 and instr13 all died in round 6 or 7 of a flip drive with an allocation that eviction could not pay. sgl-project#790 made the failure name itself: EVICTION UNDER-DELIVERED: asked for 512 tokens, the pool received 94. The tree still reports 67674 evictable tokens. A RESIDENCY CAP IS ENGAGED and is holding 63641 slot ids out of the allocator's free list. WHICH CAP, settled by arithmetic rather than by inspection. Above the shrink's cap of 137233 the id space holds only 161792 - 137233 = 24559 ids in total, so a confiscator holding 63641 cannot be that one. Above the LEVELLED cap of 40960 it holds 120832, and 63641 fits. The cap that kills was installed 95760 rows BELOW the highest live row (136720), four seconds after the cutover: 05:28:17 KV-BACKING released 160 MiB by backing 137233 rows instead of 161792 (highest live row 136720, 24145 ids withheld) 05:28:21 KV-BACKING cap agreement: exposed rows 137233 -> 40960 (group level 40960) 05:29:28 RuntimeError: Out of memory ... the pool received 94 ... holding 63641 slot ids ROOT. phase_flip_spill.recover_kv_backing (:1167-1173) reduced only [backed, -backed] and levelled every rank to the group's minimum BACKED ROWS via level_recovery_to -> reconcile_to (kv_backing_relief.py:2224+), with no reference to the live set at all. The cap therefore lands under the rows the radix tree still holds, KvRowCap._apply (:490-512) confiscates every id the peel frees above it, and the pool can never be paid again. The law was already written one function away: collective_cap_target (:227-230) returns None when the group's MAX floor exceeds the MIN capable. The recovery levelling was the one path in the chain that never made that trade. FIX. New public live_floor_rows() (:2113) exposes the reading the levelling needed and had no way to ask for; reconcile_to declines and logs when level < floor instead of engaging (:2260-2325); recover_kv_backing's payload widens to [backed, -backed, -floor] so the group can make collective_cap_target's trade, and declines loudly when the group's MAX floor is above the MIN backing, a truncated payload included. WHY NOT THE TWO OBVIOUS ALTERNATIVES. Releasing the cap at the cutover re-admits ids over pages the shrink genuinely unmapped -- the cudaErrorIllegalAddress that reverted c4e5579, quoted in _shrink_to. That trades an OOM which raises for a fault that kills every rank without raising. Refusing the carry is mis-ordered: the carried slots are not seeded, they are already live, so refusing them drops a live request's KV -- and on this specimen the cap is engaged AFTER the cutover by the recovery hook, so there is no carry left to refuse. WHAT THIS DOES NOT FIX, stated plainly. The levelling now DECLINES where it used to kill, which means divergent id spaces and flips refused by the frame ballot -- lost flips and reduced admission, never a dead rank. Getting those flips back needs PP2's capacity deficit funded: measured on instr13, PP2 alone runs a shortfall (need = floor 3034 + delta 256 + want 686 = 3976 MiB against free 3210 MiB -> deficit +766 MiB) while PP0 and PP1 sit on surpluses of 4644 and 3408 MiB. That is a planner item, tracked in evidence-665-f1/BUG_planner_corridor_capacity.md, and no hand pin is applied here. TESTS. test/registered/unit/mem_cache/test_residency_cap_flip_levelling_792.py, new, 12 tests on a real KvRowCap, a real TokenToKVPoolAllocator, a real KvBackingRelief and the real recover_kv_backing; only the VMM arena is a stub. fix reverted -> 9 failed / 3 passed, reproducing the specimen text verbatim behavioural neuter of the reconcile_to guard alone -> 3 failed / 9 passed, 0 AttributeError with the fix -> 12 passed; with neighbours 656 and 790 -> 46 passed TheLevellingStillHappensWhenItIsHonest passes in BOTH directions on purpose: it goes red if the fix disables sgl-project#656 C22-e instead of bounding it. ruff, codespell and black clean on all four files; the 2 test_black_ratchet_656.py failures are pre-existing at HEAD.

…eriving it (slice 1) Ten fixes on this branch (sgl-project#757 sgl-project#789 sgl-project#790 #791b #791c sgl-project#792 sgl-project#795 sgl-project#797 #797b #797c) all have ONE form: each rank RE-DERIVES the pass schedule -- rid set, chunk length, prefix length -- locally from its own state, and a phase flip invalidates that state non-atomically. Every "new root" was the next consumer of the same re-derivation, so the list grew instead of converging. This stops patching consumers. THE DATUM WAS ALREADY ON THE WIRE AND NOBODY READ IT. PPAdmissionEntry.extend_len (pp_admission_congruence.py:177) has crossed the wire since sgl-project#791's first commit with exactly three consumers: to_wire (:223), from_wire (:244), one log string (:824). Nothing ever built a batch from it; reconcile_pp_admission_decision returns Dict[rid, prefix_len] and drops the second number on the floor. THE MECHANISM, CORRECTED AGAINST THE LOG rather than assumed. boot_instr20.log:5171,5181-5183: PP0 ADMIT rid=6cbe2733 prefix_lens=0 chunked=1 -> 512-row chunk PP1 ADMIT rid=6cbe2733 prefix_lens=512 chunked=0 -> 333-token remainder PP1 DID receive prefix_len=0 and DID clamp prefix_indices to it -- scheduler.py:7026-7027 worked. Then add_one_req's HOST LOAD-BACK put the 512 back: needs_host_load_back() went true when the HiCache prefetch landed and schedule_policy.py:1539-1549 concatenates the recovered indices. "MAMBA-HOST-RESUME ... triggers load_back" appears on PP1 and PP2 and is ABSENT on PP0 -- that asymmetry is the bug. 845-512=333 then fitted rem_chunk_tokens whole, so the NON-chunked branch fired. The re-derivation lives INSIDE THE ADDER, after the schedule was already applied, and it re-derived BOTH numbers. DESIGN. forwarded_schedule() (pp_admission_congruence.py:500) is the pass geometry as a value: rid -> (prefix_len, extend_len) for exactly the rids `effective` names; None or a voided decision yields {}. _add_scheduled_req (schedule_policy.py:1237) EXECUTES both numbers: no rem_chunk_tokens, no page/align rounding, no host load-back, no budget veto -- the budget is still charged. The gate sits above every local veto (:1600), and add_chunked_req (:1328) gains the gate it never had at all (it is entered from scheduler.py:7004, BEFORE the admission loop). The three membership vetoes that silently narrowed -- batch_is_full (:7035), the HiCache prefetch_done skip (:7047, the instr20 race itself) and the LoRA gate (:7024) -- become refusals (scheduler.py:7190). REFUSAL IS CONTROL FLOW, NOT A RESULT CODE. PPScheduleRefused (:163) is an exception because every AddReqResult means "build a batch without this request", which is precisely the corruption. A refusal reuses sgl-project#797 end to end (_pp_refuse_forwarded_schedule, scheduler.py:6333 /:6369): sets _pp_admission_pass_voided, voids the forwarded decision, re-notes the slot expectation. No new mechanism. Inside the loop a refusal is CARRIED, not thrown, so alloc_group_end() still runs (:7025, :7118, :7169). WHY THE GUARDS CAN NO LONGER FIRE, each owed a reason: sgl-project#631's _want is extend_num_tokens, which now comes only from the forwarded extend_len with load-back suppressed, so it IS the upstream's row count by construction rather than by agreement (green arm: rows=512, batch_tokens=512). sgl-project#757/sgl-project#787 stamp and sgl-project#795 epoch were already structural; what changes is that they can no longer be correct-but-insufficient, as instr17 and instr20 both were -- every identity right, only the width wrong. Width is now an identity too. sgl-project#789 needs a membership divergence, which is now identical-or-refused. #791c's tripwire detects a self-narrowed batch, and no path creates one. HONEST CORRECTION TO THE SUBSUMPTION CLAIM: retraction does NOT become unnecessary. Physical impossibility is real -- a rank genuinely lacking KV for [local, told) cannot execute. What becomes structurally impossible is the NARROW-THEN-DETECT shape: no code path is left that builds a batch of a geometry the upstream did not name. NOT COVERED BY THIS SLICE, stated so nobody assumes otherwise: - BATCH ORDER. can_run_list follows the local waiting_queue order, the decision follows PP0's. Same rid set in a different order gives EQUAL WIDTHS and permuted rows -- silent. Covered by the full design (execute in decision order), not by this slice. - Decode batches: retract_decode (scheduler.py:7489/:7517) mutates long-lived state on a tp_cpu_group reduce that is world=1 under TP=1/PP=3. Different root, filed by sgl-project#797. - Radix eviction divergence, KV pool sizing, spec-decode draft schedules. NAMED RESIDUAL, filed at the site (scheduler.py:7118) in sgl-project#797's practice: a request admitted earlier in a loop that later refuses has taken a persistent inc_lock_ref, released on batch completion. Undoing it needs the exact IncLockRefResult (SWA/Mamba tombstone params) the adder does not keep, and a blind release makes the one thing a mismatched release worsens. Bounded: reaching that line takes a genuinely unexecutable geometry and kills the pass. DEFAULT PATH UNTOUCHED: _pp_scheduled_extents() returns None on PP0 and on every pp_size<=1 boot, so scheduled_extent_for returns None and both adders take the pre-existing arithmetic unentered; PPScheduleRefused is unraisable there. Pinned by test_no_mapping_is_the_untouched_default_path and corroborated by 56/56 on the schedule_policy neighbours. TESTS. test_pp_forwarded_schedule_791.py: 17 passed (3 live gloo arms + 14 pure), 92 s. test_red_without_the_forwarded_geometry_instr20_reappears -- can-fail, rebinding ONLY the fix's return value in the child: batch=(512,333) rows=512 mismatch=True, byte-identical to instr20 PP1 09:40:30 test_green_the_forwarded_geometry_survives_the_mid_pass_prefetch -- batch=(0,512) rows=512 mismatch=False, load_back_calls=0 test_an_impossible_geometry_raises_rather_than_narrowing -- the architectural property Neighbour set (791/791b/791c/797/631 x3) re-measured AT HEAD: 11 failed / 75 passed; after: 11 failed / 92 passed, failure names byte-identical, zero regressions, +17. schedule_policy neighbours 56 passed / 0 failed. ruff 119 = 119 at HEAD (parity), ruff format clean on everything authored, codespell identical.

Carries [sgl-project#790] carry-instrument gating and [sgl-project#777] threshold honesty. feat/797 is an ancestor of this head, so the previous stage is subsumed; kept as its own merge for per-ticket attribution. Clean merge. Touches phase_policy.py, which sgl-project#817/sgl-project#820 also touch -- those branch from 587e4c2 and are merged after, see their stage notes.

Closes the Cluster 4 defect that sgl-project#815 escalated rather than fixed: the sgl-project#677 HOLD wrapper swallowed the blocked-admission exit. sgl-project#817 inverts it into an allowlist exactly as the wrapper's own comment prescribed. Clean merge against the sgl-project#790 phase_policy.py edits.

…t#816, sgl-project#810, sgl-project#806, sgl-project#797, sgl-project#790/sgl-project#777, sgl-project#817/sgl-project#820, sgl-project#818 Merge-checklist duty for the nine stages on this branch. Entries follow each section's own house style, and the mechanism text is taken from the commits' measured evidence rather than restated from the ticket titles. §3 KV backing relief + the allocator cap -- UPDATED IN PLACE rather than given a second bullet, because sgl-project#814 and sgl-project#816 are follow-on defects OF the KvRowCap mechanism that bullet already describes: the census reading the withheld block as a leak (340262 of 465190 ids), the lift being reachable only from a cutover (one boot at 26.8% of its id space for the life of the process), and exposure exceeding the backing (417850 rows over 105413 committed, the device-side assert in masked_set_kv_buffer_kernel). §3 HiCache staging write-through ring (sgl-project#810), new bullet, plus its two companion refusals -- the unbounded-file-tier refusal and the boot preflight ledger entry, the latter being why 22.01 GB of MHATokenToKVPoolHost across three PP ranks previously reached the preflight as nothing. §7 BAR1 deadline + loud abort -- appended the sgl-project#818 peer-liveness half to the existing narrative: the gate could wait forever on a peer that no longer exists, and neither Bar1CollectiveStalled (reset by every resolved read) nor defer_stall_for_building_peer (900 s off a build marker) caught it. §12 Robustness canon -- three new families: contradictory-flag (sgl-project#806), read-back-after-construction (sgl-project#797), denylist-of-reasons (sgl-project#817, sgl-project#820). §18.3 hicache staging sizing (sgl-project#810) -- §18's own rule is that a merge adding a reusable module adds its entry in the SAME merge, and this module had none. Records the removal of fits_pinned_host_budget so it is not reintroduced. §18.6 mamba carry instrument (sgl-project#767, gated by sgl-project#790) and flip break-even N (sgl-project#777). No existing entry was contradicted. Checked before writing: none of sgl-project#814, sgl-project#810, sgl-project#806, sgl-project#772, sgl-project#797, sgl-project#790, sgl-project#777, sgl-project#817, sgl-project#820, sgl-project#818 had a catalog entry, and the one sgl-project#677 line (§19.2, RESTORE-NEVER-REBUILD) describes a different mechanism than the sgl-project#677 layout hold, so it is not stale and was left alone. Gates for the tree this documents (hermetic, CUDA_VISIBLE_DEVICES=""): battery test/registered/unit/{managers,planner,server_args,mem_cache} baseline integ @ 78d27da 44 failed c56d238 (through sgl-project#818) 30 failed, 8452 passed 0 new ids, 14 fixed 1c4eadb (through sgl-project#816) 30 failed, 8461 passed 0 new ids, same set test_barlink_abort_gate_liveness_818.py (outside the battery dirs) 10 passed ruff --select=F401,F821,UP037 and codespell: 0 new findings vs the same file set on 78d27da (16 ruff / 6 codespell exist identically on base) docs-only change; codespell clean on the catalog itself

chore: add copyright for srt

0651122

zhyncs requested review from Ying1123, hnyls2002 and merrymercy July 28, 2024 13:06

zhyncs merged commit dd7e8b9 into sgl-project:main Jul 28, 2024

zhyncs deleted the license branch July 28, 2024 13:07

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

chore: add copyright for srt (sgl-project#790)

45a52fd

Navigation Menu

Uh oh!

chore: add copyright for srt - #790

chore: add copyright for srt#790
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:license

zhyncs commented Jul 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

zhyncs commented Jul 28, 2024

Motivation

Modification

Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant