docs: make badges center by zhyncs · Pull Request #789 · sgl-project/sglang

zhyncs · 2024-07-28T12:27:43Z

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
Modify documentation as needed, such as docstrings or example tutorials.

fix 0613

…ion, not before the exchange A boot reached serving, one admissible queued request was never served, and the process died. The wedge detector reported "1 queued, 0 running, NO first token for 340-370s and no prefill chunk either" and named its own gap in the same line -- "no phase-policy corroboration seen -- the wedge class is broader than that path". It was right: this is not an admission defect at all. EVIDENCE OF RECORD. py-spy on the specimen (WEDGE_788_specimen.log:732/915/1083) catches all three ranks in a circular wait: PP0, PP1 gloo waitSend _pp_commit_comm_work (scheduler_pp_mixin.py:2465) from _pp_forward_and_process_input_requests:1007 PP2 gloo waitRecv _pp_recv_typed_dict:2709 from _pp_recv_proxy_tensors:2803 The request chain was flushed at the TOP of the pass, before the rank had posted anything else it owed its peers that iteration -- the proxy tensor-dict send and the output-ring send both come later in _event_loop_pp_body. A downstream whose progress needs one of those was therefore waiting on a rank that was itself waiting on that downstream. The chain recv is posted only at the top of a pass (:420) and nowhere else in the loop body, so a rank parked in the proxy receive cannot drain the message its upstream is blocked flushing. THE FIX IS ONE ADDITION. _pp_forward_and_process_input_requests is byte-identical to before: it still commits send_req_work, then posts the async forward, then runs process_input_requests. What is new is _pp_commit_pending_req_work, called once per iteration from _event_loop_pp_body after the proxy isend and the output commit and BEFORE the phase-flip round hook. By the time that loop returns to the top-of-pass commit, the handle has already been waited on and cleared, so the old call site is a no-op there -- and the two disaggregation loops (:618, :765), which do not call the new method, keep their previous behaviour exactly. Deleting rather than moving the old call site is deliberate on both counts: it keeps the sgl-project#633 ordering contract literally intact (commit, then forward, then process, with send_req_work holding this pass's handle on return -- pinned by test_scheduler_pp_request_order_633), and it avoids a second staging slot. An earlier shape of this fix did stage the send separately; it broke that contract and left the two disagg loops overwriting a live P2PWork handle every pass. Precedent: a7ff250 "[sgl-project#753] Flush the output sends after the exchange, not before it" made the same move for the OUTPUT channel. This is that fix for the REQUEST channel, ungated -- the hazard is not gapped-specific. NEGATIVE FINDING, RECORDED RATHER THAN BURIED. A deterministic 3-process gloo reproduction of the specimen deadlock was attempted and NOT achieved under a faithful model of the loop. Two candidate mechanisms were tested and refuted: - Eager-send asymmetry. Hypothesis: an empty request list puts one 8-byte tensor on the wire and completes eagerly, so only a large first request can block. MEASURED FALSE: a 2-process probe with a 2.0s-delayed matching recv blocked for the full delay at every payload size from 8 B to 1 MiB in this build. No eager completion at any size. - Schedule asymmetry. A linear chain+proxy relay that flushes the old handle before dispatching the new one is deadlock-free by induction (each rank's flush needs only its immediate downstream one pass behind), and dozens of real 3-process runs across depths and offset combinations all completed. So the reduced two-channel model cannot close a cycle. The production graph has an edge that model lacks: the output ring closes last-rank -> rank 0 (_pp_send_recv_and_preprocess_output_tensors:3038, last-rank branch :3012-3014, via next_first_rank_mb_id at :416/:608/:754). That, and real GPU compute skew, are the two named unconfirmed candidates for the missing ingredient. Neither is built into the test, and the gap is stated in its docstring rather than papered over. The mechanism's evidence of record therefore remains the py-spy trace above plus the sgl-project#753 precedent, and the survival boot is the integration proof. RECOVERY RUNG. The detector gains one bounded action: after a wedge stays continuously alarming past a threshold well above the report threshold (default 3x, env SGLANG_ADMISSION_WEDGE_RECOVERY_SECONDS; non-positive values fall back to the default rather than firing every poll), it makes ONE forced-admission attempt per episode through the existing corridor_admission actuator and logs it loudly either way. Its own docstring is explicit that it relieves VRAM pressure at the admission site and will NOT move a comms deadlock like this one, and that calling an actuator from the watchdog thread is a new cross-thread shape over CUDA allocator state. TESTS test_pp_chain_flush_deadlock_788.py (new; 3 real gloo processes, CPU only) - load-bearing: runtime ordering pin. The SHIPPED _pp_commit_pending_req_work is wrapped and the observed sequence asserted to be proxy_send -> chain_flush -> round_hook for every pass and every non-last rank, so the flush cannot drift back past the collective and cost it the "last blocking op of the iteration" property its own comment relies on. - can-fail: the same run with the flush relocated to the top records a different sequence, so the pin discriminates placement. - the honest negative case described above, in place of the deleted hang-repro. A test that is green or red for the wrong reason is worse than no test. test_pp_flip_leftover_proxy_757.py: harness-only repair, no assertion touched. It was silently DEAD on this branch -- its _GlooWire predated rank_in_group/world_size and the src positional recv_typed_tensor_dict now passes, so it errored 3/3 before reaching an assertion. Now 3 passed and a working regression check again. Verified green with this diff: test_pp_chain_flush_deadlock_788 and test_scheduler_pp_request_order_633 (8 passed), test_phase_policy (90 passed), test_pp_flip_leftover_proxy_757 (3 passed). PRE-EXISTING FAILURES, MEASURED NOT ASSUMED. The managers slice reads 43F/2707P here against 39F/2705P at a pristine detached ff5651a worktree. Every delta is accounted for: -3 (the 757 repair above now passes), +5 stub drift and +1 ordering-contract break, both introduced by the earlier staging-slot shape and both gone with it, +1 test_pp_drain_completeness_787 which is INTENTIONALLY RED here -- that file's fix lives on fix/787-drain-completeness and it flips green at the merge, which makes it a free integration can-fail. The residual 39 are the same interface-drift family (rank_in_group, _pp_gapped_wire) plus one stale constant, all present at base with no part of this diff. DEBTS sgl-project#789 the two disaggregation PP loops keep the top-of-pass flush and therefore still carry this hazard sgl-project#757-harness-drift repaired here; the same drift still blinds test_pp_proxy_stamp_631 and test_pp_slot_last_batch_631 sgl-project#753-stale-constant 10.923 != 11.923 in the gapped entry protocol test sgl-project#766-pointer-stale the register points sgl-project#766 at ARM_defaultfull7.log, which is a 26-line host-ledger snapshot with no Bar1CollectiveAborted and no proxy/drain lines

…ving hot paths An isolation boot wedged for 25+ minutes with 120 ADMISSION-WEDGE markers. py-spy (two dumps 25 min apart, byte-identical, so hard-stuck) put PP0's MainThread INSIDE logging.emit while PP1 and PP2 starved in pp_chain_receiver.recv for a chain send PP0 never reached. ROOT CAUSE. PP0 admitted the first radix-carrying request and reached the sgl-project#767 instrument in HybridReqToTokenPool.alloc, which logged req.mamba_pool_idx -- a 1-element CUDA tensor -- as a %s argument. The format call runs Tensor.__repr__ -> _tensor_str -> a D2H copy -> a stream synchronize, inside logging.emit, on the admission path. The device was occupied by a spinning ncclDevKernel_SendRecv, so the sync never returned. The instrument produced ZERO output on that boot: the record died mid-format, which is why the log looks like the branch never ran. THE TRAP, NAMED SO IT IS NOT RE-INTRODUCED. Every obvious way to print the value synchronizes: %s/str()/repr(), .item(), .cpu(), .tolist(), float(), int(), and f-string interpolation. A fix that swaps one for another only relocates the sync. sync_free_tensor_repr returns host-resident metadata instead -- shape, dtype, device, and id() to correlate one tensor handle across lines -- and passes non-tensors through unchanged, so a value that is sometimes a tensor and sometimes a plain int is safe either way. SWEPT THE FAMILY (sgl-project#695: expensive work inside logging arguments on serving paths), fixing six more sites of the identical shape: mamba_component.py sgl-project#767-TRACE prefix-match model_runner.py sgl-project#767-TRACE cow_and_clear SKIP and body, per extend forward pass -- hotter than the original site mamba_radix_cache.py cache_finished / cache_unfinished / match dflash_solo_pool.py:_reclaim THE ONE WITH NO GATE The dflash site matters most: every other hit sits behind an opt-in debug flag, that one runs unconditionally on decode-time draft-slot allocation under real load. It computed self._slot_epoch[victims].max().item() purely for a diagnostic string. Victims are drawn from the ascending argsort's low end, so their epoch is bounded above by self._epoch, which is already a host int -- and the message text is REWORDED to match what is now reported ("at round X ... untouched since before this round") rather than quietly printing a different number under the old wording. LISTED, DELIBERATELY NOT FIXED, with reasons rather than silence: phase_flip_output_trace.trace_round and phase_flip_resident_carry's cutover falsifier (both bounded on purpose, the second says so in its own comment), ngram_corpus.debug_result (reachable only from a __main__ demo), and dspark_planner._log_verify_lens_decision -- mechanically the same shape and genuinely hot when armed, but its whole purpose is the exact per-request values, so an identity stand-in would gut the tool rather than trim it. It wants rate-limiting, not this treatment, and is left as a named follow-up. TEST. test_admission_log_no_device_sync_790.py, hermetic and CPU-only. A tripwire monkeypatches Tensor.__repr__/__str__/item/tolist/cpu/__float__/ __int__ to raise, and the test drives the REAL HybridReqToTokenPool.alloc through the sgl-project#767 branch rather than a re-implementation. Red-first was demonstrated by reverting just the one call-site argument: the tripwire fires through alloc -> logger.warning -> emit -> format -> getMessage -> msg % args, reproducing the incident's exact path. Green after: 5 passed. A can-fail case restores the identity formatting to prove the trap still fires from inside the real branch. Verified with the sgl-project#788 and sgl-project#787 suites alongside: 12 passed. Pre-existing and NOT caused by this change: 4 failures in test_mamba_anchor_seams_747.py (AttributeError: 'MambaComponent' object has no attribute 'cache' in _raw_token_pos), confirmed identical with the change reverted. Ruff findings in mamba_radix_cache.py and model_runner.py outside these hunks are pre-existing lint debt. This is the probable cure for the isolation boot's wedge: the log sync was the linchpin edge of that cycle. It does not touch sgl-project#789 -- one relay, two transports, no shared readiness contract -- which remains open debt.

…t honour it opt out A TP=1/PP=3 boot deadlocks on the first radix-carrying request, measured twice. py-spy --locals: the last rank holds a ScheduleBatch and blocks in _pp_recv_proxy_tensors for its slot, while both upstream ranks carry cur_batch=None and report themselves idle. The mb_ids were the correct -1 stagger, so this is not a slot desync -- the ranks diverge on BATCH PRESENCE. WHY THAT CAN HAPPEN AT ALL. PP ranks are N independent schedulers that agree only by determinism: the request is chain-forwarded to every stage unconditionally (scheduler_pp_mixin.py:1069-1074), but each stage re-derives the admission verdict from its OWN queue and radix state (_get_new_batch_prefill_raw, scheduler.py:6377, first gate 6414-6417), and the proxy send is gated on that rank's own batch (scheduler_pp_mixin.py:488-501). A rank that declines therefore forwards the request but can never send the proxy its downstream is blocking on. The #616g uniformity floors that keep ranks aligned are scoped to tp_cpu_group, which has ONE member on every rank of a TP=1/PP=3 boot, so all three floors switch off (scheduler.py:4693-4703 -- its comment "One rank: nothing to diverge from" is true for TP, false for PP). WHAT THIS COMMIT ADDS: the decision model, standalone and not yet wired. PP0 is already the sole tokenizer-receipt point (request_receiver.py:143, 197-219), so it decides membership and per-request prefix length once, and the decision travels instead of being re-derived. THE DECISION CARRIES LENGTHS, NEVER POINTERS. req.prefix_indices are real slot pointers into the deciding rank's own KV pool (match_prefix_for_req, schedule_batch.py:1296-1314) and are meaningless off-rank. Prefix length drives KV REUSE, not accounting: prepare_for_extend (schedule_batch.py:2262-2266, 2306-2311) sizes out_cache_loc and, through extend_num_tokens, the activation that crosses the stage boundary. A wrong length corrupts -- garbage KV rows, or a crossing whose row count disagrees between stages even when the admitted SET agrees. Hence the asymmetric rule: local >= told truncate to told. Safe; gives up some legitimate local reuse, the same slack trade #616g already makes on the TP axis. local < told this rank cannot honour it, and the physical reason is documented: it cannot compute the missing KV in-pass, because only stage 0 holds the embedding. THE local<told PATH DEGRADES, IT DOES NOT RAISE, and that is the point. local<told is not an exotic corner on this shape -- it is what an ordinary cache hit looks like. A path that raised there would trade a silent wedge for a crash on every cache hit, which is worse than the dark cache we started from. So the request is EXCLUDED from the effective decision, exactly one warning names rank/rid/told/local, its siblings in the same batch are untouched, and the exclusion propagates without re-warning. TESTS: test_pp_admission_congruence_791.py, hermetic, 11 passed. A neutered simulation reproduces genuine three-rank divergence (80/136/200 rows), so the tests fail for the real reason rather than by construction. NOT WIRED YET, deliberately. The admission path and event loop are untouched, scheduler_pp_mixin.py is under a scope fence held by the sgl-project#789 readiness-contract work, and pp_typed_channel.py has zero edits -- the intended carrier is the typed tensor-dict channel under a new kind "admission_decision", which is keyed per mb_id and already carries a non-tensor stamp. Wiring lands with the re-queue path and the retry pin. KNOWN OPEN HOLE, named here rather than discovered later: nothing yet forces told=0 when an excluded request is retried. PP0 would re-match the same prefix, the downstream would exclude it again, and the wedge would come back as a retract loop. The retraction has to flow back to PP0 and pin that rid's next decision to told=0 (one-shot, rid-scoped) -- or PP0 learns downstream coverage from the retraction, which is already half of the congruence guard. That is a blocker for wiring, not for landing this module.

…evidence a message is coming A PP rank could enter an unbounded blocking receive for a proxy that nobody would ever send. Measured twice: the last rank holds a ScheduleBatch and blocks in _pp_recv_proxy_tensors for its slot, while both upstream ranks carry cur_batch=None and report themselves idle -- so no rank will produce the message it waits for. Health keeps answering, forward counters keep looking sane, and the process sits there until it is killed. THE CONTRACT. _pp_wait_for_proxy_readiness runs immediately before the blocking receive and demands POSITIVE evidence: the CHAN_DICT sent counter having moved past local_consumed, i.e. the upstream provably posted. On a healthy boot that evidence is already there and the function returns at once, so the receive path is unchanged. IT IS A PRESENCE SIGNAL, NOT A TIMER, and that distinction is the whole design. "Wait N seconds then give up" is sgl-project#630, a livelock this codebase already has on record: ranks that time out, retry, time out again, make no progress and look alive throughout. Here the budget is only a backstop -- the raise fires solely when the budget is fully exhausted AND the counter never moved once. A timing-out Work.wait() on the transport was never an option either: a timed-out gloo wait destroys the pair, so the peer then sees "Connection closed by peer". The inbox is consulted before the counters, and that case was found by running the regression rather than by foresight: a message already stashed for this exact consumer (for instance one the armed drain took off the wire) leaves the counters reading "caught up", so counters alone would have called a delivered message absent. TRANSPORT COVERAGE, STATED RATHER THAN IMPLIED. This covers "the upstream never posted" uniformly across the gloo-metadata and NCCL-payload halves, because a sender that never got there never advanced its counter either. It does NOT cover "posted, then stalled mid-transfer". That limitation is written into the docstring; the measured specimens are all the never-posted case. DEFAULT PATH UNCHANGED: with pp_flip_counters None -- an ordinary boot without phase flip -- the check is a no-op, verified directly rather than assumed. Budget 30 s via SGLANG_PP_PROXY_READINESS_BUDGET_S, tagged HAND PIN sgl-project#789. THIS IS A BACKSTOP, NOT THE HANDLER FOR THE COMMON CASE, and it must not be wired alone. The ordinary divergence on this shape -- one rank admits, the others do not -- would otherwise reach this contract, never move the counter, and raise after the budget: a crash on every cache hit, which is only a bounded, diagnosable version of the same fatality. That case belongs to sgl-project#791's decision model, which excludes a request no rank can honour and degrades it instead. sgl-project#791 must therefore be wired ahead of this, so that by the time the contract can fire, something genuinely different is broken. TESTS. test_pp_proxy_readiness_contract_789.py, 4 green. Its neutered case is the red and the can-fail in one: with the readiness call patched out the path is byte-identical to the pre-sgl-project#789 code, and the victim rank genuinely hangs against two idle upstreams -- proven a hang rather than a fast error by a bounded outer driver that finds it alive at the deadline with no result. Disclosed: the source edits preceded the red-first test, an ordering deviation. It is remediated by that neutered case being exactly the pristine path, not by argument. The one-line harness repairs to test_pp_drain_completeness_787 and test_pp_flip_leftover_proxy_757 bind the new method on holders that predate it; no assertion was touched. Verified together in one run: 30 passed across the 789, 787, 757, 788, 633 and 791 suites.

…n, instead of re-deriving it per rank This wires the decision model (sgl-project#791), its congruence guard (sgl-project#630) and the readiness contract (sgl-project#789) together. None of them changed behaviour until now; each was landed standalone and green. THE DEFECT. PP ranks were N independent schedulers agreeing only by determinism. The request is chain-forwarded to every stage unconditionally (scheduler_pp_mixin.py:1069-1074), but each stage re-derived admission from its OWN queue and radix state (_get_new_batch_prefill_raw, scheduler.py:6377, first gate :6414-6417), while the proxy send is gated on that rank's own cur_batch (:488-501). So a rank that declined forwarded the request and could never send the proxy its downstream was blocking on. Measured twice, deterministically, on the first radix-carrying request after health: the last rank held a ScheduleBatch on its slot while both upstreams sat idle with cur_batch=None. The #616g uniformity floors that would have kept them aligned are scoped to tp_cpu_group, which has ONE member on every rank of a TP=1/PP=3 boot (scheduler.py:4693-4703), so all three were off. WHAT NOW HAPPENS. Rank 0 -- already the sole tokenizer-receipt point (request_receiver.py:143,197-219) -- builds the decision once per admission pass, with the congruence guard clamping the told prefix length to any learned floor. The decision travels as its own typed-channel kind "admission_decision": an ordered list of (rid, prefix_len, extend_len, admitted). Downstream ranks consume it instead of re-deriving. LENGTHS TRAVEL, POINTERS NEVER DO. prefix_indices are slot pointers into the deciding rank's own pool and are meaningless off-rank, so each receiver resolves the told length against its OWN radix tree. A receiver that cannot honour it excludes that request, logs one warning naming rank/rid/told/local, and leaves its siblings untouched -- it does not raise. That path is the ordinary cache-hit shape on this rig, and raising there would have traded a silent wedge for a crash on every cache hit. THE RETURN TRIP IS PART OF CORRECTNESS, not an extra. The chain-reconciled decision flows back to rank 0 (scheduler_pp_mixin.py:721) so the guard learns the observed coverage and, on a clean pass, CLEARS the rid's floor. Wiring the decision without it would leak a floor per rid and poison that request's reuse for the life of the process. ORDERING: sgl-project#791's degrade runs before sgl-project#789's contract can fire. The contract is the backstop for a genuine protocol violation; the ordinary divergence must never reach it. On a healthy boot its raise path stays unfired, and that is a success criterion of the next instrumented boot rather than an assumption. CONSTRAINTS HELD, CHECKED AGAINST THE DIFF RATHER THAN INTENDED: - No collective on the admission path. scheduler.py:6405-6407 documents a 2026-08-17 deadlock of exactly that family and warns against it verbatim; the diff introduces no all_reduce, all_gather, barrier or broadcast. - No device tensor reaches a logging argument (sgl-project#790). - pp_size == 1 is byte-identical: the guard is None below pp_size 2 (scheduler.py:1577). NOT DONE, DELIBERATELY: the two disaggregation PP loops (:618, :765) are untouched and still carry the original chain-flush hazard; the readiness contract does not cover them either, since it lives in the proxy receive path they do not use. That remains its own open item. Verified in ONE run over ten files, 190 passed against a 187 baseline -- the delta is exactly the new integration cases. The suite list is deliberately wider than the change: a verification list scoped to the change at hand is what let a merge regression through earlier today.

…the send The proxy readiness gate refused to enter the blocking receive until the upstream's CHAN_DICT `sent` count exceeded this rank's `consumed` count. `sent` is published strictly after the post, so it says nothing while the upstream is still inside the send call. That window is empty for an ordinary isend -- but not for the first point-to-point op on a torch NCCL process group, which creates the 2-rank communicator lazily, and that creation is a rendezvous: the isend does not return until the peer enters the matching receive. Every boot therefore died on its first real prefill: PP0 cannot bump `sent` until the isend returns the isend cannot return until PP1 enters the receive PP1 will not enter the receive until `sent` bumps with the gate itself as one arc of the cycle. Boots instr7 and instr8 (2026-08-21) produced this identically; py-spy caught PP0 in isend -> send_tensor_dict -> _pp_send_dict_to_next_stage 33 s into a send it would never leave, while both downstream ranks raised "sgl-project#789 PROXY READINESS TIMEOUT ... posted 1681, consumed 1681". The message was not missing. It was being posted to the rank that refused to collect it. The gate's docstring asserted a stuck sender was "covered identically" to a sender that scheduled nothing; that assertion was the defect. Add a second counter, `attempted`, published on the line BEFORE the send call, and read only by the gate that would otherwise raise. "The upstream has irrevocably entered a send for me" is as positive a presence signal as "the upstream posted", and during a rendezvous it is the only one that exists. The module's ordering rule is unchanged and still governs `sent`; the new counter is not the phantom-message hazard in reverse, because the decision to send is already taken and unconditional at the bump, so the receiver waits on transfer and rendezvous time rather than on peer scheduling. Drain loops keep reading `sent`: they may take off the wire only what is provably already on it. Cost is one extra /dev/shm publish per dict message, measured at 13.6 us. Tests, all on this branch, measured not assumed: test_pp_proxy_readiness_rendezvous_789.py, 5 passed -- three live spawned processes, real gloo, the shipped functions, and a wire whose send blocks in a real dist.recv until the downstream enters recv, i.e. the NCCL lazy-init property in gloo primitives. The wire supplies a hazard rather than a guarantee, closing the direction of the trap where a test transport is more careful than production. Can-fail: neutering only PhaseFlipCounters.attempted, in the child, reproduces sgl-project#789 on both downstream ranks -- the metal specimen. test/registered/unit/managers full directory: 47 failed, 2753 passed. HEAD (96df16d) measured with the identical command: 47 failed, 2748 passed. The failing set is identical in both directions; the 5 added passes are the new file. The five holder/stub repairs in existing tests are interface drift only -- no assertion touched. Boot instr9, PP=3 on this rig with the reference instrumented configuration: health reached, 15 min 12 s uptime, one request served, the 8-request burst 8/8 in 3.1 s with per-request spec_accept_length mean 2.38 (sgl-project#779 gate PASS), 6 further back-to-back bursts under sustained load, 48 more requests, 0 failures, still serving afterwards. Zero sgl-project#789 raises, zero wedge markers, zero scheduler exceptions, and sgl-project#788 admission byte-identical on all three ranks including the 7-request batch. The first surviving boot of this series. Corridor is NOT passed on that boot and is not addressed here: NVML free sat flat at 5229/6612/5475 MiB per card under load against the 819-1229 MiB band, i.e. 4.4-5.0 GiB per card unclaimed against grants of 31800/18800/19800 MiB. That is a planner capacity item, and it is the first boot in this series on which the number means anything, because no earlier one ever served a request.

…eriving it (slice 1) Ten fixes on this branch (sgl-project#757 sgl-project#789 sgl-project#790 #791b #791c sgl-project#792 sgl-project#795 sgl-project#797 #797b #797c) all have ONE form: each rank RE-DERIVES the pass schedule -- rid set, chunk length, prefix length -- locally from its own state, and a phase flip invalidates that state non-atomically. Every "new root" was the next consumer of the same re-derivation, so the list grew instead of converging. This stops patching consumers. THE DATUM WAS ALREADY ON THE WIRE AND NOBODY READ IT. PPAdmissionEntry.extend_len (pp_admission_congruence.py:177) has crossed the wire since sgl-project#791's first commit with exactly three consumers: to_wire (:223), from_wire (:244), one log string (:824). Nothing ever built a batch from it; reconcile_pp_admission_decision returns Dict[rid, prefix_len] and drops the second number on the floor. THE MECHANISM, CORRECTED AGAINST THE LOG rather than assumed. boot_instr20.log:5171,5181-5183: PP0 ADMIT rid=6cbe2733 prefix_lens=0 chunked=1 -> 512-row chunk PP1 ADMIT rid=6cbe2733 prefix_lens=512 chunked=0 -> 333-token remainder PP1 DID receive prefix_len=0 and DID clamp prefix_indices to it -- scheduler.py:7026-7027 worked. Then add_one_req's HOST LOAD-BACK put the 512 back: needs_host_load_back() went true when the HiCache prefetch landed and schedule_policy.py:1539-1549 concatenates the recovered indices. "MAMBA-HOST-RESUME ... triggers load_back" appears on PP1 and PP2 and is ABSENT on PP0 -- that asymmetry is the bug. 845-512=333 then fitted rem_chunk_tokens whole, so the NON-chunked branch fired. The re-derivation lives INSIDE THE ADDER, after the schedule was already applied, and it re-derived BOTH numbers. DESIGN. forwarded_schedule() (pp_admission_congruence.py:500) is the pass geometry as a value: rid -> (prefix_len, extend_len) for exactly the rids `effective` names; None or a voided decision yields {}. _add_scheduled_req (schedule_policy.py:1237) EXECUTES both numbers: no rem_chunk_tokens, no page/align rounding, no host load-back, no budget veto -- the budget is still charged. The gate sits above every local veto (:1600), and add_chunked_req (:1328) gains the gate it never had at all (it is entered from scheduler.py:7004, BEFORE the admission loop). The three membership vetoes that silently narrowed -- batch_is_full (:7035), the HiCache prefetch_done skip (:7047, the instr20 race itself) and the LoRA gate (:7024) -- become refusals (scheduler.py:7190). REFUSAL IS CONTROL FLOW, NOT A RESULT CODE. PPScheduleRefused (:163) is an exception because every AddReqResult means "build a batch without this request", which is precisely the corruption. A refusal reuses sgl-project#797 end to end (_pp_refuse_forwarded_schedule, scheduler.py:6333 /:6369): sets _pp_admission_pass_voided, voids the forwarded decision, re-notes the slot expectation. No new mechanism. Inside the loop a refusal is CARRIED, not thrown, so alloc_group_end() still runs (:7025, :7118, :7169). WHY THE GUARDS CAN NO LONGER FIRE, each owed a reason: sgl-project#631's _want is extend_num_tokens, which now comes only from the forwarded extend_len with load-back suppressed, so it IS the upstream's row count by construction rather than by agreement (green arm: rows=512, batch_tokens=512). sgl-project#757/sgl-project#787 stamp and sgl-project#795 epoch were already structural; what changes is that they can no longer be correct-but-insufficient, as instr17 and instr20 both were -- every identity right, only the width wrong. Width is now an identity too. sgl-project#789 needs a membership divergence, which is now identical-or-refused. #791c's tripwire detects a self-narrowed batch, and no path creates one. HONEST CORRECTION TO THE SUBSUMPTION CLAIM: retraction does NOT become unnecessary. Physical impossibility is real -- a rank genuinely lacking KV for [local, told) cannot execute. What becomes structurally impossible is the NARROW-THEN-DETECT shape: no code path is left that builds a batch of a geometry the upstream did not name. NOT COVERED BY THIS SLICE, stated so nobody assumes otherwise: - BATCH ORDER. can_run_list follows the local waiting_queue order, the decision follows PP0's. Same rid set in a different order gives EQUAL WIDTHS and permuted rows -- silent. Covered by the full design (execute in decision order), not by this slice. - Decode batches: retract_decode (scheduler.py:7489/:7517) mutates long-lived state on a tp_cpu_group reduce that is world=1 under TP=1/PP=3. Different root, filed by sgl-project#797. - Radix eviction divergence, KV pool sizing, spec-decode draft schedules. NAMED RESIDUAL, filed at the site (scheduler.py:7118) in sgl-project#797's practice: a request admitted earlier in a loop that later refuses has taken a persistent inc_lock_ref, released on batch completion. Undoing it needs the exact IncLockRefResult (SWA/Mamba tombstone params) the adder does not keep, and a blind release makes the one thing a mismatched release worsens. Bounded: reaching that line takes a genuinely unexecutable geometry and kills the pass. DEFAULT PATH UNTOUCHED: _pp_scheduled_extents() returns None on PP0 and on every pp_size<=1 boot, so scheduled_extent_for returns None and both adders take the pre-existing arithmetic unentered; PPScheduleRefused is unraisable there. Pinned by test_no_mapping_is_the_untouched_default_path and corroborated by 56/56 on the schedule_policy neighbours. TESTS. test_pp_forwarded_schedule_791.py: 17 passed (3 live gloo arms + 14 pure), 92 s. test_red_without_the_forwarded_geometry_instr20_reappears -- can-fail, rebinding ONLY the fix's return value in the child: batch=(512,333) rows=512 mismatch=True, byte-identical to instr20 PP1 09:40:30 test_green_the_forwarded_geometry_survives_the_mid_pass_prefetch -- batch=(0,512) rows=512 mismatch=False, load_back_calls=0 test_an_impossible_geometry_raises_rather_than_narrowing -- the architectural property Neighbour set (791/791b/791c/797/631 x3) re-measured AT HEAD: 11 failed / 75 passed; after: 11 failed / 92 passed, failure names byte-identical, zero regressions, +17. schedule_policy neighbours 56 passed / 0 failed. ruff 119 = 119 at HEAD (parity), ruff format clean on everything authored, codespell identical.

…n a send nobody owes The PP output ring wedged with all three ranks alive and none of them able to move (specimen /spinning/evidence-665-f1/wedge_802f_1712/, PP=3, --enable-phase-flip, py-spy of all three schedulers): PP0 _pp_commit_comm_work <- _pp_commit_pending_req_work (:4071/:2262) PP1 _pp_recv_dict_from_prev_stage <- _do_recv (:5608/:6069) PP2 PpChainReceiver.recv <- recv_requests (pp_chain_receiver.py:329) PP0 is flushing the request chain, which PP1 can only take at the top of its next pass; PP1 never reaches that top because it is blocked in the output receive; PP2 waits for the chain send PP1 has not reached. Three arcs, one cycle, no timeout anywhere. THE ASYMMETRY. The two ends of the intermediate hop apply unrelated predicates. `_do_recv` decides to receive from THIS rank's own slot state; the non-last sender in `_pp_send_output_to_next_stage` decides to forward on `if pp_outputs:`, which is whatever it received LAST iteration. The last-rank hop is matched by construction because it consults `_pp_output_expected_for_slot` -- but that flag is the FIRST rank's verdict, published for PP0's arc. Nothing publishes the same thing for the intermediate hop. WHY THE EXISTING VOID CONTRACT DOES NOT COVER IT. `pp_void_forward_ payload` (sgl-project#797) already forwards a void along this hop and stops at `pp_first_retracting_rank`, on the argument that rank r and everything after it has an empty slot whose receive early-returns. That holds for the VOIDED slot -- `_pp_void_own_batch` empties it. It does not hold for a slot the resident decode path still occupies, and both void paths keep resident requests on purpose (`_pp_absorb_void_output` refuses to release them as a double-free; `_pp_void_own_batch` deliberately leaves `running_mbs` alone). The specimen shows exactly that: PP0's last act was absorbing a void for slot 0, leaving `pp_outputs` None, while PP1 logged `running=1 chunked=1` and re-entered the receive for a slot PP0 would never send to. PP0 cannot know PP1's resident set, so closing this needs a per-slot expectation every non-first rank publishes to its predecessor -- a protocol extension, and not this commit. WHAT THIS COMMIT DOES. The sgl-project#789 readiness gate is parameterised by wire kind and the output receive now passes through it. The CHAN_DICT counter was never proxy-specific -- one counter per wire, demultiplexed by `__msg_type__` after it comes off -- so the gate reads the same true statement about the same wire either way; `kind` selects only the per-(src, kind) inbox peek. The ring is cut at its one cuttable arc: PP1 waits boundedly and refuses by name instead of for ever. It does not make the missing send appear, and the error text says so. AN ALIAS, NOT A WRAPPER. `_pp_wait_for_proxy_readiness` is now a class-level alias for the same function object. About ten stand-in holders across the sgl-project#631/sgl-project#757/sgl-project#787/sgl-project#789/sgl-project#791/sgl-project#795/sgl-project#797/sgl-project#798 test family bind that name one method at a time; a delegating wrapper resolved the second name on the HOLDER and turned 9 green tests into 5 failures. Measured, then fixed. Tests: test/registered/unit/managers/test_pp_output_readiness_ring_802.py -- three real gloo processes, real PhaseFlipCounters, neutering done in the child (spawn re-executes the module, not the test body). * red arm, gate neutered: stuck_ranks == [0, 1, 2], specimen reproduced * green arm: stuck == [], PP1 reports "sgl-project#789 OUTPUT READINESS TIMEOUT" and "sgl-project#802-ring", PP0 chain-flushed, PP2 requests-received * false-positive direction: upstream really posts -> receive succeeds * no-op without counters, so the non-phase-flip default path is unchanged Mutants, both die: call site removed -> green arm red (all ranks stuck); gate raises unconditionally -> false-positive test red. Regression: baseline 787+789 = 9 passed / 0 failed; with this change 791b + 789 + 787 + 802 = 17 passed / 0 failed.

…he ring with the existing drain W4a made the PP chain receive bounded, named and resumable, but left its automatic abort off (SGLANG_PP_CHAIN_RECV_STALL_S=0), so on metal it could diagnose nothing and recover nothing. This wires it. THE DEFAULT STAYS OFF, AND THAT IS THE POINT. An idle PP rank legitimately blocks in this receive until a request arrives, so there is no duration that separates "idle" from "wedged"; a wall-clock default would SIGQUIT a healthy idle server, the same trap sgl-project#821's marker documents for its own arm. The recovery is therefore armed by EVIDENCE. THE PREDICATE. bump_attempted publishes that a rank has ENTERED a send BEFORE it posts it (phase_flip_counters.py:220-226) -- the only counter whose timing can witness a peer parked INSIDE a send rather than one that has finished. When this rank's CHAN_DICT upstream has entered more dict sends than this rank has taken off that wire, the peer is parked in a send only this rank can drain while this rank is parked in a receive that peer will never feed. That is boot_827's ring stated in counters: PP0 in _pp_commit_admission_send_work on the typed-dict channel, PP1 and PP2 in the request-relay chain receive. abort_check is consulted only at the yield, i.e. only once a receive is already overdue, so a healthy pass never reaches it. THE RING-CUT REUSES sgl-project#757 RATHER THAN INVENTING A CONSUMPTION PATH. pp_flip_drain_leftover_dicts already demultiplexes, stashes a wrong-kind message in _pp_tensor_dict_inbox where its real consumer looks, and discards only a provably void proxy -- which is exactly what makes taking the dict off the wire out of the pass's normal order safe. It runs on every disarm route already. request_receiver catches PpChainRecvStalled, runs one drain turn, and resumes the SAME posted receive, which is what ParkedWait's resumability is for. Servicing that does not clear the stall re-raises rather than spins: at that point the ring is closed for a reason this code does not model, and retrying would turn a diagnosable wedge into an invisible one. Follows sgl-project#789's shape deliberately. _pp_wait_for_dict_readiness argues the false-positive direction is the safe one for the mirror gate, and it is here too: a spurious fire costs one drain turn and a resumed receive, and the receive stays posted and framed throughout, so the late message still arrives intact. Missing a real one costs the boot. sgl-project#789 also declines to invent a new protocol for its case; this declines likewise and cuts the ring on the arc waiting for a message nobody posted. Also: _pp_flip_pass_tick publishes _pp_live_mb_id, which the drain needs to tell an owed proxy from a leftover one. TESTS (hermetic, CVD="", CPU only, no gloo, no scheduler construction) test/registered/unit/managers/test_pp_chain_abort_check_824.py 9 passed; all 9 fail pre-fix. Covers both directions: the boot_827 counter state aborts, and an idle rank is NOT aborted at 0 s or at 3600 s. Mutants killed, one per edge: entered >= taken -> the idle rank is aborted (1 failed) service-turn cap raised -> an unclearable stall stops being re-raised service hook never called -> the ring is never cut (2 failed) Run under /spinning/htsglang-gpu/.venv (torch 2.11.0+cu130, datasets 5.0.0) with PYTHONPATH leading to this worktree, verified by `import sglang; sglang.__file__`. My earlier runs used /usr/bin/python3, which lacks datasets and silently collected only part of the suite; see the corrected note in WINDOW-QUEUE.md. No boot was run. This is desk work.

…he silent hop Boot 2 of window-flip-0828 wedged silently for 10+ minutes: PP0 and PP1 blocked in gloo waitSend under _pp_commit_comm_work's naked p2p_work.work.wait() (PP1 via _pp_commit_pending_req_work), PP2 in _do_recv -- a closed three-arc cycle on a group whose own timeout is two hours. The sgl-project#753 comment in this file already described the exact shape (boot v7pp9); the ordering was fixed then, the wait stayed naked. The bound routes through the sgl-project#630/sgl-project#829 canon (hicache_collective.bounded_wait -> ParkedWait): the unbounded wait() parks on a thread and the deadline is on the JOIN. Deliberate deviation from the order's 'pass the deadline into work.wait(timeout)': sgl-project#829 retracted that design in this tree -- an expired timed Work.wait closes the gloo pair, and hicache_collective.py names _pp_commit_comm_work as a measured victim (34 of 262 boot logs). Budget 120 s (SGLANG_PP_RING_COMMIT_BUDGET_S; <= 0 = documented escape hatch, byte-for-byte pre-sgl-project#973), reasoned against the canon's constants: 12.6x the longest healthy cutover, 4x the sgl-project#789 budget, 5x under the HiCache bound, 60x under the group timeout. On expiry: RingCommitTimeout with a sgl-project#650-style peer statement naming the silent hop; transport failure is NOT converted (sgl-project#734 stays distinguishable). Helpers are module-level, not methods -- the method form broke ~12 one-method-at-a-time stand-in holders (measured: 796 5->2, 801 14->3), recorded in a code comment. Wait-site audit: :5253 bounded-now; :3270 bounded by delegation (proven by test); _do_recv/:7592 and :5545 bounded-already via the sgl-project#789 gate; parallel_state.py recv_object stays unbounded and is NAMED as its own posten (needs the resumable PpChainReceiver treatment, not a terminal bound mid-protocol). Tests: test_ring_commit_bounded_973.py, 5 arms on real 3-process gloo -- pre-sgl-project#973 naked commit HANGS (distinguished from raised via fsynced progress marker), bounded commit raises within budget with the peer statement, healthy paired traffic unchanged, escape-hatch mutant hangs again (can-fail), :3270 reachability arm. Gates: BEFORE == AFTER byte-identical on 796(5)/801(14)/797(31)/630(5+14)/829(7) plus an 18-module sibling sweep; extraction count probe OK on every run. ruff/codespell: 0 new findings.

…it travels Boot-2 ring wedge (R3a root H3): the output ring's intermediate-hop predicates serve the LAGGED slot -- the sender forwards on `if pp_outputs:` (what it took off the wire last iteration), the receiver reads its own `mbs[next_mb_id]`, committed passes earlier -- and sgl-project#951's launched posting is a SAME-SLOT closure that cannot retract a batch already resident there. The uncovered path is the void relay: `pp_void_relay_stop_rank` derives its stop from the RETRACTION structure, so a void that names no retraction (sgl-project#944's zero-offer escape: a rank that lost the request retracts nothing and launches nothing) was forwarded to a rank whose slot for it was empty, taken off the wire by that rank's next ADMISSION receive, stashed, and served POSITIONALLY to a healthy later generation's receive -- which `_pp_absorb_void_output` then emptied on one rank only, leaving the ring one message short for ever. PP2 sat in `_do_recv` unbounded because sgl-project#971's busy wire kept the sgl-project#789 gate early-returning (consumed<posted); boot 1 only survived on a counter another defect had frozen. Fix: the launched CHAIN -- the same per-hop statement sgl-project#951 consumes, kept per generation. `_PP_LAUNCHED_CHAIN_KEY` accumulates one bool per rank on the admission decision (PP0 starts it, every hop appends its own `mbs[mb_id] is not None`, the last rank records without sending, sgl-project#796), rides back on the slot's output/void via `pp_output_payload_with_return_trip`, and `pp_void_forward_payload` consults `pp_void_relay_launched_verdict`: a void travels exactly to the ranks whose own admission-time statement says they launched, never to its source. Absent chain falls back to the legacy rule byte for byte. Log-only `sgl-project#978 STALE VOID` tripwire in `_pp_absorb_void_output`. Red-first, measured 2026-08-28 against the unfixed tree (TheLaggedSlotIsBeyondTheLaunchedPosting): test_the_launched_chain_relay_stop_closes_the_lagged_slot FAILED -- stuck [0,1,2], PP2 parked in output_exchange, PP1 event `pass=7 void_absorbed slot=2` against hazard slot 1 (the mispair), with sgl-project#951 wired in AND CHAN_DICT counters armed (the readiness gate ran and early-returned: boot-2 fidelity, not boot-1's frozen-counter accident). Count check: 1 failed, 1 passed of 2. After the fix: 2 passed; the wedge stays measurable via fix_off (spawn worker forces the verdict back to legacy -- mutant on the hazard direction, reproduces the wedge). The verdict pure function is pinned exhaustively over rings 2..8; three boundary mutants (hand back to source / read own entry / short chain forwards) all fall. Suites: neighbor baseline before == after. Before: 95 passed (801, 791b, 795, 797, 951, idle-void, void-send families, 262s). After: 100 collected, 99 passed + 1 failed, the 1 a harness-stub interface drift in test_pp_retracted_pass_void_797.py (local recorder lacking the new kwarg; repaired, no assertion touched; file then 31 passed). ruff: only the one finding already present on HEAD; codespell clean. sgl-project#981 boundary, checked before push (CPG_DERIVATION_0828 (b)): this fix's predicate does NOT stand on the `_pp_output_expected_incoming` memo. The chain's write (decision recv), reset (top-of-body block) and consumption (decision send) all live inside full body iterations; the resume-slot jump's `continue` sits before the reset block but consumes nothing before the next full body resets. The by-slot chain is written only at a slot's admission and read while that generation is resident. K3's ungated admission recv and the clear-site-behind-skip fragility remain sgl-project#981.

Boot 51 died on "sgl-project#789 OUTPUT READINESS TIMEOUT: mb_id=2: upstream (rank 1) posted 9 dict message(s) on dict|output (entered 9), this rank has consumed 9". Nine posted, nine consumed, nothing outstanding -- and the rank waited for a tenth, then killed the instance for its absence. The arithmetic in the readiness gate is right; the caller is not. `_do_recv` enters the blocking output receive on `mbs[next_mb_id] is not None and not prebuilt and not can_skip` -- this rank's OWN slot state -- while the upstream forwards on `if pp_outputs:`. Two predicates on one stream, and the raise's own hint says so: "this rank decided to receive from its OWN slot state while its upstream decided to forward on `if pp_outputs:`, and nothing publishes this rank's per-slot expectation to that upstream". So the gate now reports its verdict instead of only ever killing on it. When `consumed == posted == attempted`, the upstream has stated in-band that it forwarded nothing for this slot, and the caller takes the SAME no-output exit it already takes for `target is None` two lines up -- an existing, handled path, not a new one. What this deliberately is not: nothing is dropped, because nothing was sent; nothing is refused, because there is nothing to refuse. Those two forms are both metal-falsified on this path already (corpse R 2026-08-09, and sgl-project#995 in this window, boot 15). And no token is fabricated -- `_pp_make_skip_output_ result` is NOT reused here, because its zero placeholder is legitimate only for a chunk that really ran and produced no token, which is not this case. Opt-in, default unchanged. The proxy alias keeps raising: a missing proxy is a different fact, since those hidden states are the input to this rank's own forward and declining them would compute on nothing. Evidence: desk, executed. py_compile; both `soft` parameters present; the proxy alias reaches no soft path; the declined case emits a named line with posted/entered/consumed so a decline can never become quiet. Belegstufe: DESK-BEWIESEN -- the metal proof is the boot.

…a None Two corrections to the first cut, both measured. BOTH CHANNELS. The timeout is 2/2 deterministic and it alternates wires: boot 050655 died on dict|proxy (posted 11, consumed 11), boot 053102 on dict|output (posted 9, consumed 9) -- same mb_id=2, same rank-1 upstream, both after six clean flips. Covering only `output` would have moved the death to the proxy wire, not removed it. The proxy half cannot decline and carry on, because those hidden states are the input to this rank's own forward; it VOIDS, which is what the upstream's silence states and the same rank-agreed act sgl-project#798 performs on the same fact discovered one step earlier. It voids at the point of discovery rather than signalling the caller, because two legitimate `return None` exits already sit above it and a sentinel may not share a value with a real result. VOID, NOT RETURN. The first cut returned early from `_do_recv` on a declined output, and boot 52 died on "AttributeError: 'NoneType' object has no attribute 'synchronize'" -- my own bug: the early return skips the assignment of `d2h_event`, and the caller dereferences it. The neighbouring `target is None` exit is safe only because the caller guards that call with `self.mbs[next_mb_id] is not None`. So the decline now makes that guard true by voiding the slot, which is also the honest statement: the upstream forwarded nothing, so the slot did not run. Boot 52 also showed the fix reaching its site with no sgl-project#789 raise anywhere, so the output gate itself behaved; what killed it was the exit I built, not the contract I changed. Evidence: desk, executed. py_compile; exactly one soft proxy gate and one soft output gate; the caller re-reads the slot after a proxy decline so every guard below keys on the same object. Belegstufe: DESK-BEWIESEN.

…erted with the architecture BOOT 15 CRASHED AT 10:59:42, all three schedulers, and the cause is the sgl-project#1046 cut -- reported as mine, not as a pre-existing condition. sgl-project#968 LOAD-BACK EXTENT UNREACHABLE for rid=c9d14e69...: PP0 published an extent of 4618 token(s) and this rank's load-back yielded only 0 1448 times on one rid, refusing the forwarded schedule on every rank every pass, until the ring wedged (sgl-project#789 PROXY READINESS TIMEOUT) and the schedulers died. ROOT: sgl-project#1042's "a hitless match must never clear" was CORRECT while the extent was consumed A LAP LATER by the delivery row -- the fact had to outlive the match that made it. sgl-project#1046 moved consumption into the SAME match and I carried the old rule across unchanged. The extent then outlived its own validity: rid c9d14e69 kept extent=4618 through a readmit whose match read `host_hit=0`, and the clamp demanded 4618 from a tree that could serve 0. THE LESSON IS THE LIFECYCLE LAW ITSELF: a table is valid for ONE architecture. It must be RE-DERIVED when the consumer moves, never inherited. I built the table, then broke it by moving the consumer without re-deriving it. TWO FIXES, both following from the same inversion: 1. A hitless match now CLEARS (`hitless_clear`, was `hitless_noop`). Under local consumption the extent is only valid for the match that produced it. 2. `_applied < _lb_extent` is no longer a PPScheduleRefused. That raise existed because `_lb_extent` was PP0's PUBLISHED number, so a rank that could not reach it would diverge from peers who could. Since sgl-project#1046 the extent is this rank's own, so a shortfall means only that this rank's stamp went stale -- local staleness, not divergence. It now takes what its own tree can serve and says so (`sgl-project#1048 EXTENT STALE`), which is uniform because every rank re-derives from its own tree. Leaving it a group refusal is what turned a stale number into a ring wedge. ALSO REPORTED: the boot-15 reproducibility run is CONFOUNDED, not negative. Two cell runs landed on a server that was dying and then dead (deadman CRASH 10:59:47); the repro window's `sgl-project#988 genuine=0` measures the death, not the mechanism. It is not evidence either way and is not counted as such. check_1042 updated to pin the INVERTED rule and re-run green; ruff clean apart from pre-existing E402.

…ter (rendezvous) Boot 631row2 wedged 90 s after READY: the slot-aware probe checked only posted-vs-consumed, and the FIRST p2p op on a torch NCCL group creates its communicator lazily -- the sender's isend cannot return, so posted cannot bump, until the receiver enters the recv. PP0 sat inside the isend of the first health-check frame while PP1/PP2 probed posted=0 forever: the instr7/instr8 cycle, re-created. consumed < attempted (bumped BEFORE the send) now also drains, exactly as _pp_wait_for_dict_readiness's sgl-project#789 branch does. Matched check: py_compile. Metal proof is the boot.

docs: make badges center

06c76bd

zhyncs merged commit 1f013d6 into sgl-project:main Jul 28, 2024

zhyncs deleted the doc branch July 28, 2024 12:27

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

docs: make badges center (sgl-project#789)

7c1e8a9

cherryblo added a commit to cherryblo/sglang-project that referenced this pull request Jul 2, 2026

Merge pull request sgl-project#789 from hhhh1252023/patch-10

2662e1e

fix 0613

Navigation Menu

Uh oh!

docs: make badges center - #789

docs: make badges center#789
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:doc

zhyncs commented Jul 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

zhyncs commented Jul 28, 2024

Motivation

Modification

Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant