docs: update README - #712
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
cen121212
pushed a commit
to cen121212/sglang
that referenced
this pull request
Nov 10, 2025
* Update test_utils.py * Update test_ascend_log_requests.py * Update test_matched_stop.py * Update run_suite.py * Update test_abort.py * Update test_chunked_prefill.py
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…-pong A layout may be declared unable to run only after it has HAD THE CHANCE to run. An emptiness observed within idle_locked_settle_s of entering a layout is a transient, not a verdict. THE CLAIM THAT FAILED The IDLE_LOCKED branch sits above the min-dwell check and bypasses the only anti-thrash bound, justified by a comment stating "IT CANNOT OSCILLATE ... after the flip the target runs by premise, so the same condition is false there". It oscillated. Across the 16 boot rotations of 2026-08-17 the policy produced alternating runs of 72 arms / 299 s, 12 / 31 s and 10 / 27 s twice. The premise fails on WHEN it is evaluated, not on what it says. This branch is reached on the first round after a cutover, while the just-entered layout is still empty and its carried work not yet re-admitted -- Scheduler._idle_locked_inputs is gated on _round_built_nothing, which a just-entered layout satisfies trivially. The new layout is observed in its empty transient, certified unable to run, and armed straight back. That is also where sgl-project#713's TTFT quantisation came from. The unit of delay is one whole cutover (median 2864 ms tp_to_pp, 2772 ms pp_to_tp over 486 flips), so the 0.1 / 3.1 / 5.9 s levels in the sgl-project#713 tables are how many cutovers a request sat through. The 06:19 table was taken during a 27 s run. THE VALUE IS DERIVED, NOT CHOSEN Delay from "cutover complete" to the first batch the new layout builds, over 162 cutovers: 0 s for 66 of 150, 1 s for 34, 2 s for 27, 3 s for 12, thin tail to 6 s, one outlier at 30 s. 2.0 s covers 84.7 %. The p95 (4 s) is deliberately NOT used: it exceeds the 3 s min_dwell_s this rig boots with, and a settle above the dwell would make the fast escape slower than the path it exists to bypass. Log stamps are second-resolution, so this is a bound, not an optimum, and the docstring says so. FAILURE DIRECTION, AND WHAT BOUNDS IT A genuine idle lock forming within the settle of a cutover is now delayed, by at most the settle. Bounded three ways: the value is capped at min_dwell_s at the use site, so an operator cannot configure the fast escape into being the slow one; the 180 s decode-stall cap remains the backstop that released the 09:42:39 specimen; and the settle is consulted only right after a cutover, which is the only place the transient exists. sgl-project#688's escape is preserved and pinned. phase_since, not last_flip_at: the first is when THIS layout was entered and is maintained from the OBSERVED phase, so a manual POST /phase_flip restarts it too; the second is an arm stamp that is 0 until the policy has armed once. None degrades to pre-guard behaviour rather than to an infinite settle. RED-FIRST, AGAINST THE RECORDING test_idle_locked_settle_713.py replays the recorded arms through the real policy off scripts/fixtures/d2_injector_pingpong_excerpt.txt, reusing the injector's parser so harness and suite cannot disagree about what the log says. settle disabled -> 12 of 12 arms armed, the ping-pong reproduced settle enabled -> 1 of 12 armed Not zero, and that is the point: the arm that legitimately leaves a 35 s-settled idle-locked layout survives, and only the eleven arm-backs on 0-1 s-old layouts are refused. The first version of this replay asserted all twelve were transients; the data refused, one arm being 35 s past its cutover. That arm was not noise, it was the control case, and it is now the selectivity pin. Mutation-proven: disabling the guard at source turns 3 tests red. REGRESSIONS Baselined before and after by capturing the failure set with the guard reverted. 8 failures pre-exist on this branch in this area and are untouched; the guard adds none and 705 pass. The five sgl-project#677-era pins stay green because they construct PhasePolicyState() with no phase_since -- which is also why they never caught this, and their docstring now says so instead of restating the falsified invariant. sgl-project#712 TEXT REMOVED The BOTH-BLOCKED decline used to redirect to "the state-slot bound (mamba/GDN slots)". That was never measured: it was a hypothesis authored into a log string, read back out of the log and filed as a finding, and sgl-project#712 was closed as unfounded on that evidence. The line now reports only what it knows. test_both_blocked_binding_resource_708 REQUIRED that wording, so the pin is inverted rather than relaxed -- an unmeasured cause must not be named.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
Please explain the motivation behind this PR and the goal you aim to achieve with it.
Modification
Briefly describe the changes made in this PR.
Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.