feat: add fake tag by zhyncs · Pull Request #770 · sgl-project/sglang

zhyncs · 2024-07-27T14:59:26Z

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
Modify documentation as needed, such as docstrings or example tutorials.

* support fp8 for ascend npu * adapt for deepep * clean code

…ost in the pool solve THE RULE IS RIGHT, THE SIZING WAS WRONG. `collective_kv_backing_relief` grants nothing when the MANDATORY ask exceeds what the rung can return above its admission floor, because laundering a short ask into a seam that then does not fit is worse than a clean abandon. Untouched here. What that rule cannot survive is a pool solved without the staging term: the ask is then unfundable for the life of the boot, so the rung reports it could return 1902-2037 MiB, grants 0, and logs "evicted 0 rows over 0 shrinks so far" -- 72 times in one boot -- while every flip has to find its bytes somewhere else. THE THIRD TERM, and it is none of the two already charged. The seam reserve is what a flip COSTS WHILE IT RUNS and is only knowable by measurement. The arming floor is a LEVEL THE GATE COMPARES AGAINST, derived from the stated corridor law. The staging ask is what the rung must be ABLE TO FUND for the seam to enter at all, and `project_staging_bytes` computes it exactly, because "the plan's only live-set input is the slot count, and everything else is static layout". Same class as the arming floor before it: a term the planner had every input for and did not charge, producing a healthy-looking boot that never flips, with no runtime recovery because the pool is fixed at boot. THE FUNDABLE LEG ONLY. pp_to_tp is the leg the rung may shrink for and the one whose refusal is fatal under strict purity; tp_to_pp abstains by design because its pool is about to become active again and recover_kv_backing would undo the shrink inside the same flip. Charging for a leg that never shrinks would hold memory against a payment nobody makes. 33 of the 72 observed refusals were that by-design abstention and are NOT what this fixes. EXACT OR ZERO, NEVER INVENTED. When the projection is not reachable at sizing time the post is 0 and the log says which of the two it charged, because a fabricated constant would be indistinguishable from the exact figure -- the derived_provenance defect this module already names. A cold zero means the pool is sized as it was before, which is a visible outcome rather than a silent one. Red-first on the specimen numbers from this rig's boot projection, 840 MiB (rank 0) and 1617 MiB (rank 1): 6 tests failed on the absent function, 16 pass now across sgl-project#767/sgl-project#768/sgl-project#771. ruff clean.

… the posts The authority from the previous commit answered the question correctly and nothing called it. This wires the SAFE half. _funding_post_census is strictly observational: it takes no decision, spends nothing, and returns a suffix for the refusal line that is already final by the time it runs. What it changes is that a refusal can now say WHICH posts it considered and why each paid zero, instead of "reclaimed 0 MiB from [nothing]" -- a string that means three different things (no providers, providers that paid zero, a funder that was never in the ladder's list) and on 2026-08-16 was read as the wrong one for a whole morning. The specimen is the third case: the same second that printed [nothing] also printed "KV capacity is the funder" with an exact draw. Also adds authority_from_seam_snapshot, which takes only plain integers the caller has already computed for its own census. It introspects nothing, so the refusal path does not reach into a scheduler to use it. TWO WRONG GUESSES, CAUGHT BY THE TESTS RATHER THAN BY THE LOG The census reads the rung behind a broad `except`, so a refusal can never crash while trying to explain itself. That safety has a price: a wrong attribute name is swallowed and the census goes permanently silent, which in a log is indistinguishable from one that was never needed -- the exact failure shape the draft-weights provider comment warns about a few hundred lines up. I made that mistake twice while writing this. `current_rows` / `floor_rows` / `row_bytes` were guessed and none of the three exist; the real reads are `_last_proposal_terms` (kv_backing_relief.py:1036, keys `current` and `floor_rows`), `_bytes_per_row`, and `_min_release_rows()` (:1375). Then the class was guessed as KVBackingRelief; it is KvBackingRelief. Neither error would have surfaced at runtime -- both produce a silent, empty census. TestRungAccessorContract pins all four names so a rename upstream fails loudly here instead of quietly disarming the feature. Corroboration worth recording: kv_backing_relief.py:1093 already carries a note added 2026-08-22 saying a rank's floor "can VETO the group's shrink even if a peer has a fundable plan". The veto was already known and already written down. What was missing was anything that turns it into a verdict a caller can act on. TEST RESULTS (hermetic, CUDA_VISIBLE_DEVICES="") test/srt/test_funding_authority_770.py 40 passed in 2.6 s (was 35) test/srt/mutants_770.sh 8/8 mutants KILLED phase_flip_runtime imports clean; census resolves on PhaseFlipRuntime ruff --select=F401,F821,UP037 3 findings, byte-identical to the base version at HEAD~1 (2090, 2581, 7022->7096 shifted by inserted lines) -- ZERO new findings codespell 4 pre-existing "retuned" hits at :1738-1743, none mine

… floor from its own cap; solve the arming floor; name the break-even provenance Three ordered items on top of the funding authority. A) THE PEER VETO, AT ITS SOURCE (sgl-project#812 / kein-bindender-rang) The live defect is not _floor_rows, it is what happens to its output. In _floor_ppm, kv_backing_relief.py:167: if floor_rows >= current_rows: return _SHRINK_SCALE # 100% == "no shrink" Measured 06:32:05, all three ranks carrying the SAME floor 128549 -- correct, because under PP a request's tokens occupy KV on every stage, so the live set is genuinely replicated -- against caps that are unequal BY DESIGN: PP0 backed 212992 floor 128549 -> 60.4% of its own cap PP1 backed 124928 floor 128549 -> 102.9% <- the defect PP2 backed 133120 floor 128549 -> 96.6% PP1's 102.9% clamps to 100%, the group MAX takes it, explain_kv_target computes target = max(desire, max_floor), and NOBODY shrinks -- vetoing PP0's fully fundable 84443-row plan on the very rank that needed it. The sibling _shrink_ppm documents this exact trap for the ambition side ("on an uneven fleet the smallest pool's 'no change' is the smallest number in the group and silently wins"). The lesson was never applied to the floor side. Fix, local only -- the collective FORM is untouched, no new collective on the admission path (DESIGN_679's warning stands): _floor_rows now derives its result against this rank's own backed rows and clamps DOWN to them. An unbacked row cannot be reserved, so a floor above the cap was never a meaningful quantity. The condition is logged as an UNDER-BACKED RANK -- a defect about that rank's backing, whose answer is to grow it -- and floor_exceeds_local_cap() splits the two cases _floor_ppm used to collapse: floor == cap is a HEALTHY full pool, floor > cap is the defect. They returned the identical value before, which is how one rank's local problem became a group-wide freeze. HONEST SCOPE: the clamp makes the quantity truthful at the point it is computed and names the defect. It does NOT by itself unblock the veto, because a clamped floor still equals the cap and still returns the neutral element under a PROPORTIONAL agreement. Replacing that proportion is a change to what the group agrees on -- enforcement, and metal. Ticket in the COORD. B) DEFECT A: THE ARMING FLOOR IS THE FREE VARIABLE, THE BAND IS NOT band floor 819 + seam entry reserve 512 = 1331 arming floor + arming margin 192 = 1523 band ceiling 1229 <- 294 MiB short The corridor band (819-1229 free per card under load) is a hard user rule: below it is a breach, above it boot acceptance has FAILED. It does not move. The arming floor is DERIVED and the seam entry reserve is an allowance, so the floor is what must give. solve_arming_floor() reports the largest reserve that fits -- 218 MiB against the 512 shipped -- and refuses BY NAME otherwise. It does not auto-correct: cutting the reserve changes what the seam may spend while it runs, which needs metal. Also withdraws a piece of advice that named a forbidden state. The abandon message said the flip "is retried when occupancy drops"; clearing a 1331 MiB watermark means leaving the acceptance band from ABOVE, so no amount of waiting can end it -- and 18f measured exactly that (draining the load did not lift the lock). _arming_floor_advice() now retracts the retry advice whenever the watermark is unreachable. C) sgl-project#819: 7004 IS NOT A SEED, AND THE STALENESS IS UNEVEN 7004 is a literal nowhere; break_even_tokens computes N = C / (1/X - 1/P) and 7004 is what the shipped inputs produce (asserted in the suite). The three inputs do NOT share a provenance, which is the actual finding: C flip cost SELF-CORRECTING -- FlipCostEstimator.observe() is fed real cutover durations, seeded from DEFAULT_FLIP_COST_S=3.2 X TP prefill 1681.0, env-overridable, NEVER measured at runtime P PP prefill 7245.5, same shape One input self-corrects and two cannot. A rig whose prefill ladder differs from the sgl-project#631 mainrig silently solves N against another machine's hardware unless a human sets two env vars, and nothing tells them to. BreakEvenProvenance makes that visible instead of implicit. TEST RESULTS (hermetic, CUDA_VISIBLE_DEVICES="") test/srt/test_funding_authority_770.py 48 passed (was 40) test/srt/test_floor_local_cap_812.py 16 passed (new) test/srt/mutants_770.sh 11/11 KILLED (M9/M10/M11 new) test/srt/mutants_812.sh 4/4 KILLED ruff --select=F401,F821,UP037 3 findings on phase_flip_runtime.py, identical to base (2090, 2581, 7022->7137 shifted by inserted lines) -- ZERO new codespell clean on all new files TWO MUTANTS SURVIVED FIRST CONTACT AND BOTH WERE REAL. MA3 (clamp rounds UP past the cap) survived because the fixture used cap 124928 with page 64 -- exactly divisible, so round-up and round-down agree and the test could not tell them apart. Refixtured on 124900. MA5 (drop the min() guard in _floor_ppm) is an EQUIVALENT mutant, proven not patched: the early return means the expression is only reached with floor < current, where ceil(floor*1e6/current) <= 999999 < _SHRINK_SCALE. Verified numerically; the guard is kept as defence-in-depth and the mutant is recorded in the harness rather than run, because a harness listing an unkillable mutant reports a permanent false gap.

… a correctness invariant for a funding win THE PREVIOUS COMMIT SHIPPED A REGRESSION AND THE TREE CAUGHT IT. _floor_rows clamped the floor DOWN to this rank's backed rows whenever the floor exceeded them. It looked like the ordered fix -- "derive the per-rank floor from the per-rank cap; a floor >100% of its own cap is a defect by construction" -- and the arithmetic is indeed a defect. The clamp is not its remedy. The floor IS `live set + 1 + margin + admission reserve`. Lowering it to a smaller cap does not make an under-backed rank able to shrink; it authorises a cap BELOW rows that are still in use. Measured, not argued: test_residency_cap_flip_levelling_792 ::TheLevellingMustNotCapBelowTheLiveSet ::test_the_recovery_levelling_leaves_every_rank_able_to_pay with the clamp 1 failed without the clamp 12 passed The invariant is older than this ticket and the tree already carried it. My ID-level regression is what surfaced it, and it is exactly the danger direction I had written into my own mutant rationale one commit earlier -- and then implemented anyway. WHAT STAYS. The detection: floor_exceeds_local_cap() splits the two cases _floor_ppm collapsed -- floor == cap is a HEALTHY full pool, floor > cap is an under-backed rank -- and _floor_rows now LOGS that condition by name instead of letting it vanish into a group-wide freeze. The floor itself is untouched. WHAT THIS MEANS FOR THE VETO. It is not fixed here, and the honest statement is that it cannot be fixed locally at all. An under-backed rank genuinely has nothing to give; the repair is on the BACKING side (grow it) or in the agreement shape (stop agreeing a proportion of each rank's own cap). Both are enforcement and both need metal. Ticket with acceptance criteria in the COORD. MUTANT MA1 IS NOW THE CLAMP ITSELF, so the withdrawn version cannot creep back: reintroducing it turns the suite red. Added test_the_floor_never_drops_below_the_live_set so this file cannot drift toward it either. Also ran black over the touched files (the sgl-project#656 formatting ratchet). The ratchet's own two failures are PRE-EXISTING and not mine: the files it flags are phase_flip_boot.py, phase_flip_seam_reserve.py, kv_vmm_backing.py and two test files, none of which this branch touches. TEST RESULTS (hermetic, CUDA_VISIBLE_DEVICES="") test_funding_authority_770.py + test_floor_local_cap_812.py + test_residency_cap_flip_levelling_792.py 76 passed test/srt/mutants_770.sh 11/11 KILLED test/srt/mutants_812.sh 4/4 KILLED (+1 equivalent) ruff --select=F401,F821,UP037 All checks passed codespell clean PRE-EXISTING FAILURES CONFIRMED NOT MINE, each by running the same file against the base module 481c732: test_evict_rung_floor_invariant_717.py 3 failed on base AND on mine (AttributeError '_buffers' -- the sgl-project#717 stubs, unrelated to this branch) test_r1_private_constant_gate_584.py flags VERDICT_BAR_MIB, NEAR_OOM_MIB, BYTES_PER_GB, MIN_STAGING_GB -- all in files this branch never touches test_scheduler_chunked_req_gate.py 3, unrelated test_pp_proxy_cross_epoch_mispair_795.py 1, unrelated

16g: funding_authority, solve_arming_floor at 218 instead of 512, the retraction fix to the abandon message, and sgl-project#819 provenance. Clean merge. Shares phase_flip_runtime.py with feat/822 and kv_backing_relief.py with the already-merged sgl-project#816; hunk regions checked disjoint before merging (sgl-project#770 at :6120-6972 vs sgl-project#822 at :3741-3798/:8029).

19a: the ownership authority plus three wiring sites in phase_flip_runtime.py. fe43b09 is carried as an ancestor of this head. Clean merge; see the sgl-project#770 stage for the phase_flip_runtime.py disjointness check.

…gl-project#821, sgl-project#823, sgl-project#824, sgl-project#770/sgl-project#812, sgl-project#815; close the section 18 breach Merge-checklist duty for the five wave-2 stages on this branch (fix/801, fix/824, feat/770, feat/822, fix/815). Mechanism text is taken from the commits' own measured evidence, not restated from ticket titles. WHAT WAS ALREADY THERE, checked before writing rather than after. feat/822 catalogued ITSELF mid-train -- 21ff075 carries +49 lines of this file -- so sgl-project#822 already has a complete section 3 entry including the sgl-project#816 clamp firing-rate baseline. No second sgl-project#822 entry was written. sgl-project#801, sgl-project#815, sgl-project#821, sgl-project#823, sgl-project#824 and sgl-project#770 had zero prior mentions; sgl-project#812 was name-checked inside the KV-backing-relief bullet as a consequence of the cap, with no entry of its own. SECTION 12, five new families: Said-once (sgl-project#823) -- a fact that changes during a process, reported once at the moment it first became true, is a stale label afterwards. Both instances live in _update_uniform_pool_budget: the floors' coverage said three times at startup and never again through four cutovers in 55 s, and a divergent TP queue head logged as an event when it is a duration. The mutant that makes the case: disabling the recovery edge left all 14 tests green before the streak was extracted. Blind-instrument (sgl-project#821) -- the scheduler watchdog's activity predicate reads exactly the two values a rank parked in the dict receive freezes, so the instrument best placed to see the wedge is the one structurally unable to. Non-monotone-source (sgl-project#824) -- a monotone guarantee (the protected prefix) derived from a non-monotone tracker (mamba_last_track_seqlen). Landed as a measured assertion, cache_protected_len=16384 against page_aligned_len=8192. Invariant-for-funding (sgl-project#770, sgl-project#812) -- a group-derived per-rank floor asking PP1 for 102.9% of its own cap, and the withdrawal of a floor clamp that bought funding by breaking the levelling invariant (1 failed with, 12 passed without). Stub-drift (sgl-project#815) -- ten helpers drifted behind named commits, and the rule the repair follows: faithfully or not at all. This entry also RECORDS why seven cases in test_pp_flip_slot_hold_631.py are deliberately left red -- the faithful stub is a real rank identity, which drags the whole admission-receive path in with it. That is documentation of a known state, not a claim that it is fixed. Plus a NUMBER-COLLISION note, because the log is ambiguous without it: 9d13bf0 is subject-tagged [sgl-project#824] and names its test file _824.py but is canonically register-sgl-project#823 (floor-scope reporter, strand 16f, arriving on fix/801-admission-recv-liveness); the canonical sgl-project#824 is 658d0c1 on fix/824-chunked-prefill-protected-len. No rebase, so the titles stand and the mapping is written down instead. SECTION 7: sgl-project#801's dead-peer measurement, which REFUTES the premise under the sgl-project#816 survivor analysis -- an exited or SIGKILLed peer makes the admission receive raise after 1.02 s, while alive-never-sends and SIGSTOP both block unbounded, so a rank silent for 119.7 s was not waiting on a corpse. SECTION 2: the funding authority (sgl-project#770/sgl-project#584/sgl-project#819) -- one question, posts declared by name, the three priced losses (L1 the unreachable 2638 MiB, L2 a 3437-row ask against 8192-row granularity returning a silent zero at 65% of the shortfall, L3 claimed=0 against an accounted 107 MiB), and the arming floor solved rather than asserted (512 MiB reserve needs 1523 against a 1229 ceiling = structurally unreachable; 218 is the largest that fits). sgl-project#819's break-even carries its provenance because X and P are env-overridable defaults that are never runtime-measured. SECTION 18 -- THE BREACH, CLOSED. Rule 5 says a PR adding a reusable module adds its entry in the SAME PR. Three modules had landed without one: kv_row_ownership.py (sgl-project#822), funding_authority.py (sgl-project#770), uniform_floor_scope.py (sgl-project#823). prefetch_ballot.py (#791b) had the same gap from an EARLIER merge and is added here labelled as backfilled -- the provenance is stated rather than quietly absorbed, because the checker only tests citations that EXIST, so a missing entry costs nothing and is exactly the failure mode rule 5 cannot catch. Each entry says what is NOT wired, in those words. Both authorities are so far consulted only for the arming floor, the census and the retire/audit pair; the clamp firing-rate metric, format_violations, diagnose_floor_band, uniform_absolute_floor and slack_above_uniform_floor have no production caller at this commit. Recording an unwired export as wired is how a catalog becomes a liability. One trap recorded: funding_authority's docstring at :56 names a solve_funding function that does not exist -- the solve is can_fund at :378. TWO DEFECTS FOUND WHILE WRITING, both fixed here: 1. Section 18 cited `watchdog.py:88`, which resolves to python/sglang/srt/watchdog.py -- a file that does not exist in this tree. The intended target is turnkey/watchdog.py:88 (the retired generation probe), now qualified. This was a live red test, not a cosmetic issue. 2. The sgl-project#822 section 3 entry cited kv_backing_relief.py:504 for exposure_over_backing; :504 is a docstring line and the def is at :547. The entry landed already drifted. Corrected, with the drift noted in place per the header rule that the code wins. TESTS. test/registered/unit/docs/test_building_blocks_catalog_538.py, the mechanical section-18 pointer checker: 91 passed / 1 failed BEFORE (the watchdog.py:88 breakage), 101 passed / 0 failed AFTER. The ten new cases are the ten new citations, and every one resolves. The indicator is proven in both directions without needing an injected mutant: it was red on a real bad pointer of exactly the class it claims to detect, and is green now. codespell on the file: clean.

Wave 3, stage 4 of the batched-window tree. WINDOW-QUEUE ticket W6 (strand 21a, preflight_pass=Y: 19 passed red-first with 6F/6P before the fix, 8/8 mutants killed, id-regression 75 files byte-identical with 0 new). Two commits, base 3b2bbde -- which is already ON this line (the sgl-project#770/sgl-project#812 floor-clamp withdrawal), so this stage adds no divergence of its own: b546893 [sgl-project#828] The backing dial converges the BACKING, and a post is credited by what it delivered 4235879 [sgl-project#828] Desk pre-flight for the batched window: the released band clears the gate boot_827 refused at THE DEFECT. `runtime_set_backing_tokens` branched grow-vs-shrink on `self.size` -- the EXPOSED id space -- instead of on the committed backing, so a rung that should have released reported `branch=grow` and released 0. A dial that converges the wrong quantity is not a mis-tuned dial; it is a dial attached to the wrong shaft, and the census downstream then credits a post for delivering nothing. This is the same family the catalog records in section 2 under the funding authority: a post is credited BY WHAT IT DELIVERED, not by what it was asked for. sgl-project#770 named the posts; this makes the credit honest. GATE. Battery test/registered/unit/{managers,planner,server_args,mem_cache}, hermetic under CUDA_VISIBLE_DEVICES="", one battery at a time. baseline (tip 4f2072a) 7 failed, 8585 passed, 1852 skipped, 887 s stage 1 (W1+W2) 7 failed, 8638 passed, 1852 skipped, 868 s stage 2 (W3) 7 failed, 8653 passed, 1852 skipped, 922 s stage 3 (W4a/W5/W4b) 7 failed, 8673 passed, 1852 skipped, 915 s stage 3b (sgl-project#791 stub) 0 failed, 8684 passed, 1852 skipped, 900 s this stage 0 failed, 8703 passed, 1852 skipped, 931 s NEW failure ids NONE THIS IS THE FIRST STAGE GATED AGAINST A ZERO BASELINE. Since stage 3b the line's extracted failure list is EMPTY, so "no new failure ids" here is not a comparison against a tolerated set -- it is the whole result. CITATION MAINTENANCE, carried in this commit because this stage is what forced it. W6 adds 37 lines to funding_authority.py and 71 to memory_pool.py, both of which the catalog cites by file:line, and the section-18 checker did NOT notice: it verifies that a cited line EXISTS, not that the cited SYMBOL is on it, so it stayed at 104 passed while solve_arming_floor moved :659 -> :694, diagnose_floor_band :585 -> :620 and slack_above_uniform_floor :813 -> :848, each then pointing at a comment, an `if` and a dataclass field. A sweep of every citation this train touches found the same drift elsewhere and all of it is corrected here, verified line by line against the files: scheduler.py nine citations (stage 3 added 113 lines) :4684->:4795, :4792->:4903, :4814->:4925, :4938->:5049, :4947->:5058, :4995->:5106, :5011->:5122, :7085->:7196, :7462->:7573; phase_flip_runtime.py five, :3838/:3839->:4040/:4041, :3881->:4083, :6275->:6487, :7128->:7353; memory_pool.py :4978 -> :4983 with the bound assert at :5032. Untouched and re-verified as still correct: kv_row_ownership.py, prefetch_ballot.py, uniform_floor_scope.py, tree_congruence.py, invariant_checker.py:1175, mamba_ckpt_utils.py:185, kv_backing_relief.py:162/:547, phase_policy.py:831. One drift found in the sweep is NOT this train's doing and is labelled as such: planner/placement.py:813 -> :838, in a file no stage here touches. It drifted earlier and nothing caught it, which is the point. The gap is now recorded in section 18.8 with the measurement that proves it, and the interim practice it implies is applied rather than merely described: where a file takes inserts from several tickets, the STATEMENT is cited alongside the line (done for sgl-project#821's three cur_batch_for_debug sites). Symbol-resolving the checker is registered as its own task. A note on the checker being live rather than assumed, because it caught ME while I was documenting its blind spot: writing the phrase `watchdog.py:88` into the section-18.8 prose turned the gap note itself into a section-18 citation, and the checker went 104 passed / 1 FAILED on the spot. It scans section 18 for anything of the shape `file.py:N`, prose included. The wording now names the class instead of quoting a path, and the checker is back to 104 passed / 0 failed -- re-run after the edit, not assumed. So the checker is genuinely armed for the class it covers; the gap in 18.8 is about the class it does not. codespell clean. No boot.

W22 refused every tp_to_pp seam with "reclaimed 0 MiB from [nothing]", 39 times, while the kv-slack post held 2776 MiB. "[nothing]" is the LADDER's provider list and it is honest about the ladder -- but standing alone it reads as "this rig had no memory to give", which sends the next reader to capacity planning instead of to the registry. Part of a window went to that difference. I FIRST GOT THE REMEDY BACKWARDS, and the falsifier I wrote encoded it. Its red assertion was `declared - registered == set()`: the ladder must register every declared post. The tree refutes that at phase_flip_runtime.py:8290-8294 -- "No KV provider is registered with the guard at all, BY DESIGN (the cap is a group decision and the ladder is rank-local), so the rung's bytes arrive as kv_freed BEFORE the probe". Registering kv-slack would let a rank-local ladder spend a group-decided cap, and spend it a SECOND time after the rung already paid. The gap is real; that remedy would have been a double-spend. So the acceptance assertion was corrected to pin the PROPERTY rather than a mechanism -- a refusal must not describe its sources as "nothing" while a declared post holds credit -- and the forbidden remedy is now itself a guard test (`test_kv_slack_must_NOT_be_registered_with_the_ladder`), so nobody closes sgl-project#813 the dangerous way later. THE FIX. `CorridorGuard.declare_offledger_funder(view)` names a funder the ladder may not spend. It is consulted ONLY on a refusal and ONLY for text: it frees nothing, changes no verdict, and is wrapped so a broken view cannot turn a refusal into a crash -- the rule `explain_kv_target` already states, that a diagnostic which can alter what it reports is worse than none. Unset by default, so every existing refusal line stays byte-identical. APPENDED, NOT SUBSTITUTED. The ladder's own "[nothing]" stays, because it is the truthful record of what the gate actually spent. The test pins that both facts appear TOGETHER and in that order -- a later change that drops the suffix and leaves the bare list fails it, which is the regression that matters. WIRED, not merely built -- an unwired capability is this family's whole defect. `get_corridor_guard` attaches a witness for the KV rung, deriving slack with the SAME expression the sgl-project#770 census uses (phase_flip_runtime.py:8716-8721: current - floor_rows from `_last_proposal_terms`). A witness computing slack its own way would be a fifth bookkeeper, which is what this build exists to stop. The rung stays unregistered; the big comment at phase_flip_spill.py:2515 explaining why is untouched and still true. Acceptance: the registry falsifier flipped and its xfail is lifted. On W22's numbers (want 3248, free 2420, empty ladder) the refusal now reads "... reclaimed 0 MiB from [nothing], arming floor 1255 MiB, corridor law 1024 MiB (seam staging tp_to_pp); this gate may not draw on kv-slack holds 2560 MiB -- that funder pays before the gate and its cap is a group decision, so it can never appear in the list above (sgl-project#813)". Tests: 1030 passed / 1 xfailed across funding, relief, corridor, floor, backing, ownership, arming and spill suites -- xfails 2 -> 1, the remaining one being the exposure/veto falsifier that F1 owns. 0 failures. ruff clean. Hermetic, CVD="", no boots.

…all three falsifiers First slice of the user's flip-image design: RAM holds ONE layout image plus a small overshoot, and at the flip the incoming layout streams RAM -> VRAM while the outgoing one streams VRAM -> RAM into the pages just freed. PCIe is full duplex, so the copy-back rides the idle return direction. THE COPY-BACK IS NOT WRITE-BACK. The weights are immutable and nothing is saved; it is residency PLACEMENT for the next flip, which is what a single-layout RAM budget requires. Written into the module docstring because a later reader who mistakes it for a write-back will optimise it away and break the following flip. WHY THIS AND NOT THE PARTIAL PIN: W26 proved the dual pin impossible here -- both pin arms OOM-killed in the LAUNCH phase, before any flip. One layout plus eps (~30 GiB vs ~68.7 GiB) fits AND takes the disk off the steady-state critical path, which is what reaches the physics floor; a partial pin leaves a disk share behind, and W26 measured the leg 99.8-100 % storage-bound. THIS SLICE IS THE ARITHMETIC ONLY, deliberately. The overshoot sizing and the interleaved schedule are pure functions over byte counts, so every invariant the scheme rests on is falsifiable WITHOUT a GPU -- the same split sgl-project#852's estimator and sgl-project#856(a)'s bound phrase use, and for the same reason. OVERSHOOT = size asymmetry + in-flight window, sized from the LARGER direction. The asymmetry is W26's measured one (PP0 15925.8/16362.7, PP1 8573.8/8961.3, PP2 8573.8/9481.6 MiB); a single fixed reservation has to cover whichever direction the next flip takes, so a mean is the OOM. The in-flight term is separate and pinned: an implementation returning only the asymmetry gives 0 for equal layouts and stalls immediately. ALL THREE NAMED FALSIFIERS ARE ASSERTED: * no actual overlap -- `rotation_totals` counts co-scheduled steps; a real rotation must have them and must have them as the DOMINANT shape (>90 % of steps), not as an accident of the tails. Its can-fail partner: a one-sided rotation must report zero overlap. * RAM leak across cycles -- three full A->B->A cycles must return host occupancy exactly to its start. Three, because W27-retry's leak fired on the THIRD cycle, not the first. * checksum -- verified against the real source: the image is `payload = image[:layout.total_bytes]` plus an int64 trailer, checked with `uint8_checksum(dst)` over the ARENA. That last part is what makes a D2H reproducible: bytes returned from VRAM verify exactly as bytes read from disk do, so only the 8-byte trailer is new. A FINDING THE TESTS PRODUCED, and it is why the budget test first passed vacuously: THE RAM BUDGET BINDS IN ONLY ONE DIRECTION. Pressure exists solely when the OUTGOING layout is LARGER than the incoming one, because only then does the copy-back need more RAM than the H2D frees -- PP0 copying back its 16362.7 MiB tp image while the smaller 15925.8 MiB pp image streams in leaves 436.9 MiB with nowhere to go. The opposite direction schedules cleanly at zero overshoot. Both halves are now asserted so the asymmetry is recorded rather than rediscovered. Under-sizing STALLS LOUDLY rather than proceeding: a scheduler that kept going would be holding both layouts, which is precisely the state that OOM-killed W26's pin arms. GATE (foreground, family-batched): managers core (45 PP files excluded) 3554 passed, 336 subtests, 0 failed model_executor 777 passed / 15 failed (+14 new) The 15 are the pre-existing sgl-project#815 family. ZERO new failures. NOT YET BUILT, and not claimed: the device-side execution (streams, the pinned ring registered once per sgl-project#720/sgl-project#729, the planner-priced host post per sgl-project#721/sgl-project#770), and the separately-instrumented priming flip. This slice is the plan those will execute.

feat: add fake tag

b4662c3

zhyncs requested review from Ying1123, hnyls2002 and merrymercy July 27, 2024 14:59

zhyncs merged commit de854fb into sgl-project:main Jul 27, 2024

zhyncs deleted the fake-tag branch July 27, 2024 16:22

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

feat: add fake tag (sgl-project#770)

146c513

LinyuanLi0046 added a commit to LinyuanLi0046/sglang that referenced this pull request Jun 15, 2026

[NPU][A5] Support FP8 for Ascend NPU (sgl-project#770)

d708976

* support fp8 for ascend npu * adapt for deepep * clean code

Navigation Menu

Uh oh!

feat: add fake tag - #770

feat: add fake tag#770
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:fake-tag

zhyncs commented Jul 27, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

zhyncs commented Jul 27, 2024

Motivation

Modification

Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant