Revert "Organize public APIs" by Ying1123 · Pull Request #815 · sgl-project/sglang

Ying1123 · 2024-07-30T02:40:13Z

Reverts #809

This reverts commit c8e9fed.

* Update test_ascend_L2_cache_01.py * Update run_suite.py * Update test_ascend_L2_cache_01.py

…he leak that exposed it Two defects, one symptom: seven tests in test/registered/unit/mem_cache/test_session_branch_rewind_unit.py failed with 'Default process group has not been initialized' whenever anything in test/registered/unit/managers had run first, and passed on their own. The production half. log_info_on_rank0's except branch -- its FALLBACK -- called torch.distributed.get_rank() unguarded. get_rank() throws outright without a default process group, and is_initialized() does not rule that out: it is a plain module attribute that test doubles and embedding hosts replace. When it threw, it threw out of SessionController.rewind_to (session_controller.py:514), so a log line aborted a session rewind. A rank that cannot be determined now degrades to 'log it', exactly as the never-initialized case already did. The test half. test_dynamic_chunk_profile_661.py saved and restored the process globals torch.distributed.is_available / is_initialized inside _run, which runs in _N racing threads. Thread B saved thread A's lambda as the 'original' and restored it in its finally, so is_initialized stayed pinned to True for the rest of the process. The save/restore now happens once, on the thread that owns the workers. Evidence, both directions: victim alone 13 passed polluter file + victim 7 failed, 9 passed after both fixes 16 passed polluter file alone, before/after 3 passed (unchanged) Direct probe with is_initialized pinned True and no group: the helper logs '(rank-check failed: ...)' and returns instead of raising.

…s neighbours keep measuring test_flip_arm_snapshot_746.py's test_abandon_paths_clear_behaviorally silences sglang.srt.managers.phase_flip_runtime while it drives the three abandon paths. The silence was correct; leaving it behind was not. The logger is process-global, so at CRITICAL it also swallowed the ERROR records that sgl-project#800's test_the_abandonment_names_this_rank_s_own_withhold asserts on -- that test then failed as 'no abandonment was logged', with nothing in its own file to point at. caplog raises the ROOT level, which cannot undo a level pinned on the module logger itself. Introduced 2026-08-17 by 8433ccd [sgl-project#746]. It reddens sgl-project#800's suite only in combination, which is why the sgl-project#800 line measured that suite green: the combination it measured did not contain this file. Evidence: 746 + 800 together 1 failed, 42 passed same pair, this one test deselected 42 passed, 1 deselected after the restore 43 passed

…or grew Both _bare_arena helpers build a KvVmmArena with object.__new__ on purpose -- the real constructor calls cuInit, reserves address space and compiles a C stub, none of which is under test -- and then hand-set the fields commit and decommit touch. Production grew one more on 2026-08-16 (17e7c8e, [sgl-project#464]): self._coalesce_resume, read in commit_range at kv_vmm_backing.py:667. Both helper files were last touched 2026-08-09, so every test in them has raised AttributeError ever since. Set to False explicitly, mirroring the constructor's own documented DEFAULT OFF, rather than resolved from the environment: off reproduces the per-sgl-project#330-chunk plan byte-for-byte, and that plan is what these sgl-project#631 assertions are written against. Flipping the sgl-project#464 lever on for a measurement must not silently reshape the plan under test. test_kv_arena_handle_retention_631.py 7 failed, 1 passed -> 8 passed test_kv_arena_span_ops_631.py 1 failed, 11 passed -> 12 passed

…uard actually sits on The pp_with_spec registry entry cited server_args.py:17583 and :17598. Those lines now carry sgl-project#797's provenance block; the guard itself moved to server_args.py:18958 (if self.pp_size > 1) and the spec assert to server_args.py:18973 (verified by reading the block, not by search alone). The overrides.py:2163 half of the cite is still correct and is untouched. This is the drift test_rejected_evidence_pins.py exists to catch, and the entry's own text records the previous occurrence ('the earlier :16240-16245 cite had drifted'). A rejection reason nobody can follow back to the code is a reason that stops being checkable. test_rejected_evidence_pins.py 1 failed, 5 passed -> 6 passed

…init__ builds, and re-pin the drifted evidence cite Two things this merge stage surfaced, both mechanical. test_hicache_forced_handover_unit.py's _build_cache uses HiRadixCache.__new__ on purpose -- the real constructor builds host pools, a controller thread and CUDA state, none of it under test -- and hand-sets the fields the insert path touches. sgl-project#810 added one more in __init__: self.staging_write_ring, which the whole insert path then guards on with 'is not None'. Without it every guard raised AttributeError instead of evaluating False. Set to None, the shipped state under --hicache-host-role retention (the default, and the regime these cases are written against); UnifiedRadixCache spells the same default out at unified_radix_cache.py:757. This reproduces on feat/810-hicache-staging-ring @ c089029 standalone, so it is that branch's own test debt, not a merge interaction. test_hicache_forced_handover_unit.py 9 failed, 6 passed -> 12 passed rejected.py's pp_with_spec evidence cited server_args.py:18958/:18973 as of [sgl-project#815]. sgl-project#810 inserted ~84 lines above them; the guard is now at :19042 and the spec assert at :19057 (read, not searched). Second drift of this cite in one day -- it is a line-number pin, so every merge that touches server_args.py moves it, which is exactly what test_rejected_evidence_pins.py exists to catch. test_rejected_evidence_pins.py 1 failed, 5 passed -> 6 passed

Closes the Cluster 4 defect that sgl-project#815 escalated rather than fixed: the sgl-project#677 HOLD wrapper swallowed the blocked-admission exit. sgl-project#817 inverts it into an allowlist exactly as the wrapper's own comment prescribed. Clean merge against the sgl-project#790 phase_policy.py edits.

Four commits from 16f, each tested and pushed separately: [sgl-project#801] a dead PP peer is not the wedge (the transport already reports it), [sgl-project#821] the watchdog could not see the wedge it was best placed to diagnose, the ballot-divergence observability commit, and the uniformity-floor scope reporter. NUMBERING, recorded because the titles do not match the register: the head commit 9d13bf0 is titled '[sgl-project#824]' but canonically it is the register's sgl-project#823 artefact (the floor-scope reporter). Register sgl-project#824 is 18f's chunked-prefill protected-len crash fix, merged as its own stage below. No rebase -- the commit titles stand as written and the mapping is documented here, in the COORD and in the catalog. Clean merge. Shares test_collective_family_siblings_610.py with fix/815 (not yet merged): checked at hunk level, sgl-project#801 adds two class-level counters to BudgetHarness (:465) while sgl-project#815 fixes _budget_state_stub (:540/:557) -- different symbols, different drift instances, no collision.

Two commits closing the rest of the sgl-project#815 stale-stub debt that 17b rooted but left standing with a verdict: eight test helpers that had drifted behind the code they measure, plus the two pp_* stubs that 17b had judged unfixable without a sgl-project#791-shaped rewrite and that turned out to be fixable faithfully. Test-only change. Shares test_collective_family_siblings_610.py with the already-merged fix/801: checked at hunk level before merging -- sgl-project#801 adds two class-level counters to BudgetHarness (:465), sgl-project#815 fixes _budget_state_stub (:540/:557). Different symbols, different drift instances, no collision.

… stub These 7 tests had been red on an AttributeError: the loop under test asks `not self.pp_group.is_first_rank` (scheduler_pp_mixin.py:1343) and the test-local `_Group` stub carried `is_last_rank = True` and nothing else. THE CHEAP REPAIR IS THE TRAP. Adding `is_first_rank = True` beside it turns all 7 green immediately -- and makes this rank the FIRST and the LAST stage of a three-stage pipeline at once, which no rank of a pp_size=3 ring can be. Every branch keyed on either role would then take the wrong arm while the suite reported green. That is the sgl-project#630 lesson exactly: an unfaithful stub does not merely fail to catch a defect, it encodes the defect's assumption and then certifies it. So the stub now carries a REAL position in a REAL ring. `_Group(rank, pp_size)` derives both roles from that position, and rank 2 is the default because that is what the old lone `is_last_rank = True` was reaching for: the last stage, which skips the proxy-send block that is not under test. The difference is that it is now last WITHOUT also claiming to be first, so the admission-decision branch takes the arm a real last rank takes. The collaborators on that arm are stubbed to the cheapest thing that keeps the control flow real -- they are not the subject; the loop CONTROL is, and it still comes unbound off the mixin. `assert_faithful_pp_roles` makes the forbidden combination loud rather than silent, and `_Rank` runs it on construction. THE RED-FIRST LOGIC IS INVERTED HERE, and that is the honest shape for a repair to a suite that was already red. The proof is in two directions: * the 7 tests go GREEN with the faithful stub (18 passed total); * a PLANTED first==last stub goes RED -- 16 of 18 fail. That is the can-fail, and it is what stops the cheap repair being reintroduced. Four new tests carry that second direction, including one that is easy to forget: `test_the_faithful_role_is_load_bearing_not_decorative` asserts the non-first admission arm is ACTUALLY entered. Without it the added stubs could be dead code and "faithful" would be doing no work -- the suite would be green because the branch was never reached, which is the state it was in before, only quieter. TESTS (hermetic, CVD="", CPU only, no CUDA, no distributed) test_pp_flip_slot_hold_631.py 18 passed (was 7 failed / 7 passed). Mutant killed: `is_first_rank` forced True -> 16 failed. Battery re-measured under /spinning/htsglang-gpu/.venv (datasets 5.0.0, full collection), over the 7 files that carried every failure: base 500be7e 22 failed / 35 passed HEAD before this commit 22 failed / 35 passed HEAD with this commit 15 failed / 46 passed -7, exactly the tests repaired here, and this branch still adds none. Of the 15 remaining, 3 (test_pp_slot_last_batch_631) are an artefact of this branch's base being pre-sgl-project#815: fix/815-rest-stubdrift landed on the line at 4f2072a, while base 500be7e sits on the earlier ancestor 21ff075. They disappear when this stage merges onto the line. No boot was run. This is desk work.

…gl-project#821, sgl-project#823, sgl-project#824, sgl-project#770/sgl-project#812, sgl-project#815; close the section 18 breach Merge-checklist duty for the five wave-2 stages on this branch (fix/801, fix/824, feat/770, feat/822, fix/815). Mechanism text is taken from the commits' own measured evidence, not restated from ticket titles. WHAT WAS ALREADY THERE, checked before writing rather than after. feat/822 catalogued ITSELF mid-train -- 21ff075 carries +49 lines of this file -- so sgl-project#822 already has a complete section 3 entry including the sgl-project#816 clamp firing-rate baseline. No second sgl-project#822 entry was written. sgl-project#801, sgl-project#815, sgl-project#821, sgl-project#823, sgl-project#824 and sgl-project#770 had zero prior mentions; sgl-project#812 was name-checked inside the KV-backing-relief bullet as a consequence of the cap, with no entry of its own. SECTION 12, five new families: Said-once (sgl-project#823) -- a fact that changes during a process, reported once at the moment it first became true, is a stale label afterwards. Both instances live in _update_uniform_pool_budget: the floors' coverage said three times at startup and never again through four cutovers in 55 s, and a divergent TP queue head logged as an event when it is a duration. The mutant that makes the case: disabling the recovery edge left all 14 tests green before the streak was extracted. Blind-instrument (sgl-project#821) -- the scheduler watchdog's activity predicate reads exactly the two values a rank parked in the dict receive freezes, so the instrument best placed to see the wedge is the one structurally unable to. Non-monotone-source (sgl-project#824) -- a monotone guarantee (the protected prefix) derived from a non-monotone tracker (mamba_last_track_seqlen). Landed as a measured assertion, cache_protected_len=16384 against page_aligned_len=8192. Invariant-for-funding (sgl-project#770, sgl-project#812) -- a group-derived per-rank floor asking PP1 for 102.9% of its own cap, and the withdrawal of a floor clamp that bought funding by breaking the levelling invariant (1 failed with, 12 passed without). Stub-drift (sgl-project#815) -- ten helpers drifted behind named commits, and the rule the repair follows: faithfully or not at all. This entry also RECORDS why seven cases in test_pp_flip_slot_hold_631.py are deliberately left red -- the faithful stub is a real rank identity, which drags the whole admission-receive path in with it. That is documentation of a known state, not a claim that it is fixed. Plus a NUMBER-COLLISION note, because the log is ambiguous without it: 9d13bf0 is subject-tagged [sgl-project#824] and names its test file _824.py but is canonically register-sgl-project#823 (floor-scope reporter, strand 16f, arriving on fix/801-admission-recv-liveness); the canonical sgl-project#824 is 658d0c1 on fix/824-chunked-prefill-protected-len. No rebase, so the titles stand and the mapping is written down instead. SECTION 7: sgl-project#801's dead-peer measurement, which REFUTES the premise under the sgl-project#816 survivor analysis -- an exited or SIGKILLed peer makes the admission receive raise after 1.02 s, while alive-never-sends and SIGSTOP both block unbounded, so a rank silent for 119.7 s was not waiting on a corpse. SECTION 2: the funding authority (sgl-project#770/sgl-project#584/sgl-project#819) -- one question, posts declared by name, the three priced losses (L1 the unreachable 2638 MiB, L2 a 3437-row ask against 8192-row granularity returning a silent zero at 65% of the shortfall, L3 claimed=0 against an accounted 107 MiB), and the arming floor solved rather than asserted (512 MiB reserve needs 1523 against a 1229 ceiling = structurally unreachable; 218 is the largest that fits). sgl-project#819's break-even carries its provenance because X and P are env-overridable defaults that are never runtime-measured. SECTION 18 -- THE BREACH, CLOSED. Rule 5 says a PR adding a reusable module adds its entry in the SAME PR. Three modules had landed without one: kv_row_ownership.py (sgl-project#822), funding_authority.py (sgl-project#770), uniform_floor_scope.py (sgl-project#823). prefetch_ballot.py (#791b) had the same gap from an EARLIER merge and is added here labelled as backfilled -- the provenance is stated rather than quietly absorbed, because the checker only tests citations that EXIST, so a missing entry costs nothing and is exactly the failure mode rule 5 cannot catch. Each entry says what is NOT wired, in those words. Both authorities are so far consulted only for the arming floor, the census and the retire/audit pair; the clamp firing-rate metric, format_violations, diagnose_floor_band, uniform_absolute_floor and slack_above_uniform_floor have no production caller at this commit. Recording an unwired export as wired is how a catalog becomes a liability. One trap recorded: funding_authority's docstring at :56 names a solve_funding function that does not exist -- the solve is can_fund at :378. TWO DEFECTS FOUND WHILE WRITING, both fixed here: 1. Section 18 cited `watchdog.py:88`, which resolves to python/sglang/srt/watchdog.py -- a file that does not exist in this tree. The intended target is turnkey/watchdog.py:88 (the retired generation probe), now qualified. This was a live red test, not a cosmetic issue. 2. The sgl-project#822 section 3 entry cited kv_backing_relief.py:504 for exposure_over_backing; :504 is a docstring line and the def is at :547. The entry landed already drifted. Corrected, with the drift noted in place per the header rule that the code wins. TESTS. test/registered/unit/docs/test_building_blocks_catalog_538.py, the mechanical section-18 pointer checker: 91 passed / 1 failed BEFORE (the watchdog.py:88 breakage), 101 passed / 0 failed AFTER. The ten new cases are the ten new citations, and every one resolves. The indicator is proven in both directions without needing an injected mutant: it was red on a real bad pointer of exactly the class it claims to detect, and is green now. codespell on the file: clean.

Wave 3, stage 1 -- the first stage of the batched-window tree. Carries WINDOW-QUEUE tickets W1 (sgl-project#825/sgl-project#827, release-walk crash guard) and W2 (sgl-project#826, solved arming floor), which the queue requires to ship in the SAME tree: W1 lets the boot survive the flip walk and W2 lets the flip actually arm, so validating either alone repeats boot_827's wasted window. Four commits, base 21ff075 (the feat/822 stage): bb09574 [sgl-project#825] Prove the prefix trees congruent at the one aligned point 80ca356 [sgl-project#826] Adopt the solved arming floor, or refuse the boot by name e0895cf [sgl-project#825/sgl-project#827] Withdraw the tree reset: it crashed on resident lock refs 500be7e [W2] Pre-flight: arming reachability must include the measured seam draw ANCESTRY COMPUTED, NOT INFERRED FROM THE BRANCH NAME. merge-base with the tip is 21ff075, i.e. this branch does not carry fix/815-rest-stubdrift. That matters only if the two touch the same files, and they do not: sgl-project#815 is test-only across ten helper files, none of which are the five test files here. CONFLICT LAGE: none, and checked rather than hoped. `git merge-tree --write-tree` against the tip returned a single tree with no conflict block before the merge was attempted, and the merge itself reported "Automatic merge went well". The file surface is DISJOINT from wave 3's other stage (fix/801-void-send-contract, which touches only scheduler_pp_mixin.py), so the two stages cannot interact textually. W2's ticket is preflight_pass=N and that is NOT a reason to keep it off the line. The queue's N is CONDITIONAL: the solved arming floor is reachable only while the seam's measured draw stays at or below the solved 218 MiB reserve, and the measured draws on this rig are 162/387/436/907 MiB. That gates the WINDOW, not the merge -- the actuator sits behind SGLANG_ARMING_FLOOR_SOLVED, so merging it changes no default. Delivering one bootable tree is this strand's job; whether the window runs is the operator's. The branch's own pre-flight suite is worth naming because it refuses a false green rather than manufacturing one: test_arming_reachability_preflight_w2.py asserts `band_floor + max(reserve, measured_draw) + margin <= band_ceiling` and records that the obvious W2 assertion -- 819 + 218 + 192 = 1229 <= 1229 -- is true and useless, because it silently assumes a measured draw of zero, which is the case that does not occur. SECTION 18 DUTY, DISCHARGED IN THIS MERGE. This stage adds a new reusable module, managers/tree_congruence.py, so rule 5 applies to this merge the same way it applied to the four modules catalogued in 9f1af20. The entry is in this commit (docs/dev/FEATURE_CATALOG.md, section 18.5). The reusable part is not the prefix-tree logic but the equality-by-MIN-reduce trick: a digest sent as the pair (d, -d) makes the group agree iff group_min == -group_neg_min, which turns "are all ranks equal?" into two MIN-reduces over a collective that offers only min. The entry states the gate honestly: DETECTION is ungated and always runs, while the RECOVERY is behind SGLANG_TREE_RECONCILE (phase_flip_runtime.py:3777, unset = OFF) because the reset it performs is what crashed on resident lock references and was withdrawn rather than repaired in place. GATE. Battery test/registered/unit/{managers,planner,server_args,mem_cache}, hermetic under CUDA_VISIBLE_DEVICES="", PYTHONPATH set to this worktree, one battery at a time. baseline (tip 4f2072a) 7 failed, 8585 passed, 1852 skipped, 887 s this stage 7 failed, 8638 passed, 1852 skipped, 868 s NEW failure ids NONE (comm against the baseline list is empty) fixed vs baseline 0 The +53 passes are this stage's own new cases; the failure set is unchanged and is still exactly the seven known ones. The baseline was re-measured in this worktree rather than inherited from strand 17c, and came out byte-identical to its list: the same seven cases, all in test_pp_flip_slot_hold_631.py, left red on purpose (see the stub-drift family in section 12 of the catalog). The gate is not vacuum-green: all five suites this stage adds land INSIDE the battery directories and are therefore actually collected -- test_arming_floor_actuator_826.py, test_arming_reachability_preflight_w2.py, test_tree_congruence_825.py and test_tree_congruence_wiring_825.py under managers, test_lock_release_walk_detached_827.py under mem_cache. Section-18 pointer checker (test_building_blocks_catalog_538.py): 104 passed, up from 101, the three new cases being the three new tree_congruence citations. COVERAGE CHECK, because a battery that silently fails to collect is a gate that measures nothing. The W4/W5 strand reported that its worktree cannot collect the managers suite at all -- 92 modules import `datasets`, which is not installed there. That does NOT apply to this gate, and it was verified in the positive direction rather than inferred from the absence of an error message: the venv here has datasets 5.0.0, the battery log contains zero collection errors and zero ModuleNotFoundError, and an explicit --collect-only over test/registered/unit/managers reports 3436 tests collected. The numbers above therefore cover the managers suite in full. No boot. Serving stays down; this tree is the input to ONE batched window, not a reason to open one.

Wave 3, stage 3 of the batched-window tree. WINDOW-QUEUE tickets W4 (PP ring wedges immediately after an abandoned flip) and W5 (sgl-project#821 did not instrument the receive path that actually wedged), both preflight_pass=Y. Four commits, base 500be7e -- which is EXACTLY this tree's stage-1 branch, so this stage sits directly on top of W1/W2 with no divergence to reconcile: 2efb933 [sgl-project#824 W4a] Bound the PP chain receive without tearing the stream it guards dd92de4 [sgl-project#824 W5] Name the arm that tripped the watchdog, and instrument the path that wedged 430e468 [sgl-project#824 W4b] A zero-iteration armed window must not move the slot ceb79d6 [sgl-project#824 W4a] Arm the chain-recv recovery on STATE, and cut the ring with the existing drain GATED AGAINST ceb79d6, NOT 430e468. The branch head moved while this stage's first battery was already running, so that run was measuring a superseded commit and was DISCARDED rather than reported -- a gate that certifies a commit the tree does not carry is worse than no gate. The in-flight battery was stopped by explicit PID (no pkill), py-spy dumped first per standing rule and found healthy mid-test rather than wedged, and its partial log kept aside as w3c_stale430.log. A foreign battery belonging to another session was running on the box at the same time and was NOT touched. The final W4a commit arms the recovery on STATE (`bump_attempted`, entered >= taken as the evidence predicate) and cuts the ring through the existing sgl-project#757 drain, with the wall-clock default left at 0. W4b is the ROOT of the pair, and it lands in the same mechanism as this branch's own known-red neighbours: an armed window that ran ZERO slot iterations must not advance mb_id. The boot_827 specimen recorded exactly that -- "rank 0 ran 0 slot iteration(s) (armed at mb_id=0, disarmed at mb_id=1)" -- one line before the ring stopped dead and stayed silent for 31 s until the health check noticed. That is the sgl-project#631 defect class with the spread reduced to a single rank: arm on one slot, leave on another, then re-enter the pipeline wherever you happened to stop. TOUCH SURFACES, computed rather than assumed. This stage shares scheduler_pp_mixin.py with both earlier wave-3 stages and with wave 2, and it extends sgl-project#821's own suite (test_pp_wedge_watchdog_is_honest_821.py) rather than duplicating it -- W5 is the continuation of sgl-project#821, by the same argument sgl-project#821 made, so touching that file is intended and not a collision. It also adds to mem_cache/hicache_collective.py, the file carrying sgl-project#734's `waited < timeout_s * 0.95` discriminator that strand 17c's 622 falsification identified as the fragile part. That last point is what makes this gate non-vacuous rather than merely green: `test_pp_sync_rendezvous_630.py` -- the suite that went red under the 622 merge for precisely this threshold, and the reason 622 is still off the line -- sits INSIDE the battery. If this stage disturbed that discriminator, the gate would say so in the same way it said so for 622. NOTE ON A PATH, because it looks like a contradiction and is not: this stage modifies python/sglang/srt/utils/watchdog.py, while commit 9f1af20 fixed a catalog citation that resolved against python/sglang/srt/watchdog.py. Those are different files; the catalog's intended target was turnkey/watchdog.py:88 (the generation probe retired by user order), which is where it now points. GATE. Battery test/registered/unit/{managers,planner,server_args,mem_cache}, hermetic under CUDA_VISIBLE_DEVICES="", one battery at a time. baseline (tip 4f2072a) 7 failed, 8585 passed, 1852 skipped, 887 s stage 1 (W1+W2) 7 failed, 8638 passed, 1852 skipped, 868 s stage 2 (W3) 7 failed, 8653 passed, 1852 skipped, 922 s this stage 7 failed, 8673 passed, 1852 skipped, 915 s NEW failure ids NONE (comm against the baseline list is empty) fixed vs baseline 0 THE +20 IS ACCOUNTED FOR EXACTLY, not assumed. Four new suites adding only 20 passes is low enough to be worth checking, because a suite that silently skips looks identical to a suite that passes cheaply. Two facts settle it: the skipped count is 1852 in every run of this train -- baseline, stage 1, stage 2 and this one -- so nothing new is being skipped; and running the four suites standalone reports exactly 20 passed, matching the battery's delta to the case. They run, and they are collected. COVERAGE, and this stage is the reason to state it explicitly. The W4/W5 strand reported that its OWN worktree cannot collect the managers suite -- 92 modules import `datasets`, which is not installed there. That limitation is NOT present in this gate, verified in the positive direction rather than inferred from a missing error line: datasets 5.0.0 is importable in this venv, the battery logs carry zero collection errors and zero ModuleNotFoundError, and an explicit --collect-only over test/registered/unit/managers reports 3436 tests collected. So this stage's no-regression claim rests on a fully collected managers suite, which the strand's own local run could not provide. Its four new suites land inside the battery directories and are therefore actually collected: test_pp_ring_abort_recovery.py, test_pp_wedge_watchdog_names_the_arm_824.py, test_pp_zero_iteration_window_slot_824.py and test_pp_chain_abort_check_824.py, all under managers. ON PROVENANCE, if this stage ever shows a new failure id: the branch is PRE-sgl-project#815 (its base 500be7e sits on 21ff075), so the last_batch-631 failures visible in the strand's own worktree are expected to DISAPPEAR on this line, which carries fix/815. A new id here would therefore have to be checked against that provenance before being attributed to the strand's change. In this run none appeared, so the question stays hypothetical. No boot. This tree is the input to ONE batched window.

…ip deferred with its invariant named W27-retry leaked 152 rows per retract+drop cycle and died on the third. This is the root, derived from the tree code rather than from an API guess. `MambaRadixCache.reset` (mamba_radix_cache.py:555) installs a NEW `TreeNode()` as root and zeroes `full_evictable_size_` / `full_protected_size_`. It frees no device row -- the old tree is simply dereferenced, and the rows its nodes held are orphaned. It is a BOOKKEEPING reset: right for a teardown where the pool is reset too, wrong for a seam that keeps serving. That is why the detector saw `evictable=22` (the NEW tree) while 152 rows belonged to nobody, visible only as a total mismatch. The call that actually returns rows is `evict` -> `evict_full`, whose leaf path frees through `token_to_kv_pool_allocator.free`. So the drop is EVICT-THEN-RESET (`drop_prefix_tree_returning_rows`). THE EVICTION IS LEGITIMATE ONLY BECAUSE OF THE FENCE. sgl-project#703 has already persisted these prefixes to the canonical store and the new layout re-reads them; without that, this would be data loss. That is why the seam order is fence -> retract -> drop and not any permutation of it, and it is written into the function so a later reader cannot reorder it innocently. BOTH DANGER DIRECTIONS ARE PINNED, because only one of them is loud: * orphaning -- the metal defect, modelled with the accumulate-per-cycle shape it actually had (it fired on the THIRD cycle, not the first); * DOUBLE-RETURN -- silent where the leak was loud. The test allocator raises on a second free, so an evict-and-also-free implementation fails here rather than corrupting the pool quietly. A deliberate contract change: `build_cutover_release` no longer hands back the bare `tree_cache.reset`, and `test_seam_order_856` was updated to assert that it does NOT -- the bare reset is the tempting one-liner and it is exactly what leaked. THE EMPTY WAVE LOOP IS DEFERRED, WITH THE INVARIANT NAMED. W27-retry measured 16 empty waves costing ~314 ms plus 753.6 ms for retract+consume+drop. Skipping the loop looks like free seam time and is not safe yet: `finalize_wave` calls `dst.restore_backing(layers)`, which marks the destination layers RESIDENT independently of whether any KV moved. Skipping would leave the pool answering NO to `backing_is_resident`. The cheap successor is one `restore_backing(all layers)` outside the loop; it changes backing semantics and needs its own test, so it is a follow-up and not a line here. THE LOAD DRIVER IS FIXED, and its bug was a measurement trap worth recording. W27-retry's driver reported real token counts on requests that never connected: curl failed (dead-port smoke confirms code='000', rc=7, NO file written) and the parser then read `/tmp/w26_resp_<tag>.json` -- a LEFTOVER from the W26 window, same tag scheme, same prefix -- and reported W26's numbers as this run's. Two fixes: a unique prefix plus `rm -f` before the curl, so a failure can only ever yield a MISSING file and never someone else's numbers; and a health check that ABORTS after 3 consecutive failures. Mock-smoked against the dead server: exit 3 at round 4, instead of the 640 rounds in 45 s the broken one spun. GATE (foreground, family-batched, per the sgl-project#749 lesson; no background waiters): managers core (45 PP files excluded) 3554 passed, 18 skipped, 336 subtests, EXIT=0 PP family, one file per process 45/45, 0 failures mem_cache 1710 passed, 361 subtests, 0 failed server_args 776 passed, 191 subtests, 0 failed model_executor 763 passed / 15 failed The 15 are the pre-existing sgl-project#815 family, verified earlier against HEAD (identical 15/754 set). ZERO new failures.

…all three falsifiers First slice of the user's flip-image design: RAM holds ONE layout image plus a small overshoot, and at the flip the incoming layout streams RAM -> VRAM while the outgoing one streams VRAM -> RAM into the pages just freed. PCIe is full duplex, so the copy-back rides the idle return direction. THE COPY-BACK IS NOT WRITE-BACK. The weights are immutable and nothing is saved; it is residency PLACEMENT for the next flip, which is what a single-layout RAM budget requires. Written into the module docstring because a later reader who mistakes it for a write-back will optimise it away and break the following flip. WHY THIS AND NOT THE PARTIAL PIN: W26 proved the dual pin impossible here -- both pin arms OOM-killed in the LAUNCH phase, before any flip. One layout plus eps (~30 GiB vs ~68.7 GiB) fits AND takes the disk off the steady-state critical path, which is what reaches the physics floor; a partial pin leaves a disk share behind, and W26 measured the leg 99.8-100 % storage-bound. THIS SLICE IS THE ARITHMETIC ONLY, deliberately. The overshoot sizing and the interleaved schedule are pure functions over byte counts, so every invariant the scheme rests on is falsifiable WITHOUT a GPU -- the same split sgl-project#852's estimator and sgl-project#856(a)'s bound phrase use, and for the same reason. OVERSHOOT = size asymmetry + in-flight window, sized from the LARGER direction. The asymmetry is W26's measured one (PP0 15925.8/16362.7, PP1 8573.8/8961.3, PP2 8573.8/9481.6 MiB); a single fixed reservation has to cover whichever direction the next flip takes, so a mean is the OOM. The in-flight term is separate and pinned: an implementation returning only the asymmetry gives 0 for equal layouts and stalls immediately. ALL THREE NAMED FALSIFIERS ARE ASSERTED: * no actual overlap -- `rotation_totals` counts co-scheduled steps; a real rotation must have them and must have them as the DOMINANT shape (>90 % of steps), not as an accident of the tails. Its can-fail partner: a one-sided rotation must report zero overlap. * RAM leak across cycles -- three full A->B->A cycles must return host occupancy exactly to its start. Three, because W27-retry's leak fired on the THIRD cycle, not the first. * checksum -- verified against the real source: the image is `payload = image[:layout.total_bytes]` plus an int64 trailer, checked with `uint8_checksum(dst)` over the ARENA. That last part is what makes a D2H reproducible: bytes returned from VRAM verify exactly as bytes read from disk do, so only the 8-byte trailer is new. A FINDING THE TESTS PRODUCED, and it is why the budget test first passed vacuously: THE RAM BUDGET BINDS IN ONLY ONE DIRECTION. Pressure exists solely when the OUTGOING layout is LARGER than the incoming one, because only then does the copy-back need more RAM than the H2D frees -- PP0 copying back its 16362.7 MiB tp image while the smaller 15925.8 MiB pp image streams in leaves 436.9 MiB with nowhere to go. The opposite direction schedules cleanly at zero overshoot. Both halves are now asserted so the asymmetry is recorded rather than rediscovered. Under-sizing STALLS LOUDLY rather than proceeding: a scheduler that kept going would be holding both layouts, which is precisely the state that OOM-killed W26's pin arms. GATE (foreground, family-batched): managers core (45 PP files excluded) 3554 passed, 336 subtests, 0 failed model_executor 777 passed / 15 failed (+14 new) The 15 are the pre-existing sgl-project#815 family. ZERO new failures. NOT YET BUILT, and not claimed: the device-side execution (streams, the pinned ring registered once per sgl-project#720/sgl-project#729, the planner-priced host post per sgl-project#721/sgl-project#770), and the separately-instrumented priming flip. This slice is the plan those will execute.

…NE image The wiring slice. Slice 2 built and proved the executor; this makes it the real path and removes the two-image allocation it was built to replace. BOOT ALLOCATES ONE HOST IMAGE, NOT TWO. `PhaseFlipStacks` carried `image_pp` AND `image_tp` for the life of the process -- that is the dual pin, and W26 OOM-killed BOTH its arms in the LAUNCH phase, before any flip ran. It now carries a single `rotation_image`, sized max(pp, tp) + trailer, plus `image_holds` naming which layout is resting in it. `image_from_tensors` grew an `out=` arm so the boot snapshots straight INTO that buffer instead of beside it, and the PP image is released the moment the arena carries the PP layout. The boot's two-image PEAK is unchanged (the PP snapshot necessarily predates layout_tp, so the max cannot be known earlier); what changes is the STEADY STATE, which is where the design's budget lives: one layout + overshoot. `refill()` ROTATES. Both directions go through `rotate_arena`: the target layout streams RAM -> VRAM while the outgoing one is placed back into the pages it frees. The high-water commit stays exactly where it was and means MORE now than it did -- the leg touches both layouts structurally rather than only via a recovery arm. ONE PATH, NOT TWO. The priming fill is the same call with `outgoing_bytes=0`: nothing in the arena worth keeping, so the copy-back has zero length and it degenerates to the plain contiguous H2D it always was, carrying `priming=True` so it can never be averaged into a warm number (P4). A buffer holding the wrong layout is NOT a fallback -- under a single-image budget there is no second source to fall back to, so it is an invariant violation and it refuses. The three-leg `moves_nothing` guard and the directional budget law are untouched. WHAT THIS GIVES UP, stated because it is a real loss and not an oversight. `arena_refill`'s `restore=(other_layout, other_image)` rewrote the ACTIVE layout from its own separate image on a checksum mismatch, so an abort left both layouts byte-exact. That arm NEEDS a second lifetime image, which is precisely the pin W26 proved impossible here. With one buffer it cannot exist: a mismatch now declares the arena undefined and refuses loudly instead of silently serving it. `test_restore_never_rebuild_677` is rewritten to pin the replacement rather than deleted -- its actual invariant (a flip may pay COPY time, never BUILD time) is unchanged and now also covers the copy-back, which is the new most-likely place to reach for a rebuild. TEST CHURN, all of it contract-driven and none of it widened away: * test_restore_never_rebuild_677 -- restore arm -> refusal; new pin that the copy-back places the outgoing layout by COPY under the build fence. * test_arena_high_water_631 -- stub moved from `arena_refill` to `rotate_arena`; the high-water need is now max(incoming, outgoing) structurally instead of via `restore[0]`. * test_acceptance_emitters_758 -- same seam move; the sgl-project#758 emitter is still asserted to fire, which is what that file exists for. * test_phase_flip_boot (scheduler) -- one max-sized image; the arena is now PRIMED with PP, because the first refill copies those bytes back and the second streams them in, so an unprimed arena would round-trip garbage. GATES (foreground, family-batched, PYTHONPATH pinned to this worktree): managers batch B (64 PP files, one process each) 697 passed, 0 failed model_executor 15 failed / 803 passed -- the same three sgl-project#815 files managers batch A 2641 passed; 1 failure + 1 collect error, both identical at HEAD (+1 pass = the split restore test) mem_cache (chunked) 442 / 7f+474p / segfault / 24f+1192p -- byte-identical to HEAD, segfaulting chunk segfaults at HEAD too scheduler test_phase_flip_boot 31 passed new wiring suite 13 passed ZERO new failures. ruff F/E7/E9 clean on every new and changed file; phase_flip_boot 1 -> 1 (pre-existing unused `pack_into_arena`). server_args was not re-run: nothing in this slice touches it, and its ledger post landed and was gated in the previous commit. DESK-PROVEN: the rotation is byte-exact through the REAL `PhaseFlipStacks. refill` over CPU tensors, in both directions and across three full A->B->A cycles, with the buffer and the arena returning to their exact starting content each time; the wrong-layout refusal and the corrupt-image refusal both fire and both have passing can-fail partners. NOT CLAIMED: nothing here has run on a GPU. The W26-instrument re-measurement (warm 1-2 s both directions, read_s ~0, the verdict leaving STORAGE-BOUND) is what the proof window is for.

…aHostUnregister before the pages go back FOUND ON METAL, in the W28 proof window (specimen /spinning/evidence-665-f1/SPECIMEN_w28_a1_hostregister712.log). The boot died during HiCache's host KV allocation: cudaHostRegister failed (rc=712, part or all of the requested memory range is already mapped) for ptr=0x76a88d7d8000 size=5612142592 MY DEFECT, introduced by the wiring slice. `_alloc_host_image` page-locks through `cudaHostRegister`. Until W28 every host image lived for the life of the process, so nothing ever had to be given back and no release path existed. The chunk rotation frees the PP boot image once the arena carries that layout (`del image_pp`), which returned the pages to the process allocator while CUDA still had the RANGE registered. The next large host allocation landed on it and its own registration failed. The registration is a process-wide fact about an ADDRESS RANGE, not a property of the tensor. Freeing without unregistering leaves a landmine at an address nobody owns any more, and the victim is whoever mallocs next -- which is why the traceback pointed at HiCache, a component with nothing to do with the flip. `release_host_image` undoes the allocation in the right order: cudaHostUnregister first, then the caller drops the reference. It also returns the pinned-registry post, which previously could only be given back on an allocation FAILURE (`_LIVE_IMAGE_POSTS` tracks it per live image) -- correct while images were lifetime objects, wrong once one is freed at boot. WHY NO UNIT TEST PINS THIS: the failure needs a real CUDA host registration and a subsequent large host allocation landing on the freed range. Both hermetic halves (post accounting, call order) are cheap to assert and the ordering is stated in the function; the RACE ITSELF is a metal property. Recorded here rather than claimed as covered. GATE: model_executor 15 failed / 803 passed -- the same three pre-existing sgl-project#815 files, ZERO new. scheduler test_phase_flip_boot 31 passed, rotation wiring 13 passed, restore_never_rebuild 8 passed. ruff F/E7/E9 unchanged (1 pre-existing unused `pack_into_arena`); format clean. PROVEN BY THE FIX: attempt 2 booted past this point, reached readiness, served a real generation and executed a warm rotation. The window's remaining findings are in /spinning/gpu-arb/W28-RESULT.md.

…ath, and name where the leg's time goes CLOSES A GAP I FILED AGAINST MYSELF. `_timed_arena_refill` discarded the `RotationStats` it got back, so on metal `overlapped_steps` -- the duplex falsifier this whole scheme rests on -- was NOT OBSERVABLE for a warm flip. W28 could therefore score it only as NO DATA. Both the stats line and a phase line are now logged per flip leg. THE SECOND HALF IS THE HONEST ONE. W28's leg reported LINK-BOUND (read 0.000s / h2d-wait 0.114s over 512 chunk(s), drain 0.001s) on a 4.833 s rotation. Those terms are 2.4 % of it. The phrase named a bound nobody had observed -- one number wearing several meanings, inside the dominant term of the seam, which is the class sgl-project#851 exists to remove. `RotationPhases` splits the leg into save / d2h-issue / h2d-issue / wait / ring / checksum / plan and, crucially, a NAMED `UNACCOUNTED` residual, so the unexplained mass has to land somewhere instead of being distributed over the parts that happen to be measured. `dominant()` lets the residual COMPETE with the real phases: if the leftover is the biggest term, the instrument says it does not know rather than crowning the largest thing it can see. `phases_reconcile` refuses a leg leaving more than 10 % unexplained -- chosen, not derived, and chosen so that W28's own 97.6 % could not have passed it. HOST-SIDE PHASES ARE WHAT RECONCILE, and that follows from the measurement, not from taste: `h2d_wait_s` was near zero, so the host was never blocked on the device, so whatever consumed the wall clock was work the host thread did itself. Device spans answer a different question and are deliberately excluded from the sum -- adding a device span to a host total is how a reconciliation is made to "pass" without meaning anything. A CPU replay puts `d2h_issue` on top. That is a LEAD, not a finding, and it is labelled as one: on CPU those copies are eager by construction. DEFERRED BY USER DECISION (2026-08-24): ~4 s per flip is acceptable for now, so the 4 s -> 1-2 s hunt is filed as W30 rather than chased here. What W30 still needs is written into the ticket: CUDA-event device spans (staggered read after the drain, never sync-in-loop) and the two can-fail mutants. ALSO FIXED, and it was mine from the previous slice: the sgl-project#758 emitter test replaced `rotation_executor.rotate_arena` and never restored it, so a green file made the scheduler suite fail behind it. setUp/tearDown now restore it. `Tuple` was used unimported (masked at runtime by `from __future__ import annotations`, caught by ruff). W29 IS TURNKEY, and the runnable path is the point of this slice: * /spinning/gpu-arb/w29_load.sh -- ALTERNATES concurrent large-prompt BURSTS with low-concurrency decode HOLDS. W28's driver was serial deep decode, which builds no prefill backlog at all, so the policy went TP-sticky after one flip; the user caught that from the outside. Bursts aim well above the bar. * /spinning/gpu-arb/boot_w29.sh -- the sgl-project#630 PP x disk-HiCache warmup wedge is AVOIDED with `--skip-server-warmup`, and the alternative is named and rejected: dropping `--hicache-storage-backend file` would delete the geometry-free disk STORE that is a prefix's only route across the flip (sgl-project#706), i.e. the premise of the no-KV design. * BREAK-EVEN REPRICED FROM THE MEASURED SEAM. N ~= 2188 tokens per second of seam (18614/8.50 = 2190 and 39500/18.06 = 2187 agree). The measured flip is 6.410 s slowest rank, round trip 12.82 s -> N ~= 28,050, not ~39,500. The bar is ~29 % LOWER, so the policy should flip MORE readily. Derived by proportion from the two known pairs, said plainly so it is not mistaken for a first-principles repricing. GATE (foreground, family-batched): model_executor 15 failed / 803 passed -- the same three pre-existing sgl-project#815 files, ZERO new managers (chunked) 3184 passed; 1 failure + 1 collect error, both identical at HEAD mem_cache (chunked) 7f+474p / 24f+1192p -- byte-identical to HEAD scheduler test_phase_flip_boot 31 passed; rotation suites 95 passed combined ruff F/E7/E9 clean on the changed files; format clean.

Revert "Organize public APIs (#809)"

9069b47

This reverts commit c8e9fed.

Ying1123 merged commit db6089e into main Jul 30, 2024

Ying1123 deleted the revert-809-organize-public-apis branch July 30, 2024 02:40

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

Revert "Organize public APIs" (sgl-project#815)

c23e7ad

amote-i pushed a commit to amote-i/sglang that referenced this pull request Dec 8, 2025

update test_ascend_L2_cache_01.py (sgl-project#815)

6b26cd0

* Update test_ascend_L2_cache_01.py * Update run_suite.py * Update test_ascend_L2_cache_01.py

Navigation Menu

Uh oh!

Revert "Organize public APIs" - #815

Revert "Organize public APIs"#815
Ying1123 merged 1 commit into
mainfrom
revert-809-organize-public-apis

Ying1123 commented Jul 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Ying1123 commented Jul 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant