add minicpm support by Titan-p · Pull Request #602 · sgl-project/sglang

Titan-p · 2024-07-08T08:13:58Z

Adapted from https://raw.githubusercontent.com/vllm-project/vllm/56b325e977435af744f8b3dca7af0ca209663558/vllm/model_executor/models/minicpm.py

python -m sglang.launch_server --model-path /path2/miniCPM-bf16 --chat-template chatml --trust-remote-code

merrymercy · 2024-07-09T06:46:12Z

@Titan-p It is merged. Thanks for the contribution!

Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

…-project#601 and sgl-project#628 Merge target: d958a63 "Return bad request for malformed DSV4 tool arguments (sgl-project#602)". Excludes b4de164 (sgl-project#601 "Fix DSV4 C128 state mapping lifecycle") and f59f166 (sgl-project#628 "Sync clean SM90 FP8+FP4 MegaMoE from main") per user instruction. Brings in 13 upstream commits including: - 3e31718 [HiCache] DeepSeek-V4 host-mem L2 + NVMe/disk L3 (sgl-project#564) - 6dddb6d Fix DSv4 draft extend replay forward batch (sgl-project#569) - 9d212a9 add observability (sgl-project#570) - 104a38d Bytedance/deepseek v4 fix draft extend padding (sgl-project#573) - d958a63 Return bad request for malformed DSV4 tool arguments (sgl-project#602) - 8 docker/codex build hardening commits Conflict resolutions (all 8 files stability-critical, memory-related): 1. tree_component.py / full_component.py / swa_component.py / mamba_component.py -- commit_hicache_transfer signature: keep HEAD's typed (insert_result, pool_storage_result) kwargs. Callers in unified_radix_cache.py explicitly pass these; their **kw variant is strictly a superset but less safe. Also drop their side's manual `kw.get(...)` extraction in mamba PREFETCH branch since the params are now directly bound. 2. hybrid_cache_controller.py -- extra_host_mem_release_queues type annotation: keep HEAD's Queue[torch.Tensor] (more precise). 3. swa_memory_pool.py -- their side added a full duplicate SWATokenToKVPoolAllocator class inline in this file, but our fork moved the class to allocator/swa.py in an earlier refactor. Drop their duplicate; instead apply sgl-project#573's free_swa fix (guard against negative sentinels and out-of-range indices, expand page-aligned free) directly to allocator/swa.py where the class actually lives. 4. hybrid_pool_assembler.py: - kvcache.compress_state_pools indexing: keep either form (addition is commutative); use HEAD's ordering for local consistency. - C128 state pool registration: keep HEAD's explicit non-registration with the "C128 State pool intentionally not registered" comment. Our HEAD's upstream sgl-project#27293 deliberately opted out; their side (from before sgl-project#601) still registers it. Silently reintroducing the pool would regress the fix and pressure host memory. - attach_hybrid_pool_to_unified_cache body: keep HEAD's _select_strategy(...).build() dispatch (from upstream sgl-project#26295) over their 245-line inline if-elif chain. Our _DeepSeekV4Strategy already handles the deepseek_v4 stack; the strategy signature already accepts storage_backend / storage_extra_config / storage_prefetch_threshold. 5. unified_radix_cache.py (24 conflicts) -- keep HEAD wholesale: - HEAD has 10 methods theirs lacks (all from upstream sgl-project#26881, sgl-project#27285, sgl-project#27391, sgl-project#27489, sgl-project#27554, sgl-project#27655 L3 support + our PR sgl-project#27010 port: prefetch_sync_thread, ack-driven sync, handle_prefetch_result, _drain_async_work, _finish_write_through_ack, etc.) - Their only extra method (terminate_prefetch(rid)) was deliberately removed by our PR sgl-project#27010 port. - HEAD's load_back_threshold=10 matches hi_mamba_radix_cache and hiradix_cache (theirs=256 was an inconsistent local tuning). - HEAD passes total_len to InsertResult (dataclass field exists) and priority to UnifiedTreeNode (LRU correctness). - HEAD's write_backup early-returns on written==0 (upstream sgl-project#27108 safety fix); theirs drops the check. - HEAD's _drain_queue is blocking with qsize MIN-reduce (our PR sgl-project#27010's lockstep drain); theirs is old get_nowait+Empty pattern that would desync PP/TP. - sgl-project#564's L2/L3 additions in this file are functionally covered by upstream sgl-project#26881 (already in HEAD via ep_main). No net feature loss from taking HEAD. Verified: AST parse OK for all 9 mem_cache files, compileall exit=0 on managers + mem_cache trees. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

planner_corridor_mib() returning 0 is only correct while the reserve demand carries BOTH the NVML carve-out and the user reserve. That coupling lived in a docstring, so removing a carrier would have surfaced as a quiet boot under the user's floor rather than a red test. The invariant is now stated at planner_corridor_mib -- including the repair rule (put the term back; do NOT make the corridor non-zero, which cannot know a card's carve-out or the user's number and would double-count wherever the demand still carried them) -- and pinned by TestTheCorridorZeroIsCoupledToTheDemand, which delta-proves each carrier through the real reserve_demand_per_gpu. CAN-FAIL PROVEN by mutating production code and observing red, not asserted: remove the user reserve from reserve_demand_per_gpu -> 2 failed (test_carrier_one..., test_and_therefore_the_corridor_adds_nothing: 'AssertionError: 0 != 1024') remove the carve-out term from build_card_ledgers -> 4 failed (test_carrier_two...: 'AssertionError: 0 != 93', plus the three term tests) both mutations reverted; 386 passed on mem_ledger + registry afterwards A first attempt at this proof produced a false green: the mutation heredoc ran 'python', which does not exist here, so nothing was patched and the suite passed for the wrong reason. Recorded because it is the exact failure mode the proof exists to catch -- a gate that cannot fail looks identical to a gate that passes.

…it defaulting to 0 Found by the acceptance window, not by the tests: the fixed boot's demand moved by exactly +1024 on all three cards -- the user reserve and nothing else. The carve-out term priced at ZERO in production while every unit test passed. Cause: _build_card_ledgers read the field as getattr(card, 'reserved_mib', 0), and the card object the boot resolves is _RankGpuCard (built from memory_info_for_uuid), which had no such field. Every test built CardFacts by hand with reserved_mib set, so the tests exercised the endpoint and never the chain. A quiet default turned a missing link into a plausible number. observed budgets 25530 / 16274 / 15403 (carve-out priced at 0) intended budgets 25012 / 15849 / 14978 (carve-out priced) difference 518 / 425 / 425 = exactly the per-card carve-out Fix, end to end: MemoryInfo carries reserved_bytes (read via the same v2 path as DeviceInfo); _RankGpuCard.reserved_mib is a REQUIRED field, so a card type that forgets it raises TypeError instead of pricing zero; _build_card_ledgers reads the attribute directly, no getattr default. New tests pin the CHAIN rather than the endpoint -- MemoryInfo carries it, _RankGpuCard refuses to be built without it, and _build_card_ledgers' source contains no getattr default. The last one reads source deliberately: this defect had no runtime signature to assert against, which is precisely how it shipped. test_reserve_semantic_split's boot-path expectation moves 4222 -> 4647 (+425, once per card, not per rank). That assertion is the one that would have caught this had its Card stub carried the field; it does now. TESTS: mem_ledger + registry 389 passed; ruff clean on the new file. NOTE: the currently running boot (pgid 18302) has the HALF-applied fix and its measurements are reported as such -- superseded, not presented as the result.

…asurement boot is cheap SGLANG_VRAM_FLIGHT_TRACE now takes "1"/"all" or a comma-separated rank list. A SCOPE, not a cap: an armed rank still records its whole boot with nothing dropped. The reason to have it is that the ring is uncapped by design and its host-RAM cost scales with a boot's allocation count on a swapless box, so the pressure to reach for max_entries -- the knob that produced the sgl-project#602 wrap -- has to be answered by a different lever. A rank scope costs the unscoped ranks their trace visibly; max_entries costs every rank its oldest events silently. This makes the measurement boot stageable. Marks alone (SGLANG_VRAM_FLIGHT_DIR, ~4 ms per rank per boot, no trace) already yield the complete per-post resident dataset; the trace is only needed to break a post down by call site, and can then be armed on one rank. TESTS - 236 green across the mem_ledger suite (was 232; four scope tests added, including that "0" is read as a rank name and not as a boolean off, which would otherwise produce a boot with no trace at all). - GPU chain test re-run on the 5090: ALL PASS, 16 checks. - The wiring pin from the previous commit caught this commit's own change of arm_process_trace's signature and had to be updated. That is the pin working.

…tead of modelling it The demand model overpredicts by 4664 / 993 / 2701 MiB on this rig (sgl-project#602 window), i.e. up to 4.7 GiB of KV pool per card is given away, because no MODELED term has ever been checked against a MEASURED one. This adds the instrument that produces the measurement. It records and attributes; it sizes nothing and writes no number into the ledger. Three sources, because no single one covers the posts. 1. Phase marks. mark() reads torch.cuda.memory_stats and NVML at eight named boot boundaries (BOOT_PHASES) and appends one JSON line per mark. The difference between two consecutive marks IS the post allocated between them. Measured cost: 0.6 ms per mark, ~4 ms per rank per boot. 2. Process-start allocation recording, armed in run_scheduler_process before the first CUDA allocation. max_entries is NOT capped: the sgl-project#602 capture set it to 100000, came back exactly full, and therefore silently held only its final 10.7 s. The window is instead bounded by an EVENT -- the snapshot is dumped and recording stopped once every runner in the process is up -- so nothing inside the window is dropped and serving does not pay +1.7 us per allocation (3.7 vs 2.0 us, measured) for the life of the process. Dumped once per process, not per runner: under speculative decoding the target and the NEXTN draft both capture graphs, and a snapshot taken at either one's capture_end is missing the other's. 3. The non-torch remainder, per PID. Each mark records what NVML says THIS pid holds on the card, so non_torch = nvml_self - torch_reserved is measured rather than left over from two card-level totals. This also settles TERM_PARENT_CONTEXT by direct observation instead of by gap inference. CORRECTION TO THE sgl-project#602 HANDOFF. It advises reading device_traces and NOT segments[].blocks[].frames. That is right for those captures and the reason is worth keeping straight: torch fills block frames only for blocks allocated after recording began, so a post-boot arming starves that structure (80 of 2046 blocks, 3 of 25142 MiB). Armed at process start it becomes the EXACT resident attribution, because it is keyed on the blocks the allocator still holds rather than on a ring of events. Both structures are parsed, and both parsers return a Coverage verdict beside their numbers -- the sgl-project#602 aggregation produced a tidy, sorted, entirely truthful 3 MiB answer to a 25142 MiB question with nothing in the output saying so. MEASURED, and correcting an assumption of my own: torch.cuda.is_initialized() is NOT "a primary context exists". On this rig, pinned to the 5090: after import torch -> False / NVML 0 MiB; after importing any sglang.srt module -> True / NVML 0 MiB; after the first 1-byte allocation -> True / NVML 500 MiB. Verified in the same run that current_device(), memory_stats() and _record_memory_history() bind nothing either, which is what makes it safe for a mark to read them at process start. The direct question is answered by NVML's per-process list and recorded as holds_device_context. FINDING, not fixed here: DemandInputs.parent_binds_cuda_context is hardcoded False at engine.py:467, so TERM_PARENT_CONTEXT is never charged on the production path -- only test_full_demand_reserve_593 ever sets it True. On the reference rig the parent/tokenizer binds a context on the TP0 card, so this is an UNDER-charge of about one context. It is left alone until the measurement boot observes the parent directly in NVML's per-process list; correcting a term against an inference is what this task exists to stop. TESTS - test_flight_recorder_605.py: 26 tests green (hermetic, CUDA_VISIBLE_DEVICES=99). - Whole mem_ledger suite: 232 green, no regression. - CAN-FAIL PROVEN by mutating production code and observing red, four times: coverage ignoring unframed blocks -> 3 red; python_site returning frames[0] -> 3 red; churn ignoring orphan frees -> 2 red; deleting the kv_pool_sized call site -> 1 red. Restored to green after each. - Parsers run against the real sgl-project#602 captures, not fixtures: 2046 blocks / 80 framed / 25142 MiB reserved / 100000 entries / 40 orphan frees / 31 MiB peak outstanding, all asserted. The 31 MiB independently reproduces the handoff's corrected figure. - CHAIN TEST ON A REAL CARD (scripts/vram_ledger/flight_selftest.py, 5090 by UUID, ~570 MiB, corridor untouched): ALL PASS, 16 checks. It proves what no fixture can -- arming binds no context, marking binds no context, and a snapshot from a process-start arming reaches COMPLETE coverage with the probe allocation attributed to its own source line. Its first cut asserted on torch's is_initialized flag and reported four failures against a correct instrument; that is where the measurement above came from. - Wiring pinned against the serving tree's own source, because sgl-project#602 closed with twelve green fixture tests beside a production carrier that lacked the field. Proves presence, not execution; execution on the serving path needs a boot. NOT YET DONE: no measurement boot has run, so there is no per-post dataset for this rig yet and demand model v2 is not started.

…asurement boot is cheap SGLANG_VRAM_FLIGHT_TRACE now takes "1"/"all" or a comma-separated rank list. A SCOPE, not a cap: an armed rank still records its whole boot with nothing dropped. The reason to have it is that the ring is uncapped by design and its host-RAM cost scales with a boot's allocation count on a swapless box, so the pressure to reach for max_entries -- the knob that produced the sgl-project#602 wrap -- has to be answered by a different lever. A rank scope costs the unscoped ranks their trace visibly; max_entries costs every rank its oldest events silently. This makes the measurement boot stageable. Marks alone (SGLANG_VRAM_FLIGHT_DIR, ~4 ms per rank per boot, no trace) already yield the complete per-post resident dataset; the trace is only needed to break a post down by call site, and can then be armed on one rank. TESTS - 236 green across the mem_ledger suite (was 232; four scope tests added, including that "0" is read as a rank name and not as a boolean off, which would otherwise produce a boot with no trace at all). - GPU chain test re-run on the 5090: ALL PASS, 16 checks. - The wiring pin from the previous commit caught this commit's own change of arm_process_trace's signature and had to be updated. That is the pin working.

…ys honest Precondition for arming SGLANG_VRAM_FLIGHT_DIR on every production boot. The mark file is APPENDED -- that is what lets a crashed boot keep the boundaries it did reach -- so a permanently armed recorder makes one file hold many boots, and a reader that returns the whole file returns marks from several process lifetimes. phase_deltas over that computes a difference from one boot's last mark to the next boot's first: a number that describes nothing, printed in the same table and the same units as the real posts. Left alone, permanent arming would have made the instrument's output less trustworthy the longer it ran. - Every mark carries boot_id, derived from the LAUNCHER (this process's parent pid + start time) rather than from this process, so all ranks of one boot agree on it without a collective. A per-process uuid would tag each rank's file differently and make cross-rank reading impossible. - read_marks returns the latest boot by default, a named boot on request, and the whole file only when explicitly asked with boot="all". - phase_deltas SKIPS the seam and logs it, rather than reporting it. - list_boots + `attribute_flight.py boots` say which boots a directory holds; `phases --boot` selects one. MEASURED, for the retention question: one rank's boot is 8 marks / 5800 bytes (725 B per mark), so a 3-rank boot costs 17 KiB and 1000 boots cost 16.6 MiB. No rotation policy is proposed for the marks, because at that size the condition does not bind and an unused knob is one more thing to get wrong. Snapshots are the opposite case (~18 MB each) and are only written when the trace is armed, which stays window-only. TESTS - 241 green across the mem_ledger suite (was 236; six boot-scoping tests). - CAN-FAIL PROVEN twice: making phase_deltas span the seam -> 1 red (4 deltas where 3 are real); making read_marks return the whole file -> 2 red. - attribute_flight.py smoked end to end against a two-boot directory: `boots` lists both, `phases` shows only the latest. - Rebased onto 7044be2 (production tree: sgl-project#602 ledger + sgl-project#603 hoist) before all of the above; the rebase was clean and every call site survived it.

…CL term supply path + profile resolution in the constructor) First successful --enable-vram-ledger boot. Operator review: core hunks read (bracket is a no-op unarmed, engine path exception-guarded, server_args hunk idempotent in the shared ledger constructor); mem_ledger suite self-run hermetically on the hunter tree: 295 passed. Fill acceptance 1 of 3 cards -- residual gap is uneven-DCP token quantisation (sgl-project#602 fill side), not a ledger term.

Covers the demand model and all twelve terms with provenance, the pre-boot probe, both calibration flows (activation footprint per recipe, NCCL buffers per communicator signature), the flag split from the legacy --rank-auto-reserve-mib path, flight-recorder verification, and known limits (legacy production recipe, unpriced ~70 MiB load transient, the uneven-DCP token-vector quantisation gap and its sgl-project#602 ownership, co-location vs the NCCL probe). Every command and flag cited against the code (HTML comments); citations spot-checked against engine.py, server_args.py, probe.py.

Two boots one flag apart, plus the procedure for reading them. Recorded as scripts rather than as a transcript because the acceptance has to be repeatable by someone who was not in the window. The README states why the arms run the FP8 32768 recipe and not production: the ledger's activation and graph-capture terms are keyed on an activation profile digest, and the digest cached on this rig was measured for that recipe. Corridor mode on the production recipe would leave those terms UNBOUNDED and refuse the boot -- correct, but not an acceptance run. It also states the two-sided pass condition explicitly, because only stating the floor invites the failure this task exists to remove: a boot that holds 1024 MiB by leaving 4 GB idle has not passed.

…ndow The cards carry a user-facing translator stack on the tenant reserve layout RESERVE=13000,4200,4200. Restoring the default serving layout instead would hand that headroom to the KV pool and leave the translator tenants nowhere to boot into, so the restore step names the layout explicitly. Also records the known watchdog gap: its auto-restart falls back to RESERVE=auto, so starting the watchdog alone does not reproduce the tenant layout. The tenant recipe has to be booted explicitly first, then the watchdog armed on top of it.

…l-side: exact divisor-block solver for the KV unit) Desk-complete slice: corridor mode of --rank-kv-ratio solves the exact argmax of C(u) = u * sum(E_r // u) via divisor-block enumeration; floor enters as capacity E_r = min(P_r, Q_r), never as an objective term; env pin wins; refusals never silent. 49 new hermetic tests green on this tree; the 3 red tests in test_dcp_context_ceiling.py are PRE-EXISTING on base 6e86366 (verified identically red there, _PoolRowStub AttributeError) and untouched by this branch. GPU acceptance stays OPEN (window request filed: FP8/32768 recipe, then restore to the tenant layout) -- gate flips only after that window.

…priced per rank into the sgl-project#593 reserve and sgl-project#602 corridor late terms — inherited 70 MiB honestly labelled CALIBRATED with a measured replacement path via the flight recorder; communicator_groups declaration corrected both directions with an AST contract test against parallel_state; runbook 16.8 reserve semantics + 16.9 fill side documented OPEN)

…false zero residue The fill side of sgl-project#602 stayed open because the instrument that was supposed to name the posts was wrong in two ways on the ship config. Both were found by reading the marks of a real ship boot (1353495-1786609875, 2026-08-13, --tp-size 1 --pp-size 3 --rank-gpu-memory-mib 31800,14000,15600). RANK COLLISION. Marks are filed under the TP rank. The ship config runs TP=1 with PP=3, so the TP rank is 0 in all three processes and all three append to flight_marks_rank0.jsonl. read_marks keyed its result on the rank field, so three processes on three different cards came back as one timeline, and differencing it produced posts no card ever paid -- a 20480 MiB card's 482 MiB CUDA context billed to the 32607 MiB card. Grouping is now by pid, which the writing process stamps and which cannot collide while it lives. The boot is also resolved across all rank files instead of per file, since ranks do not stop writing at the same instant. FALSE ZERO RESIDUE. non_torch_bytes was written as max(0, nvml_self - reserved). On this config torch reports up to 7162 MiB MORE reserved than NVML says the process holds, so the subtraction went far negative and the field read 0 -- publishing "no CUDA context, no NCCL buffer, no JIT workspace on this card" for a rank whose context measured 886 MiB. A floor built to absorb sub-MiB quantisation was swallowing gigabytes. The two cases are separated: non_torch_measurable says whether the subtraction is a measurement at all, and unbacked_reservation_bytes measures how far reserved exceeds resident. The new report closes the per-card identity exactly on all three cards of the ship boot: nvml_total = carve_out + resident + foreign + free, drift 0 MiB. NOT claimed here: that the unbacked reservation causes the underfill. The CUDA path of get_available_gpu_memory reads torch.cuda.mem_get_info (the driver's free) after an empty_cache, not torch's reserved counter, so that chain is falsified rather than asserted. Tests, PYTHONPATH=/spinning/wt-605-fr/python, python3 -m pytest: - test/registered/unit/mem_ledger/test_fill_side_605.py: 4 passed. Red first: 3 of 4 failed against the pre-fix recorder (rank grouping "1 != 3"; KeyError non_torch_measurable; unbacked_reservation_bytes absent from a mark whose non_torch_bytes read 0 on 27308 MiB resident). - Can-fail proof for the unbooked-allocation falsifier: a mutant that sets the residual line to 0 fails with "!! IDENTITY BROKEN by 300 MiB"; reverting the mutant restores 4 passed. - test/registered/unit/mem_ledger/: 357 passed, 1 failed, 1 collection error. Both pre-existing and untouched by this change: the 612 communicator-group declaration test and a missing 'datasets' module, neither of which references flight_recorder, read_marks or the report. - ruff: all checks passed. black: clean. codespell: clean.

…olve the seam it moves THE FIRST CUT OF THIS SOLVER PRICED AN 18 GiB RUNNER AT NOTHING. pp_cut.py had no NEXTN / draft term at all. Charging the recorder's raw `weights_draft` post instead -- 18226 MiB on rank 0, ~10.8 GiB on each 3080 -- made the model declare the CURRENTLY RUNNING configuration infeasible, which is the loudest available proof that the gross number is the wrong one. THE NON-ADDITIVITY IS THE WHOLE TERM. The two runners do not both occupy the card. Between sizing the target pool and loading the draft weights the process RELEASES and rebuilds into the same space, and the recorder measures exactly that as `inter_runner_gap`: -15320 / -8546 / -8550 MiB. So the draft runner's real cost is `weights_draft - overlap_credit` = 2906 / 2250 / 2250 MiB, an order of magnitude under the gross figure. Both halves are kept on `DraftResidency` so a reader sees the SIZE of the correction rather than being handed a small number to trust. MEASURED OR REFUSED, NEVER DEFAULTED (sgl-project#606). A missing recorder post is not zero. Zero is a real value here -- "this deployment has no draft runner" -- and the two are numerically identical, so only the caller can separate them. `draft_residency_from_flight` raises `DraftResidencyUnavailable` on an empty directory or a boot with no draft posts, and `PPCutInputs.draft_runner_present` makes the term MANDATORY per rank when a draft runner is declared. The reading is provenance-stamped with pid, card uuid, boot id and source directory, and grouped by PID rather than rank for the reason `flight_recorder.read_marks` documents: under --tp-size 1 --pp-size 3 all three processes file under TP rank 0. CALIBRATION ACCEPTANCE, AS A TEST RATHER THAN A CLAIM. With the credit the running config prices FEASIBLE (floor 753449 tokens at its own seam fixed point); with the raw weights it prices infeasible. Both directions are pinned, so the day the correction stops mattering the test says so instead of passing vacuously. THE SEAM SENSITIVITY CAME BACK POSITIVE, SO THE SEAM IS NOW SOLVED. The brief said the per-token seam scaling could stay documented unless it flips the cut. It flips it. Pricing the seam at the LIVE arena while solving for a larger one understates it on precisely the stage the solve wants to load: seam priced at the live arena (471638 tok) -> cut [31, 16, 17] seam priced at its own fixed point -> cut [30, 17, 17] A one-shot solve therefore returns a cut optimal for a seam demand that the cut itself invalidates. `solve_pp_cut_for_kv_floor_at_seam_fixed_point` re-prices the seam at each candidate arena until the arena stops moving (4 iterations here), and `world_kv_floor_at_seam_fixed_point` scores a GIVEN cut the same way -- comparing a fixed-point candidate against a one-shot incumbent would credit the candidate with the correction. THE ANSWER, on the live budgets, recorder-measured draft residency, and the live KV bytes/token (1908.2 B/token/attn-layer, derived from the boot's own logged K+V over 471638 tokens): incumbent [28,20,16] at its fixed point 753449 tokens solved [30,17,17] at its fixed point 976476 tokens attn [7,4,5] honest delta +223027 tokens (+29.6 %) So [30,17,17] SURVIVES calibration -- but not by the route that first produced it: the uncalibrated solve reached it by accident, the calibrated one-shot gives [31,16,17], and only the fixed point returns to [30,17,17]. The agreement is a coincidence of two corrections, not a confirmation, and it is recorded that way. TESTS. Red-first: 13 of 14 fail against the previous commit (the API does not exist); 20 green now with the fixed-point class added. Covered: both posts read separately and netted; provenance stamped; absence raises on an empty directory, on a boot with no draft posts, and at PPCutInputs construction when a declared draft runner has no measurement; declared-absent is still allowed to mean zero; the draft term lowers the floor rather than raising it; the calibration acceptance in both directions; the re-solve still beats the shipping cut, covers all 64 layers and matches a full enumeration; the fixed point converges, disagrees with the one-shot solve, charges MORE seam than the live figure, and still wins when the incumbent is scored on equal terms; a mis-sized seam vector is refused. NO REGRESSIONS: test/registered/unit/planner 2529 passed (2509 before, +20 = exactly the new cases), 2 failed -- the same two pre-existing test_rejected_evidence_pins failures, verified against a reverted tree in the previous commit. Hermetic throughout (CUDA_VISIBLE_DEVICES=""). STILL NOT BOOTED. The remaining uncalibrated inputs are the per-rank transient and fixed overhead, which still come from the sgl-project#485 reference bench rather than this boot; the absolute floor should not be quoted as a live prediction until those are read from the recorder too. What HAS closed is the term that made the model call the running configuration unrunnable.

…, and the cut moves again THE TERMS. `fixed_overhead_mib` and `transient_mib` were still the sgl-project#485 reference bench's, measured on a different checkpoint. `residency_terms_from_flight` reads both by pid, provenance-stamped, with the same refuse-on-absence separation the draft term uses. FIXED OVERHEAD IS SETTLED BY THE BOOT MARKS: what is resident at `boot_complete` and is not weights, not the KV pool and not the draft runner -- CUDA context, NCCL init, attention workspace, graph capture, boot tail. Measured 1610 / 1222 / 1198 MiB. THE TRANSIENT IS NOT SETTLED, AND THE READER SAYS SO. The serving marks give the draw over the window they cover: 742 / 440 / 584 MiB over 996 s. Law 31's worst measured state is 1989-3148 MiB on a 22-minute soak, and a gate fed the gentler number admitted cuts metal then broke the corridor on, twice. So the reader returns `observed_transient_mib` with its sample count and window and hard-codes `covers_worst_load_state=False` -- nothing in the marks proves the window was a soak. The caller charges `max(observed, worst known)`, so the sgl-project#485 soak value survives as a floor and the gate cannot get cheaper by looking at a shorter window. That is asserted, not just documented. THE REFUSAL FIRED ON REAL DATA WHILE THIS WAS BEING WRITTEN. Reading /spinning/flight_605 without a boot id hit an in-flight boot (pid 3505168, no `boot_complete` yet) and raised instead of differencing an unfinished boot into a confident overhead. That is the sgl-project#606 lesson doing its job on live input rather than in a fixture. THE CALIBRATION ACCEPTANCE HOLDS: with the measured overhead charged, the RUNNING configuration still prices feasible (624865 tokens at its own seam fixed point). AND THE CUT MOVES AGAIN, WHICH IS THE ANSWER TO THE QUESTION ASKED: uncalibrated [30,17,17] + draft term, seam one-shot [31,16,17] + draft term, seam fixed point [30,17,17] + overhead and transient (this) [31,16,17] So [30,17,17] does NOT survive the third calibration. Chasing that oscillation is what produced the finding that matters: THE THREE CUTS ARE A TIE, ON BOTH OBJECTIVES. Under full calibration [30,17,17], [31,16,17] and [29,18,17] score the IDENTICAL KV floor (851960) and the IDENTICAL makespan (1.0804 s), because stage 2 is the bottleneck on both in all three and stage 2 is the same in all three. The solver was returning a deterministic tie-break, not a preference, and reporting either vector as "the answer" would have implied a distinction the objective cannot make. **The load-bearing change is stage 2: 16 -> 17 layers.** The stage0/stage1 split is free. TICKET_602_METAL.md recommends 31,16,17 only because it leaves the largest margin on stage 1 (0.9221 s against a 1.0804 s bottleneck), and says plainly that preserving `...,17` is what preserves the result. Fully calibrated, both cuts at their own seam fixed point: incumbent [28,20,16] 624865 tokens solved [31,16,17] 851960 tokens +227095 (+36.3 %) WHAT IS STILL NOT CALIBRATED, and it is why the ticket makes the absolute number a DIAGNOSTIC rather than a gate: `nonlayer_weight_mib` (embedding on stage 0, lm_head on stage 2) and `state_mib` (the GDN/mamba state pool) are both still ZERO, and against the recorder the model's per-stage weight total is short by 1.63x / 1.43x / 1.90x -- the ratio being highest on the LAST stage is the signature of the missing lm_head. The measured floor will land materially below 851960 and the boot must not fail for it. TESTS. Red-first: 7 of 13 fail against d9ed47b; 13 green now. Covered: overhead differenced from the boot marks; the observed transient as peak over boot_complete; provenance including the sample count; the reader refusing to call a short window a worst case; absence raising on no marks, on a boot with no serving marks, and on a boot with no boot_complete; the running config still feasible; charging the overhead lowering the floor; the WORST load state being charged rather than the observed one; and the solve converging, beating the shipping cut on equal terms and covering all 64 layers. NO REGRESSIONS: test/registered/unit/planner 2542 passed (2529 before, +13 = exactly the new cases), 2 failed -- the same pre-existing test_rejected_evidence_pins pair. Hermetic throughout (CUDA_VISIBLE_DEVICES=""). TICKET_602_METAL.md added: one boot, one flag, hard acceptance on the corridor, the arming floor and a rising world MIN; the absolute floor recorded as a diagnostic; and an explicit scheduling constraint against F4-r4's sgl-project#689 window, because re-cutting moves per-stage arena and seam geometry -- the quantities his peer-fit and arena-occupant measurements are about.

…mmendation reverses THE SHORTFALL POINTED AT THE NON-LAYER WEIGHTS AND IT WAS RIGHT. Against the recorder the model's per-stage weight total was short by 1.63x / 1.43x / 1.90x, highest on the LAST stage -- the signature of a missing lm_head. `pp_cut` already HAD every term (`embedding_weight_bytes`, `lm_head_weight_bytes`, `replicated_weight_bytes`, `state_bytes_per_linear_layer`); the sgl-project#485 fixture was passing ZERO for all four. This was an input calibration, not a model gap. `checkpoint_weight_terms` reads them from the safetensors HEADERS -- no payload touched, no accelerator needed -- which is what `PPCutInputs` already tells callers to do rather than deriving them from config formulas. WHAT THE RECONSTRUCTION FOUND. The checkpoint carries a VISION TOWER (879 MiB) and an MTP head (405 MiB) that are resident on EVERY stage. Modelling them as first-stage-only leaves stages 1 and 2 short by exactly their size, which is how they were found. With them as replicated payloads the per-stage weight identity closes to -3 / +7 / +11 MiB against the recorder. TWO CORRECTIONS FOUND BY CHASING THE REMAINING GAP, both real defects in what this branch had already shipped: * `residency_terms_from_flight` was subtracting the draft runner's own KV motion (`kv_pool_sized` with draft=True, -740 MiB on rank 1) out of the overhead. The cost model prices only the TARGET KV arena, so that term was then represented NOWHERE and the reconstruction sat 740 MiB high on that rank. The overhead now absorbs it and the identity is exact against what the model can actually price. * `budget_mib` was being fed `--rank-gpu-memory-mib`. The corridor, the seam and the transient are all measured against the CARD (NVML free), not against a pool-sizing flag; on rank 1 the flag (18800) is smaller than at-rest residency plus the corridor, so the model called a running boot unrunnable. The budget is the card's allocatable size. Fully calibrated the identity closes to -53 / +8 / +39 MiB (0.2 %), and the model now REPRODUCES THE FLOOR THE LIVE BOOT SIZED: 480010 predicted against 471638 measured, 1.8 % high. That is what turns the floor from a diagnostic into a gate, and TICKET_602_METAL.md is updated to a hard floor acceptance with a +/-5 % tolerance earned from that demonstrated error. AND THE RECOMMENDATION REVERSES. The previous ticket said `31,16,17` and `+227095` tokens (`+36.3 %`). Both are RETRACTED. With the real 476 MiB linear layer, moving layers onto a stage costs far more than the bench-priced model believed: solved cut [29,19,16] (stage 2 KEEPS 16 layers) every 17-on-stage-2 cut INFEASIBLE incumbent 480010 -> solved 497245 +17235 tokens, +3.6 % So the whole term-2 prize is single-digit percent, not thirty. The earlier number was an artifact of the uncalibrated weight model, and the ticket now says plainly that a 3.6 % gain may not justify a boot slot -- deferring is a legitimate outcome. This is exactly why the metal arm was withheld. THE BOUNDARY, NAMED RATHER THAN APPROXIMATED. The recorder does NOT separate the GDN/mamba state pool from the KV pool; both land in one `kv_pool_sized` post. The three stages fit that post as linear in ATTENTION-layer count with a residual constant of -132 MiB and a spread under 3 MiB, which leaves no room for a term scaling with LINEAR layers (stage 0 has 21 of them, stage 2 has 12). `state_bytes_per_linear_layer` is therefore 0 on the strength of that fit, and the fit is pinned as a test so a checkpoint that sizes GDN state differently fails here instead of silently. Separately: the sgl-project#485 bench transient (1346/1120/982) is NOT transferable to this checkpoint -- charging it prices the running configuration infeasible, which is how it was found to be wrong. The observed 996-second window (742/440/584) is the only measurement of THIS deployment; a soak-length window should replace it and would move the gate. TESTS. Red-first: 9 of 11 fail against 7f2f8c6. 14 green now, including three guarded live-data cases: the checkpoint reconstruction matching the recorder on every stage, the model reproducing the live floor within 5 %, and two pins on the reversal itself -- stage 2 keeps 16 layers, and the reclaim stays under 15 % (above that on this rig has meant the weight model went uncalibrated again). NO REGRESSIONS: test/registered/unit/planner 2556 passed (2542 before, +14 = exactly the new cases), 2 failed -- the same pre-existing test_rejected_evidence_pins pair. Hermetic throughout (CUDA_VISIBLE_DEVICES="").

…d the two open decisions The handoff artifact for the merge queue. Nine commits on fix/602-fill-side, what each is, and the two things that are NOT decided: * R' semantics for the cold seam. 5301b94 derives and announces the cold per-token slope but the reserve stays inactive, so cold boots still size floor-only. `SeamReserve.active` needs `id_space > 0` -- a measurement anchor a derivation does not have -- and the anchor-free `solve_pool_tokens` has no live caller, so the budget it solves against is boot-path design. F4-r4's call. * sgl-project#602 metal arm, deferred by the operator at +3.6 %. Also records the retraction in full: the earlier 31,16,17 / +227095 (+36.3 %) recommendation was an artifact of bench-priced weights and is withdrawn in favour of 29,19,16 / +17235 (+3.6 %), stage 2 keeping 16 layers. MERGE-QUEUE EVIDENCE, asked for and answered NO: cherry-picking 0274bed (sgl-project#681) into a scratch copy of this tree does NOT green the two PrefillAdmissionBudgetTest reds. They fail identically with `AttributeError: 'BudgetHarness' object has no attribute '_local_mamba_avail'`. That is a STALE TEST HARNESS, not a product defect -- `Scheduler._local_mamba_avail` is real (scheduler.py:4257, called from _update_uniform_pool_budget at :4067) and the harness stand-in never binds it. Unrelated to sgl-project#681's fundable_extend_floor. Scratch dropped. Docs only; no code touched.

…oor; the cut is withdrawn F4-r4 REFUSED A METAL ARM ON THIS SOLVER AND HE WAS RIGHT. His census boots measured 471303 tokens on the incumbent cut where the model predicted 361566 -- -23.3 %, far outside the +/-5 % gate this ticket carried. The defect is not a mis-calibrated constant. The two numbers answer different questions: corridor-safe floor funds the WORST measured load transient (law 31) the sizer's pool funds NO load transient, because the pool is sized BEFORE any seam has ever run In his regime the worst load state is a SEAM on EVERY rank -- SEAM_TP_TO_PP 2168 MiB on rank 0, SEAM_PP_TO_TP 700 / 932 on ranks 1 / 2, two to three times the prefill-triggered scalars -- so the gap is ~29 %. Holding one against the other is a category error and the gate was doing exactly that. MY OWN 1.8 % AGREEMENT WAS PARTLY LUCK, and this is the part worth recording: the transient I charged came from the sgl-project#485 PREFILL bench (1346/1120/982), not from a seam census, so it was small enough that the category error stayed inside the gate. A regime whose worst state is a seam exposes it immediately. The agreement was not evidence the model was right. THE FIX IS A SEPARATION, NOT A RETUNE. * `StageCost.transient_mib` no longer carries the SUM of the worst-load peak and the cut-invariant overhead. They are charged by different questions, so `fixed_overhead_mib` is its own field. `resident_mib` still counts both, so every existing corridor-safe consumer is byte-neutral (2574 planner tests pass, +13, no behaviour change). * `world_predicted_pool` -- sizer-equivalent, transient excluded. THE ONLY output that may be gated against a measured `max_total_num_tokens`. * `world_corridor_safe_floor` -- worst transient funded, its own meaning, never conflated. `world_corridor_safe_floor` is a real function and NOT an alias of `world_kv_floor`, which was the second thing this exposed: `world_kv_floor` also refuses a stage whose headroom is negative at the INPUT arena. Correct for the solver -- a cut that does not fit the arena being searched is not a candidate -- and wrong for scoring an incumbent, because a MEASURED arena is full by construction, headroom sits at ~0, and a model error of either sign flips it to "infeasible" so the capacity cannot be reported at all. Every comparison returned None until the two gates were separated. The solver keeps its gate; scoring gets a form without it, so the two outputs differ by the transient term ALONE. CALIBRATED AND VALIDATED on F4-r4's census (/spinning/evidence-665-f1/ census-602/, solver cherry-pick 299f292), measured pool 471303: world_predicted_pool 468984 -0.5 % inside the gate world_corridor_safe_floor 333645 -29.2 % the number that was compared AND THE RECOMMENDATION IS WITHDRAWN, which is the finding this commit exists for. Re-solved on his terms, the INCUMBENT cut is the global optimum over all 1953 contiguous cuts: [28,20,16] 468984 <- incumbent, best [29,19,16] 439256 -6.3 % <- what this ticket recommended [30,17,17] 346105 -26.2 % <- what the revision before it recommended There is no cut to arm and no gain to measure on that regime. The +3.6 % was a property of MY boot's terms, not a portable result: attention layer 374.24 vs 362.3 MiB, replicated payload 920.45 vs 1284, residuals 4874.6/3001.4/3005.2 vs 1678/482/482. A cut solved on one regime must not be armed on another, and the ticket now says so as a precondition rather than a caveat. TESTS. Red-first: 10 of 13 fail before the split. Green now. Falsified by restoring the conflation -- 4 go red and the gate case reports -29.2 %, reproducing the class of error F4-r4 measured. Covered: the two outputs differ under a seam transient and coincide only at zero transient; the pool prediction inside the gate; the corridor-safe floor pinned OUTSIDE it so the category error cannot return quietly; corridor and seam RESERVE binding both while the transient binds only the floor; the solver gate refusing a measured arena while the scoring form reports there; and the cost object reporting the two terms separately with residency unchanged. planner 2574 passed, 2 failed (the same pre-existing test_rejected_evidence_pins pair). Hermetic. NO DEPLOY.

…o-arm verdict The handoff artifact carried revision 2's +3.6 % recommendation and the ±5 % gate as if both still stood. Neither does, after F4-r4's census boots: the gate was comparing a corridor-safe floor against a measured pool (~29 % apart in a seam-worst regime, read -23.3 %), and re-solving on his terms makes the INCUMBENT cut the global optimum, with the recommended 29,19,16 6.3 % worse. Also records the standing rule the ticket now carries: re-solve on the regime being booted and check world_predicted_pool against its measured pool before any arm. A cut solved on one regime must not be armed on another. Docs only.

…mall flip and unmeasured THE MEASUREMENT, from 291 same-regime PHASE-FLIP DONE lines (and 3955 unique across all logs). The part NOT covered by read+exchange+write is: 123 live slots total 2453 ms unaccounted 1999 ms 81 % 440095 live slots total 4290 ms unaccounted 2287 ms 53 % FLAT at 2.0-2.35 s across a 3600x range of occupancy (123 -> 440095 live slots, 708 -> 3.4M cells). Regressed: unaccounted ~ live slots has slope 0.0005 ms/slot and R2 0.255 -- i.e. it barely correlates with size at all, which is what makes it a fixed cost rather than a small-flip artefact. Total ~ live slots gives intercept 2754 ms, R2 0.871. So the fixed cost, not the movement, is what floors every window sgl-project#677's economics can solve, and it is the term sgl-project#692 must price depth against. WHERE IT WAS HIDING, and I had it wrong first. My initial reading blamed the per-wave backing swap (release_wave/restore_wave). It is NOT the residual: `t_write0` is taken BEFORE those calls and `write_ms` accumulates after them, so the backing swap has been inside `write_ms` all along. Reading the timer placement rather than the call order is what corrected it. The three timers cover the WAVE LOOP ONLY. What fell outside is the tail: _pool_census("pre-cutover") phase_flip_runtime.py:6538 for fn in self._pre_cutover_fns: fn(...) :6539-6544 EXTRA MOVERS -- the weights arena refill and the GDN state leg _cutover_fn(direction) :6545 the group step _pool_census("post-cutover") :6547 THE MOVERS ARE OCCUPANCY-INDEPENDENT BY CONSTRUCTION -- the weights arena refill is the same bytes whatever the KV live set holds -- which is the leading explanation for a residual that does not move with occupancy. That is a hypothesis this commit makes MEASURABLE rather than one it asserts. SO THE TAIL IS TIMED, split movers vs cutover because they have different fixes, and both are reported on the DONE line and in `last_stats`. A residual that has to be regressed across boots cannot be priced per flip; a reported number can, and sgl-project#677/sgl-project#692 both need it per flip. WHAT THIS COMMIT DOES NOT DO. It does not reduce the cost. The reduction candidate is the wave count -- `_flip_waves` (:4024) is "A PURE FUNCTION OF THE REPLICATED LAYER MAP AND THE DIRECTION", so W=16 is paid at 123 live slots exactly as at 440095, even though the staging transient waves exist to bound is trivial there. The module's own docstring already pre-authorises the landing spot: "each extra wave costs one more exchange round trip. If a measurement ever shows the round trips dominating, W=8 is the place to stand, not W=1" (:4083-4089). That A/B needs NO code: `SGLANG_FLIP_SEAM_WAVES` (:2568) already overrides W. It is left unshipped deliberately -- lowering W trades ms for staging MiB, which lands on the arming floor and the corridor, and sgl-project#602 showed that budget is regime-dependent. Hard-coding a policy from one regime's numbers is the mistake that ticket already made once. HONEST LIMIT ON THE PER-WAVE TERM. Fitting unaccounted = a + b*W across the W=4 and W=16 populations gives b ~ 66 ms/wave, a ~ 997 ms. That fit is CONFOUNDED: the W=4 samples are release-first and the W=16 samples restore-first, so it mixes wave count with regime and must not be quoted as a per-wave price. The clean number will come from the env A/B above, within one regime -- which is exactly what the new movers/cutover fields will report. TESTS. 9 cases, source-level because the alternative is driving a full flip: the movers clock opening before the pre-cutover census and closing after the mover loop; the cutover clock wrapping the cutover; the three original timers untouched; both fields in `last_stats` and in the DONE line. Plus an AST arity guard on the DONE line -- specifiers counted against arguments -- because a %-format mismatch raises at the moment the flip completes, the worst possible place to learn it. Falsified: dropping one argument makes it report "17 format specifiers and 16 arguments". managers 2093 passed, 0 failed (2084 before, +9). Hermetic. NO DEPLOY.

…e two commits to hold MERGE_NOTES_602.md rewritten to cover all 18 commits: what each fixes, its test evidence, whether it touches runtime, and whether an equivalent patch is already on the serving line (verified with `git cherry`, not by message matching -- four are: c41645c, ce60358, 658ea3a, 84b0171). DRY RUN: clean. Merged into `integration/r2` -- the live line, since the serving tree descends from its tip a73a0d8 -- in a throwaway worktree, `--no-commit --no-ff`, then aborted and the worktree dropped. Zero conflicts, zero unmerged paths, so nothing was pre-resolved because nothing needed it. Verified semantically as well as textually: on the MERGED tree, managers 2093 passed / 0 failed and planner 2574 passed / 2 failed, the two being the same pre-existing test_rejected_evidence_pins pair that is already red on the base. THE FACT THE OPERATOR NEEDS, and it is not in the commit count: `7936bc4850` is NOT an ancestor of integration/r2, so merging this branch drags in its whole base lineage -- 115 commits, of which 18 are mine and 97 are the hotfix/677 work (sgl-project#662 x20, [PhasePolicy] x18, sgl-project#677 x8, sgl-project#678 x7, sgl-project#679 x6, ...), 129 files, +22731/-762. Approving this merge is approving that lineage, most of which is not mine to vouch for. If only this work is wanted it must be cherry-picked rather than merged. NOT ATOMIC, and it splits cleanly into four groups with an order: (1) the four already on serving -- merging them only reconciles integration with what is already running; (2) the three sgl-project#624 test-only drift guards, which take managers from 4 failures to 0 and should land early so the line stays green during review; (3) desk tool + docs, all planner/pp_cut.py and markdown, imported by no serving path; (4) hold. HOLD, two commits, both runtime and neither on the serving line: * e21e87f (sgl-project#690) touches the seam hot path and changes the PHASE-FLIP DONE format. Already queued to land on deploy WITH the W=8/W=4 probe after the sgl-project#694 soak verdict; merging it into integration first puts it in front of the soak meant to measure it. * 5301b94 (sgl-project#685) touches the boot sizing path. Announce-only today and abstention-guarded, but unsoaked, and the R' decision it waits on is not made. Nothing else in the chain can move serving behaviour. Docs only; no merge performed, no deploy, scratch worktree removed.

… dry run, and the vouch The notes were written at 3c984ad and had gone stale in three ways. Chain: 18 commits at e21e87f -> 22 at 5939d0e. Added sgl-project#697 (8fb86ef), sgl-project#441 (5e0fa1e), sgl-project#524 (5939d0e) and the revision-1 docs commit. Runtime-touching is now 8 of 22, and sgl-project#441 is recorded as NOT runtime despite its ticket -- it adds a handover script and one test guard. Target: integration/r2 has moved a73a0d8 -> a157bf1, so the dry run was re-run rather than carried over. Still zero conflicts. Scope shrank with the target's advance: 115 -> 109 commits, 129 -> 124 files, +22731 -> +22196. Merged-tree tests: managers 2093 passed / 0 failed; planner plus the three new test files 2594 passed / 2 failed. Those 2 are PpWithSpecEvidenceTest in test_rejected_evidence_pins.py and were re-verified as pre-existing by aborting the merge and running that file on clean a157bf1 -- same 2 failed, 2 passed. The chain introduces no new failure. Because the target tip moved, this was re-checked instead of quoting revision 1's finding. Holds: sgl-project#690 (e21e87f) and sgl-project#697 (8fb86ef) are in F4-r4's bundle, so revision 1's objection to landing sgl-project#690 ahead of the sgl-project#694 soak is spent; both lift to group 1 on his boot commit. sgl-project#524 (5939d0e) joins the hold list -- desk-verified only, never on metal, A/B window-gated. sgl-project#685 (5301b94) stays held on the unmade R' decision. Vouch (new section 5): the four F4-r4 cherry-picks are byte-identical to the originals by git patch-id --stable -- f630947/c41645c8c9, 7c58aba/ce6035884d, f1f31d2/658ea3ac11, de92bb6/84b0171fa6 -- so preferring the originals is free and the only real risk is double attribution. 1073702 is excluded: patch-id 8595e66885ac matches nothing here, it is diagnostic scaffolding and not mine to vouch for. Section 6 records that 5af1531 (sgl-project#696) and c738ef5 (sgl-project#689) are F4-r4's line, not this branch, and are listed only because they lift holds; sgl-project#696 is what was DoSing the lanes. Docs only. No merge performed; the dry-run worktree was removed.

…tor co-solved The sgl-project#602 capacity solve withdrew the cut for the POOL objective. The question actually asked -- more prefill on the 5090 -- is a time objective and was never solved for. This adds it, and then corrects its own first framing. Timing model, calibrated from the measured 49.2/154.8/116.4 ms at [28,20,16] (1.757/7.740/7.275 ms per layer). Two objectives are reported side by side because they disagree: SERIAL sum(layers_r * ms_r) and PIPELINED max over stages. Backtest: the model reproduces the calibration point exactly, and the hand-arithmetic anchor holds -- [42,12,10] gives 239.43 ms against 320.4, i.e. 1.338x, matching the 1.34x expected by hand. Result table under the rank0 cap (x_serial / x_pipelined vs incumbent): [28,20,16] 1.000 / 1.000 incumbent [42,12,10] 1.338 / 1.667 the anchor [42,11,11] 1.341 / 1.818 DOMINATES the anchor on BOTH axes [42, 1,21] 1.367 / 1.013 serial-optimal and useless once pipelined The serial optimum is a degenerate cut that collapses pipelined throughput, so reporting only one objective would have recommended it. [42,11,11] is the candidate to arm. Rank0 cap recomputed rather than quoted: at 724.3 MiB/layer against a 31,800 MiB budget, 42 layers leave 1,379 MiB (corridor OK), 43 leave 655 MiB (CORRIDOR VIOLATED), 44 overflow by 69 MiB. So DESIGN_691's "about 42" is confirmed, and the binding constraint at 42 is the CORRIDOR, not raw weight overflow -- 43 fits the card and still must be refused. Co-solve correction (user, restating sgl-project#320/sgl-project#492): revision 1 priced a "pool cost" per candidate while holding the KV token vector PINNED. That is single-family optimization and sgl-project#485 forbids it. Layers moved to rank0 free exactly their weight bytes on the rank they left, and the uneven-DCP / rank-kv-ratio machinery relocates the displaced KV share onto those bytes. Rank0's cap therefore bounds rank0's SHARE of the token split, not the world pool. The conservation is exact, not approximate, and the tests prove it rather than assert it: total VRAM is fixed and the same 64 layers of weights exist wherever they sit, so total free bytes are invariant; under DCP token-sharding one token costs kv_per_token_per_layer * total_layers wherever it lands. World pool is therefore identical across every feasible cut, and the only residual is second-order -- seam/staging in both directions plus TP-phase redistribution -- which is ITEMIZED and whose sum equals the delta exactly (no fudge term). Scope note owed to the earlier verdict: "29,19,16 is 6.3 percent worse" held only under a pinned vector. Co-solved, that comparison does not stand as a capacity argument. Calibration limit stated in the model docstring and pinned by a test: one measured cut gives one point per rank and cannot separate per-layer slope from fixed per-stage cost. fixed_ms defaults to zero, the OPTIMISTIC end, so every speedup here is an upper bound. A second measured cut per rank would pin it. Tests: 15, red first. Planner regression 41 passed across the 702 and 602 suites. ruff clean. No deploy, no arm.

…mall flip and unmeasured THE MEASUREMENT, from 291 same-regime PHASE-FLIP DONE lines (and 3955 unique across all logs). The part NOT covered by read+exchange+write is: 123 live slots total 2453 ms unaccounted 1999 ms 81 % 440095 live slots total 4290 ms unaccounted 2287 ms 53 % FLAT at 2.0-2.35 s across a 3600x range of occupancy (123 -> 440095 live slots, 708 -> 3.4M cells). Regressed: unaccounted ~ live slots has slope 0.0005 ms/slot and R2 0.255 -- i.e. it barely correlates with size at all, which is what makes it a fixed cost rather than a small-flip artefact. Total ~ live slots gives intercept 2754 ms, R2 0.871. So the fixed cost, not the movement, is what floors every window sgl-project#677's economics can solve, and it is the term sgl-project#692 must price depth against. WHERE IT WAS HIDING, and I had it wrong first. My initial reading blamed the per-wave backing swap (release_wave/restore_wave). It is NOT the residual: `t_write0` is taken BEFORE those calls and `write_ms` accumulates after them, so the backing swap has been inside `write_ms` all along. Reading the timer placement rather than the call order is what corrected it. The three timers cover the WAVE LOOP ONLY. What fell outside is the tail: _pool_census("pre-cutover") phase_flip_runtime.py:6538 for fn in self._pre_cutover_fns: fn(...) :6539-6544 EXTRA MOVERS -- the weights arena refill and the GDN state leg _cutover_fn(direction) :6545 the group step _pool_census("post-cutover") :6547 THE MOVERS ARE OCCUPANCY-INDEPENDENT BY CONSTRUCTION -- the weights arena refill is the same bytes whatever the KV live set holds -- which is the leading explanation for a residual that does not move with occupancy. That is a hypothesis this commit makes MEASURABLE rather than one it asserts. SO THE TAIL IS TIMED, split movers vs cutover because they have different fixes, and both are reported on the DONE line and in `last_stats`. A residual that has to be regressed across boots cannot be priced per flip; a reported number can, and sgl-project#677/sgl-project#692 both need it per flip. WHAT THIS COMMIT DOES NOT DO. It does not reduce the cost. The reduction candidate is the wave count -- `_flip_waves` (:4024) is "A PURE FUNCTION OF THE REPLICATED LAYER MAP AND THE DIRECTION", so W=16 is paid at 123 live slots exactly as at 440095, even though the staging transient waves exist to bound is trivial there. The module's own docstring already pre-authorises the landing spot: "each extra wave costs one more exchange round trip. If a measurement ever shows the round trips dominating, W=8 is the place to stand, not W=1" (:4083-4089). That A/B needs NO code: `SGLANG_FLIP_SEAM_WAVES` (:2568) already overrides W. It is left unshipped deliberately -- lowering W trades ms for staging MiB, which lands on the arming floor and the corridor, and sgl-project#602 showed that budget is regime-dependent. Hard-coding a policy from one regime's numbers is the mistake that ticket already made once. HONEST LIMIT ON THE PER-WAVE TERM. Fitting unaccounted = a + b*W across the W=4 and W=16 populations gives b ~ 66 ms/wave, a ~ 997 ms. That fit is CONFOUNDED: the W=4 samples are release-first and the W=16 samples restore-first, so it mixes wave count with regime and must not be quoted as a per-wave price. The clean number will come from the env A/B above, within one regime -- which is exactly what the new movers/cutover fields will report. TESTS. 9 cases, source-level because the alternative is driving a full flip: the movers clock opening before the pre-cutover census and closing after the mover loop; the cutover clock wrapping the cutover; the three original timers untouched; both fields in `last_stats` and in the DONE line. Plus an AST arity guard on the DONE line -- specifiers counted against arguments -- because a %-format mismatch raises at the moment the flip completes, the worst possible place to learn it. Falsified: dropping one argument makes it report "17 format specifiers and 16 arguments". managers 2093 passed, 0 failed (2084 before, +9). Hermetic. NO DEPLOY.

…tor co-solved The sgl-project#602 capacity solve withdrew the cut for the POOL objective. The question actually asked -- more prefill on the 5090 -- is a time objective and was never solved for. This adds it, and then corrects its own first framing. Timing model, calibrated from the measured 49.2/154.8/116.4 ms at [28,20,16] (1.757/7.740/7.275 ms per layer). Two objectives are reported side by side because they disagree: SERIAL sum(layers_r * ms_r) and PIPELINED max over stages. Backtest: the model reproduces the calibration point exactly, and the hand-arithmetic anchor holds -- [42,12,10] gives 239.43 ms against 320.4, i.e. 1.338x, matching the 1.34x expected by hand. Result table under the rank0 cap (x_serial / x_pipelined vs incumbent): [28,20,16] 1.000 / 1.000 incumbent [42,12,10] 1.338 / 1.667 the anchor [42,11,11] 1.341 / 1.818 DOMINATES the anchor on BOTH axes [42, 1,21] 1.367 / 1.013 serial-optimal and useless once pipelined The serial optimum is a degenerate cut that collapses pipelined throughput, so reporting only one objective would have recommended it. [42,11,11] is the candidate to arm. Rank0 cap recomputed rather than quoted: at 724.3 MiB/layer against a 31,800 MiB budget, 42 layers leave 1,379 MiB (corridor OK), 43 leave 655 MiB (CORRIDOR VIOLATED), 44 overflow by 69 MiB. So DESIGN_691's "about 42" is confirmed, and the binding constraint at 42 is the CORRIDOR, not raw weight overflow -- 43 fits the card and still must be refused. Co-solve correction (user, restating sgl-project#320/sgl-project#492): revision 1 priced a "pool cost" per candidate while holding the KV token vector PINNED. That is single-family optimization and sgl-project#485 forbids it. Layers moved to rank0 free exactly their weight bytes on the rank they left, and the uneven-DCP / rank-kv-ratio machinery relocates the displaced KV share onto those bytes. Rank0's cap therefore bounds rank0's SHARE of the token split, not the world pool. The conservation is exact, not approximate, and the tests prove it rather than assert it: total VRAM is fixed and the same 64 layers of weights exist wherever they sit, so total free bytes are invariant; under DCP token-sharding one token costs kv_per_token_per_layer * total_layers wherever it lands. World pool is therefore identical across every feasible cut, and the only residual is second-order -- seam/staging in both directions plus TP-phase redistribution -- which is ITEMIZED and whose sum equals the delta exactly (no fudge term). Scope note owed to the earlier verdict: "29,19,16 is 6.3 percent worse" held only under a pinned vector. Co-solved, that comparison does not stand as a capacity argument. Calibration limit stated in the model docstring and pinned by a test: one measured cut gives one point per rank and cannot separate per-layer slope from fixed per-stage cost. fixed_ms defaults to zero, the OPTIMISTIC end, so every speedup here is an upper bound. A second measured cut per rank would pin it. Tests: 15, red first. Planner regression 41 passed across the 702 and 602 suites. ruff clean. No deploy, no arm.

… the sizer must NOT claim it VERDICT: no dead reservation found. The gap is the arming floor (deliberate) plus min-rule residue (structural), so this is the labeled accounting table the brief asks for in that case, not a forced fix. Per-rank, from the sgl-project#704/sgl-project#707 instrument chain rather than modelled: rank budget weights mamba rest holdback allowed used slack PP0 31800 16064.5 916.5 14819.5 6690.1 8129.5 5964.7 2164.8 PP1 18800 10061.8 654.3 8083.4 3562.7 4520.7 4260.5 260.2 PP2 19800 10699.8 523.3 8576.3 5167.9 3408.4 3408.4 0.0 Total slack 2,425 MiB, and the shape is the finding: it is almost all on PP0, and the BINDER PP2 has exactly zero. HOLDBACK is deliberate, established in sgl-project#707: the pool is capped so the resting free column still holds the arming floor. Sizing below it is the boot that holds the corridor and never flips (sgl-project#656 E/G). Not claimable. SLACK is structural and still not claimable BY THE SIZER. The pool is one global token count, so a non-binding rank cannot spend its extra allowance without the binder moving -- PP0's 2,164.8 MiB is not booked-and-untouched, it is unreachable at this cut. A sizer that claimed it would hand the pool tokens PP2 cannot back, and PP2 is the rank that OOMs. That is the sgl-project#593-family direction the brief warns about, so I wrote NO red-first test for it: the falsifier would have to assert a wrong answer. WHERE THE CAPACITY ACTUALLY IS: the slack is a property of the CUT, and that is already solved on another strand. sgl-project#702 rev5 found the incumbent [28,20,16] is not pool-optimal -- the binding rank switches, so moving layers OFF the binder raises the pool, and [30,18,16] gives +20 percent AND 1.11x pipelined prefill. Claim it by rebalancing the cut, not by relaxing a reserve. WHAT I DO NOT CLAIM: the ticket's 2.0 / 5.7 / 3.7 GiB per card is a DIFFERENT operating point from the instrumented boot, and the instruments that would attribute it per-term only exist on boots carrying 2a6305d / 5f3e61f / f55c1a8. Mapping this table onto those three numbers is the cross-boot arithmetic this strand has already retracted twice, so I did not. Ask for the next boot to capture the three instrument lines at that operating point and the table re-runs against it directly -- a window item. One thing that CAN be said without it: against the measured arming floors (1728 / 1825 / 2467 MiB) the per-card free exceeds the ~1024 MiB corridor target BY CONSTRUCTION, because on a flip-enabled boot the arming floor is the binding level, not the corridor. Reading the gap against 1024 overstates it on every rank. Docs only. No sizer change, deliberately.

…ce in one harvest plan, with a proven runner Consolidated from the comp4 gate run (progress.662-F4-r5), WINDOW_TICKET_745/755, NOTE_747 par.8-9, NOTE_738/755, TICKET_727 and the operator ledger. Two hard incompatibilities shape the plan: WINDOW_TICKET_745 Arm 1 excludes the checkpoint interval that sgl-project#758's anchor-cadence observable requires (-> ARM I hicache harvest, ARM II = one flag more), and sgl-project#713's TTFT<3s needs a quiet router while every other loaded gate needs the soak backlog (-> sgl-project#713 is the idle sub-phase BEFORE the backlog, not a separate boot; the 06:44Z soak driver is the load source per the Lastprobe rule). ARM I phases: load-time (sgl-project#738 no-99G-plateau, file-backed-image reclaim), idle-quiet (sgl-project#713, health, corridor), loaded (Gates A/B/C, - sgl-project#757 race-holds, sgl-project#748 all three shapes, sgl-project#744/sgl-project#717 rung-funded flip, - sgl-project#690 refill census, sgl-project#758-2 mamba host resume, WT_745's three lines, corridor minima), teardown (image reclaim). ARM II adds --mamba-checkpoint-interval 8192 for sgl-project#758-1 anchor cadence + NOTE_747 par.8.1-8.3. SEPARATE windows named with reasons: sgl-project#727 four-boot A/B, WT_755 slots A/B (pool-geometry confound), sgl-project#755 metal retraction (mechanism not built -- nothing to measure), sgl-project#709, sgl-project#735 Step-2. sgl-project#602 and sgl-project#536/sgl-project#537 carried as HONEST unresolved slots (owner-held detail / not found with acceptance shape) rather than invented readouts. Runner run_window_ladder.sh: PASS/FAIL/UNOBS table from boot log + live server; never boots, never kills, never touches the soak driver; soak-tolerant by construction (loaded checks are log observations, the one latency check runs only in --phase idle, and choosing that phase IS the operator's quiet-router assertion). Missing-emitter cases (anchor cadence) report UNOBS, never FAIL -- absence of an instrument is not absence of the property. Mock-smoked per the desk rule, both directions: a fixture built from the comp4 specimen lines reproduces the real run's verdicts exactly (GATE-C crash, both sgl-project#748 shapes + vacuous relief, sgl-project#757 sentence = 5 FAIL; sgl-project#744/sgl-project#690 PASS; exit 1) and a clean fixture goes fully green incl. the ARM II cadence line (exit 0). bash -n clean, codespell clean. Nothing was booted.

…le, as a rule The rejected register answers "was this CONFIGURATION tried and settled". It has no answer for a different question that turns out to matter more: is the number I am about to run traceable to work that was later withdrawn. An investigation can be retracted while its outputs keep riding, because nothing connects the two -- the value is just a number in a launch line by then, with no lineage attached to notice. The shipped uneven-DCP token vector is the proof. 29,19,16 comes from sgl-project#602, whose per-card attribution was withdrawn and re-derived more than once (NOTE_602_fill_side_attribution.md:67, "already had to retract twice"), so the slack figures it was proportioned from no longer stand. It has been the active vector on every boot since, and every one of those boots measured its own per-rank capacity, computed a better vector, printed it, and threw it away. The rule, stated so it can be enforced rather than remembered: AN ACTIVE VECTOR MUST NEVER ORIGINATE FROM A RETRACTED INVESTIGATION. "Active" carries the weight. A retracted value that is merely present is harmless -- as a sizing seed it is superseded in-process by the measured optimum before anything serves on it. A retracted value that is PINNED, or that is seeded and then fails to be superseded, is the number the server actually runs, and that is refused. So the gate belongs where a vector becomes active, and the seed path gets re-checked after the install attempt instead of waved through. Matching is layered, because a rule that fires only when the operator volunteers the incriminating fact is not a rule. Mode 1: a declared --*-provenance naming a retracted investigation is refused whatever the value, which is the durable general form. Mode 2: with no provenance declared, the gcd-reduced vector is matched against the values the retraction recorded -- without this the rule would be silent on precisely the vector that motivated it, since the launch shipping 29,19,16 declares no lineage at all. Mode 2 matches gcd-reduced so 58,38,32 cannot dodge it, and a stated clean lineage short-circuits it, so it stays a fallback for an unstated lineage rather than an override of a stated one. A measured install stamps PROVENANCE_MEASURED and is never refused -- the remedy must not be refused by the rule it satisfies. This commit is the register and the predicate. The boot-path refusals that consume them are the next commit; landed separately so the gate arrives with its can-fail evidence rather than alongside a wiring change. Tests: test/registered/unit/planner/test_retracted_provenance_797.py, 21 passed. Can-fail EXECUTED, both directions: neutralising the mode-2 value loop turns 2 red (the shipped vector and its scaled spelling stop being caught); making the predicate ignore its declared provenance turns 3 red. Restored and re-verified 21 passed. ruff check clean and ruff format clean on both files. No behaviour change yet: nothing calls into this module, so every existing path is byte-identical.

…a retracted study 214a9f6 landed the register and the predicate with nothing calling them. This wires them into resolve_cp_token_ratios, the single resolver every boot passes through, so the rule stops being a fact about a module and becomes a refusal. A PINNED vector whose provenance names a retracted investigation -- or, with no provenance declared, whose gcd-reduced value matches one a retraction recorded -- now raises RetractedProvenanceError before anything is sized. Both explicit doors are covered: SGLANG_UNEVEN_TOKEN_VECTOR and the --rank-kv-ratio a,b,c pin. The falsifier is the vector this rig shipped: 29,19,16 from sgl-project#602, pinned, with no lineage declared, refused by value. A SEED is permitted past this point and warned about instead, and that is the rule applied consistently rather than softened. The rule is about the ACTIVE vector; a seed is superseded in-process by the measured optimum before anything serves on it, so for a seed the question is only answerable AFTER the install attempt. That check is the remaining piece and it is named here rather than implied: a seed that fails to be superseded is precisely the boot this task exists to end, and it must be refused at the install site. Two flags carry the lineage, both published to the environment like the role and for the same reason -- the resolver runs inside the flip's second stack build, where the boot's ServerArgs object is not the one consulted. --uneven-token-vector-provenance is published ONLY when stated, because "not stated" is a meaningful answer: it is what ARMS the fallback value match. Publishing "None" as a string would read as a declared lineage and disarm the gate, which is the failure mode this whole task is about. _token_vector_role gets one definition, env over flag, and an EMPTY env value reads as "not stated here, ask the flag" rather than as "pin". That is deliberate and it is the 2026-08-19 lesson: an empty override silently meaning the default is how a blank SGLANG_UNEVEN_TOKEN_VECTOR rode along for days unnoticed. A gate that switches itself off when handed an empty string is the same defect wearing different clothes. Tests: test_retracted_vector_boot_refusal_797.py, 12 passed, new, all driving the real resolver rather than the predicate -- the sgl-project#182 lesson, where a token-vector honesty guard was correct and sat on a branch no server could reach. Can-fail EXECUTED twice: deleting the refusal call in the env branch turns 5 red; making _token_vector_role hard-default an empty env to "pin" turns the empty-override test red. Restored and re-verified 12 passed. The sibling suite test_retracted_provenance_797.py stays at 21 passed. Regression, matched HEAD, both runs hermetic (CUDA_VISIBLE_DEVICES="") and identical selection: test/registered/unit/distributed/ at HEAD 57778d4 in a clean worktree gives 81 failed / 2997 passed; with this change, 81 failed / 3009 passed. Same failure count, and the +12 passed are exactly this commit's new tests. Sampled failures are pre-existing GroupCoordinator/parallel_state attribute errors unrelated to the token vector, and 81 matches the pre-existing baseline 95fdc54 recorded. Noted honestly: this repo's reporter emits no "FAILED id" summary lines for these, so the sets were compared by count and by sampling rather than id-by-id. ruff check and ruff format clean on the touched files; the two format hunks ruff still reports in distributed/utils.py are pre-existing and outside this change. Default path unchanged: with no explicit vector, no provenance and no retraction match, the resolver behaves exactly as before.

…alue match sgl-project#797 layered two matching modes and returned out of the first one. A declared provenance naming an investigation that is NOT in the retraction register hit `return by_investigation(declared)` -> None, and mode 2's value match never ran. So the shipped 29,19,16 -- from retracted sgl-project#602, the vector the whole gate exists for -- passed under any unrelated lineage. Not an exotic input. The env twin SGLANG_UNEVEN_TOKEN_VECTOR_PROVENANCE outlives the process that set it, so one earlier launch in the same shell disarms every later one. The gate protects the tree-spec-DCP class from serving on withdrawn evidence; a value-blind gate protects nothing. FIX DIRECTION: the modes ADD, they do not replace. Mode 1 may convict a vector whose value looks innocent; it may never acquit one whose value does not. A declared, non-retracted provenance now falls THROUGH to the value match instead of returning. SCOPE, and the boundary is deliberate rather than an oversight. PROVENANCE_MEASURED keeps its short-circuit. It names no investigation -- it asserts that this boot profiled the value itself -- so a value match there would refuse a measurement for resembling the withdrawn estimate it replaced. That exemption is sgl-project#797's own and is tested directly at test_retracted_vector_boot_refusal_797.py::TestWhatMustNotBeRefused797; widening past it would have turned a green sibling red, which is a different ticket, not this one. Named here rather than changed silently. RED FIRST, both axes. Before the fix: 6 failed / 6 passed -- the 6 red are the new-behaviour cases, the 6 green are mode 1, the no-false-positive side and the measured exemption, so the file is not blanket-red. After: 36 passed across the new suite plus both sgl-project#797 suites (boot refusal, seed liveness), i.e. the widened refusal regresses neither sibling. THE MUTANT, as the ticket names it: provenance = 'sgl-project#797' (real, not retracted) over vector 29,19,16 (retracted). Caught, at the predicate and at the boot path through resolve_cp_token_ratios -- sgl-project#182's rule, a gate no boot arrives at is not a gate. The gcd-written form 58,38,32 is caught on the same path, so the fix is not one rewrite wide. CLASS SWEEP. The shape is "a declared lineage short-circuits a value check". planner/rejected.py, the register next door, has no provenance concept at all and cannot carry it. distributed/utils.py:690 reads like the same shape and is not: it is the env-over-flag fallback that PRODUCES the declaration, and None there arms the value match rather than disarming it. retracted.py was the only instance. Two comments the fix made false are corrected in the same commit: the module docstring's "mode 1 short-circuits before any value comparison happens" and mode 2's description as the UNDECLARED-provenance mode.

…guessing Boot 24 died of a CUDA OOM at `fla/chunk_o.py:146 torch.zeros_like(v)` inside the GDN extend kernel, minutes into serving, with 21.69 MiB of its budget left. The corridor guard had emitted CANNOT FULLY HOLD THE CORRIDOR FLOOR three times on PP0 and the third one landed in the same second as the OOM. THE INSTRUMENTS TO ANSWER THIS ALREADY EXISTED IN THIS TREE AND HAD NEVER BEEN ARMED ON A BOOT. `flight_recorder.arm_process_trace` is wired correctly at scheduler.py, ahead of `Scheduler(...)` -- the ordering that makes per-block stacks attributable at all (sgl-project#602: armed post-boot it covered 80 of 2046 blocks). What was missing is not a mechanism, it is a RUN. So this adds no new bookkeeping; it opens the existing one at the two moments that matter. WHAT CHANGES, three call sites and one flag: * `HOLD_ENV` / `hold_trace_through_serving()` (sgl-project#1054). The window closes at `boot_complete` today, and that is right for the RESIDENT posts -- they are all in place by then, and serving would otherwise grow an uncapped ring for the life of the process. It is wrong for the TRANSIENT one: the allocation that killed boot 24 only happens under real prefill depth, i.e. only after the window shuts. The flag is read at the ONE disarm site, not inside `disarm_process_trace`, so the verb keeps meaning "stop recording" for every other caller; what is conditional is the boot's decision to stop. The held case logs its own price (uncapped ring, host RAM grows for the life of the run) and says it is never for an acceptance boot. * The scheduler's exception handler dumps the allocation snapshot. The death is the one moment the allocator state is worth most and the one moment nothing captured it: boot 24's traceback names the LINE and says nothing about who held the other 31 GiB. It imports `flight_recorder` ITSELF rather than reusing the name bound in the try block -- an exception that beats that import would otherwise raise NameError inside the crash handler and report "could not dump" for a boot that never armed anything. * The corridor guard dumps on its FIRST dip per process. That line is the last honest warning before this class of death, and nothing recorded who held the memory when it fired. First dip only: the onset is the interesting state and a dump per dip would write gigabytes under load, which is how a diagnostic becomes the outage. * AND THE GUARD STOPS ASSERTING AN ALL-CLEAR IT CANNOT SUPPORT. The line said "the allocation itself fits, so this is a dip and not an OOM". Boot 24 refuted that in the emitting line's own second. The predicted trough comes from THIS pass's providers and does not bound what a later kernel allocates on the same device -- the GDN transient has no term in it. The sentence now states the scope of its claim, carries the refutation with its date, and ends as a warning with an unmeasured residual. INDIKATOR-GESETZ: an indicator is a finding only once it has been checked that it measures what it claims. Every new path is a no-op unless the recorder is armed (`dump_trace` returns None), so an ordinary boot pays an attribute lookup. ROOT BEFORE EFFECT * CLASS: a modeled term that no instrument ever measured, while the instrument that could measure it shipped disarmed. The ledger has been printing "6 terms neither measured nor bounded" in 80 of 82 boots. * SIBLING SWEEP: `phase_flip_runtime.py:10452` emits a same-named warning for the flip seam entry. It is NOT the same instrument and is deliberately left alone -- it already carries a MEASURED term ("this rank's worst MEASURED draw of %d MiB"), which is exactly what the corridor_guard line lacked. Boot 24's three lines were the corridor_guard one (they name a device and an arming floor). * FUTURE CHECK: `devtools/check_1054_memory_truth.py`, hermetic, no CUDA. It asserts the wiring structurally (AST, not substring): the disarm sits in the else of the hold test, exactly one disarm site remains, arm still precedes `Scheduler(`, the handler dumps and imports for itself, the guard dumps on dip 1, and the retracted all-clear no longer terminates the warning. It caught one real defect while being written -- the first version of its last assertion searched for the retracted sentence and failed on the retraction that QUOTES it; re-aimed at the assertive form, and the reason is in the check. EVIDENCE TIER: DESK-PROVEN. 11/11 green. The metal proof is the diagnostic boot; nothing here is a ship number and the launcher labels the run DIAGNOSTIC-ONLY in its header.

…osts THE OBJECTION THIS ANSWERS IS THIS MODULE'S OWN. `arm_process_trace` passes no `max_entries` on purpose: sgl-project#602's capture came back exactly full at 100000, holding its final 10.7 seconds, and a ring that wraps SILENTLY turns "this post has no allocation event" into an untrue statement. That argument is about SILENCE, not about caps, and it stops applying the moment the wrap is COUNTED. A cap became necessary with sgl-project#1054's held-open window. The ring then grows for the life of the process on a swapless box that has already host-OOM-killed serving WITHOUT foreign load (sgl-project#1038: oom_kill 6->7 across one boot death), where serving carries oom_score_adj=500 and is therefore the preferred victim -- so the cgroup counter would name the victim and never the cause. A diagnostic boot that kills the box measures nothing. THE DROP COUNT IS DERIVED, NOT GUESSED, and this is the part that makes the cap admissible. torch reports no drops, but `memory_stats()["allocation.all.allocated"]` is a monotone count of every allocation the process has made. Sampled at arm time and again at each dump, the difference is how many events the ring was OFFERED; everything past the cap fell out of it. `ring_loss_report()` prints that with every snapshot, beside the artifact's path, so a reader opening the pickle six boots from now does not have to reconstruct whether it wrapped. Three states, never two: a complete window, a named number of dropped events, or UNKNOWN when torch's counter was unreadable -- and UNKNOWN is never rendered as "none dropped", because an unmeasurable loss is not an absent loss. Scope kept honest in both directions: * Uncapped remains the DEFAULT and is still called the better record. The cap is opt-in, for the one configuration that needs it. * A malformed cap value falls back to UNCAPPED, never to a zero-entry ring -- the failure mode of `int(raw)` on a typo would otherwise be a silently empty trace that looks like a boot which allocated nothing. * The module docstring's "max_entries is therefore never capped here" is corrected in place rather than left to contradict the code. * The ring is BYCATCH for this campaign anyway: the load-bearing artifacts are the snapshots at the corridor guard's first dip and in the crash handler, whose per-block stacks come from the live allocator's segments, not from the ring. Launcher side (uncommitted, /spinning/gpu-arb): the diagnostic branch arms ONE rank by default -- rank 0, the rank boot 24 died on -- rather than all three, so one uncapped-by-default post does not become three; caps the ring at 3,000,000 entries; and runs `devtools/diag_host_preflight.sh` first, which prices the ring against `free -g` and the sgl-project#721 16 GiB floor and REFUSES the launch rather than discovering the shortfall by dying. Measured now: 109 GiB available, ring bound 1 GiB at that cap, 92 GiB headroom. The pre-flight states what it does NOT cover -- it prices the ring only, not the serving posts and not the unmeasured transient the boot exists to measure, so a PASS is not a promise that this boot cannot host-OOM. FUTURE CHECK: `devtools/check_1054_memory_truth.py` grew four assertions -- uncapped by default, cap read when set, malformed cap falls back to uncapped, and UNKNOWN never rendered as zero drops. 15/15 green. EVIDENCE TIER: DESK-PROVEN.

add minicpm support

f54f2a0

merrymercy merged commit 710f614 into sgl-project:main Jul 9, 2024

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

add minicpm support (sgl-project#602)

71bf59f

hzwzwzw pushed a commit to hzwzwzw/sglang that referenced this pull request Jul 2, 2026

Return bad request for malformed DSV4 tool arguments (sgl-project#602)

d958a63

Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

Navigation Menu

Uh oh!

add minicpm support - #602

add minicpm support#602
merrymercy merged 1 commit into
sgl-project:mainfrom
Titan-p:main

Titan-p commented Jul 8, 2024

Uh oh!

merrymercy commented Jul 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Titan-p commented Jul 8, 2024

Uh oh!

merrymercy commented Jul 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants