Fix the default argument of OpenAI Chat completion by merrymercy · Pull Request #605 · sgl-project/sglang

merrymercy · 2024-07-09T08:52:33Z

Fixes #582

Bump the pinned DeepEP commit (ci_install_deepep.sh and docker/Dockerfile) from 9af0e0d0 to d4f41e4 (v1.2.1-32), the first pin that ships EPv2 ElasticBuffer (EPv2 was introduced in DeepEP sgl-project#605 / b306af0). d4f41e4 is a descendant of the old pin and still exports the legacy v1 Buffer, so the existing deepep backend keeps working while the epv2 backend can import ElasticBuffer. EPv2 links NCCL symmetric memory (nccl::NCCLSymmetricMemoryContext), so the build needs an NCCL shipping the symmetric-memory headers (verified with nvidia-nccl-cu13>=2.30.7).

Bump the mainline DeepEP pin from 9af0e0d (v1.2.1, 2025-09-15) to 567632d, the last commit before the EPv2 repo reorg (sgl-project#605). This picks up deepseek-ai/DeepEP#458 (hidden-dim 3072 low-latency kernels), the use_fabric buffer kwarg (sgl-project#217), and NVSHMEM 3.5 runtime fixes (sgl-project#564), while keeping the pre-reorg layout the Dockerfile build steps expect. Also updates the matching checkout in ci_install_deepep.sh. Context: sgl-project#24502.

…asurement boot is cheap SGLANG_VRAM_FLIGHT_TRACE now takes "1"/"all" or a comma-separated rank list. A SCOPE, not a cap: an armed rank still records its whole boot with nothing dropped. The reason to have it is that the ring is uncapped by design and its host-RAM cost scales with a boot's allocation count on a swapless box, so the pressure to reach for max_entries -- the knob that produced the sgl-project#602 wrap -- has to be answered by a different lever. A rank scope costs the unscoped ranks their trace visibly; max_entries costs every rank its oldest events silently. This makes the measurement boot stageable. Marks alone (SGLANG_VRAM_FLIGHT_DIR, ~4 ms per rank per boot, no trace) already yield the complete per-post resident dataset; the trace is only needed to break a post down by call site, and can then be armed on one rank. TESTS - 236 green across the mem_ledger suite (was 232; four scope tests added, including that "0" is read as a rank name and not as a boolean off, which would otherwise produce a boot with no trace at all). - GPU chain test re-run on the 5090: ALL PASS, 16 checks. - The wiring pin from the previous commit caught this commit's own change of arm_process_trace's signature and had to be updated. That is the pin working.

…tead of modelling it The demand model overpredicts by 4664 / 993 / 2701 MiB on this rig (sgl-project#602 window), i.e. up to 4.7 GiB of KV pool per card is given away, because no MODELED term has ever been checked against a MEASURED one. This adds the instrument that produces the measurement. It records and attributes; it sizes nothing and writes no number into the ledger. Three sources, because no single one covers the posts. 1. Phase marks. mark() reads torch.cuda.memory_stats and NVML at eight named boot boundaries (BOOT_PHASES) and appends one JSON line per mark. The difference between two consecutive marks IS the post allocated between them. Measured cost: 0.6 ms per mark, ~4 ms per rank per boot. 2. Process-start allocation recording, armed in run_scheduler_process before the first CUDA allocation. max_entries is NOT capped: the sgl-project#602 capture set it to 100000, came back exactly full, and therefore silently held only its final 10.7 s. The window is instead bounded by an EVENT -- the snapshot is dumped and recording stopped once every runner in the process is up -- so nothing inside the window is dropped and serving does not pay +1.7 us per allocation (3.7 vs 2.0 us, measured) for the life of the process. Dumped once per process, not per runner: under speculative decoding the target and the NEXTN draft both capture graphs, and a snapshot taken at either one's capture_end is missing the other's. 3. The non-torch remainder, per PID. Each mark records what NVML says THIS pid holds on the card, so non_torch = nvml_self - torch_reserved is measured rather than left over from two card-level totals. This also settles TERM_PARENT_CONTEXT by direct observation instead of by gap inference. CORRECTION TO THE sgl-project#602 HANDOFF. It advises reading device_traces and NOT segments[].blocks[].frames. That is right for those captures and the reason is worth keeping straight: torch fills block frames only for blocks allocated after recording began, so a post-boot arming starves that structure (80 of 2046 blocks, 3 of 25142 MiB). Armed at process start it becomes the EXACT resident attribution, because it is keyed on the blocks the allocator still holds rather than on a ring of events. Both structures are parsed, and both parsers return a Coverage verdict beside their numbers -- the sgl-project#602 aggregation produced a tidy, sorted, entirely truthful 3 MiB answer to a 25142 MiB question with nothing in the output saying so. MEASURED, and correcting an assumption of my own: torch.cuda.is_initialized() is NOT "a primary context exists". On this rig, pinned to the 5090: after import torch -> False / NVML 0 MiB; after importing any sglang.srt module -> True / NVML 0 MiB; after the first 1-byte allocation -> True / NVML 500 MiB. Verified in the same run that current_device(), memory_stats() and _record_memory_history() bind nothing either, which is what makes it safe for a mark to read them at process start. The direct question is answered by NVML's per-process list and recorded as holds_device_context. FINDING, not fixed here: DemandInputs.parent_binds_cuda_context is hardcoded False at engine.py:467, so TERM_PARENT_CONTEXT is never charged on the production path -- only test_full_demand_reserve_593 ever sets it True. On the reference rig the parent/tokenizer binds a context on the TP0 card, so this is an UNDER-charge of about one context. It is left alone until the measurement boot observes the parent directly in NVML's per-process list; correcting a term against an inference is what this task exists to stop. TESTS - test_flight_recorder_605.py: 26 tests green (hermetic, CUDA_VISIBLE_DEVICES=99). - Whole mem_ledger suite: 232 green, no regression. - CAN-FAIL PROVEN by mutating production code and observing red, four times: coverage ignoring unframed blocks -> 3 red; python_site returning frames[0] -> 3 red; churn ignoring orphan frees -> 2 red; deleting the kv_pool_sized call site -> 1 red. Restored to green after each. - Parsers run against the real sgl-project#602 captures, not fixtures: 2046 blocks / 80 framed / 25142 MiB reserved / 100000 entries / 40 orphan frees / 31 MiB peak outstanding, all asserted. The 31 MiB independently reproduces the handoff's corrected figure. - CHAIN TEST ON A REAL CARD (scripts/vram_ledger/flight_selftest.py, 5090 by UUID, ~570 MiB, corridor untouched): ALL PASS, 16 checks. It proves what no fixture can -- arming binds no context, marking binds no context, and a snapshot from a process-start arming reaches COMPLETE coverage with the probe allocation attributed to its own source line. Its first cut asserted on torch's is_initialized flag and reported four failures against a correct instrument; that is where the measurement above came from. - Wiring pinned against the serving tree's own source, because sgl-project#602 closed with twelve green fixture tests beside a production carrier that lacked the field. Proves presence, not execution; execution on the serving path needs a boot. NOT YET DONE: no measurement boot has run, so there is no per-post dataset for this rig yet and demand model v2 is not started.

…asurement boot is cheap SGLANG_VRAM_FLIGHT_TRACE now takes "1"/"all" or a comma-separated rank list. A SCOPE, not a cap: an armed rank still records its whole boot with nothing dropped. The reason to have it is that the ring is uncapped by design and its host-RAM cost scales with a boot's allocation count on a swapless box, so the pressure to reach for max_entries -- the knob that produced the sgl-project#602 wrap -- has to be answered by a different lever. A rank scope costs the unscoped ranks their trace visibly; max_entries costs every rank its oldest events silently. This makes the measurement boot stageable. Marks alone (SGLANG_VRAM_FLIGHT_DIR, ~4 ms per rank per boot, no trace) already yield the complete per-post resident dataset; the trace is only needed to break a post down by call site, and can then be armed on one rank. TESTS - 236 green across the mem_ledger suite (was 232; four scope tests added, including that "0" is read as a rank name and not as a boolean off, which would otherwise produce a boot with no trace at all). - GPU chain test re-run on the 5090: ALL PASS, 16 checks. - The wiring pin from the previous commit caught this commit's own change of arm_process_trace's signature and had to be updated. That is the pin working.

…ys honest Precondition for arming SGLANG_VRAM_FLIGHT_DIR on every production boot. The mark file is APPENDED -- that is what lets a crashed boot keep the boundaries it did reach -- so a permanently armed recorder makes one file hold many boots, and a reader that returns the whole file returns marks from several process lifetimes. phase_deltas over that computes a difference from one boot's last mark to the next boot's first: a number that describes nothing, printed in the same table and the same units as the real posts. Left alone, permanent arming would have made the instrument's output less trustworthy the longer it ran. - Every mark carries boot_id, derived from the LAUNCHER (this process's parent pid + start time) rather than from this process, so all ranks of one boot agree on it without a collective. A per-process uuid would tag each rank's file differently and make cross-rank reading impossible. - read_marks returns the latest boot by default, a named boot on request, and the whole file only when explicitly asked with boot="all". - phase_deltas SKIPS the seam and logs it, rather than reporting it. - list_boots + `attribute_flight.py boots` say which boots a directory holds; `phases --boot` selects one. MEASURED, for the retention question: one rank's boot is 8 marks / 5800 bytes (725 B per mark), so a 3-rank boot costs 17 KiB and 1000 boots cost 16.6 MiB. No rotation policy is proposed for the marks, because at that size the condition does not bind and an unused knob is one more thing to get wrong. Snapshots are the opposite case (~18 MB each) and are only written when the trace is armed, which stays window-only. TESTS - 241 green across the mem_ledger suite (was 236; six boot-scoping tests). - CAN-FAIL PROVEN twice: making phase_deltas span the seam -> 1 red (4 deltas where 3 are real); making read_marks return the whole file -> 2 red. - attribute_flight.py smoked end to end against a two-boot directory: `boots` lists both, `phases` shows only the latest. - Rebased onto 7044be2 (production tree: sgl-project#602 ledger + sgl-project#603 hoist) before all of the above; the rebase was clean and every call site survived it.

…t posts The payout step. "Internal demand overpredicts by 4664 / 993 / 2701 MiB" is a fact that funds no fix: it says the model is wrong without saying WHICH term is wrong. This puts each modeled term beside the measurement that brackets it and prints the error per term. - contract.py writes the MODELED ledger into the flight directory at boot (no-op unless the recorder is armed, before the fit/refuse verdict so a refusing boot is inspectable). Without it the two halves of the comparison never exist at once: the ledger is logged and gone, the marks arrive later. - reconcile.py maps terms to phase gaps through TERM_TO_POST, a TABLE with a stated basis per entry, so a reader can reject a mapping he disagrees with instead of reverse-engineering it from arithmetic. - `attribute_flight.py reconcile <dir>` prints it. TWO WAYS SUCH A TABLE COULD LIE, both pinned: ABSORBING. A measured byte no term claims, folded into the nearest term, makes the table balance and the unmodeled post disappear. The residuum is therefore computed independently and printed as its own row whether it is 4000 MiB or 0. PRETENDING TO MEASURE. A term with no bracketing boundary is UNMEASURED, not 0-measured. Reporting 0 would manufacture a full-size overprediction on a term that may be perfectly correct -- TERM_NCCL_BUFFERS is exactly that case today. Two more honest refusals fell out of the mapping: the pool-sizing gap holds the KV pool AND the state pool, so the KV pool (the ledger's residual) is subtracted -- and when the gap is SMALLER than the pool the ledger placed in it, the gap is not split at all, because the pool did not get what was budgeted and a negative state pool is nonsense reported as fact. TERM_ACTIVATION is labelled a LOWER BOUND: the first forward is not necessarily the deepest the rank will see. TERM_PARENT_CONTEXT gets a DIRECT settlement rather than an inference: bytes held on the card by any process that is not a rank of this boot, from NVML's per-process list. A rank-only card yields a MEASURED zero, which is a different statement from "we did not look". TESTS - 254 green across the mem_ledger suite (was 241; 13 reconciliation tests). - Two of those tests failed on first run and the CODE was right both times: the residuum reported 1325 and 300 MiB where I had predicted 900 and 0, and in each case it had named exactly the posts I had left out of my own fixture ledger. Expectations corrected, not the code. - CLI smoked end to end against a synthetic boot: a deliberately inflated capture term reads +2701 MiB against a 900 MiB measurement, the unmodeled attention workspaces land in a 300 MiB residuum, and the NCCL term reads UNMEASURED. NOT YET RUN against a real boot; no rig numbers exist here. The mapping bases are claims about where allocations happen and the first real boot is what tests them.

…, not where one flagset looks FIRST REAL EXECUTION FALSIFIED THE PAYOUT. Production booted the recorder tree at 21:11, wrote all three ranks' marks correctly, and produced NO modeled ledger, so reconcile had nothing to compare against. Two independent defects, either of which alone would have wasted the boot. 1. THE DUMP SAT ON A FLAG-GATED PATH. It was placed in enforce_boot_contract, which is reached only from _vram_ledger_non_kv_per_gpu, which sits inside `if vram_ledger_enabled()`. Production runs --rank-auto-reserve-mib auto with enable_vram_ledger False, so it takes ledger_full_demand_per_gpu and never calls the contract at all. server_args.py states this outright in its own comment: "the gated ledger path has always added it (_vram_ledger_non_kv_per_gpu), this one did not, and production runs THIS one". I had that comment in front of me while writing the dump. The fix is not a better caller. It is that a caller was the wrong KIND of place: the dump now sits in engine.build_card_ledgers, the single function that constructs a ledger, whose own docstring says both paths share it. No reserve path, flagset or future caller can route around it. 2. THE LAUNCHER AND THE RANKS WOULD HAVE DISAGREED ON THE BOOT ID. The ledger is built during argument resolution in the LAUNCHER; the marks are written by the RANKS. boot_id derived from the parent process gives a rank its launcher (right) and the launcher its shell (wrong), so the two halves would have been filed under different ids and reconciliation would have matched nothing even with defect 1 fixed. The launcher now publishes the id through SGLANG_VRAM_FLIGHT_BOOT_ID and the ranks inherit it. The published value equals what the parent-based derivation produces, so nothing that already works changes. TESTS - 263 green across the mem_ledger suite (was 254; 9 new). - The new tests EXECUTE the constructor rather than inspecting the source. A source-level assertion would have passed for the shipped defect too: the call was present, just somewhere production never went. That is the whole lesson of this commit and it is why the test calls build_card_ledgers and looks for the file. - CAN-FAIL PROVEN twice against the exact shipped defects: putting the dump back behind the contract -> 3 red, including "no ledger written by the constructor" and the ledger/marks id-agreement test; making the launcher derive its id from its parent -> 1 red. - Also pinned: contract.py may not mention the recorder at all, so nothing Stage 1 needs can drift back behind that gate. - A dump failure never costs the caller its ledger (asserted against an unwritable directory); unarmed boots write nothing. Found while making the fix: my scripted removal of the old dump truncated contract.py, taking kv_pool_mib_per_rank with it. The suite caught it on the next run. Restored from git and redone. NOT YET RUN against a real boot. The next armed boot is what proves this, and it proves it by producing a ledger_<boot_id>.json beside the three mark files.

…rank runs two runners The 21:11 boot's marks show 14 phases per rank, not 8: under speculative decoding the target AND the NEXTN draft each run initialize(), so pre_weight_load / weights_loaded / kv_pool_sized / capture_begin all fire twice. Only capture_end carried the runner tag, which was enough to read the capture posts and not enough for the rest. MEASURED on rank 0 (5090), why this matters: target weights 13124 MiB, then reserved DROPS 154 MiB, then draft weights 2052 MiB. reconcile's gap rule takes the first occurrence of the opening phase and the last of the closing one, so the state-pool gap (weights_loaded -> kv_pool_sized) currently spans the draft's weight load and would charge 2052 MiB of draft weights to the mamba/GDN state pool. The number would have looked like a large underprediction in a term that is fine. This commit only makes the data separable; the per-runner gap arithmetic in reconcile.py is the next step and is not in here. TESTS: 263 green, unchanged. This is a tagging change; the boot that proves it is the next armed one, and it proves it by showing draft_worker on all 14 marks instead of 2.

…t skipped Crashes 9, 10 and 11 (2026-08-05) all ended in the same unhelpful shape: ranks 0 and 1 standing in one collective, rank 2 standing in a different one, and nothing anywhere saying which collective rank 2 had SKIPPED. The tracebacks report where each rank was STANDING, which is downstream of the defect. The per-rank counts already existed -- the prefill line prints "tp.all_reduce 243.4/129x" -- they were simply never compared ACROSS ranks. This adds that comparison as a standing instrument. It is measurement, not mitigation: it changes nothing about what the ranks do and cannot repair a divergence. It reports, so the next crash arrives with its cause attached. Counting: one dict increment per collective, at the four GroupCoordinator dispatch sites, keyed by the family names the collective clock already derives (tp.all_reduce, dcp.all_gather, ...). Placed AFTER the clock's re-entry guard so each collective is counted exactly once. Guarded by a module-level bool resolved at import, so the hot path costs one bool read plus one dict increment. Two reporting paths, for two different failure states: 1. Scheduler._census_tick -- the detector. Runs on tp_cpu_group (gloo), NEVER the device/BAR1 path, so a wedged device group cannot silence the instrument meant to explain it. Rides the same unconditional once-per-iteration point as the sgl-project#603 reduce, so its round counter is replicated and the cadence gate opens on the same round for every rank. Default every 50 iterations (~2 s in production), an order of magnitude inside the ~30 s spin deadline: the divergence is named while the ranks are still healthy, long before the abort. 2. format_local_census, called from barlink_bar1.check_aborted before it raises -- the wedge-proof half. Takes NO collective on purpose: by abort time a peer is very likely dead (in all three crashes the third rank aborted ~30 s after the first two), so a cross-rank exchange would hang exactly when the evidence is wanted. Each rank logs its own counts; the three lines are diffed afterwards. Warn-never-raise throughout: a failed comparison is logged and dropped, and the abort-path dump can never mask the real error. ARMED BY DEFAULT, like the sgl-project#605 flight-recorder marks -- an instrument that must be switched on before a crash explains no crashes. SGLANG_COLLECTIVE_CENSUS=0 disables counting; SGLANG_COLLECTIVE_CENSUS_INTERVAL tunes or (<=0) disables only the periodic comparison, leaving the abort dump armed. Tests: test/registered/unit/distributed/test_collective_census_583.py Hermetic (no CUDA, no process group, no model). The injected divergence is crashes 9/11 in numbers: [129, 129, 128] on tp.all_reduce. * 14 passed * can-fail proof: breaking only the count comparison in _diff turns 4 of them red * pins the missing-key case (a rank that skipped a family's ONLY collective has no key at all -> must read as zero, not drop the family) * pins armed-by-default, the kill switch, the replicated cadence gate, and warn-never-raise on a wedged/absent group * regression: census + retract-583 + 603 + retract-fcfs = 35 passed * ruff: 0 new findings (parallel_state 0/0, barlink_bar1 2/2, scheduler 89/89, schedule_batch 40/40); new files ruff-clean and formatted; codespell clean Crash 11 (21:26, boot 11, tree WITHOUT the retraction fix) is byte-for-byte crash 9's shape and NOT crash 10's: all three ranks are inside run_batch (scheduler.py:2032), ranks 0/1 at communicator.py:1179 and rank 2 at linear.py:2340, with check windows 1 and 4. So the `if batch:` fork is not involved there, and crashes 9/11 are a site INSIDE the forward, distinct from the retraction site closed for crash 10. The census is what will name it.

…false zero residue The fill side of sgl-project#602 stayed open because the instrument that was supposed to name the posts was wrong in two ways on the ship config. Both were found by reading the marks of a real ship boot (1353495-1786609875, 2026-08-13, --tp-size 1 --pp-size 3 --rank-gpu-memory-mib 31800,14000,15600). RANK COLLISION. Marks are filed under the TP rank. The ship config runs TP=1 with PP=3, so the TP rank is 0 in all three processes and all three append to flight_marks_rank0.jsonl. read_marks keyed its result on the rank field, so three processes on three different cards came back as one timeline, and differencing it produced posts no card ever paid -- a 20480 MiB card's 482 MiB CUDA context billed to the 32607 MiB card. Grouping is now by pid, which the writing process stamps and which cannot collide while it lives. The boot is also resolved across all rank files instead of per file, since ranks do not stop writing at the same instant. FALSE ZERO RESIDUE. non_torch_bytes was written as max(0, nvml_self - reserved). On this config torch reports up to 7162 MiB MORE reserved than NVML says the process holds, so the subtraction went far negative and the field read 0 -- publishing "no CUDA context, no NCCL buffer, no JIT workspace on this card" for a rank whose context measured 886 MiB. A floor built to absorb sub-MiB quantisation was swallowing gigabytes. The two cases are separated: non_torch_measurable says whether the subtraction is a measurement at all, and unbacked_reservation_bytes measures how far reserved exceeds resident. The new report closes the per-card identity exactly on all three cards of the ship boot: nvml_total = carve_out + resident + foreign + free, drift 0 MiB. NOT claimed here: that the unbacked reservation causes the underfill. The CUDA path of get_available_gpu_memory reads torch.cuda.mem_get_info (the driver's free) after an empty_cache, not torch's reserved counter, so that chain is falsified rather than asserted. Tests, PYTHONPATH=/spinning/wt-605-fr/python, python3 -m pytest: - test/registered/unit/mem_ledger/test_fill_side_605.py: 4 passed. Red first: 3 of 4 failed against the pre-fix recorder (rank grouping "1 != 3"; KeyError non_torch_measurable; unbacked_reservation_bytes absent from a mark whose non_torch_bytes read 0 on 27308 MiB resident). - Can-fail proof for the unbooked-allocation falsifier: a mutant that sets the residual line to 0 fails with "!! IDENTITY BROKEN by 300 MiB"; reverting the mutant restores 4 passed. - test/registered/unit/mem_ledger/: 357 passed, 1 failed, 1 collection error. Both pre-existing and untouched by this change: the 612 communicator-group declaration test and a missing 'datasets' module, neither of which references flight_recorder, read_marks or the report. - ruff: all checks passed. black: clean. codespell: clean.

…ed spread Records what the recorder produced once it stopped misfiling ranks and stopped publishing a floored zero (eb5ace7). The fill-side table closes the per-card identity exactly on all three cards of ship boot 1353495-1786609875 -- nvml_total = carve_out + resident + foreign + free, drift 0 MiB -- with a corridor overshoot of +3569 / +1134 / +1830 MiB. The answer to "why is this not computed exactly beforehand" is a spread over 14 boots of one config: cuda_context_and_comm has spread EXACTLY 0 (888 / 482 / 482 MiB, every boot), while kv_pool_target moves 1364-2408 MiB and tracks the free-at-last-mark spread almost one for one. The corridor overshoot is not made of hidden consumers; it is made of the pool being sized differently each boot. Also names what the unbacked reservation is, which was this shift's open risk: the KV pool is a CUDA VMM arena (mem_cache/kv_vmm_backing.py, "256 GiB virtual; free until committed") exposed as a torch.cuda.MemPool through CUDAPluggableAllocator and committed incrementally against a watermark, so torch's reserved counts the pool's LOGICAL size while NVML counts committed pages. Their difference is the uncommitted span. That reframes the fill side: the free VRAM is space the arena has not committed, so the lever is the commit watermark, which C22 already localised. Register: C23 (the reserved-counter causal chain, falsified), C24 (the false zero residue), C25 (rank vs pid under PP), C26 (what reserved means here). Not claimed: that raising the watermark fills the corridor. The recorder does not yet mark committed bytes; that is named as the next stage. Tests: test_fill_side_605.py 4 passed. Docs-only otherwise.

…posts it cannot support Stage 4 of the flight recorder: mem_ledger/measured.py reads the recorder's boot history and offers per-card residuals to the ledger, off unless SGLANG_VRAM_LEDGER_MEASURED is set. The rule that makes this worth having is the refusal. Measured over the ship boots of 2026-08-13, the posts of one unchanged config fall into two populations: cuda_context_and_comm at spread EXACTLY 0 (888 / 482 / 482 MiB over 20 boots) and kv_pool_target at spread 1364-2408 MiB. A median exists for both and looks equally like an answer. Installing the second would convert variance into false precision, so precedence is not "prefer any measurement" but "prefer a measurement stable across at least MIN_BOOTS boots"; anything wider is logged as DECLINED with its spread and the modelled term stands. Real-data result, no GPU touched: cuda_context_bytes = 888 / 482 / 482 MiB, derived from the boot history instead of from the GPU probe that calibration.py otherwise spends device seconds on. python -m sglang.srt.mem_ledger.probe --from-flight-recorder DIR reports what the history supports and what it refuses. The import sits inside that branch, so the boot path never reaches this module and is byte-identical with the flag absent. Tests, PYTHONPATH=/spinning/wt-605-fr/python: - test_measured_source_605.py: 8 passed, including off-by-default (inert without the env var, active with it) and too-few-boots. - Can-fail proof: a mutant whose stability verdict ignores the spread fails both TestWidePostIsRefused cases; reverting restores 8 passed. - test/registered/unit/mem_ledger/: 365 passed, 1 failed. The failure is the pre-existing 612 communicator-group declaration test, untouched here. - Validated against the real history at /spinning/flight_605: 20 boots, three cards, spread 0 on every card. - ruff, black, codespell clean.

…ots, and the corridor under load R2. Four things that were argued in R1 are now measured, on a boot of the ship configuration (1464299-1786612548, argv from the s485 capture). COMMIT WATERMARK AS A POST. KvVmmArena keeps a read-only census (arena_census, a WeakValueDictionary and not a WeakSet so observing an arena cannot keep it alive) and mark() records kv_arena_reserved / backed / retained per phase. The chain "commit watermark -> resident -> free -> corridor" is now a measured quantity rather than a plausible story. Measured at first_forward: the 5090 holds 31182 MiB of arena reservation against 22690 MiB committed, i.e. 8492 MiB reserved and unbacked, beside 3463 MiB free on the card. retained is 0 on this boot and is reported separately from backed on purpose -- parked handles are memory NVML charges the process for while backed does not. LEDGER DUMP ON SHIP BOOTS. The modelled ledger was never built on the shipped config: it pins --rank-gpu-memory-mib, which is the pin path, and the pin path skips the planner, so fourteen boots of recorded marks sat beside zero modelled counterparts and reconcile.py had nothing to compare. __post_init__ now builds one for the RECORD after the configuration resolves, gated on the recorder already being armed and wrapped whole. The ledger is discarded: never returned, never consulted for a size. First ledger_<boot>.json on a ship-config boot in this chain was written by this commit. CORRIDOR UNDER LOAD, 100 ms. The boot marks are snapshots and cannot answer a law about a continuous floor. corridor_trace.py samples in-process (bounded ring, off unless SGLANG_CORRIDOR_TRACE_MS is set) and corridor_sampler.py samples the same quantity from OUTSIDE the ranks, which needs no call site in the scheduler. Both reduce on the MINIMUM, because a floor is broken by the worst instant and not the typical one. Measured over 75 s, 750 samples per card, under five real generations: 3080 5c64 free min 1542 MiB margin +518 no breach 5090 31d7 free min 2801 MiB margin +1777 no breach 3080 62db free min 2062 MiB margin +1038 no breach ERRORS FIRST, THE INSTRUMENT'S OWN: the first version of the sampler cost 25250.9 us per card per sample and ran at a 75.7% duty cycle, because the registry's memory_info_for_uuid opens an nvmlInit/nvmlShutdown pair and scans every device by UUID on each call -- thirty init/shutdown cycles per second at this cadence. Handles are now resolved once per run: 28.9 us per card, 0.0867% duty, 0 overruns. The cost is published in every summary rather than assumed, and the two runs agree within ~150 MiB despite the 874x difference, which is what makes the cheap reading trustworthy. Tests, PYTHONPATH=/spinning/wt-605-fr/python: - test_corridor_trace_605.py: 8 passed. Can-fail proof: a mutant reducing on the mean instead of the min fails the single-trough breach case. - test/registered/unit/mem_ledger/: 373 passed, 1 failed -- the pre-existing 612 communicator-group test, which references none of this. - ruff clean on every file touched here. server_args.py carries 357 pre-existing ruff errors, identical count in the untouched ship worktree. - Real generations against the measured boot: 5/5 http 200, 320 tokens each.

…demand is exact R1's headline is withdrawn by measurement. It reported a per-post spread over "14 boots of one unchanged config" and concluded the corridor overshoot was boot-to-boot nondeterminism in KV pool sizing. The 14 boots were >=11 different configs -- the sgl-project#656 shift was editing --phase-flip-tp-vector between them -- so R1 pooled distinct configurations into one distribution and read the variance as randomness. The conclusion was drawn as wide as the question instead of as wide as the sample. Grouped by full post signature the same 14 boots give 11 signatures, one with four members, and those four are byte-identical on every post and every card. A fifth boot taken on the same captured argv but a DIFFERENT code tree reproduced it exactly: resident 28436 / 18002 / 16674 MiB. Given its configuration the boot's demand is exact and repeatable to the byte, and the only quantity that moves at all is the NVML-unattributed driver line. Also settled here: the 702 MiB weights_target spread is between-config, not a layout lottery (three cards rise together by identical +28/+674 increments, which no re-partition can do, since a re-partition conserves the total); the residual's three suspected tenants are excluded by NVML's own per-process list showing exactly one pid per card; and the corridor holds under load with the binding card's worst instant at 1542 MiB, +518 over the law. The number the KV-universe shift asked for: the arena carries 6.8-8.5 GiB of reserved-but-uncommitted address space per card, so the commit watermark is not capacity-limited, and the safely claimable amounts at the under-load worst instant are +1777 / +518 / +1038 MiB. Register: C27 (the retraction), C28 (weights layout falsified), C29 (the pin path is why the ledger never dumped), C30 (residual candidates excluded). Docs and register only. Ship restored via res-r5 and verified with a REAL generation (120 tokens, http 200), not a health 200.

… then externally killed An earlier note would have left a successor believing this shift ended with serving up. It did -- health 200 at 09:31:00Z and a REAL generation of 120 tokens, not a health 200 alone -- and then the instance took an external SIGKILL at 09:32:32Z as the sgl-project#656 shift opened a new window. The instance's own log rules out the kernel OOM killer (oom_kill still 9, MemAvailable 57872 MiB). No attempt was made to boot into their live window: two instances on one set of cards is an OOM, which the capture-replay script refuses by design.

…project#605 VRAM flight recorder) Merge source origin/feat/vram-flightrecorder-605 @ 848ccb6 onto the step-1 tip 70302cf. Conflicts: 0. 18 files, 2566 insertions, 217 deletions. Carries: per-boot attribution with drift 0 (marks grouped by PID, not by TP rank -- under PP the TP rank is 0 in all three processes and rank-grouping billed a 3080's context to the 5090); the commit watermark posted as a recorded quantity via KvVmmArena.arena_census(); continuous under-load mode; and the ship-boot ledger dump -- ServerArgs.__post_init__ now builds a modelled ledger FOR THE RECORD, gated on the recorder already being armed and discarded immediately, because the ship config pins --rank-gpu-memory-mib, which takes the pin path, which skips the planner, so 14 boots of measured marks had sat beside zero modelled counterparts. Flag-gated throughout: with SGLANG_VRAM_FLIGHT_DIR unset the dump returns on its first line. Both source branches of MERGE-R7 touch python/sglang/srt/server_args.py and docs/dev/631/CONTRADICTIONS_REGISTER.md. Both merged without conflict and both sides were verified present in the merged files afterwards rather than assumed: the kv-universe seam/vacate comment blocks and the sgl-project#605 _dump_observation_ledger are all present, and the register is +526 lines / -0 against 481411a, which is exactly 518 (kv-universe) + 8 (flight recorder). TEST RESULTS -- same CPU-only desk protocol as step 1 (CUDA_VISIBLE_DEVICES=99, PYTHONPATH=<worktree>/python, pytest --color=no, one directory per invocation). | suite | BASE 481411a | after step 1 | after step 2 | |----------------------|-----------------|--------------|--------------| | sgl-project#631 flip family | 1116 passed | 1116 passed | 1116 passed | | unit/managers | 9F 1357P 18S | 9F 1357P 18S | 9F 1357P 18S | | unit/mem_ledger | 1F 359P | 1F 359P | 1F 379P | | unit/model_executor | 15F 594P | 15F 594P | 15F 594P | | unit/server_args | 615P | 615P | 615P | | unit/turnkey | 116P | 116P | 116P | | unit/utils | 46F 348P 4S | 46F 348P 4S | 46F 348P 4S | | unit/docker | 4P | 4P | 4P | | new-656 arm | (files absent) | 4F 56P | 4F 56P | Every failure count identical to baseline at every step; the one mem_ledger failure is the same inherited test_communicator_group_contract_612.py case before and after. +20 mem_ledger passes are this step's three new sgl-project#605 test files. Total new tests across both merges: +80 collected, 0 new failures. Lint, base vs merged over the 37 touched .py files: ruff 473 -> 472 (one error removed in phase_policy.py, none added; all 15 new modules ruff-clean). black 26.1.0, the version pinned in .pre-commit-config.yaml: 9 files dirty before, 9 after, but the SETS DIFFER -- this branch's reformat of server_args.py is not churn, it FIXES pre-existing drift (server_args.py, model_runner.py and kv_vmm_backing.py were black-dirty at 481411a and are clean now), while three newly added test files land dirty. Six files are dirty on both sides. Recorded, not fixed: a merge shift does not reformat, and the standing defect is that the pre-commit hook is not being run on this line. codespell 9 -> 11; the two new hits are both the word "unparseable" (a valid variant) and the pre-existing "schedul" hit at CONTRADICTIONS_REGISTER.md is the deliberate 15-character TASK_COMM_LEN truncation from MERGE-R5 and must not be "fixed".

…el collision Both R7 source branches extend docs/dev/631/CONTRADICTIONS_REGISTER.md and the union merged cleanly (+526 lines, 0 deletions), but the merged file carried EIGHT ambiguous identifiers, which is a silent defect in the one document whose purpose is that a claim can be looked up. feat/vram-flightrecorder-605 appended eight table rows labelled C23-C30. Every one of those labels was already in use at 481411a for an unrelated contradiction: C23-C26 the four '### C<n>' sections further down the file (the SGLANG_UNEVEN_TOKEN_VECTOR reading, the sgl-project#695 recipe's rank discovery, the sgl-project#695 exact-size pin, sgl-project#644's allocator residual) C27-C30 narrative entries the body itself cites 4, 9, 3 and 6 times The briefing for this shift anticipated a four-label collision (C23-C26). It is eight: C27-C30 are load-bearing too, which is only visible by grepping the body rather than the headings, since the incumbents are prose entries and the arrivals are table rows -- the two forms do not textually conflict, so git merged them without a murmur. Resolved by renaming the ARRIVING side, which has no inbound references in this file, to a namespace that cannot collide: C23->C605-1 ... C30->C605-8, in arrival order. Verified programmatically that all eight row bodies are byte-identical apart from the label; nothing was dropped, merged or reworded, and no incumbent entry was touched. A note above the table records the mapping, and both sgl-project#605 handoffs, which name their own labels in prose, carry a pointer to the shipped ones. Docs only: no code path, no test, no count changes.

…n eight-label register collision Records what merged (feat/kv-universe-656 @ 4157aad, then feat/vram-flightrecorder-605 @ 848ccb6, both --no-ff, both conflict-free), the suite counts at every step against the frozen baseline /spinning/wt-merge-r7-base @ 481411a, the register union verification (+542 lines, 0 deletions), and the follow-ups carried from both branch handoffs. Errors first, and the three worth reading before quoting anything else: 1. The C-label collision the briefing sized at four labels is EIGHT. The sgl-project#605 rows arrive as TABLE ROWS while the incumbent C23-C30 are '###' sections and prose entries, so git merged all eight silently and the heading grep -- the check that was legitimate in R6, when only one branch touched the file -- reports the file clean. C28 alone is cited nine times in the body. Resolved by renaming the arriving side to C605-1..C605-8; bodies verified byte-identical apart from the label. 2. Four cases in test/srt/test_phase_flip_serving_proof_gate.py -- the file pinning the quarantine-constant removal, i.e. step 1's central claim -- cannot pass under the CPU-only merge protocol at all: they need a visible accelerator. Proven inherited by running the identical arm against the untouched source worktree. So that gate did NOT run in this merge, and the branch's own "239 passed" was measured with a device visible. 3. The sgl-project#605 branch's server_args.py reformat is NOT churn. It is a black 26.1.0 pass that fixes pre-existing drift: server_args.py, model_runner.py and kv_vmm_backing.py were formatter-dirty at 481411a and are clean now. Reverting those hunks to keep the diff tight would have re-broken the file. Six files remain dirty on both sides -- the pinned pre-commit hook is not running on this line. Also records that the stale local feat/route-a-631 ref is now THREE rounds behind and still checked out in another strand's worktree, so R8 must keep using origin/* as merge sources and HEAD:refs/heads/* for pushes. Docs only.

…ng the ledger TASK 1 (weights on the pin path) and TASK 4 (residual recalibration). COMPLETENESS CHECK, red-first. reconcile.completeness_failures() names any card whose model-weights term is priced at zero or absent. Run against the real shipped dump ledger_1464299-1786612548.json it fails on all three cards, which is the red the brief asked for: model weights (shards) on GPU 0/1/2: priced at 0 MiB. This is the PIN PATH signature -- pinning --rank-gpu-memory-mib skips the planner that computes the shard vector, so the term is built and dumped as a zero rather than refused. A REFUSAL PASSES THE CHECK. Only a silent zero fails. That distinction is the whole point: a zero is indistinguishable from a price. WEIGHTS ON THE PIN PATH. _build_card_ledgers now accepts weight_mib_per_rank and history; both default to None and None reproduces the old behaviour exactly, so the sgl-project#593 sizing paths are byte-identical. Only the observation ledger passes them -- a sizing path must not change what a boot reserves as a side effect of an instrument being armed. PIPELINE PARALLELISM RETURNS None, NOT A NUMBER. The ship configuration is pp_size=3 / tp_size=1 with --pp-layer-ratio 28,20,16, so the split is a LAYER split, and per-layer bytes are not uniform on a hybrid Qwen3.5/3.6 checkpoint (interleaved GDN and full attention). Solving the measured episode-1 posts 13674/8325/9293 MiB for a uniform per-layer cost gives 488/416/581 MiB per layer -- the model does not close, so it is not shipped. None reaches the ledger as no term at all, which the completeness check then names. HISTORY INTO THE LEDGER. build_card_ledgers(history=...) gives measured-over- inherited precedence for the two terms the boot history can speak to, and refuses rather than averages a wide one: hardware residual band charged at its HIGH, superseding the window- 2026-08-06 probe (664 -> 902 MiB on the 5090) load transient REFUSED and UNBOUNDED; the inherited 70 MiB is explicitly NOT used as a fallback A refused residual band keeps the probe calibration: a wide history is a reason to distrust a summary of the history, not to discard a direct measurement. Tests: 15 new across test_history_into_ledger_605.py (8) and test_pin_path_weights_605.py (7), all red first. The byte-identical-default test is the one protecting every existing caller. Suite: 531 passed, 1 failed (inherited test_communicator_group_contract_612, also failing at base cd71ec3). black 26.1.0 clean; ruff 0 on every touched file, same as base.

…a caller TASK 3 (missing boundaries) and TASK 6 (corridor_trace production call site). NCCL BOUNDARY. TERM_NCCL_BUFFERS has been in the ledger taxonomy since sgl-project#595 with nowhere to be measured, so it read UNMEASURED on every boot ever recorded. BOOT_PHASES gains nccl_init_begin/nccl_init_end and model_runner marks them around the whole group-building block. The end mark sits AFTER the phase-flip secondary groups, not after initialize_model_parallel: those groups are communicators too, and a boundary drawn before them would price a launch's NCCL buffers at whatever fraction the primary topology happens to build. GDN SCRATCH GETS A NAMED BOUNDARY BUT NO CALL SITE, deliberately. The GDN prefill scratch is allocated inside a forward pass, not at boot, so there is no honest boot-phase home for it; inventing one would put a mark on the hot path to satisfy a table. TERM_TO_POST names gdn_scratch_begin/end so the reconciliation reports the term as UNMEASURED-with-a-named-missing-boundary -- a to-do with an address rather than a permanent blank. CORRIDOR CALL SITE. R2 open item 1: the sampler was tested, ready and callerless -- a module the tree carried and never ran. Scheduler gains _corridor_trace_tick(), called beside _census_tick(). It arms ONCE (a sampler per iteration would be one NVML thread per scheduler round), never raises, and never retries a failed arming. Off by default in two independent ways: the state lives in class-level attributes so an un-armed scheduler carries no per-instance attribute at all, and corridor_trace.start() returns None unless SGLANG_CORRIDOR_TRACE_MS is set. Tests: 9 new (test_corridor_trace_call_site_605.py), red before the change with AttributeError on Scheduler._corridor_trace_tick. The tick is exercised by binding the REAL Scheduler method to a stub, not by mocking the method under test, so a rename on the production side fails this file. Three tests pin the off-by-default property and two pin never-raise/never-retry. The existing phase-contract tests (every declared phase has a call site; no call site invents a phase) both accept the new pair. Suite: 424 passed, 1 failed (inherited test_communicator_group_contract_612). black 26.1.0 clean.

…re the model was TASK 5. Re-ran reconcile.py against the same boot 1464299 data with the corrected mapping, and the headline is a retraction of the first run's own table rather than a verdict on the model. Two of the first run's five findings were instrument defects, not model defects: weights 27800 MiB measured on a card holding 26364 MiB in total -- the row summed two loads across a free. Correct: 13850 (largest of three episodes 13674/13850/2074). transient 0 MiB measured at the one mark where the quantity is structurally zero. Real peak 13392 MiB. And the totals row subtracted a BUDGETED KV pool (29927 MiB) from a MEASURED footprint whose arena actually backed 21130, driving measured demand to -3045 MiB. Now read from the arena census; demand reads 5752/4113/3961 MiB. ACCEPTANCE. Every row is now matched, matched-within-a-recorded-band, UNMEASURED-with-the-boundary-named, or REFUSED-with-the-refusal-quoted. The refused activation and graph-capture terms were absent from the first table entirely and are now rows. THE WEIGHTS ROW DOES NOT CLOSE, AND IS NOT FAKED. pp_size=3 with --pp-layer-ratio 28,20,16 makes the split a LAYER split; per-layer bytes are not uniform on a hybrid checkpoint (solving the measured posts for a uniform cost gives 488/416/581 MiB per layer, a 39% spread). The estimator is named as open; what ships is the completeness check that fails loudly on the pin path's silent zero. Docs: docs/dev/631/HANDOFF_LEDGER_RECONCILE.md, docs/dev/631/RECONCILE_SECOND_RUN.md Register: C605-9..C605-12 appended (summed-weights measurement, the 70 MiB transient in both value and mapping, the residual recalibration, the budgeted-KV-pool subtraction). Evidence: /spinning/evidence-631/ledger-r1/ (rerun_reconcile.py, reconcile_second_run.txt, boot_history_bands.txt, RECONCILE_SECOND_RUN.md). Suite: 424 passed, 1 failed vs base 379 passed, 1 failed (same inherited test_communicator_group_contract_612). black clean, codespell clean.

…econciled nothing Follow-up on five live boots in /spinning/evidence-631/acceptance-656/flight/. All four verified fixed against boot 1917721-1786622304. Two this branch already covered; two it did not. 1. THE TOOL HAD RECONCILED NOTHING FOR AN ENTIRE RELEASE. read_marks was correctly changed to PID keying (the ship config runs --tp-size 1 --pp-size 3, so all three processes file under TP rank 0) and attribute_flight._reconcile() was never updated: it handed the pid dict into marks_by_rank.get(int(rank)). Pids 1918126..8 cannot equal ranks 0..2, so EVERY card matched nothing and the tool printed "The ledger names no card whose rank left marks" and exited 1 -- read by two shifts as a fact about the boot. THE SILENT SKIP WAS A PINNED CONTRACT and is the accomplice: a test asserted an unmatched rank is skipped rather than invented, which made a total keying mismatch look like a quiet edge case. reconcile() now re-keys via marks_by_rank_from_pids() -- card uuid, else the caller's key, else the MAXIMUM rank the marks carry (process-level phases all report 0, written before the runner knows its pipeline rank; verified on two boots) -- cross- checks nvml_total_bytes against the card, and REFUSES on any card left unmatched. CardVramLedger.to_json now emits the uuid so the join is identity rather than inference. 2. MEASURED DEMAND WENT NEGATIVE: -2023 MiB on the 5090, -180 on a 3080. My fix already subtracted the measured pool but FELL BACK to the modelled budget when the arena census was absent -- the same defect wearing a condition. Fallback removed; absent an arena census the demand refuses loudly. The obvious second source is rejected by measurement: the target runner's weights_loaded -> kv_pool_sized growth contains the KV pool AND the state pool (7720 vs 6916 MiB on boot 1464299). 3. _field_bytes TOOK THE FIRST MARK = always the target runner. The peak fix covered only the transient. Live boot 1917721 reads the hardware residual at 886 MiB on the target and 896 on the draft. A field term is a LEVEL, so it is now the MAXIMUM across runner partitions; summing would be wrong and process-level phases are unchanged. Falsifier (target 0 / draft nonzero) pinned. 4. not_applicable IS NOT UNMEASURED. NCCL buffers are correctly 0 under barlink; reporting that as UNMEASURED invites a successor to hunt for the boundary this branch just added, measure 0, and conclude the recorder is broken. Renders "N/A -- not applicable to this launch (barlink ...)". LIVE ACCEPTANCE, boot 1917721-1786622304: demand -2023 / 158 / -180 -> 5740 / 4075 / 3919 MiB residual 886 / 480 / 480 -> 896 / 496 / 496 MiB (draft runner) NCCL UNMEASURED -> N/A (barlink) CLI exit 1, zero cards -> all three cards rendered Output: /spinning/evidence-631/ledger-r1/reconcile_live_boot_1917721.txt Tests: 13 new (test_reconcile_caller_keying_605.py), fixtures transcribed from the live boot, all red first. Two legacy tests pinned falsified premises and are updated in place with the falsification in their docstrings. Suite: 437 passed / 1 inherited failed (base 379/1); with turnkey 553/1. Register: C605-13..C605-17. black 26.1.0 clean, ruff 0 on every touched file.

…ject#605 reconcile-loop closure) Reconcile-loop closure for the sgl-project#605 VRAM flight recorder, merged at 20e2e37 onto cd71ec3 (the MERGE-R7 tip both lines carried). Carried: pin-path weight completeness check (the pin path skips the planner, so the ledger's largest post shipped as 0 MiB on every pinned boot; None now reaches the ledger as no term at all and reconcile names it loudly, where a zero would have passed every check by looking like a price); the load transient boundary and its refused constant; the NCCL phase boundary (nccl_init_begin/end around the whole group-building block, end mark placed after the phase-flip secondary groups because those are communicators too) with N/A(barlink) rendering; hardware-residual recalibration from 472 boots of recorded history (refuse-wide rule, charge the band HIGH never the mean); the corridor_trace.py production call site in Scheduler._corridor_trace_tick, flag-gated default OFF in two independent ways; the caller-keying fix in attribute_flight.py (a PID-keyed dict was passed into a rank-keyed lookup, so every card matched nothing and the tool reconciled ZERO cards for an entire release while printing that as a fact about the boot); measured-pool subtraction with loud refusal instead of a silent fallback to the modelled budget; _field_bytes max-across-runners; card uuid in the ledger dump. Register: appends C605-9..C605-17, no collision with the incumbent C605-1..C605-8 that MERGE-R7 created (9 lines added, 0 deleted). Tests documented in docs/dev/631/HANDOFF_MERGE_R8.md.

…mmunicators the ledger was missing MERGE-R9 12.6, the one inherited mem_ledger red, and load-bearing rather than cosmetic: initialize_phase_flip_secondary_groups builds flip_tp, flip_dcp and flip_pp over the same world (parallel_state.py:3502-3512), and sgl-project#605 put the nccl_init_end mark AFTER that call on purpose (model_runner.py:1985) -- so the first boot to measure TERM_NCCL_BUFFERS measures those three communicators while the term's signature does not know they exist. Undeclared, they hit the OOM direction twice: their buffers were charged to no ledger row, and they left nccl_signature unmoved, so a figure measured on a non-flip launch was silently reusable on a flip one. Both directions of the contract now hold. The over-declaration arm was already green and stays green: flip_pp is emitted with world_size=1 (the call site passes pp_size=1) and classify_communicator_groups -- not this inventory -- is what decides a single-rank group allocates nothing. Deciding it at the inventory would hide the group the moment the call site passes a different pp_size. flip_dcp is emitted on the CONSTRUCTION SITE's condition (dcp_size > 1 of the SECONDARY set), not on "dcp is enabled": the secondary set's DCP width is len(--phase-flip-tp-vector), and the primary --dcp-size has nothing to do with it. The width is read from the vector the call site itself reads, and server_args pins len(vector) == pp_size (server_args.py:7708), so the inventory cannot drift from the launch. A malformed vector falls back to pp_size instead of raising -- server_args is the judge of argv and the ledger runs before it; a bad flag must not arrive as a memory-ledger stack trace. TEST RESULTS (venv /spinning/htsglang-gpu/.venv, CUDA_VISIBLE_DEVICES=99, PYTHONPATH=<worktree>/python, pytest --color=no): base 99a4b0a test/registered/unit/mem_ledger/ 1 failed, 437 passed FAILED test_communicator_group_contract_612.py:: TestTheDeclarationNamesEveryGroupTheRuntimeBuilds:: test_no_runtime_group_is_missing_from_the_declaration -> ['flip_dcp', 'flip_pp', 'flip_tp'] != [] here test/registered/unit/mem_ledger/ 444 passed, 0 failed Name-diffed: the base red is gone and the delta is exactly the six new pins in TestThePhaseFlipSecondaryGroupsAreDeclared. No test changed verdict in the other direction.

… unattributed WHAT THE GAP COST, MEASURED. On 2026-08-16 at 02:36:30 an instance died with GPU 0 ... 76.38 MiB is free ... Process 1920108 has 4.29 GiB memory in use Naming that process took hours: log archaeology, then a pid-clock interpolation across two boots' `boot_id` fields (pid 1851351 -> 02:10:44, pid 1924983 -> 02:38:10, 44.7 pids/s, so pid 1920108 lands at 02:36:22). It turned out to be a test harness on the serving card. Every fact needed to answer that in ONE LINE was already computed. `_nvml_view` returns `nvml_processes`, the full pid->bytes map of everyone on the card, on every mark. The recorder simply stops marking: its last boot post is `first_forward`, and the failure was 36 minutes later. NOT A DUPLICATE OF THE sgl-project#605 CORRIDOR SAMPLER, and the difference is durability. That sampler does run during serving, at 100 ms, and does call `_nvml_view`: * it keeps a fixed-size RAM ring, so it dies with the process that crashes -- which is the one process whose state the post-mortem needs; * its `Sample` retains free/self/reserved/allocated and DISCARDS the per-pid map it just read, so it cannot name a foreign holder even while running; * it was not armed on the boot that died. Marks are appended to a FILE and survive the crash. That is not a theoretical advantage: the surviving boot marks are exactly what made the pid clock calibratable after the process was gone. A SEPARATE FILE, DELIBERATELY. The boot ledger's readers pair marks BY POST NAME (`reconcile` asks for the `weights_loaded -> kv_pool_sized` delta, and for `kv_arena_backed_bytes` at `boot_complete`). A boot post is a unique boundary; a serving sample is a time series, and thousands of the latter in that file would turn a table of posts into a log with posts in it. So the series goes to `flight_serving_rank{n}.jsonl` and every existing consumer of `flight_marks_rank{n}.jsonl` is untouched -- 467 mem_ledger tests confirm it. ONE RECORD BUILDER, TWO DESTINATIONS. `mark` grew an internal `_filename` rather than gaining a second copy of the record layout. A duplicated schema is precisely how the field a post-mortem needs ends up present in one file and missing in the other -- which is the shape of the defect this commit exists to close. PACED, ON THE MONOTONIC CLOCK. The call site runs once per scheduler iteration, thousands of times a second; the pacer (default 30 s, `0` disables) is what makes that affordable, and the cost when the recorder is unarmed is one dict lookup. Wall time is unusable here: an NTP step backwards would stall the series and a step forwards would flood it, and the boot that most needs the record is the long-lived one whose clock is being corrected. WHERE IT IS CALLED, AND WHY THERE. Beside `_corridor_trace_tick` in `get_next_batch_to_run`, on that line's existing argument -- every rank reaches it exactly once per round -- so the cadence is replicated and the per-rank files line up round for round, which is what makes them comparable across ranks. Unlike its neighbours it needs no collective and takes no branch: it is write-only and cannot make two ranks disagree about anything. It is NOT on a batch-conditional path, because an idle rank losing its card to a foreign process is exactly the 2026-08-16 case. THE CALL-SITE TEST EARNED ITS PLACE IMMEDIATELY. The first version of the tick referenced `flight_recorder` as if it were a module-level import; it is imported inside `run_scheduler_process`. That is a `NameError` on every iteration, and the bare `except Exception` around it turned the whole instrument into one that silently never runs -- the exact failure mode this task exists to prevent, in the code meant to prevent it. `test_the_tick_marks_with_this_rank` failed and named it. The import now sits OUTSIDE the guard so it fails loudly, only the call is guarded, and the guard logs once at WARNING rather than staying silent. The recorder's own `BOOT_PHASES` comment states the lesson this file follows: "twelve green fixture tests passed while the production carrier lacked the field they all built by hand." TESTS. `test_flight_serving_marks_684`, 17 cases, hermetic (CPU only, no CUDA): - RED FIRST: all 13 API cases failed with `no attribute` before the change. - The acceptance property reproduces 02:36:30 exactly -- this process at 26.65 GiB and a foreign pid at 4.29 GiB on one card with 80 MiB free -- and asserts the foreign holder is named IN THE RECORD ON DISK. - Pacing, monotonic-clock behaviour, `0` disables, a malformed interval falls back rather than silencing the series, an unarmed process writes nothing. - The boot ledger keeps one mark per post while the series accumulates. - An NVML failure still leaves a timestamped record; a write failure and a raising probe never reach the serving loop. - Call site: taken off the REAL Scheduler class, so a rename fails this file; pins `self.ps.tp_rank` (the Scheduler has no `tp_rank` -- a predecessor's assumption that it did raised on every rank), and pins the position beside the corridor tick. Suites: 467 mem_ledger, plus 557 across mem_ledger + the 681/682 guard files.

…he fix was on disk THE LEDGER COULD NOT ARBITRATE, TWICE IN ONE NIGHT. It reported `model weights (shards) = 0 MiB` on all three cards while they held 16196 / 10194 / 10832 MiB, marked three posts UNBOUNDED and said `fits=False` everywhere. Neither the sgl-project#683 attribution ("who took the card") nor the sgl-project#678 sizing question could be settled from it; both were answered by hand. (a) WHY THE ZERO, AND WHERE THE NUMBER WAS. Not an unwired term, not a fingerprint that never matches: `reconcile.completeness_failures` already names it -- the shipped config pins `--rank-gpu-memory-mib`, the PIN PATH, and the pin path skips the planner that computes the shard vector. The term is built from an all-zero vector and dumped as a price, indistinguishable in the JSON from a model that needs no weights. The flight recorder was already measuring it. The reserved-bytes delta between `pre_weight_load` and `weights_loaded` IS the shard footprint, per card, every boot. This reads that. Provenance gets a fifth kind, `MEASURED` -- not MODELED (no formula of ours produces it) and not CALIBRATED (not a cached probe under a fingerprint; re-measured every boot). That is the kind the standing directive asks for: demand from measurement, not estimate. DETECTION STAYS WHERE IT ALREADY WORKS. The engine does NOT add a second refusal for the zero case. `completeness_failures` calls it "the PIN PATH signature" and `require_complete` raises on it; a refusal inside the engine would change the verdict of every card whose shard vector is legitimately absent at that point in the boot, including the pre-planner ledger whose residual is the RANK BUDGET rather than the KV pool. I built that refusal first, watched it break five unrelated suites, and took it back out -- the detector is the acceptance, pinned in both directions: it stops firing once the post is measured and keeps firing on a fingerprint with no marks to stand in. (b) THE FITS SPLIT, A PRECISION AND NOT A WEAKENING. `fits` was `not unbounded and committed <= total`, so ONE inherently-unbounded transient made every card unfittable forever. The arithmetic side already passed (2848 against 32607); `fits=False` was caused purely by a non-empty `unbounded`. The load transient refuses on EVIDENCE -- 563 boots, a 0-18486 MiB spread, above the 50% refusal rule -- and that refusal is correct and permanent, so the flag was conflating "a post could not be priced" with "the card is overcommitted". A transient is a peak ABOVE the resident set, not a claim on the card. It now travels in `unbounded_transient`, rendered beside the verdict WITH its evidence, and does not block the fit. Resident refusals still do: pinned in both directions, including a synthetic unbounded resident post forcing fits=False. Nothing is estimated in either bucket. (d) THE ACTIVATION POST IS AN UPPER BOUND, LABELLED AS ONE. The serving marks sample `allocated_peak_bytes` under real load -- the counter `probe_activation.py` reads -- but never reset it, and a reset at mark time would change what every other reader of those fields sees. So what this publishes is a monotone envelope since process start, above the resting set, over-stated by an unknown amount and said so in the term's own derivation. Over-reserving is the safe direction; the failure this ledger exists to end is the under-charge. A per-phase point estimate needs the reset and is future work. THE INSTRUMENT BUILT TWO TASKS AGO IS WHAT MADE (a) AND (d) POSSIBLE, and it is live: 519 serving samples spanning 03:56:48 to 05:27:02 on the running boot, plus the boot-scoped marks. No new measurement path was added -- these are differences between marks the recorder was already taking. (c) STAYS GATED, AND THE INVESTIGATION RESOLVED -- see the report. Measured across the same two capture marks: reserved delta 184/182/182 MiB against an NVML free drop of 282/324/324, so the reserved-bytes delta is structurally blind to 98-142 MiB per card (the private-pool component an OOM here named directly). Feeding `capture_mib` would under-charge. The probe does not need replacing, the FIELD does -- the honest quantity is the NVML free delta these same marks already record -- but the refusal also carries an inherited "3.3-3.8x low" that does not reproduce on this config (~192 estimate against 282-324 measured, i.e. 1.5-1.7x), so that has to be re-derived rather than carried. The finding is recorded in the code; nothing is fed. TESTS. 13 new hermetic cases on the reader, red-first (all 13 failed on the missing API), including: the first load is the shard post and not the drafter or the second layout, cards kept apart by uuid, an unpaired load yielding nothing rather than a guess, no marks giving an empty answer rather than a zero, and the serving envelope labelled upper-bound-only. Plus the completeness detector pinned in both directions and the transient split pinned in both directions. Suites: 482 mem_ledger, 2415 across mem_ledger + unit/managers. The 4 failures in that sweep are pre-existing and unrelated, parity-proven earlier against the untouched tree.

… allocator's THE ALLOCATOR'S BOOKS ARE STRUCTURALLY BLIND TO PART OF A GRAPH CAPTURE. Measured on this rig 2026-08-16, across the same capture_begin -> capture_end marks: card reserved delta NVML free drop blind to 5090 184 MiB 282 MiB 98 MiB 3080 182 MiB 324 MiB 142 MiB 3080 182 MiB 324 MiB 142 MiB The gap is the private-pool component CUDA graphs take, which an OOM on this rig named outright ("71.21 MiB allocated in private pools"). So the post is fed from the NVML free delta, MEASURED provenance, re-measured every boot and never cached across regimes -- the same rule the weights term follows. BOTH NUMBERS ARE KEPT. `capture_reserved_mib` sits beside `capture_mib` in the record and in the term's own derivation, so the blindness stays visible to the next reader instead of decaying into folk knowledge about "why we use NVML here". THE STALE FACTOR IS WITHDRAWN, NOT SILENTLY REPLACED. The refusal carried "3.3-3.8x low" from the 2026-08-05 window. It does not reproduce on this configuration: the captured-tokens estimate reads ~192 MiB here against a measured 282-324, i.e. 1.5-1.7x. The ESTIMATE stays rejected -- an under-charge is the direction that OOMs -- but the factor is re-derived from this boot's own marks rather than carried, and the refusal text now says so. That is the same disposal the 550k pin got in sgl-project#678, for the same reason: a number that outlives the regime it was measured in is not evidence. CONTAMINATION IS A PRECISION CAVEAT, NOT A SAFETY ONE, and it is recorded as such. NVML free is card-wide, so a foreign process allocating during the capture window inflates the reading. That is an OVER-charge: it costs KV pool, never a boot -- the same direction that justifies the activation upper bound. A STRUCTURAL FLOOR, NOT A THRESHOLD. The driver cannot release less than the allocator took, so an NVML delta BELOW the reserved delta on the same window is physically impossible. That is an instrument fault, not a small capture: `capture_fault_mib` appears instead of `capture_mib` and the term posts UNBOUNDED loudly, quoting both readings, rather than feeding a number already known to be wrong. A boot whose marks lack the pair at all posts UNBOUNDED too. WHAT THE LEDGER NOW SAYS, from the running boot's records on disk (hermetic, no card touched): card demand before -> after KV pool before -> after fits 5090 1824 -> 19188 29759 -> 12395 False -> True 3080 1325 -> 12561 18131 -> 6895 False -> True 3080 1325 -> 13158 18131 -> 6298 False -> True weights 16196 / 10194 / 10832 MiB (was 0 on all three) capture 262 / 324 / 324 MiB (allocator-only: 164 / 182 / 182) activation <=906 / <=718 / <=677 MiB (upper bound, labelled) Every card prices its resident side fully and fits with honest numbers. The load transient still refuses -- correctly, on 563 boots of evidence -- and travels as a risk band that does not block the verdict. TESTS. Four new hermetic cases on the capture reader: the fed number is the driver's while the allocator's is kept beside it, an impossible reading becomes a fault rather than a price, a boot without the pair prices nothing, and the no-op second capture round of a flip boot is skipped rather than read as a zero-cost capture. 485 mem_ledger, 2418 across mem_ledger + unit/managers; the 4 failures there are pre-existing and unrelated, parity-proven against the untouched tree.

…ribution, rank-2 root 1. RESTORE, NEVER REBUILD -- the pin (test_restore_never_rebuild_677.py). A fence patches the BUILD entry points (weights_arena.allocate_arena, weights_arena.pack_into_arena, torch.cuda.CUDAGraph) to raise, and the REAL production mover PhaseFlipStacks.refill runs under it on both legs, including the checksum-mismatch restore arm -- the one branch that touches the arena twice and is likeliest to reach for a rebuild. arena_refill is deliberately NOT fenced: it is the copy the flip exists to perform, and a pin that fenced it would pass by forbidding the work. Can-fail arms: every entry point is shown to actually raise, and a planted mover calling allocate_arena is caught. Pinning production rather than a re-implemented loop is the point -- a re-implementation keeps passing while production drifts (sgl-project#624). 2. PEAK-INSTANT ATTRIBUTION -- PhaseFlipRuntime._record_seam_peak, emitted on the sgl-project#605 flight-recorder channel (not a new one: it already carries the torch view, the NVML view and the boot id, and is append-only). Placed at _staging_affordable, the instant the flip's demand is weighed against free VRAM -- earlier the buffers do not exist, later the decision is already taken. Carries staging_bytes, refill_destination_bytes, graph_workspace_bytes, the reserve, driver free, allocator cached free, and a SIGNED unattributed_bytes. Two choices, both pinned and both mutation-proven: unmeasured components are None and never 0 (a zero reads as "costs nothing" -- the sgl-project#606 defaulted- measurement defect), and the residual is signed, because a negative one means the named terms OVER-count, a different defect that max(0, ...) would hide. Guarded end to end: an instrument on the seam path may cost a line, never a cutover. 3. THE RANK-2 ANOMALY -- the sgl-project#685 candidate is REFUTED. sgl-project#685 (0e50e48, f1774d7) is an UnboundLocalError use-before-bind on a variable named 'cell' in the cold seam-pricing branch; it has no arena-tail content and no 1456 MiB figure exists anywhere in source or records. The live candidate is arena GROWTH, from the sgl-project#690 image sizes: rank 0 PP 12619.6 TP 9614.9 growth 0.0 seam draw 909 rank 1 PP 9014.0 TP 9614.9 growth 600.9 seam draw 1006 rank 2 PP 7211.2 TP 9614.9 growth 2403.7 seam draw 1648 Rank 0's PP layout is the larger, so it never grows; rank 2 grows four times what rank 1 does. The ordering matches the draws exactly and no other per-rank term does. Stated as a CANDIDATE, not a finding: the excess over rank 0 is 16% of the growth on rank 1 and 31% on rank 2, so growth explains the order and not the size. Instrument 2 settles it without a dedicated experiment -- it emits exactly this quantity at the peak instant. FILED, NOT BUILT: TICKET_718_contiguous_destination_extent.md -- making a peer's destination extent contiguous so a BAR1 write lands in place and the receive buffer stops existing. The largest structural reduction available, and the only one that removes a buffer rather than relocating it; it touches the reshard wire format and the pool allocator together, so it belongs to a deliberate design pass. The ticket names what must be decided, what must not be assumed (it trades allocator freedom for staging bytes, and that freedom is what keeps admission working under fragmentation), and makes a live reading from instrument 2 its prerequisite. Tests, hermetic (CUDA_VISIBLE_DEVICES="", no GPU, no serving contact): test_restore_never_rebuild_677.py 7 passed + 3 subtests test_seam_peak_attribution_677.py 8 passed MUTATION PROOF: null->0 and signed->floored fails 3 of 8 FULL SUITE compared by failing NODE ID against base b786858, ANSI stripped (counts alone hid a real regression earlier in this branch): base 13 failed / 2181 passed; now 13 failed / 2208 passed regressions: NONE. The 13 are pre-existing. ruff + codespell clean on all changed files. Desk only. Live readings ride the later review boot.

…ure what warm-up costs TWO PIECES THE DESIGN OWES, both consequences of carrying no KV. 1. VALIDATE-EARLY REFUSAL (sgl-project#806 pattern, ServerArgs.__post_init__). The flip retracts every resident request and DROPS the prefix tree, so the next phase starts with an empty device tier and restores prefixes by read-through. With --enable-hierarchical-cache off there is nothing to read through: the sgl-project#703 flip-time writeback has nowhere to persist, the retracted prefixes are gone, and every conversation re-prefills from scratch on every flip. That is a correctness-shaped cost, not a tuning one, and it is INVISIBLE at runtime -- the flip completes, the requests complete, and only the token bill says anything happened. A silent, expensive, correct-looking failure is what a launch gate is for. Placed beside sgl-project#806's own check and after `materialize_declarations`, for the reason sgl-project#806 documents: the hierarchical cache is switched by handlers above, so an earlier check would pass a launch this one refuses. sgl-project#806 refuses the flip that cannot ENUMERATE what it must move; this refuses the flip that cannot RESTORE what it deliberately drops. NO FALLBACK IS OFFERED AND THE MESSAGE SAYS SO -- a test asserts the phrase. Reviving the mover here would reintroduce the seam this ticket retired, the staging reserve behind W25's 33 refused arms, and the resident carry that crashed three ranks in sgl-project#825. Both exits are named instead, because which one is right depends on what the operator wanted. 2. THE WARM-UP LEDGER (managers/warmup_latency.py), the metric the user named: not "rows carried" but served-request latency by rounds since cutover. PRIOR ART CHECKED AND IT IS A NEAR MISS, not a hit: `regime_classifier.PhaseDwellGate.rounds_since_flip` has the rounds-since-flip concept but is a GATE deciding whether a flip may happen, and carries no latency. Also searched: phase_flip_runtime for warm/latency/post-cutover, the seam census (times the seam, not what follows it), the sgl-project#605 flight recorder (the seam's own peaks). Nothing measures this. Geometric bands (<=1, <=4, <=16, <=64, steady) because the CLAIM is that the cost concentrates in the first rounds and decays; a mean cannot be right or wrong in a way that shows that. Every band reports as a ratio against THIS instance's own steady state, so the figure needs no remembered number from another boot to be readable. The can-fail directions are the file's point. `None` survives wherever nothing was compared -- a ratio against an absent control reads as "no warm-up cost" while meaning "nothing was compared", the sgl-project#606 shape in the one number this ticket is judged on. "Has not flipped" never folds into "has flipped and settled". An empty ledger still says it has nothing to say. HALF-WIRED, AND SAID PLAINLY. `note_cutover` is wired at the cutover. The request feed is NOT: request latency is assembled in `tokenizer_manager`, a DIFFERENT PROCESS, so feeding it is a cross-process change rather than a line. Recorded as an open integration point instead of guessed at -- a wrongly-wired instrument reports a number, which is worse than reporting none, and this is the number the proof window judges the design on. TESTS (hermetic, CUDA_VISIBLE_DEVICES=""): test_phase_flip_needs_hicache_856.py in server_args/ -- 776 passed overall test_warmup_latency_856.py 13 passed, 5 subtests managers -k "phase_flip or seam or warmup or uncached" 564 passed, 28 subtests, 0 failed ruff format clean; ruff check adds ZERO errors -- verified per file from the repo root against HEAD: server_args 358 -> 358, phase_flip_runtime 0, warmup_latency 0, corridor_admission 0. RUFF MEASUREMENT CORRECTION, since a previous commit reasoned from it: ruff must be run FROM THE REPO ROOT or it does not find pyproject.toml and reports a different rule set entirely (phase_flip_runtime.py read as "110 errors" from a stray cwd and "All checks passed" from the root). The earlier 144-vs-144 comparison was taken with the same cwd on both sides, so its conclusion -- zero added -- stands; the absolute number was just scheduler.py's 102 plus schedule_batch.py's 42.

…eport-only-Vorstufe) == BEFUND, DER DIESEN SCHNITT AUSLOESTE == `initialize_decoupled_kv_group` (parallel_state.py:3676) hatte NULL Aufrufer im ganzen Baum -- praesent-aber-unverdrahtet. Zwei Quellen: devindex symbol_profile meldet 0 aufgeloeste Caller UND 0 textuelle Reads; die Volltext-Gegenprobe ueber python/ findet ausserhalb von parallel_state.py nur einen Doku-String in mem_ledger/engine.py:678. Gleichzeitig gated `SGLANG_DECOUPLED_KV` bereits die POOL-Haelfte (model_runner_kv_cache_mixin.py:3352). Genau diese Asymmetrie IST die sgl-project#1007-Booby-Trap: die Pool-Haelfte zu armen dimensionierte einen Plan fuer eine Gruppe, die nie gebaut wurde. `refuse_pp_dcp_combination` (:3653) benennt diese Gruppe im eigenen Refusal-Text als den designierten Weg ("For token-sharded KV under PP use the #704b decoupled-KV group"). Die Sperre bleibt unberuehrt: sie verweigert das stille Vehikel (dcp_size ueberladen), nicht die Faehigkeit. == WAS DIESER SCHNITT TUT == (a) Aufrufer in ModelRunner.init_torch_distributed, an derselben Stelle und aus demselben Grund wie der sgl-project#631-Flip-Block darueber: der Create ist ein KOLLEKTIV, also muss das Erreichen rang-uniform sein -- ein Server-Arg ist das per Konstruktion. Vor der sgl-project#605-Flight-Recorder-Marke, damit die NCCL-Bilanz diese Kommunikatoren mitzaehlt. (c) `--enable-decoupled-kv`, DEFAULT AUS. Er baut die Gruppe und SONST NICHTS. `set_decoupled_kv_active` wird bewusst NICHT gerufen, `_DECOUPLED_KV_ACTIVE` bleibt False, die B1-Verzweigung in `get_dcp_group` (:2854) bleibt tot, die Pool-Haelfte ist nicht angefasst. Report-only-Vorstufe: die Gruppe existiert, liest sie aber niemand. == EIN PASSENDER DESK-CHECK (Fehlerklasse: Flag-Registrierung + Rangplan) == py_compile waere hier blind gewesen. Geprueft wurde stattdessen: 1. Flagge als dataclass-Feld registriert, Default False -> True/False 2. plan_decoupled_kv_ranks(w=3,tp=1,pp=3) = [[0,1,2]] (Flip-Form) 3. plan_decoupled_kv_ranks(w=3,tp=3,pp=1) = [[0],[1],[2]] (pure TP, degeneriert korrekt: B1 ist ein PP-Mechanismus) 4. CLI akzeptiert --enable-decoupled-kv -> ACCEPTED == BOOT-BEWEIS (boot_855_704bgroup2_0840f82601_0830_165321) == * "#704b decoupled-KV group built over pipeline ranks [[0, 1, 2]]" auf ALLEN DREI Raengen (PP0/PP1/PP2) -- Manifest-Gleichheit hielt * "manifest DIVERGES"-Refusal: 0 * "initialize_decoupled_kv_group was never called"-Assert: 0 * `set_decoupled_kv_active` trapsafe bare 0 / genuine 0 -> Routing nie gearmt * Serving liefert: "capital of France is" -> " Paris. Paris is the largest" ERSTER BOOT-VERSUCH SCHEITERTE EHRLICH und ist Teil des Befunds: Bar1WindowRefused -- die neue Gruppe 'decoupled_kv:0' erbte das 24-MiB-Default-Fenster, waehrend nur 12 MiB BAR1 frei waren (world 23 + pp 95 + flip_tp 47 + flip_dcp 31 bereits gehalten). Das Refusal ist KORREKT ("an explicit window is an instruction ... silently serving a smaller one would make every later size decision rest on a number nobody chose"). Behoben durch ein eigenes Fenster DECOUPLED_KV_0=8, nicht durch Abschalten von barlink. Merkposten fuer jeden weiteren Gruppenbau auf dieser Form: BAR1 ist mit ~196 MiB in vier Gruppen nahezu erschoepft. BELEG-STUFE: BOOT-BEWIESEN, dass die Gruppe auf allen Raengen entsteht und das Serving unveraendert laeuft. NICHT behauptet: dass token-gesharde KV unter PP damit funktioniert -- die Pool-Haelfte liest weiterhin dcp_size/attn_dcp_rank (unter PP 1 und 0) und ist der naechste, separate Schnitt (b), der ohne GO nicht gebaut wird.

Fix default argument of OpenAI Chat completion

cde134b

merrymercy merged this pull request into main Jul 9, 2024

merrymercy deleted the fix-chat-completion branch July 9, 2024 08:52

merrymercy mentioned this pull request Jul 9, 2024

OpenAI ChatCompletionRequest max_tokens defaults to None causing error #582

Closed

merrymercy added a commit that referenced this pull request Jul 9, 2024

Fix the default argument of OpenAI Chat completion (#605)

da2e5d6

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

Fix the default argument of OpenAI Chat completion (sgl-project#605)

d4ca624

dmvevents mentioned this pull request May 5, 2026

moe: add DeepEP V2 ElasticBuffer support to MoE token dispatcher #24443

Closed

5 tasks

Navigation Menu

Uh oh!

Fix the default argument of OpenAI Chat completion - #605

Fix the default argument of OpenAI Chat completion#605
merrymercy merged 1 commit into
mainfrom
fix-chat-completion

merrymercy commented Jul 9, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

merrymercy commented Jul 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

merrymercy commented Jul 9, 2024 •

edited

Loading