Turn on flashinfer by default by Ying1123 · Pull Request #578 · sgl-project/sglang

Ying1123 · 2024-07-01T21:58:33Z

No description provided.

… objective Four coordinator answers folded in; two of them close holes in the first draft. R9 (NEW, user 2026-08-05) -- every prefill/draft/decode(verify) cut is itself a load change, and an ENFORCER must actually cause a different layout per load. Flip granularity becomes a PRICED AXIS rather than a blanket rule: per knob, quantify the flip cost and classify by two checkable criteria. A knob is tick-flippable exactly when flipping it costs neither a CUDA-graph recapture nor a byte moved; everything else flips at regime boundaries with hysteresis. Grounded, not asserted: sgl-project#439 cold-expert compute assignment is verified VRAM-neutral in tree (expert_heat_migration.py:31 names "the same size before and after" as the sgl-project#439 sizing latch's invariant), which is precisely the zero-data-movement property that puts it on the tick side. Within a tick the layout is fixed by definition, so the right tools are overlap/behaviour adaptation (sgl-project#128/sgl-project#199, sgl-project#125, sgl-project#274, sgl-project#156) -- not lesser substitutes for flipping, but the correct instrument at that timescale. F9 is the falsifier for the silent error: a wrongly-regime-classified zero-move knob never demonstrates the gain it could have made, so nothing ever surfaces it. F9b guards the opposite mistake (a 3-6 s recapture at tick granularity would spend the whole tick budget on recapture). F9c asserts on ACTUATED state, not on a logged decision, because sgl-project#578 is exactly the case where the decision existed and the actuation did not. SLICE 0 (sgl-project#578) -- verified in tree, not taken on report: regime_runtime.py:911 calls planner_candidates(server_args) without solve_fn, so regime_stages.py:383 takes the None branch, returns [], and the stage table permanently holds the booted stage alone. The docstring at regime_stages.py:357 says it outright -- "it is not broken, it is unfed". The enforcer R9(i) requires therefore EXISTS AS CODE AND ACTUATES NEVER, which made every actuated requirement in the first draft unreachable. Promoted from unknown risk to a named prerequisite slice. The work is wiring plus a SolverAnswer->Stage mapping (key_solver.solve already exists and is objective-aware), but it needs plan_inputs, base plan, per-rank budgets and RigRates -- i.e. a card probe. So slice 0 shares TICKET_582 gate (a) as a prerequisite and should be scheduled into the same window; recorded. R4 -- answered, [CONFIRM] removed. The objective is JOINT (maximum compute AND maximum VRAM throughput/latency at all times), expressed as two axes rather than collapsed into one scalar on the user's behalf. Dominance on disagreement is decided by the running task's phase: decode is memory-bound so bandwidth IS throughput there, prefill is compute-bound. The weights come from the same regime signal R7 consumes. F4 is a pair for the same reason as F7: a fixed scalar objective passes exactly one of the two regimes. sgl-project#551 -- resolved as in-tree sgl-project#553, and my find is confirmed: DESIGN/ANALYSE only, no mechanism. Slice 5 therefore BUILDS elastic co-residency as the planner's tenant-event actuator. Stated plainly in the slice plan and §5 so the plan cannot be misread as driving something that exists. Also recorded: R9's knob inventory is incomplete by construction (only the knobs the addendum named are classified); a full sweep with a cost per knob is part of slice 4b, since an unclassified knob is an unpriced assumption and R9 forbids those. And "3-6 s" for recapture is an observed range, not a model -- R5's movement-cost work must cover it too.

Seventh addendum, and the first requirement here carrying a MEASURED cost rather than a hypothetical one: production showed all three ranks waiting 1.2 s+ on a 74k-cached prefill, with collective wait running 2-3x compute. R12 is R6 applied to the one vector still frozen at boot. (a) At LOW occupancy the planner selects the PERFORMANCE-optimal vector -- KV concentrated so DCP prefix-gathers are minimal. Not a micro-optimisation: the gather share grows with cached context, so at low fill the spread vector buys capacity nobody is using and pays for it in a collective floor on every prefill. sgl-project#588 decomposes the measurement; R12 is what the planner does with it. (b) The shift is CONTINUOUS and BIDIRECTIONAL. This generalises sgl-project#287's pressure ladder, and the generalisation IS the removal of its trigger-on-pressure-only design: a ladder that fires only when memory gets tight can never take the low-fill performance win, because low fill is not a pressure event. Refusing the return leg is the "surplus sits idle" failure in vector form -- capacity released, but the performance it was blocking never reclaimed. (c) Carriers verified in tree: sgl-project#287 ladder stages; sgl-project#297 phase-boundary KV resharding, delta move measured <1 s (DESIGN_297_kv_resharding.md:83, and the TEIL_HOT rung in registry/rungs.py prices it "<1 s, one dial re-raise"); sgl-project#363/sgl-project#578 regime controller as actuator now that slice 0 binds its feed. Priced movement-bound under R11's gates, and R11e is exactly the comparison that matters here: a sub-second reshard removing a 1.2 s per-prefill floor pays for itself in two prefills. F12a (near-empty + long prefix -> concentrated, gather share must drop), F12b (past the capacity knee -> spread), F12c (on drain -> move BACK). F12c is the one a pressure-only design fails while passing F12b, which is why the falsifier is again a pair, as with F4 and F7. Slice plan gains 4e (both legs; hermetic F12a/b/c, GPU gate is the gather-share drop under sgl-project#588).

integration line Four add/add conflicts. The branch is an OLDER SNAPSHOT of the same audit, and the governing fact for all four files is that its central claims have since been overtaken: it asserts that several features are UNWIRED, and they are wired now. test/registered/unit/test_unwired_features_421.py -- the conflicted TEST file, resolved so no coverage is lost. The branch carries four pin classes (TestKvPressureLadderAutoIsUnreachable, TestOffloadRegisterProfileIsUnreachable, TestColdTierShmIsUnreachable, TestMemTierRegistryHasNoConsumers) that the integration line deliberately RETIRED, each replaced by a named positive test that asserts the wiring and pins the call site. Resurrecting them would not have added coverage, it would have re-asserted four absences that no longer hold. Verified in the tree rather than taken on the retirement notes' word: `cold_tier_fetch` is imported from production at `layers/moe/fused_moe_triton/layer.py:1447` and `layers/moe/expert_offload.py:2711`, and `memtier.registry`/`memtier.tiers` at `model_executor/short_term_offload_register.py:113-114`. All five replacement test files exist on disk. So the branch's 104 unique lines here are superseded content, not dropped assertions. Corrected while resolving: the F4 retirement note cited its replacement at `test/registered/unit/moe/test_cold_tier_wiring_394.py`; the file is at `test/registered/unit/layers/moe/test_cold_tier_wiring_394.py`. docs/dev/AUDIT_421_UNWIRED.md (5 hunks), scripts/dev/audit_421/README.md (1) and scripts/dev/audit_421/detC.py (6) -- resolved hunk by hunk, not by blanket side-selection, and the branch side was found superseded in every one. detC.py in particular: the integration side adds sys.path bootstrapping so the script finds its sibling astlib.py, memoises four whole-tree derivations per Index, and flushes progress to stderr. Same computed results, so no behavioural divergence to adjudicate. The README's discarded sentence ("trust the STRONG tier") is contradicted by later evidence the integration line records, and its numeric claim survives verbatim at AUDIT_421_UNWIRED.md:317 -- checked before discarding. One stale citation corrected in the audit doc, section B.9: it claimed present-tense that the sgl-project#363/S8 planner `solve_fn` seam is unbound and cited a ratchet test `TestPlannerFeed::test_the_seam_is_unbound_in_production`. Task sgl-project#578 BOUND that seam -- `managers/regime_runtime.py:912` now passes `solve_fn=_planner_solve_fn(scheduler)` -- and the cited test does not exist; it was inverted into `test_regime_act.py::TestPlannerFeed::test_the_seam_is_bound_and_refuses_without_measurement`. Both facts re-verified against this tree. The historical narration is kept and a "Resolution (task sgl-project#578)" paragraph appended, matching how the same document already handles F4 and F6. Two inconsistencies noted and deliberately NOT edited, because neither is a merge-side disagreement and neither number can be re-derived from the tree: the memoisation speed-up is quoted as ~35 min in the audit and README but ~50 min in detC.py's own comment; and B.8 says "three of the eight pins are retired" while naming four classes. Tests: test_unwired_features_421.py 2 passed. sgl-project#631 flip family 1095 passed / 0 failed. detC.py py_compile OK, ruff clean.

…erning answer is a refusal First execution of --pp-solve-cut end-to-end through the wired handler on this rig. The pass ran on all three cards and put measured rates on disk keyed by UUID; the gate then answered, and under RUNSHEET 6b the answer is a REFUSAL, which per 5.1b outcome (3) IS the result. No W2 window was owed and none ran. MEASURED (m584/card_rates_by_uuid.json): 5090 GPU-31d7ef41 0A:00.0 203.57 TFLOPS 1661.7 GB/s 3080 GPU-5c648f96 05:00.0 51.14 TFLOPS 716.6 GB/s 3080 GPU-62dbbae1 0B:00.0 50.81 TFLOPS 717.4 GB/s All THROTTLED, correctly: this rig runs deliberate power caps (200/320 W, 400/600 W), so these ARE its operating rates. THE BORROWED RATES WERE STALE. Against s50's numbers that R12 injected by hand, bandwidth reproduces to the decimal (3080 717.4 vs 717.4) while GEMM does not (5090 -12.2 %, 3080 -22.5 %) -- the direction and rough magnitude of the 2026-08-05 power-target reduction. The 37,14,13 admission rested on pre-power-cap compute rates. A second, independent reason the old numbers do not describe this rig, alongside 6a's change-of-cut retirement. P2, pool 280000, budget 31400,19300,19300: M0 (most demanding) REFUSED M1 ADMITS 37,14,13 W1 ADMITS 37,14,13 The admitted cut is UNCHANGED at 37,14,13 despite the -12/-22 % rate move, so on this shape the choice is driven by memory feasibility, not compute balance. 6b takes the most demanding census: REFUSAL. Independently, the M1/W1-admitted cut fails C2' against the pooled-worst transient by -3643.3 MiB, so the refusal does not depend on which route a reader prefers. LEVER ARITHMETIC, swept against the REAL census rather than a fixture: L1 --rank-gpu-memory-mib EXHAUSTED BY PHYSICS. Refuses at the 5090's entire NVML total (32607), and refuses even with every card at full nameplate (32607,20480,20480): 39968 MiB needed against 48684 usable. The aggregate FITS by 8716 MiB and the solve still fails -- it is a CONTIGUITY/PACKING result, which R12 suspected and this measures. There is no budget left. L3 --max-total-tokens CONFIRMED NOT A LEVER, now on metal: 280000 -> 20000, every one refused. 5.1a asserted this from a fixture proxy. L2 seam THE ONLY RECOVERING LEVER. Threshold ~515 MiB against the measured 1838 -- a 1323 MiB / 72 % reduction -- and it then selects a THIRD cut, 29,19,16. Independently close to 5.1a's fixture- derived "~500 MiB". Any throughput number would again be retired. sgl-project#363 FLIP TARGETS, counted (bonus, not the protocol): card probe now visible to the planner's own lookup YES, 3 cards with rates flip targets from planner candidates 0 ... with per-stage measurements present 2 stages, 1 flip target The card-rate half CLEARS sgl-project#363's blocker-one precondition -- the PlannerFeedUnavailable('no card probe on disk') every rank logged is gone -- and does NOT produce flip targets, because build_stage_table refuses a solved-but-unmeasured candidate (sgl-project#578/sgl-project#360). The sgl-project#363 verdict predicted this hermetically; it is now confirmed with a real probe on disk. The remaining gap is the OTHER half of sgl-project#584's pass: per-stage A-vs-A gain, band and instrumented flip cost. Q2b proves the machinery works the moment those exist. Evidence /spinning/evidence-631/m584/: card_rate_pass.log, card_library.json, card_rates_by_uuid.json, nvml_identity.csv, p2_measured_rates.txt, p2_levers_m0.txt, p2_levers_binding_fine.txt, p2_levers_ceiling.txt, p2_seam_lever_m0.txt, p2_seam_threshold.txt, check_363_flip_targets.txt, and the drivers that produced each.

…, and a lift has THREE twins Part 2 completes the sweep begun in 46e3e30. Scope as stated there: 181 candidate lines over python/sglang/srt (tests excluded), 170 fork-owned. HEADLINE: the TREE is healthy on this axis. Zero DANGLING guards -- every ticket citation resolved to a real commit history or a checkable artifact -- and zero guards needing a lift decision. The rot is concentrated in the planner's REJECTION REGISTER, which part 1 covered. THE ONE VERIFIED STALE-TEXT, fixed: server_args.py:2275 said park-tier routing happens at exhaustion sites "once the sgl-project#236 budget lands". sgl-project#236 landed; park_instead_of_demote is live and called at kv_session_offload.py:3404. The conditional clause is dropped. A CLASSIFICATION I REJECTED AFTER CHECKING IT MYSELF, and the reason it matters more than the row does. The sweep flagged barlink_path_rates.py:21 ("the measurement does not exist yet") as stale because sgl-project#278 closed and load_nccl_reference is implemented. That is wrong: the LOADER existing is not the MEASUREMENT existing. The scripts that would produce the data exist (scripts/gpu_battery/s06_nccl_reference.*, scripts/probe/nccl_reference.py), no data file does, and the consumer at :448 explicitly handles that case by keeping the paths PLACEHOLDER. The docstring is CORRECT. Fixing it would have replaced a true statement with a false one -- which is the exact failure mode this axis is most likely to commit, so it is recorded rather than quietly dropped. THE SHARPEST FIND, and it is on this audit's own axis turned against the test suite: test/registered/unit/server_args/test_phase_flip_args.py:88 still asserts that enable_hierarchical_cache=True raises a ValueError matching "sgl-project#630". sgl-project#703 stage 2 REMOVED that blocker. The test is RED on train/0817-control right now -- verified failing at the untouched base commit with both my edits reverted, so it is not this sweep's doing. That upgrades a lesson the sgl-project#703 lift already half-knew. That lift was careful about TWINS: it moved the parse-time and runtime clauses together because "fixing only the runtime clause is not enough ... both must move together or the flag is still unusable". THERE WAS A THIRD TWIN. The test pinning the gate did not move, so a careful lift still left behind a red test asserting the presence of what it had just deliberately removed. So the lift-evidence bar in the audit gains a fifth item: the tests that pin a gate move WITH it. Two twins were not enough; there are three. NOT FIXED HERE: rewriting a test assertion is not behaviour-neutral for the suite, so it falls outside this sweep's one permitted write. Filed as its own item, with the repair named (drop the sgl-project#630 row from that loop and point at test_hicache_bounded_waits_630.py, which the code comments already name as the active protection). COUNTS, with one discrepancy I am NOT resolving by picking a side: I counted 21 provenance-shaped lines inside my 181-line candidate union; the sweep counted 187 across the whole tree without that pre-filter. Different denominators, both defensible, neither reconciled line by line. Both agree on the shape -- the large majority of #NNN mentions are settled narration and only a handful are live conditions that could expire. Stated rather than averaged. STILL-VALID substantive set spot-checked: the HiCache phase-rebind refusal (nothing in the tree ever sets the host pool it waits for), the sgl-project#578 stage-table refusal (reads liftable in isolation; the full trace shows sgl-project#584's canon fills what it can and this raise is the designed fallback for what it cannot), the sgl-project#452 CUDA-graph refutation (reconfirmed three times, never reversed), the KVSO/HiCache opt-in gate, and topk>1 under uneven DCP. Two loose citations named: layers/dcp/owner.py:127 and layers/attention/triton_backend.py:432 both cite "sgl-project#76", which is an unrelated ticket -- scale analogies written as if they were pointers. The guards are valid; the citations misdirect a reader exactly as the drifted anchors do. Suite: server_args 663 passed, 91 subtests, 1 failed -- the pre-existing test_v1_blockers_named above, verified identical at the untouched base. Catalog updated same step. codespell clean.

THE QUESTION WAS "DOES sgl-project#721'S HOST-POSTEN-LEDGER EXIST", AND THE ANSWER IS (a) WITH TWO NAMED HOLES -- NEITHER OF WHICH IS WHAT sgl-project#721 ASKED FOR. IT EXISTS AND IT IS WIRED. `mem_cache/pinned_host_budget.py` is a named, individually-priced posts ledger, and it IS a hard boot precondition: three paths reach `joint_pinned_host_error` from `ServerArgs.__post_init__` and raise `ValueError` -- server_args.py:6937 -> _handle_kv_session_offload -> :8173 server_args.py:6962 -> _handle_phase_flip -> :8416 -> :8961 server_args.py:7019 -> _handle_hicache -> :8916 -> :9029 Four posts are priced there over CONFIGURED numbers (HiCache host tier, HiCache staging host tier x ranks, phase-flip rotation staging ring x ranks, kv-session-offload spill pool), and eight more register at runtime in the workers (memory_pool_host x4, read_buffer_pool, weights_arena, and the two halves of the phase-flip staging pin this branch added in db1b07e). So it is not (b) "nobody calls it" and not (c) "an intention". BUT sgl-project#721'S TWO PRESCRIPTIONS WERE BOTH DECLINED, AND BOTH DECLINES ARE RIGHT. * "16G floor against `free -g`" -- not implemented, and must not be: `free -g` is lxcfs-synthesised in this container. That was settled in #871b, which moved the boot gate onto the sgl-project#407 owner instead. * "sonst Posten schrumpfen" -- deliberately not implemented. The module's own charter: "The guard SUMS, it never caps. Nothing in this module shrinks a request. Silently capping a pinned pool moves the failure from a boot-time message that names both posts to a later allocation whose victim the OOM killer chooses." That is a better answer than the measure it declines, and it is the reason the ledger prices every post separately -- so an operator can shrink the right flag himself. THE ONE REAL DEFECT, AND IT IS A COUNTER DESCRIBED AS AN ACTUATOR (sgl-project#464/sgl-project#578/ sgl-project#852). `check_and_register_pinned_post` carried this, about the largest claimant in the system: "`_register_image_post` declares the post BEFORE `_alloc_host_image` allocates ... deliberately, so the registry refuses an over-commitment at the DECLARATION rather than discovering it at the allocation." THE REGISTRY REFUSES NOTHING THERE. `register_pinned_post` is a bare dict write with no comparison in it; `_register_image_post`'s own first line says "Never raises"; and `grep -c check_and_register_pinned_post weights_arena.py` -> 0. So the ~27 GiB of phase-flip host weight images are COUNTED, never ADMITTED. THE BEHAVIOUR IS CORRECT AND STAYS. sgl-project#695 chose it at the call site and wrote why: "Registered, not CHECKED: a new refusal path here could break a boot that works today, and the diagnosis this is for is served by the number being present, not by a veto." Registering early still makes those bytes visible to every LATER admission, which is the half that matters. Only the comment was wrong -- and a comment promising a veto that does not exist is worse than none, because the next person sizing a pinned pool against this module would leave exactly the headroom that post already spent. WHY A TEST AND NOT JUST A RE-COMMENT. Prose cannot be regression-tested; the CONTRACT can. The split is now pinned in both directions -- counting never refuses however absurd the post, admitting refuses and names every post, a refused post is not left in the registry, and weights_arena uses the counting entry point only. If someone later wires the admitting one in there, that is a real decision with a boot-refusal consequence and it surfaces here instead of in a boot. A RULE I PROVED ON MYSELF, THIRD INSTANCE IN ONE SESSION. The first version of the prose test asserted the false sentence was ABSENT from the module -- and failed against the FIXED file, because a correction has to QUOTE the claim it corrects. Same shape as #871a's floor-promise assertion and #871b's /proc/meminfo probe assertion. The general rule, now written into the test: ABSENCE OF PROSE IS NOT TESTABLE where a correction must restate what it corrects. Prose gets one positive marker; the behaviour tests carry the contract. THE WEIGHTS TRANSIENT, DECIDED WITH REASONS AS ASKED. It is a SPIKE, not a standing post, and the existing treatment is already the right compromise: it is registered at declaration (so concurrent posts see it), and taken back both when the allocation raises (weights_arena.py:1044) and when the image is freed (:1082). Booking it as a permanent resident would refuse too early; leaving it invisible would be the sgl-project#721 outcome. What it is NOT is admitted -- and turning that on is a behaviour change with a boot-refusal consequence that needs a loaded box to justify, not a desk. LOAD REMAINS UNMEASURED, AND THAT IS IN THE CODE, NOT ONLY HERE. #871b's 0.24 GiB delta between the raw and honest readings is an IDLE measurement. A precondition gate bites under load. No load figure is asserted anywhere as a threshold. TESTS (hermetic; no boot; no card touched). CUDA_VISIBLE_DEVICES="" verified at the process, `nvidia-smi --query-compute-apps` empty throughout, cards 0/0/0. * red-first: 1 of 6 red against the uncorrected module, extracted == summary. * can-fail in the danger direction: making the COUNTING entry point refuse (a counter grown into an actuator) reddens `test_counting_never_refuses_however_absurd_the_post`. Green restored. * turnkey + this suite together: 175 passed, 0 failed. * partitioned tier-2 gate: wide 3775 / narrow 292 / serial 353 passed, 24 exclusions reported by the runner, names=1 agrees with the summary on every lane. The single failure -- test_pp_admission_wraparound_never_blocks::test_blocking_wraparound_wedges_the_ring -- is PRE-EXISTING: on the clean base 726f11e that module fails TWO tests, i.e. MORE than here, and it touches nothing in this commit. * test/registered/unit/mem_cache: 2 failed / 1808 passed. The two are the known test_acceptance_emitters_758 RefillTiming pair; +6 over 1802 is exactly this suite. NOT ESTABLISHED. Whether the ~27 GiB image post SHOULD be admitted -- that needs a box tight enough for the answer to differ, i.e. load. Whether the four launcher posts plus eight runtime posts are the COMPLETE set of pinned claimants: I enumerated every `PinnedHostPost(` and every register call in the tree, but a pinned allocation that never posts at all is invisible to that method by construction. And ZFS cache, named in sgl-project#721's list, is kernel-side and cannot be a process post in this ledger at all.

… >30s) -- aber das Instrument ist im Stall STRUKTURELL BLIND, und die Haelfte dagegen habe ich nur halb gebaut == WAS GEBAUT WURDE == Instrumentierung des gloo-all_reduce in `_update_uniform_pool_budget` (scheduler.py:6185ff): Zeitmessung um den Aufruf, WARNING je abgeschlossener Wartezeit >30s, Census-Dump (kollektiv-frei) >300s, plus ein publizierter `_uniform_reduce_since`-Zeitstempel. KEIN Timeout am Work-Handle -- der sgl-project#630-Grund steht im Kommentar: ein abgelaufenes wait(timeout) laesst den Handle in undefiniertem Zustand, "report-only" waere dann nicht mehr semantik-erhaltend. Stattdessen das Muster, das die Kette schon benutzt (on_blocked_recv): Zeitstempel publizieren, Fremd-Thread liest. Promotionskriterium steht als Anti-sgl-project#578-Klausel im Kommentar. Werkzeug-Check diesmal FEHLERKLASSEN-PASSEND (die Lehre vom letzten Zug): AST-Pruefung, dass `Scheduler` keine frozen/slots-Dekoratoren traegt, dass `time` importiert ist und dass `format_local_history(rank)` modulweit mit genau einem Argument existiert. Alle vier gruen, dann Boot. == PHASE-1-ERGEBNIS, gesunde Sequenz == NULL Wartezeiten >30s ueber sechs bestandene dataextract-Fragen. Das ist ein sauberes Ergebnis und die halbe Datengrundlage fuer die Phase-2-Deadline: im gesunden Betrieb dauert dieses Kollektiv nie auch nur 30 s. DE-06 lief in 35,0 s durch (Strandungslaeufe: 81,6 / 95,6 s). == UND DANN DIE EHRLICHE HAELFTE == Bei DE-07 strandete es erneut -- und mein Instrument meldete NICHTS. Der Grund ist strukturell und ich haette ihn beim Entwurf sehen muessen: die Messung laeuft NACH dem Aufruf. Bei einem echten Hang kehrt das all_reduce nie zurueck, also wird weder die Zeit gemessen noch die 300s-Schranke je erreicht. DAS INSTRUMENT SIEHT GENAU DAS EREIGNIS NICHT, dessen Obergrenze es finden sollte. Die Haelfte, die es koennte, habe ich ANGEFANGEN UND NICHT ZU ENDE GEBAUT: `self._uniform_reduce_since` wird publiziert -- und NIEMAND LIEST ES. Das ist exakt die sgl-project#578-Klasse (armiert, nie ausgefuehrt), die ich im selben Commit im Kommentar verboten habe. Ich habe die Regel aufgeschrieben und im selben Zug gebrochen. == LIVE-CATCH BEIM STALL == Zwei Raenge, zwei verschiedene Kollektive, gleichzeitig: 976272 active broadcast <- broadcast_pyobj (utils/common.py:2845) 976273 idle all_reduce <- _update_uniform_pool_budget (scheduler.py:6220) Dritter Befund derselben Form. Die Divergenz-Diagnose steht. == WAS PHASE 1 JETZT BRAUCHT (kein Bau ohne GO) == Der Leser fuer `_uniform_reduce_since`. Es gibt bereits einen Thread, der genau so etwas tut -- `admission-wedge-watchdog` (scheduler_components/invariant_checker.py:1377), der schon `_pp_blocked_recv_since` liest. Derselbe Leser um den Reduce-Zeitstempel erweitert, mit dem kollektiv-freien Census-Dump als Aktion, ergibt den Report-Pfad, der im Hang tatsaechlich feuert. DAS ist Phase 1 fertig; was ich gebaut habe, ist nur die gesunde Haelfte davon. Phase 2 (CRASH/STOP) bleibt gesperrt: Kriterium (ii) -- "der Report-Pfad hat nachweislich gefeuert" -- ist unerfuellt, weil es ihn im Hang noch nicht gibt. BELEG-STUFE: Bau BOOT-BEWIESEN (Checks gruen, Boot gesund, greedy kohaerent). Gesunde Verteilung BOOT-GEMESSEN (n=6 Fragen, 0 Samples >30s). Der Instrumenten-Blindfleck ist BOOT-BEWIESEN durch sein Schweigen im Stall. Serving wieder oben, health 200.

merrymercy requested changes Jul 2, 2024

View reviewed changes

Comment thread README.md Outdated

Ying1123 force-pushed the ying-flashinfer branch from 086f007 to 53c6ea3 Compare July 2, 2024 08:44

update

98f982e

Ying1123 force-pushed the ying-flashinfer branch from 53c6ea3 to 98f982e Compare July 2, 2024 09:24

Ying1123 merged commit 9380f50 into main Jul 2, 2024

Ying1123 deleted the ying-flashinfer branch July 2, 2024 09:25

hnyls2002 mentioned this pull request Jul 3, 2024

fix the broken server args #585

Merged

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

Turn on flashinfer by default (sgl-project#578)

0c0a1dd

Navigation Menu

Uh oh!

Turn on flashinfer by default - #578

Turn on flashinfer by default#578
Ying1123 merged 1 commit into
mainfrom
ying-flashinfer

Ying1123 commented Jul 1, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Ying1123 commented Jul 1, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants