test: negative value testing for frequency, presence penalizers by vhain · Pull Request #995 · sgl-project/sglang

vhain · 2024-08-09T04:16:03Z

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Continued from #973

Frequency and presence penalties accept negative values, but the test only covered positive values.

Also @zhyncs mentioned about adding negative test cases, but #973 got merged before I add these tests.

Modification

Adds negative test cases for both penalizers.

Checklist

1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
3. Modify documentation as needed, such as docstrings or example tutorials.

…project#995)

…resident at all Boot 14 (cf16281) died on PP1 after 39 s: `assert self.chunked_req is None`, scheduler.py:9784 in `_get_new_batch_prefill_raw`. Third recorded fundstelle of this family after :9286 (sgl-project#951) and :9367 (sgl-project#959). ROOT, and it corrects my own attribution in the boot-14 register entry. The invariant behind that assert is held "by ARITHMETIC, not by a check" (scheduler.py's own comment). sgl-project#959 therefore gave the two `add_one_req*` mint sites an explicit `chunked_req_outstanding` check, and SKIPPED the third -- `PrefillAdder._add_scheduled_req`, the forwarded-schedule execution path -- reasoning at schedule_policy.py that it "already has its own (`carried_chunk`)". That is the guard-comment-names-the-hazard trap. `carried_chunk` answers "is THIS request the resident continuation". The invariant needs "is there a resident continuation AT ALL". It covers a request being re-announced; it does not cover a DIFFERENT named request becoming a second continuation while the first is resident. Two of three sites guarded, one not. Boot 14 is that gap on metal: the resident continuation survived `add_chunked_req`, so `chunked_req_outstanding` was True and both sibling sites correctly refused -- and this site minted anyway, on another rid the same forwarded schedule named. sgl-project#994 EXPOSED THIS, IT DID NOT CREATE IT. Boot 13 never reached the line because the sgl-project#791 geometry refusal killed every pass before a batch was built; sgl-project#994 removed that refusal. My register entry attributed the death to sgl-project#994's effect on `rem_chunk_tokens` arithmetic. That was wrong: both sibling sites were already guarded and did refuse. The arithmetic was not the hole; the missing third guard was. Withdrawn here rather than left standing. FIX: the sibling guard, at the site that lacked it. WHY A PASS REFUSAL AND NOT A REQUEST SKIP -- the danger direction, which is the whole question here. On a forwarded schedule this rank may NOT drop a named request: the upstream's hidden states for it are already on the wire, which is what this same method already raises `PPScheduleRefused` for a few lines above. Running the chunk WITHOUT announcing it is worse -- the continuation would be untracked and re-prefilled next pass, the double prefill the standing law forbids. So the disposal is the one this path already owns: refuse the PASS by name, let sgl-project#791/sgl-project#797 void and re-derive. It cannot starve: the resident continuation is consuming chunks, and when it finishes `chunked_req` is None and the schedule is executable. This is the same direction sgl-project#959 chose ("the resident continuation is never the one to give way; the fresh admission is") -- not the sgl-project#858 mid-prefill wedge, which would be clearing `scheduler.chunked_req` instead. FUTURE CHECK: `grep -c 'if self.chunked_req_outstanding:'` over schedule_policy.py is now 3 and equals the number of `new_chunked_req` writers. A fourth writer that skips it is visible as an inequality rather than as a boot death. Execution proof (speed mode, one instrument per link): `note_second_continuation_refused(req, "_add_scheduled_req")` counts it and the refusal text names itself, so boot 15 measures whether the guard fires at all -- absence of the assert alone would be green-by-absence. Evidence: desk. py_compile; local-import dominance and guard-before-raise- before-mint verified by source order in the loaded module; guarded-site count 3 == mint-site count 3. Belegstufe: DESK-BEWIESEN.

…the pass Withdraws sgl-project#995's `PPScheduleRefused`. The guard was right, the disposal was wrong, and boot 15 measured it. BOOT 15 (473f3ad, 22:12-22:13): 175 `sgl-project#995 FORWARDED SCHEDULE UNEXECUTABLE` on exactly ONE rid (ddb6f38b…) in ~40 s, 4 batches total, rank 2 at 0% GPU while ranks 0-1 burned on corridor-reclaim. That is a LIVELOCK, not a stall: a voided pass computes nothing, so the resident continuation never advanced and each following pass rebuilt the identical refusal. It is the same shape this branch already records as "Boot 3 spent 514 refusals on that asymmetry". MY OWN STARVATION ARGUMENT WAS THE DEFECT. sgl-project#995's comment claimed "it cannot starve -- the resident continuation is consuming chunks". I took that sentence from sgl-project#959, where it is true because that path SKIPS a request and lets the pass run. On a pass REFUSAL nothing runs, so the premise is false. Borrowing a justification across paths without a coverage check is the guard-comment class this window has now produced twice. SECOND REASON, independent: the `except PPScheduleRefused` handler in scheduler.py documents a deliberately-open lock_ref leak (requests admitted earlier in the same loop keep their `inc_lock_ref` when the batch never completes) and justifies leaving it open with "Bounded: it takes a genuinely unexecutable geometry to reach this line at all". This condition is ordinary, not exotic, so the refusal converted a bounded leak into a per-pass ratchet: protected nodes, falling evictable_size, a slow wedge instead of a crash. WHY NOT ANNOUNCING LOSES NOTHING, which is what the raise got wrong. `new_chunked_req` / `scheduler.chunked_req` is LOCAL bookkeeping for a chunk this rank decided itself, so the next pass can find and resume it. Under a FORWARDED schedule that job is the upstream's: `_add_scheduled_req` receives `prefix_len` and `extend_len` fresh every pass (its own "LAST CHUNK OR NOT IS ALSO THE SCHEDULE'S TO SAY"), so the continuation is re-established from the decision whether or not this rank remembered it. The chunk runs as decided. No dropped named request, no re-prefill, and the single field keeps its one occupant so the assert stays intact. The guard itself stands: three `new_chunked_req` writers, three `chunked_req_outstanding` checks. `note_second_continuation_refused(req, "_add_scheduled_req")` still counts every occurrence, so the condition stays visible without costing a pass. Credit: the refusal direction was challenged by a desk agent on control-flow reading (lock-ref leak reachability and the livelock) before boot 15 confirmed both. Recorded because the challenge was right and the metal agreed with it, not with me. Evidence: desk + the boot-15 metal specimen that condemns the previous version. py_compile; guard-before-mint order and refusal-text absence verified in the loaded module; guarded-site count 3 == mint-site count 3. Belegstufe: sgl-project#995's refusal METALL-WIDERLEGT; this replacement DESK-BEWIESEN.

Boot 16 (996fbf4, 22:21:48, boot_943bx_996fbf4aca_0828_221614.log) died 68 s after first load. PP1 and PP2 both logged sgl-project#987 FILL-ADOPT rid=da614e20... local=8446 -> upstream=8447 appended=1 tail=[271] and PP1 then took scheduler_pp_mixin.py:2147 RuntimeError: sgl-project#631 PROXY LEFTOVER REFUSED: a proxy stamped mb_id=1 seq=17 rows=4096 epoch=2 arrived while this rank is on mb_id=2 in flip epoch 2 -- the SAME-epoch branch, i.e. a proxy for a pass this rank had already left. All three ranks down. THE ROOT IS NOT WHERE sgl-project#995 LOOKED. `_add_scheduled_req` decided whether a forwarded extent finishes its request with last_chunk = prefix_len + extend_len >= local_fill_len under a comment claiming that was "the schedule's to say ... arithmetic on forwarded integers rather than re-taken against a local budget". Two of the three integers are forwarded. The third is not: `local_fill_len` is `len(full_untruncated_fill_ids)`, rebuilt from THIS rank's origin + output_ids (+ carried tail) on every pass by `Req._refresh_fill_ids` (schedule_batch.py:1326 -- unconditional, ahead of the tree_cache gate at :1355). The verdict was a rank-local quantity wearing a forwarded one's clothes, and the fill is exactly the quantity this seam is known to disagree about. THE GUARD COVERED ONLY HALF THE DISAGREEMENT. `schedule_refusal_reason`'s third clause refuses `prefix + extend > local_fill_len` -- the decision asking for more than this rank holds. The opposite skew, this rank holding MORE than the rank that decided, passes every clause and silently flips "last chunk" into "mint a continuation". `adopt_carried_fill` cannot close it either: it only APPENDS (:1628-1633), so it lifts a short follower up to the decider and never brings a long one down. 8446 vs 8447 is that skew, and the continuation nobody decided on is what put the leftover proxy on the wire. THE FIX IS A CARRY, NOT A FOURTH CLAUSE. `PPAdmissionEntry` gains `last_chunk`, set by the deciding rank in `build_pp_admission_decision` from the same `fill_carry_for` reader that already publishes `fill_len`, projected by `forwarded_last_chunk` (the third projection of the one decision object, alongside `forwarded_fill_carry` and `_pp_forwarded_schedule_from`, so all three empty on the same passes and can never name different rid sets), and executed at the receiver. Tightening the refusal clause to `!=` was the available alternative and is wrong: `prefix + extend < local_fill_len` is the normal case for every legitimate middle chunk, so that refuses every pass. BOTH READERS OF THE VERDICT MOVE TOGETHER. `add_chunked_req` asks the same question one method later ("is this still the chunked request") and answered it by reading `extend_range.end` back off the request -- equivalent only while BOTH sides derived it locally. It now reads the same carried source via `_told_last_chunk`, or the two halves of one fact would drift, which is this ticket's own defect class one level up. `None` MEANS SAY NOTHING. A legacy sender, an unreadable fill, PP0 itself and every pp_size<=1 boot all yield no entry in the map, and both readers fall back to the pre-sgl-project#996 local derivation unchanged. RATCHET, same diff: the three `new_chunked_req = req` sites were bare assignments, so a second mint in one pass would have SILENTLY overwritten the first -- the overwritten request left in can_run_list with a partial extend range, tracked by nobody, re-prefilled next pass (the double prefill the standing law forbids, with no assert anywhere on the path). They now go through `_mint_chunked`, which asserts `new_chunked_req is None` and names both rids and the site. Nothing has been observed doing this; `chunked_req_outstanding` is set once per pass and knows only about the RESIDENT continuation, so neither existing watcher could see it. This is the check the family (sgl-project#951, sgl-project#959, sgl-project#995, sgl-project#996) kept discovering it did not have. DESK EVIDENCE, the boot-16 skew reproduced end to end and falsified in the dangerous direction: producer holding 8446 with a chunk landing on its end publishes last_chunk=True; a receiver holding 8447 derives False locally and would mint, and executes True with the carry. Old behaviour mints, new does not. Import smoke on all three modules; ruff clean on all 265 added lines (the 125 findings in these files are pre-existing and untouched). NOT yet boot-proven -- that is the next boot's job, and the sgl-project#996 log line fires exactly when the carry changes the verdict so the seam is readable directly.

Boot 22 -- the only configuration of this window that reaches decode (27 prefill / 157 decode) -- died after 55 s on PP1: sgl-project#631 PP proxy/batch mismatch: received hidden_states with 253 row(s) for a batch of 302 token(s) (bs=1) READING PROBE FIRST, because four of five suspicious number pairs today turned out to be unit confusions. This one is NOT: `_hs.shape[0]` is the leading dimension of the hidden-state tensor (one row per token) and `_want` is `forward_batch.input_ids.shape[0]` (one entry per token). Same axis, two vocabularies. The guard is sound and it is EARNED -- its own comment records the failure it replaced: "a decode batch of one request was computed on a 2048-row chunked-prefill hidden state, and the mismatched pair reached causal_conv1d_update -- an out-of-bounds write into another request's conv state." Loosening it is not on the table. WHAT WAS MISSING. The sender has declared its width since sgl-project#631 Variant B: `_pp_proxy_stamp` builds (mb_id, seqno, ROWS, epoch). Nobody reads element 2. Every stamp consumer compares only pass identity -- which the #791c note in this very file already calls out as insufficient: "Every identity above answers 'which PASS is this from'; none answers 'which BATCH is it of'." And `_pp_recv_proxy_tensors` POPS the stamp before the payload reaches the one check that does answer the batch question, so that check raises with an anonymous counterparty. This change carries the sender's stamp to that check and prints it. It splits two failures the old text could not: sender_rows == received rows -> payload intact, the PAIRING is wrong (a leftover from another pass); sender_rows != received rows -> the payload itself is wrong (assembled from the wrong result, or altered in transit). An absent stamp prints as "sender unstamped", itself a fact. CARRIED AS AN ATTRIBUTE, NEVER A DICT ENTRY. The pop site is explicit about why: PPProxyTensors' slice path maps `v[key]` over every dict entry and cuda-graph buffer copies iterate the dict, so a tuple left there "would slice to nonsense rather than raise -- the worst available outcome". An attribute is invisible to both, and the smoke asserts it stays out of `.tensors`. PURELY ADDITIVE, DELIBERATELY. No comparison is added at the receive site and nothing is refused that was not refused before. Deriving this rank's expected width there would be a second independent derivation of the very quantity whose disagreement IS the defect, and a wrong derivation would manufacture false refusals -- the mistake sgl-project#995's first version made, which boot 15 punished with a 175-refusal livelock on one rid. The one correct comparison already exists downstream; this only lets it name its counterparty. CLASS NOTE. Three sites were found independently today with one shape: a rank-local predicate deciding a group-uniform quantity (schedule_policy's admission arithmetic, the PP hold actuator, and this send/recv boundary -- where the code writes it out itself: "the recv is guarded by THIS rank's cur_batch while the upstream's send is guarded by the UPSTREAM's, and nothing enforces that the two agree"). The durable fix direction is that the DECISION travels with the PAYLOAD. Here it already does; it was simply never read. This commit is the diagnosis half; the enforcement half needs the next specimen to say which of the two failures above it is. Evidence: desk. py_compile on both files; smoke asserting PPProxyTensors accepts the attribute (not slotted/frozen), that it does NOT leak into `.tensors`, and that both message arms format. Belegstufe: DESK-BEWIESEN. Boot on the Boot-22 configuration is the test -- sgl-project#993 is reachable nowhere else, because no other cell serves long enough to reach the transition.

The fix form is settled by boot 26's evidence: sender rows == received rows with mb_id and epoch both matching, so the payload is intact and the PAIRING is wrong, and `pp_proxy_stamp_names_pass` cannot separate the two messages. The seqno can -- it was built for exactly this ("distinguishes two messages for the SAME slot, which is exactly the pair a stranded leftover creates") and no consumer has ever read it. THIS COMMIT DOES NOT ARM IT, and that is the point. The dangerous direction is not "a leftover gets through", it is "a VALID message is refused". sgl-project#995's first version proved the cost on metal: a refusal on the live path with no way onward produced 175 refusals on ONE rid in 40 s, four batches, and a dead window. So the can-fail is built on that side first, exactly there and not on the comfortable side. WHAT IT MEASURES: whether the width available at the receive site agrees with the width the downstream guard will use. If it ever disagrees on a message that is NOT a leftover, an armed check here would kill good passes, and that must be known BEFORE the check exists rather than after a boot dies of it. WHY THE COMPARISON IS TRUSTWORTHY, and this is the part that separates it from sgl-project#994's mistake: it is NOT a second derivation of the same idea. `ForwardBatch` is constructed with `input_ids=batch.input_ids` (forward_batch_info.py:709), so `self.mbs[mb_id].input_ids` IS the tensor `model_runner.forward` measures -- the same object, read at two points. sgl-project#994 truncated a geometry that a parallel computation then re-derived differently, and that is what made it a producer instead of a fix. Verified in the smoke rather than assumed, because the constructor could change. NOTHING IS REFUSED: the probe cannot raise (asserted in the smoke), the pass still runs, and the downstream width check in `model_runner.forward` remains the only gate. Disagreements are counted and named with both widths and the sender's (mb_id, seq, epoch); agreements are counted so the log states the denominator instead of leaving "no hits" ambiguous between "never disagreed" and "never ran". WHAT ARMING WILL NEED, stated now so the next step is not re-derived: a DEFINED ACTION for the rejected message -- drop the leftover and receive again -- and a defined state for the waiting receiver. Detection is half the work; surviving is the other half, and the half sgl-project#995 skipped. STILL OPEN, filed and not chased before the boot: WHY a leftover carrying this slot's own identity exists at all. This change makes the mispairing visible and will make it preventable; it does not explain the leftover. Evidence: desk. py_compile; smoke asserting the probe body contains no raise, and that ForwardBatch is still built from `batch.input_ids` so the same-object claim holds. Belegstufe: DESK-BEWIESEN.

sgl-project#798 calls itself "RANK-AGREED BY CONSTRUCTION" and it is -- for the DECISION. What it does afterwards is not. `_pp_void_own_batch` deliberately leaves `running_mbs` alone, "because the pass simply did not run, and it decodes again next pass from the state it still holds". That retention is taken rank-locally, it changes what the SLOT MEANS for the group, and nobody tells the upstream. Boot 46 read the result: sender and receiver both on slot 1, naming two DIFFERENT requests -- ('054d07eb', 8192, 8470) = 278 rows against ('d72fbd70', 8192, 8494) = 302 tokens. Each side internally consistent; both identity checks passed, because `mb_id` is the slot and not the occupant. The surplus-receive reading of the same shape is excluded: it needs the receiver BEHIND in slot index, and acceptance at :9017 proves the indices were equal. Unconditional retention keeps a corpse -- once the upstream has moved the slot on, nothing will ever be computed for the held request again, so the retention protects no live work, only itself. Unconditional release is equally wrong, and that is why this is not a revert: while the upstream still names the held request, its resident decode work is exactly what the void was written to preserve. The condition is the fix. The signal is already on the wire, so this reads and adds nothing: `_pp_admission_incoming_effective` is the upstream's rid set for the pass, and it is reset to None unconditionally at the top of every pass ("a pass that receives nothing must inherit neither half"), so a stale read is impossible by construction and no freshness stamp is needed. WHAT IS CUT HERE IS THE CONDITION, NOT THE RELEASE, and not out of caution in general. `pp_queue_orphaned_chunked_req` carries an explicit legality argument for a request displaced out of `self.chunked_req` -- "no door leads to it at all". A resident decode request is not that animal: it is reachable through `running_mbs`, and queueing it would be the double admission the same docstring prohibits. Dropping wedged (corpse R), refusing wedged (sgl-project#995); a third form that double-admits would be the third bill. Which of the two kinds occupies the slot at the moment of divergence cannot be read from the desk, so the probe reports it, and that field decides the shape of the release. Denominator incremented before any return, every exit a named reason, every reason counted -- so "did not fire" stays separable from "was never asked". Emitted on the existing off-path admission site beside sgl-project#998/#997d, never on the receive path. Evidence: desk, executed. All six exits reachable plus out-of-range; MOVED-ON can fire; seen == sum(reasons) == 8. Belegstufe: DESK-BEWIESEN.

Boot 51 died on "sgl-project#789 OUTPUT READINESS TIMEOUT: mb_id=2: upstream (rank 1) posted 9 dict message(s) on dict|output (entered 9), this rank has consumed 9". Nine posted, nine consumed, nothing outstanding -- and the rank waited for a tenth, then killed the instance for its absence. The arithmetic in the readiness gate is right; the caller is not. `_do_recv` enters the blocking output receive on `mbs[next_mb_id] is not None and not prebuilt and not can_skip` -- this rank's OWN slot state -- while the upstream forwards on `if pp_outputs:`. Two predicates on one stream, and the raise's own hint says so: "this rank decided to receive from its OWN slot state while its upstream decided to forward on `if pp_outputs:`, and nothing publishes this rank's per-slot expectation to that upstream". So the gate now reports its verdict instead of only ever killing on it. When `consumed == posted == attempted`, the upstream has stated in-band that it forwarded nothing for this slot, and the caller takes the SAME no-output exit it already takes for `target is None` two lines up -- an existing, handled path, not a new one. What this deliberately is not: nothing is dropped, because nothing was sent; nothing is refused, because there is nothing to refuse. Those two forms are both metal-falsified on this path already (corpse R 2026-08-09, and sgl-project#995 in this window, boot 15). And no token is fabricated -- `_pp_make_skip_output_ result` is NOT reused here, because its zero placeholder is legitimate only for a chunk that really ran and produced no token, which is not this case. Opt-in, default unchanged. The proxy alias keeps raising: a missing proxy is a different fact, since those hidden states are the input to this rank's own forward and declining them would compute on nothing. Evidence: desk, executed. py_compile; both `soft` parameters present; the proxy alias reaches no soft path; the declined case emits a named line with posted/entered/consumed so a decline can never become quiet. Belegstufe: DESK-BEWIESEN -- the metal proof is the boot.

…inert/unerreichbar -- die Suche zieht auf S1. Und die Inventur haelt am Pin NICHT. Fokussierter Durchgang auf den zwei einzigen als erreichbar+UNKLAR gefuehrten S0-Intake-Gates. Kein Schnitt gebaut: es gab nichts zu schneiden. == G-06 `drain_recovery_request` -- INERT, NICHT die Wurzel == Zeile haelt am Pin (scheduler.py:2860 in `process_input_requests`, ein resolved Caller). Der Code liest `RECOVERY_CHANNEL_ATTR` und gibt bei None sofort zurueck; den Kanal legt AUSSCHLIESSLICH der Watchdog-Post-Pfad an. Ohne erkannten Wedge ist das ein getattr und ein None -- die `waiting_queue` wird nicht angefasst. Empirisch am Strandungs-Boot (boot_855_fix1027_..._112900.log): Kanal NIE angelegt. WEDGE-RECOVERY genuine 0, RecoveryOutcome 0, Attribut 0. NEBENBEFUND, Lehrstueck fuer die sgl-project#995-Regel: 'recovery channel' zaehlt bare 31 / genuine 0 -- reine Prosa-Erwaehnungen. Ein blanker grep -c haette hier 31 "Treffer" gemeldet und das Gate faelschlich als aktiv ausgewiesen. VERDIKT: nicht die sgl-project#1028-Ursache. Und KEIN Loesch-Kandidat auf Kanal-Ebene: der Kanal ist die Thread-Grenze (Watchdog -> Scheduler-Thread) eines fork-eigenen Features, kein zweiter Datensatz der waiting_queue. Die Inventur-Beschreibung "Recovery-Request vor den Input-Strom injiziert" ueberzeichnet den gesunden Pfad. DESIGN-FORK, GEMELDET STATT ENTSCHIEDEN: das FEATURE Wedge-Recovery ist seinerseits eine Kompensationsschicht fuer fork-eigene Wedges, und das ist per Upstream-Minimal-Gesetz ein Loesch-Kandidat auf FEATURE-Ebene. Loeschen aendert aber nutzer-sichtbares Verhalten (ein gewedgtes Rig erholt sich dann nicht mehr selbst). Das ist ein echter Wert-Trade -> Operator-Entscheid. == G-04 Prefetch-Gruppenvotum -- UNERREICHBAR auf der Ship-Config == Die Inventur nennt scheduler.py:5085,5139. Am Pin steht dort etwas anderes (`routed_experts_start_len` bzw. ein Prefetch-Docstring). Ueber den EMITTIERTEN Literal statt ueber die Zeile relokalisiert: der Code lebt in `disaggregation/decode_hicache_mixin.py:152-197`, also im PD-Disagg-Pfad. Unser Boot faehrt `disaggregation_mode='null'`; der Mixin wird nur von `disaggregation/decode.py` verwendet. Marker `anchor_no_vote` im Strandungs-Boot: bare 0 / genuine 0. VERDIKT: auf dieser Form nicht erreichbar, also nicht die sgl-project#1028-Ursache. Die "decisive check"-Frage der Inventur (ueberlebt ein Konsument des Votums G-32?) ist auf der Ship-Config gegenstandslos; sie bleibt fuer die Disagg-Form offen. == META-BEFUND, der die sgl-project#968-Arbeitsweise betrifft == BEIDE von mir geprueften S0-Zeilen hielten am Pin NICHT: G-06 ueberzeichnet die Wirkung, G-04 nennt die falsche DATEI und eine falsche Erreichbarkeits-Spalte. INVENTAR-1003 steht auf `feat/969-deletion-cut @ 90c5ada`, einem fremden Branch, und ist einen Tag alt. Die Inventur bleibt wertvoll als Karte, WO zu suchen ist -- ihre Erreichbarkeits-Spalte traegt am Pin aber KEINEN Schnitt ohne Gegenpruefung. Fuer den restlichen sgl-project#968-Umbau gilt damit: jede Zeile wird vor einem Schnitt per emittiertem Literal am Pin relokalisiert und per Marker-Zaehlung am Boot-Log auf Erreichbarkeit geprueft, nie per Zeilennummer uebernommen. (Das ist die Determinations-Schnappschuss-Regel, jetzt mit zwei Belegen.) == FOLGE FUER sgl-project#1028 == Station S0 ist als Verlustort AUSGESCHLOSSEN: von sieben Fork-Gates sind drei strukturell unerreichbar (G-01/02/03), zwei lassen den Request per Verdikt ohnehin durch (G-05/G-07), und die beiden UNKLAR-Zeilen sind inert bzw. unerreichbar. Die Suche zieht auf S1 (ROUND ENTRY / Admission) weiter -- die Station, an der auch das Admission-Verdikt sitzt, das nach der sgl-project#968-Order zu PP0 gehoert. Damit faellt der sgl-project#1028-Einstieg mit dem Umbau-Kern zusammen, statt daneben zu liegen. BELEG-STUFE: DESK-BEWIESEN (Code am Pin) + BOOT-BEWIESEN (Marker-Zaehlung trapsafe am Strandungs-Log). Kein Boot ausgegeben, kein Code geaendert.

…anchor, and measure the loss KV is DIVISIBLE, the recurrent (mamba/GDN) state is POINTLIKE. Every length-shifting operation in the load-back chain is KV-built, so the extent it produces lands, for mamba, on a boundary with no state behind it -- and reusing a KV prefix there resumes a scan from a state that consumed different tokens. That is silently wrong, and today it is answered ALL-OR-NOTHING: the whole load-back falls and the whole prompt re-prefills. The user's grant is the opposite: re-prefill at most one HiCache chunk, from the KV position that matches the most current GDN state. WHAT THIS COMMIT DOES * Publishes the anchor depth the match walk already computed but discarded. `MatchResult.state_anchor_depth` is the absolute depth of `best_match_node`, i.e. of the deepest node EVERY component accepted -- mamba's acceptor being `is_resume_candidate` itself. The predicate is REUSED, never re-derived; a second anchor rule would be the second bookkeeping the upstream-minimal law rejects, and sgl-project#747 records what two anchor lineages do to each other. * `MatchResult.key_match_depth` beside it: how far the KEY matched with no validator asked. The PAIR is what separates "the prefix was never stored" from "the prefix is here and its anchor died with an evicted node" (sgl-project#1039). One hit number collapses them. * `state_aligned_load_back_len()` -- ONE writer, called by BOTH PPAdmissionEntry constructors, rounds PP0's extent DOWN to the anchor and counts what it gave back. `None` (no state-bearing component) keeps upstream byte-for-byte; `0` (no anchor on this path) is the CORRECT "load back nothing" under leaf-only mamba data, never an error. * `zero_match_result` zeroes the anchor depth too -- a refusal that reaches back to the root has no anchor left to point at. Same contract as sgl-project#1018 fixed for `cache_protected_len`. THREE VERIFICATIONS THAT CHANGED THE BUILD (desk-proven at this tree) * mamba_component.py's `best_depth` is NOT the number to plumb: its block needs `SGLANG_MAMBA_CKPT_STRICT_RESUME` (default False) AND `cache_controller is None` (false under HiCache), and it sums DEVICE value chunks, which diverge from the accepted-anchor depth exactly when `separate_device_match` is on. * The extent is ALREADY state-aligned RANK-LOCALLY: `host_hit_length` is measured from `best_match_node` (full_component.py), and that node is only set where the mamba validator accepted. A rank-local rounding is therefore a no-op, and is not built as one. * The real misalignment is CROSS-RANK, at the S1 clamp: PP0's extent is aligned in PP0's tree, the cut lands in the receiving rank's. That seam is MEASURED here (the FIX-3 refusal now prints the local anchor depth and whether it AGREEs with the told extent), not yet fixed -- the population decides the design. INSTRUMENTS (each names its denominator) * `sgl-project#1040 EXTENT STATE-ALIGN`: kv / extent / loss / anchor_depth / device_len / key_match_depth / class, with running totals. The round-down loss is kept APART from tokens forgone because no anchor existed -- pooling them inflated loss_max from 200 to 300 on the check fixture, which is how it was caught. * `sgl-project#988 LOADBACK` gains the KV/MAMBA split: kv_applied vs mamba_restored vs kv_only. KV moved while the restore refused is a HALF SUCCESS with a named link, never a green one. * `sgl-project#968 LOAD-BACK ROW APPLIED` gains `anchors=` and `retract_survivors=`. * `reset_for_retract` stamps retraction survival -- the free pre-measurement for the lifecycle half (sgl-project#1039); monotone, never cleared, nothing stale behind it. CHECK (speed mode: one matched check, not a battery) devtools/check_1040_rounding.py -- hermetic, CUDA-free. Named failure class of this edit: unit/coordinate error (absolute anchor depth vs the row's delta), off-by-chunk at the boundary, empty candidate set treated as an error. Trips the rounding on empty / below / exactly-at / above / pure-KV / no-hit, and walks `_state_anchor_depth` over a stub chain to pin the unit. ALL PASS, after it failed on the pooled loss denominator and that was fixed. py_compile + import smoke on all seven files; ruff clean on the three most-edited. cell_1039_ingeneration.sh: stage (iii) moved off `sgl-project#968 LOAD-BACK PROMOTED`, which the row collapse DELETED (0 live emitters, only a prose mention left -- the sgl-project#995 trap, it would have read 0 forever), onto `sgl-project#968 LOAD-BACK ROW APPLIED`. Adds the KV/mamba split, the measured loss median/max, the anchor-absent population, and a zombie test over the four abolished markers.

…e consumer it never had Boot 26 died at a sgl-project#631 proxy/batch mismatch three seconds after a cutover: 976 rows for a 558-token batch, sender rows == received rows, mb_id and epoch matching, identical geometry stamps. pp_proxy_stamp_names_pass accepts that pair by construction and names the case itself as its HONEST RESIDUAL -- 'within ONE epoch a leftover a whole ring-cycle stale still names this slot and is still accepted'. pp_loop_size is 3 on this boot form, so a pass three laps older lands on the same slot in the same epoch. The same docstring dismisses seq: 'FIFO delivery already makes it monotone, so it discriminates nothing a receiver can predict.' That is true of the wire and false of the inbox -- and take_typed serves the inbox BEFORE the wire. Stashing is the only mechanism on this channel that can reorder a pass, so seq does discriminate, at exactly the door it was declared useless for. So no new pass id is minted. _pp_proxy_seq already increments once per stamped proxy on the sender, and a rank has exactly one upstream, so received seqs are strictly increasing per (epoch, sender). A PP0-issued id beside it would be second bookkeeping next to an existing truth. The fix is a CONSUMER for the counter, not another counter. - take_typed/recv_typed_tensor_dict take an optional accept/on_reject pair. A rejected entry is DROPPED and the drain CONTINUES to the next, ending in 'fall through to the wire'. No branch refuses without a way onward. - The guard is armed for proxies and on the INBOX ONLY. The wire is FIFO and filtering it would be dead weight and the boot-killing direction: a wrong predicate there is a wedge (sgl-project#995 v1 -- 175 refusals on one rid, dead window). - One site advances the mark, downstream of both sources; fed from the inbox alone it would stand still on the common path. - Unreadable stamp, absent seq or absent mark all accept: the guard may only remove what is provably older than a pass already consumed. - Drops are counted and logged loudly (sgl-project#800's rule: a retired message is a lost payload if any consumer did in fact owe it). Red-first on the boot-26 specimen, 9 tests. Two behavioural mutants, both asserted-applied and both killed: guard-disabled (3 failed) and reject-without-a-way-onward (2 failed, the sgl-project#995 v1 death shape as its own can-fail). py_compile green, import smoke binds all five methods and both channel entry points, ruff clean on the touched files. DESK-PROVEN, not boot-proven. Boot 27 is the metal. Analysis: DESIGN_968_PP0_AUTHORITATIVE_STATIC_HALF.md 5b, TICKET_1057.

…our ways it dies are tests The convergence proof established the one place: get_next_batch_to_run + admission run as a COPY on every rank (TP proven by py-spy on a live PP1, PP by scheduler_pp_mixin.py:4402 plus metal #969N ADMIT on PP0 and PP1 in the same pass), and 14 of 16 rank-divergence blockers are one divergent INPUT into it. This removes the divergence at the OUTPUT instead of chasing inputs. WHERE CONGRUENCE IS ACTUALLY LOST, sharpened this session: pp_chunked_local_match argues the geometry is 'CONGRUENT BY CONSTRUCTION' because every rank advances a chunked request through the same add_chunked_req sequence. Sound in phase -- and no coverage at a POST-CUTOVER RE-ADMISSION, where prefix_indices is reseeded from each rank's OWN HiCache hit (schedule_batch.py:1758). Boots 26 and 27 both died within seconds of a cutover. That is the hole. THE RULE: told geometry is the DECISION and is rank-uniform; whether a rank reads its prefix bytes or RECOMPUTES them is EXECUTION and is rank-local. A rank holding less than it was told does more work; it does not present a different batch. NO REFUSAL AND NO VOID EXIST IN THIS MODULE, structurally rather than by discipline -- both were tried and both killed a boot (boot 15/sgl-project#1048: 1448 refusals on one rid until the ring wedged, and sgl-project#1048's own fix deleted that raise; sgl-project#995 v1: 175 refusals, dead window). An overshoot is made HARMLESS instead of impossible, which is also why no coverage feed is needed: the ordered Arm A wanted a MIN over per-rank coverage facts returned on the lap, and those never come home -- observed_local has exactly ONE writer, the dataclass default, because reconcile_pp_admission_decision has 0 call sites and is deliberately dark. Recompute buys the same uniformity at the standing sgl-project#939 one-chunk price without lighting that second mechanism. Absent fact = no adoption, never a local substitute: an older sender, a stand-in or a pass PP0 did not name is bit-for-bit the old behaviour. EVIDENCE -- 12 tests, red-first on boot 27 verbatim (told 4094/4096 against local prefixes 0 / 2596 / 4094 / 13376), and the four danger directions of this seam as executable assertions. All four mutants anchor-asserted and KILLED: 1 refusal-without-a-way-onward (boot-15 shape) -> 7 failed 2 apply-skipped (the tree's present state, i.e. boot 27) -> 4 failed 3 per-rank re-derivation of the uniform decision -> 3 failed 4 shortfall changes the group-visible geometry -> 6 failed ruff clean. Behaviour: none yet -- nothing calls this module. NEXT SLICE, named so it is not re-derived: producer (PP0 stamps its admitted (prefix, extend) into the row it already sends -- _pp_admission_last_built_decision, scheduler.py:10828), carrier (live and relayed verbatim already), consumer (stash at the receive point beside the sgl-project#1058 observation, promote one lap later), apply (the re-admission consult, schedule_batch.py:1718-1790). PP0 applies the LAPPED row like every follower and never its own fresh number -- 632ba36 states that explicitly and it is what keeps PP0 from being the divergent rank.

…t and instrument every writer Boot weg2sn5r produced what LOOKED like a complete trail and was not. The specimen request drew active=26, ping-pong [27,28] and replacement 29, donated 27 to a node -- and then freed [9,8]: station=alloc_pingpong rid=7c0f09f364aa mamba_slot=[27, 28] buf=[27, 28] station=donate rid=7c0f09f364aa donate_idx=0 old=27 new=29 buf_before=[27, 28] station=backup rid=node9 mamba_slot=[27] node=9 station=free rid=7c0f09f364aa mamba_slot=[26] node=19 station=free_pingpong rid=7c0f09f364aa mamba_slot=[9, 8] keep_idx=None buf=[9, 8] Identical on all three ranks; request 2 repeats it and frees [9,26], where 26 is request 1's ACTIVE slot, freed seconds earlier. Slots 28 and 29 are never freed and reach no node -- the orphans, at the measured one-per-long-request rate. The short HEALTH_CHECK control allocs [3,4] and frees [3,4], matched, which is the discriminator for "long multi-chunk only". THE WRITE THAT SWAPPED THE BUFFER IS NOT IN THAT TRAIL, AND MY OWN INSTRUMENT IS WHY. `note_924d` keys on `(subject, station)` and prints ONE line per pair. That is right for a census and wrong for a lifecycle: a request whose ping-pong buffer is allocated TWICE shows a single alloc line, and the second allocation -- the one that strands the first pair -- is dropped by the key. The trail read as complete precisely because the suppressed event was the one being hunted. Same family as the sgl-project#995 prose trap and the denominator law: an absence produced by the instrument's own key is not an absence. Ruled out from this boot: RETRACTION. All 39 `sgl-project#1042 EXTENT LIFECYCLE` lines carry `retract=0`, so `reset_for_retract` did not fire on either request, and the "retracted and re-admitted" hypothesis is dead rather than merely unconfirmed. INSTRUMENT ONLY, no behaviour change: * `note_924d` gains `dedup=False`, which puts a sequence in the key so every occurrence prints (the 8192-pair cap still applies). * the lifecycle stations -- `alloc`, `alloc_pingpong`, `donate`, `free_pingpong` -- are switched to `dedup=False`. * a new `buffer_write` station at every remaining writer of `req.mamba_ping_pong_track_buffer`: the streaming session's park and restore (`streaming_session.py`) and `reset_for_retract` (`schedule_batch.py`), each printing the buffer it is about to replace. Wrapped in try/except at the CALL SITE, because the previous instrument bug proved `note_924d`'s "never raises" does not cover argument construction. ruff: 43 findings across the three touched files, identical on the baseline tree; none added.

…after both went red on prose Both failures were the INSTRUMENT, not the product, and both are the same recorded class one level up. (1) The no-second-inventory pin grepped the module source for 'named_parameters' and matched the module's OWN DOCSTRING, which says in prose that it no longer walks it. That is the sgl-project#995 prose-marker trap: an explanation of what was removed read as the thing itself. It now walks the module's AST and asks which identifiers the CODE uses. (2) The W-code census matched the mint's own provenance COMMENT ('W90 enumerated from ...') and reported a second holder on a tree with exactly one. It now collects non-docstring string literals via the AST, so a W-code written in a comment or a docstring is the code being talked about and never a holder, while 'W_CODE = "W90"' and the launcher help text -- both real holders, both strings -- still count. The launcher pop-list pin moved to the AST in the same pass, unprompted: it was passing, but it was a text scan that would also have matched the comment explaining the pop, so it could have passed on a tree where the entry was gone and only its justification stood. That is sgl-project#1273 B4f's lesson applied before it costs something rather than after.

…he join answers the source side ROOT, verified at 15cf96c and measured on weg2xsn20: the source side does not tear in the resolver, it tears at the question. weight_exchange_shadow's ptr_of (:3336-3344) answers only for THIS rank of THIS group, and the hook chooses which group is asked for the source (:3329-3331); on hook=destination/authoritative that is the PEER, so weight_exchange.py:1811 writes src_ptr=None on every descriptor and weight_exchange_bounce.py:746 turns it into W74. XSN20's POINTER-PROFILE reproduces the split with no exception (source 515/515, authoritative 0/505, destination 0/231; log_grep bare 34 / genuine 34 / prose 0). A process cannot read a peer's data_ptr(). SECOND FINDING, first order: the product has never built a cross-group plan. The inventory is shard_axis=REPLICATED (:3237) and BOTH GroupLayouts are tp_size=1 (:3332-3334) -- the on-card diagonal, as derive_leg_plan's own docstring says (:3121, 'cross-group knowledge no single rank holds'). WHAT IS BUILT, and it duplicates no loader logic: * weg2/xchg_manifest.py -- each rank writes down what its OWN loader decided, after materialisation, keyed by the PUBLISHED identity manifest_entry(...) (weight_exchange_shadow.py:2125, the same tuple card_manifest_entries and seam_digest key on). No header arithmetic, no repack rule, no second walk: the inventory handed in is the one the caller already built. Files land in the shared dump directory (SGLANG_PHASE_FOOTPRINT_DUMP) under the region's OWN boot nonce -- no third env pair for two facts that already have one. * join_manifests -- P and D joined over param_name. The shard AXIS is READ off the join (equal columns + rows summing to the source's = ROWS; the mirror = COLS; identical = REPLICATED; anything else refuses), so the unsharded extent is the destination's rows summed. Refusals only where the join is incomplete: no counterpart = W74, shape contradiction / itemsize / schema version = W68. No new W-code. * plan_from_join -- the destination is a REAL TP group (tp_size=len(cards) with the join's own per-tensor dst_widths, which _blocks_of:1575 consumes), and refuse_diagonal_layout pins it: src_resolved=N/N on a tp_size=1 plan is refused by name, because that is the shape this slice replaces. * --weg2-xchg-legs pp_to_tp|tp_to_pp|both. Default both = byte-identical. Direction is derived from (hook, group) alone, so no new plumbing; the names are IMPORTED from layers/dcp/phase_flip_plan.py:43-44, not retyped. A skipped leg prints WEG2-XCHG-LEGS legs_skipped=<n> reason=direction-knob with both directions -- silence reads like a lane that ran clean. Published by prepare_xchg_env and POPPED by build_env, so an inherited shell value can never halve a boot's legs. TRAP CAUGHT WHILE WIRING: SchedulerWeightUpdaterManager is @DataClass(kw_only=True, slots=True) (:307), so the skip counter may not live on self -- that write would raise AttributeError inside the flip leg, which is sgl-project#1329's own shape. It lives in weight_exchange beside _PROFILE_LEGS, and a test pins it over the AST. EVIDENCE: red-first ImportError / 0 collected on 15cf96c, 30 passed on this tip. Execution smoke scripts/weg2/xchg_manifest_smoke.py drives the product call sites on the real geometries (44,10,10 / 17,7,8 / INT8-W8A8) and reads verdict=PASS checks=13/13 with src_resolved=384/384 over 384 descriptors for 128 tensors, every tensor cut across all three destination ranks, and _missing_pointer finding no hole. Launcher: flag parsed, bad value refused by choices, env published on the armed arm and 0 keys on ring. Two of my own test defects are recorded rather than quietly fixed, same class both times: a source grep for 'data_ptr(' went red on the module DOCSTRING that names it to explain it, and the slots assertion went red on the production COMMENT that names the forbidden write. Both are the sgl-project#995 prose trap inside a unit test; both now read the AST.

…n marker (sgl-project#995 class) [22-fix2]'s W99 explained the transition path with the sentence "a matching key would admit it with a printed WEG2-MODEL-IDENTITY LEGACY line" which quotes the legacy emitter's OWN MARKER inside a REFUSAL. So a census of "how many legacy admissions did this boot make" scores 1 on a log that carries ZERO of them and only this refusal. That is not hypothetical: the train seat read `LEGACY 1` off the default-form dry run and reported it, on a run whose genuine count is 0. Counter-check on the same log, anchored on the log prefix: grep -c "WEG2-LAUNCH WEG2-MODEL-IDENTITY LEGACY" -> 0 (default form) grep -co "WEG2-MODEL-IDENTITY LEGACY" -> 1 (the prose) sgl-project#995's trap, re-created by the file whose whole job this week was to stop instruments from lying. The marker moves out of the prose and the sentence describes the line instead: "a matching key would admit it with a printed legacy-identity line naming the record". The EXPLANATION stays -- dropping it to dodge the census would trade one blindness for another, since an operator reading a bare W99 still cannot otherwise tell a record with a transition path from one without. THE INVARIANT, now pinned: count(lines carrying the LOG-PREFIXED marker) == count(legacy events) and no W99 line carries the marker at all BOTH FORMS ARE FIXTURED, and that is load-bearing rather than thorough: the trap only appears in the form that REFUSES. Red-first on 9cecce5 the new class fails on the DEFAULT-form subtest and passes on the serving-form one -- a test written against the happy form alone would have stayed green straight through the defect. FORM KEYS NAMED IN THE ASSERTIONS, because an rc or a count without its form key is not a number ([16]'s lesson, handed back to this seat by the train seat for exactly this commit). Measured on this box today, on THIS commit: 24eb56724d73 serving form (xsn27 launch.sh argv + --dry-run) rc=0, ARM 1, W99 0, prefixed 2 == bare 2 d0de83d152f0 default form (TREE_ARG=... scripts/weg2/boot_weg2.sh --dry-run) rc=2, W99 1, prefixed 0 == bare 0 Both counts now agree in both forms; before this commit the default form read 0 prefixed against 1 bare. The default form's rc=2 is NOT a sgl-project#1362 finding: it carries W48 Weg2RingFormMismatch (form key d0de83d152f0 != source bceee92dc56b) and has since long before this ticket -- the xsn25 record already states that the desk default-argv dry run refuses where the real boot argv funds, 4.71 GiB apart. Under a form whose ring table came from another boot, the form key is the only identity statement available and it says "different form"; the legacy path is correctly closed there. The fixtures are SYNTHESISED FROM THE CODE, not pasted from a log: the test calls the same `resolve_image_terms` / `refuse_foreign_image` the launcher calls and prefixes the results itself. A pasted log is a photograph of a tree that may since have moved. TESTS: 38 total in this file (+4, one with 2 subtests per form). Red-first on 9cecce5: 3 failed / 3 passed, with both SUBFAILED rows naming form key d0de83d152f0 and neither naming 24eb56724d73. Guard suites (1328/1335/1263 W-code uniqueness): 39 passed. ruff: unchanged, the 6 findings on host_ledger.py are present at the parent.

…n marker (sgl-project#995 class) [22-fix2]'s W99 explained the transition path with the sentence "a matching key would admit it with a printed WEG2-MODEL-IDENTITY LEGACY line" which quotes the legacy emitter's OWN MARKER inside a REFUSAL. So a census of "how many legacy admissions did this boot make" scores 1 on a log that carries ZERO of them and only this refusal. That is not hypothetical: the train seat read `LEGACY 1` off the default-form dry run and reported it, on a run whose genuine count is 0. Counter-check on the same log, anchored on the log prefix: grep -c "WEG2-LAUNCH WEG2-MODEL-IDENTITY LEGACY" -> 0 (default form) grep -co "WEG2-MODEL-IDENTITY LEGACY" -> 1 (the prose) sgl-project#995's trap, re-created by the file whose whole job this week was to stop instruments from lying. The marker moves out of the prose and the sentence describes the line instead: "a matching key would admit it with a printed legacy-identity line naming the record". The EXPLANATION stays -- dropping it to dodge the census would trade one blindness for another, since an operator reading a bare W99 still cannot otherwise tell a record with a transition path from one without. THE INVARIANT, now pinned: count(lines carrying the LOG-PREFIXED marker) == count(legacy events) and no W99 line carries the marker at all BOTH FORMS ARE FIXTURED, and that is load-bearing rather than thorough: the trap only appears in the form that REFUSES. Red-first on 9cecce5 the new class fails on the DEFAULT-form subtest and passes on the serving-form one -- a test written against the happy form alone would have stayed green straight through the defect. FORM KEYS NAMED IN THE ASSERTIONS, because an rc or a count without its form key is not a number ([16]'s lesson, handed back to this seat by the train seat for exactly this commit). Measured on this box today, on THIS commit: 24eb56724d73 serving form (xsn27 launch.sh argv + --dry-run) rc=0, ARM 1, W99 0, prefixed 2 == bare 2 d0de83d152f0 default form (TREE_ARG=... scripts/weg2/boot_weg2.sh --dry-run) rc=2, W99 1, prefixed 0 == bare 0 Both counts now agree in both forms; before this commit the default form read 0 prefixed against 1 bare. The default form's rc=2 is NOT a sgl-project#1362 finding: it carries W48 Weg2RingFormMismatch (form key d0de83d152f0 != source bceee92dc56b) and has since long before this ticket -- the xsn25 record already states that the desk default-argv dry run refuses where the real boot argv funds, 4.71 GiB apart. Under a form whose ring table came from another boot, the form key is the only identity statement available and it says "different form"; the legacy path is correctly closed there. The fixtures are SYNTHESISED FROM THE CODE, not pasted from a log: the test calls the same `resolve_image_terms` / `refuse_foreign_image` the launcher calls and prefixes the results itself. A pasted log is a photograph of a tree that may since have moved. TESTS: 38 total in this file (+4, one with 2 subtests per form). Red-first on 9cecce5: 3 failed / 3 passed, with both SUBFAILED rows naming form key d0de83d152f0 and neither naming 24eb56724d73. Guard suites (1328/1335/1263 W-code uniqueness): 39 passed. ruff: unchanged, the 6 findings on host_ledger.py are present at the parent. (cherry picked from commit 9c22254)

…the type and not in the CLI weg2xsn27 loaded the vision tower on all six ranks and replicated it threefold on P: 879 MiB per P card in the manifest, 2.63 GiB across the host ring, measured. No Weg-2 form serves images. THE FLAG THE ORDER FIRST NAMED WOULD HAVE KILLED THE LAUNCH. `--language-only` is the encoder-disagg RECEIVER: `server_args.py:17996` refuses it without `--encoder-urls`, and `:6288` runs the EncoderBootstrapServer the receivers fetch from. Setting it on P and D drops no tower; it raises at startup. The zero hits in launcher.py were not an oversight. AND THE REAL OFF-SWITCH WAS NOT SPELLABLE. `enable_multimodal` is declared `Optional[bool]` and its `False` branch is live -- `model_config.py:436-447`, read at `server_args.py:14112` (`is_multimodal and not language_only`) -- but the CLI generated a bare `store_true` with `nargs=0`. So `False` existed in the type and in the config and NO CALLER COULD SAY IT: `--enable-multimodal` could only ever turn it ON, and passing nothing is what every boot did. That is the defect this commit fixes, and it is one line of shape rather than behaviour: `enable_multimodal` becomes `argparse.BooleanOptionalAction`, the same form six other fields in this file already use. The tri-state is UNCHANGED -- None (auto), True (force on), False (force off) -- and only the missing spelling `--no-enable-multimodal` is added. No model-name list: adding the architecture to `mm_disabled_models` would be the sgl-project#1362 fossil class, a constant with no model reference. THE KNOB. `--weg2-vision off|resident`, default `off`, emits `--no-enable-multimodal` into `common_flags` -- so BOTH groups, from the one builder they share, and `resident` is byte-identical to every boot through weg2xsn27. `resident` is kept for the day the tower becomes a 7th rank transient between the flips; that is sgl-project#1356 proper and NOT this slice. IMAGES ARE REFUSED BY NAME, not routed. `handle_generate` answers `W101 Weg2VisionRefused` with 501 (well-formed request, not implemented by this deployment -- a 400 would blame the caller's input). The count is STRUCTURAL, over `messages[].content[]` parts whose `type` is an image, never a string search: a prompt that merely contains the words "image_url" is text and routes normally. That is the sgl-project#995 prose trap one layer up, and a test pins it. The refusal names the way out, because upstream's own words for the alternative are that "image inputs would run an uninitialized vision tower" -- i.e. the failure mode is plausible WRONG TEXT, not an exception, which is the one shape that must never be reached silently. THE FORM-KEY CONSEQUENCE, NAMED AND MEASURED: off=5f4984090baf, resident=be96c42cdd6b. The flag is in the argv and deliberately NOT in FORM_KEY_EXCLUDED_FLAGS -- a tower that is absent IS a different weight statement. So the FIRST boot of the text-only form has no same-form predecessor, solves its ring from ITS OWN stem, and **W48 Weg2RingFormMismatch is EXPECTED once -- not W20**. Inheriting the multimodal form's Sigma H would price 2.63 GiB of tower this form does not hold. TESTS (10, red-first on 7d7e549: 8 failed / 2 passed). Among them: the tri-state has all three spellings; `--language-only` is pinned as NOT the mechanism, so the refusal that sent this slice back once cannot be undone quietly; `resident` emits neither multimodal flag; the two forms hash differently and the flag is not excluded from the key; and the prose trap. Dry-run smoke of the argv builder: off carries the flag, resident does not. ruff: 358 findings on server_args.py against 359 at the base -- one fewer, none new. (cherry picked from commit 2cc618b)

test: negative value testing for frequency, presence penalizers

182bdf6

Ying1123 merged commit 95a2801 into sgl-project:main Aug 9, 2024

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025

test: negative value testing for frequency, presence penalizers (sgl-…

74123c3

…project#995)

Navigation Menu

Uh oh!

test: negative value testing for frequency, presence penalizers - #995

test: negative value testing for frequency, presence penalizers#995
Ying1123 merged 1 commit into
sgl-project:mainfrom
tmfi-us:ryan/test/srt/sampling/penaltylib/add-negative-test-case

vhain commented Aug 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

vhain commented Aug 9, 2024

Motivation

Modification

Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants