fix: do not default fair mixed-prefill on TP3 or TP4

.env.example

@@ -254,9 +254,10 @@ GLM53_SUPPRESS_STOPS_IN_REASONING=1
 # (issue #6). skip = decode-only steps (starves newcomers until A finishes);
 # N>0 = cap mixed prefill tokens (alignment-aware); 0 / off = no isolation;
 # fair = time-share (decode-only stretches + bounded prefill turns). Default
-# on TP=2/3/4 since 2026-09-15 (v5; docs/diditfix.md). Set skip to restore
-# decode-only isolation. v5: fixed+per-token step-cost fit, largest chunk
-# that fits MAX_STEP_MS, prompt step-bounded probe, bounded contention credit.
+# on TP=2 since 2026-09-15 (v5; docs/diditfix.md). TP=3 stays 0; TP=4 stays skip.
+# Set skip to restore decode-only isolation. v5: fixed+per-token step-cost fit,
+# largest chunk that fits MAX_STEP_MS, prompt step-bounded probe, bounded
+# contention credit.
 GLM53_MIXED_PREFILL_CHUNK=fair
 # Used when CHUNK=fair. Identical on every rank. CHUNK well below MNBT so
 # decode rows still fit the mixed step. 128/3584 alignment is unsafe without

.env.tp3.example

@@ -51,9 +51,10 @@ NCCL_DEBUG=WARN
 # --- serve (TP=3) --------------------------------------------------------
 TP=3
 NNODES=3
-# Mixed-prefill while a peer is decoding (issue #6). fair is the default on
-# every launcher; skip restores decode-only isolation. TP=3 fair is unmeasured.
-GLM53_MIXED_PREFILL_CHUNK=fair
+# Mixed-prefill while a peer is decoding (issue #6). 0 = off is the TP=3
+# default (fair is unmeasured here). This file is sourced after .env, so the
+# explicit 0 wins over a TP=2 .env that sets fair. Set fair to opt in.
+GLM53_MIXED_PREFILL_CHUNK=0
 GLM53_FAIR_PREFILL_CHUNK=256
 GLM53_FAIR_PREFILL_SHARE=0.20
 GLM53_FAIR_PREFILL_MAX_INTERVAL_MS=2000

.env.tp4.example

@@ -73,11 +73,12 @@ CONTAINER_WORKER3=glm53-exl3-tp4-w3
 #   MAX_NUM_BATCHED_TOKENS  1024/2048/4096/8192: cold prefill on TP=4 is the same
 #                  (fabric-bound, ~1.15k tok/s at 282k), 8192 makes the first
 #                  token of a coding request 20 s under a 24k prefill. Keep 2048.
-#   GLM53_MIXED_PREFILL_CHUNK  fair is the default (same as TP=2/3). skip makes
-#                  a short question wait for the whole running generation
-#                  (330 s behind a 12k answer); off answers it in 0.5 s and the
-#                  running decode keeps ~90 % of its speed (TP=4 receipt; fair
-#                  itself is unmeasured on TP=4).
+#   GLM53_MIXED_PREFILL_CHUNK  skip is the TP=4 launcher default (fair is
+#                  unmeasured here). skip makes a short question wait for the
+#                  whole running generation (330 s behind a 12k answer); off
+#                  answers it in 0.5 s and the running decode keeps ~90 % of
+#                  its speed. This file is sourced after .env, so the explicit
+#                  value wins over a TP=2 .env that sets fair.
 #   SPEC_METHOD / DFlash2 on TP=4 (2026-09-03): a 150-min soak at the 8-seq cap with
 #                  96k prompts in the mix hung the engine 3/3 times (31-78 min) with the
 #                  draft on; the fatal step was a 96k chunked prefill sharing steps with
@@ -98,7 +99,7 @@ GPU_MEM_UTIL=0.75
 MAX_NUM_SEQS=8
 DFLASH_TOKENS=3
 MAX_NUM_BATCHED_TOKENS=2048
-GLM53_MIXED_PREFILL_CHUNK=fair
+GLM53_MIXED_PREFILL_CHUNK=skip
 GLM53_FAIR_PREFILL_CHUNK=256
 GLM53_FAIR_PREFILL_SHARE=0.20
 GLM53_FAIR_PREFILL_MAX_INTERVAL_MS=2000

README.md

@@ -877,7 +877,7 @@ that are now documented/enforced:
 | `KV_CACHE_DTYPE` | `fp8` | packed `fp8_ds_mla`; not `nvfp4`, not bf16 |
 | `DEFAULT_MAX_NEW_TOKENS` | `65536` | Omitted-only output-token default (`1..1000000`) for chat and completion requests, implemented by `overlay/patch_default_max_new_tokens.py`. Explicit `max_tokens`/`max_completion_tokens` overrides this default; independent server, platform and remaining-context caps still apply. Empty preserves stock model/server defaults and caps. Does not reserve admission capacity or fix long-prefill contention; admission is chunk-based. Caller exports (including empty) override `.env`. TP=2 launcher only; `start-tp4.sh` is unchanged. |
 | `GLM53_APC_RETENTION_INTERVAL_SWA` | *(unset)* | TP=2 DFlash2 drafter retention. Empty inherits global retention with ordinary priority; explicit `0` keeps reachable boundaries and enables draft-only eviction priority; positive values must be multiples of 3584, at most 1,000,000. Requires `SPEC_METHOD=dflash` and the hybrid prefix overlay. TP=4 rejects a non-empty value. Qualify retention, branching, and draft acceptance for the chosen global/SWA pair; see [measurements](docs/apc-retention-qualification.md) |
-| `GLM53_MIXED_PREFILL_CHUNK` | `fair` (`start.sh`, `start-tp3.sh`, `start-tp4.sh`, `.env.example`, `.env.tp3.example`, `.env.tp4.example`) | Mixed-prefill policy while a peer decodes. **`skip` starves prefills until decode ends** (the reported multi-minute newcomer freeze). `N>0` caps mixed chunks with hybrid alignment support; `0`/`off` disables isolation (admits newcomers in ~1 s but collapses the incumbent 10–36× on TP=2). `fair` v5 allocates decodes first, charges only prefill that contends with a decoder, fits a fixed-plus-per-token step cost, runs the largest chunk that fits `GLM53_FAIR_PREFILL_MAX_STEP_MS`, and gives a newcomer one prompt probe. Measured on TP=2 (reporter recipe, thinking essay at ~24 tok/s): 2k newcomer first token ~12 s, 30k newcomer ~164 s while the essay still streams, incumbent keeps ~80–90% of its in-run rate. TP=3 and TP=4 inherit the same default; those topologies were not re-measured. See [receipts](docs/diditfix.md) and [design](docs/astra-fix.md). |
+| `GLM53_MIXED_PREFILL_CHUNK` | `fair` on `start.sh` / `.env.example`; **`0` on `start-tp3.sh` / `.env.tp3.example`**; **`skip` on `start-tp4.sh` / `.env.tp4.example`** | Mixed-prefill policy while a peer decodes. **`skip` starves prefills until decode ends** (the reported multi-minute newcomer freeze). `N>0` caps mixed chunks with hybrid alignment support; `0`/`off` disables isolation (admits newcomers in ~1 s but collapses the incumbent 10–36× on TP=2). `fair` v5 allocates decodes first, charges only prefill that contends with a decoder, fits a fixed-plus-per-token step cost, runs the largest chunk that fits `GLM53_FAIR_PREFILL_MAX_STEP_MS`, and gives a newcomer one prompt probe. Measured on TP=2 (reporter recipe, thinking essay at ~24 tok/s): 2k newcomer first token ~12 s, 30k newcomer ~164 s while the essay still streams, incumbent keeps ~80–90% of its in-run rate. TP=3 and TP=4 stay off-fair until measured. See [receipts](docs/diditfix.md) and [design](docs/astra-fix.md). |
 | `GLM53_FAIR_PREFILL_CHUNK` | `256` | Probe chunk until timing samples exist. Afterwards fair v5 fits a fixed-plus-per-token step cost from solo and mixed samples and targets the largest ladder rung (128..2048) whose estimated step fits `GLM53_FAIR_PREFILL_MAX_STEP_MS`, saving credit for it instead of spending on small chunks (every prefill-bearing step costs ~0.3 s fixed on this kit, so 128-token steps ran at ~70 tok/s under v4). Base scheduler token/input and long-prefill caps still apply. |
 | `GLM53_FAIR_PREFILL_SHARE` | `0.20` | Credit accrual fraction of accounted busy engine wall time (`0..1`), using a host timing proxy. Whole mixed-step cost is charged; queued async spans are counted once. |
 | `GLM53_FAIR_PREFILL_MAX_INTERVAL_MS` | `2000` | Age at which an already-served prefill may borrow one step-bounded chunk after all shared debt is repaid; a never-served newcomer gets that probe promptly. Resource, credit, and step limits can defer service beyond this age. |

docs/diditfix.md

@@ -16,7 +16,7 @@ A ~30k unique newcomer on the same recipe also finished **during** the essay (**
 
 Incumbent decode did **not** collapse to the reporter’s 0.8–2.8 tok/s under `CHUNK=0`. On the essay, overlap rate stayed **~20.6–20.9 tok/s** vs ~24.7 solo (about **1.18–1.20×**). In-run, vs the 10 s of decode before B: **1.06×** (2k B) and **1.21×** (30k B).
 
-`start.sh`, `start-tp3.sh`, `start-tp4.sh`, `.env.example`, `.env.tp3.example`, and `.env.tp4.example` default `GLM53_MIXED_PREFILL_CHUNK` to **`fair`**. Revert with `skip`. TP3 and TP4 were not retested here.
+`start.sh` and `.env.example` default `GLM53_MIXED_PREFILL_CHUNK` to **`fair`**. TP=3 stays **`0`**; TP=4 stays **`skip`**. Opt in on those launchers with `fair`.
 
 | Claim | `skip` | v4 `fair` (prior) | **v5 `fair` (this retest)** |
 |---|---|---|---|

overlay/patch_scheduler_decode_floor.py

@@ -15,8 +15,8 @@
                The cap is fed into hybrid Mamba alignment so N < block_size
                still makes sub-block progress. 128 still stalls ~10 tok/s.
   0 / off    — disable the extra isolation policy.
-  fair       — service-time mixing (default on TP=2/3/4 since 2026-09-15,
-               v5). Decode-only
+  fair       — service-time mixing (TP=2 default since 2026-09-15, v5;
+               opt-in on TP=3/TP=4). Decode-only
                steps between prefill turns; at most
                GLM53_FAIR_PREFILL_MAX_CHUNKS chunks per turn (default 1).
                Only prefill that contends with a decoder is charged (solo

start-tp3.sh

@@ -382,10 +382,10 @@ READY_TIMEOUT="${READY_TIMEOUT:-3600}"
 # 1 = suppress client stop strings until </think> (DSpark #42 class).
 GLM53_SUPPRESS_STOPS_IN_REASONING="${GLM53_SUPPRESS_STOPS_IN_REASONING:-1}"
 # Mixed-step prefill policy when a peer is already decoding (issue #6).
-# fair = time-share mixing (default since 2026-09-15, overlay v5);
-# skip = do not mix; N>0 = cap tokens; 0 / off = no isolation.
+# 0 = off (TP=3 default; fair is unmeasured on this topology);
+# skip = do not mix; N>0 = cap tokens; fair = time-share mixing (opt-in).
 # Fair knobs are forwarded on every rank even when CHUNK is not fair.
-GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-fair}"
+GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-0}"
 GLM53_FAIR_PREFILL_CHUNK="${GLM53_FAIR_PREFILL_CHUNK:-256}"
 GLM53_FAIR_PREFILL_SHARE="${GLM53_FAIR_PREFILL_SHARE:-0.20}"
 GLM53_FAIR_PREFILL_MAX_INTERVAL_MS="${GLM53_FAIR_PREFILL_MAX_INTERVAL_MS:-2000}"

start-tp4.sh

@@ -279,10 +279,10 @@ READY_TIMEOUT="${READY_TIMEOUT:-3600}"
 # 1 = suppress client stop strings until </think> (DSpark #42 class).
 GLM53_SUPPRESS_STOPS_IN_REASONING="${GLM53_SUPPRESS_STOPS_IN_REASONING:-1}"
 # Mixed-step prefill policy when a peer is already decoding (issue #6).
-# fair = time-share mixing (default since 2026-09-15, overlay v5);
-# skip = do not mix; N>0 = cap mixed prefill tokens; 0 / off = no isolation.
+# skip = do not mix (TP=4 default; fair is unmeasured here);
+# N>0 = cap mixed prefill tokens; 0 / off = no isolation; fair = time-share.
 # Fair knobs are forwarded on every rank even when CHUNK is not fair.
-GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-fair}"
+GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-skip}"
 GLM53_FAIR_PREFILL_CHUNK="${GLM53_FAIR_PREFILL_CHUNK:-256}"
 GLM53_FAIR_PREFILL_SHARE="${GLM53_FAIR_PREFILL_SHARE:-0.20}"
 GLM53_FAIR_PREFILL_MAX_INTERVAL_MS="${GLM53_FAIR_PREFILL_MAX_INTERVAL_MS:-2000}"

tests/test_numeric_config.py

@@ -183,15 +183,24 @@ def run(extra: dict[str, str]) -> subprocess.CompletedProcess[str]:
         source = launcher.read_text()
         assert 'GLM53_FAIR_PREFILL_MAX_STEP_MS="${GLM53_FAIR_PREFILL_MAX_STEP_MS:-1000}"' in source
         assert '-e "GLM53_FAIR_PREFILL_MAX_STEP_MS=$GLM53_FAIR_PREFILL_MAX_STEP_MS"' in source
-        assert 'GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-fair}"' in source
+        if launcher.name == "start-tp3.sh":
+            assert 'GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-0}"' in source
+        elif launcher.name == "start-tp4.sh":
+            assert 'GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-skip}"' in source
+        else:
+            assert 'GLM53_MIXED_PREFILL_CHUNK="${GLM53_MIXED_PREFILL_CHUNK:-fair}"' in source
         guard = guard_source(launcher)
         for value, expected in (("1000", 0), ("0", 1)):
             script = guard + '\nGLM53_FAIR_PREFILL_MAX_STEP_MS="$1"\n' + '_glm53_canonical_positive_int GLM53_FAIR_PREFILL_MAX_STEP_MS "$GLM53_FAIR_PREFILL_MAX_STEP_MS" 600000\n'
             checked = subprocess.run(["bash", "-c", script, "test", value], capture_output=True, text=True)
             assert bool(checked.returncode) == bool(expected), (launcher, value, checked.stderr)
-    for env_example in (ROOT / ".env.example", ROOT / ".env.tp3.example", ROOT / ".env.tp4.example"):
+    for env_example, chunk in (
+        (ROOT / ".env.example", "fair"),
+        (ROOT / ".env.tp3.example", "0"),
+        (ROOT / ".env.tp4.example", "skip"),
+    ):
         text = env_example.read_text()
-        assert "GLM53_MIXED_PREFILL_CHUNK=fair" in text, env_example
+        assert f"GLM53_MIXED_PREFILL_CHUNK={chunk}" in text, env_example
 
 
 def test_restart_validates_before_stop() -> None: