scheduler: integrate #246, #221 and #180 as decode-floor v7 (#283)

CHANGELOG.md

@@ -247,6 +247,40 @@ There were no git tags for 1.0.0–1.4.0; 1.5.0 is the first cut named as a rele
 
 ### Fixed
 
+- Fair-prefill scheduler integration (#283): `overlay/patch_scheduler_decode_floor.py`
+  moves to decode-floor **v7**, combining the three scheduler directions of
+  #246 (@beastllama), #221 (@surlebeat) and #180 (@krunkosaurus) in one patcher.
+  - A selected WAITING request that vLLM refuses KV allocation no longer stalls
+    a runnable prefill. It keeps being retried and recovers once
+    memory frees (#246).
+  - Explicit request priorities still rank fair-prefill candidates under
+    `--scheduling-policy priority`. FCFS is unchanged (#221).
+  - Installer migration follows #180's fail-closed contract. Pristine, v1, v2,
+    v5 and v5+priority (the #221 layout on `main` until now) installations
+    migrate to the same bytes as a fresh v7 install, and a re-apply is a
+    verified no-op. v3/v4 and unknown version markers, drifted, duplicated,
+    marker-only and unmarked helpers, and a duplicate policy wrapper are
+    refused without writing.
+  - `overlay/patch_mamba_align_chunking.py` accepts decode-floor v7 as well
+    as v5.
+  - #180's concurrency canary (`tests/check_concurrent_agents.py`) and its CPU
+    progress regressions (`tests/test_prefill_concurrency.py`) land with it.
+
+  Qualified live on a TP2 pair with a pre-registered baseline → candidate →
+  baseline run (installed scheduler `97c90a18…`, 2026-09-30):
+  - PASS: KV-blocked admission progress, the #180 contention canary (5/5 per
+    arm), TheGrill routine prefill, and the CPU migration matrix.
+  - Inconclusive, because the two baseline runs differ from each other by more
+    than the frozen tolerance: decode, concurrency and mixed TheGrill cells,
+    and temperature-0 output identity. No TheGrill gate regressed.
+  - Known limitation under `--scheduling-policy priority`: a short
+    high-priority prompt can wait about 85 s behind a larger KV-blocked
+    request of the same priority. This is tracked separately.
+  - `start-tp3.sh` and `start-tp4.sh` load the same two overlays, but only TP2
+    was qualified live.
+  - The proposed TP=2 `CHUNK=0` default is **not** part of this change;
+    launcher defaults are unchanged.
+
 - `start-tp4.sh` forwards `EXL3_FAT_GROUPED` to all four ranks (#286) and
   defaults it to `1` (E3) with the coupled `EXL3_TEMP_ROWS_FUSED` default (32
   with E3, 256 with E2), matching `start.sh` and `start-tp3.sh`. Before, the

Dockerfile

@@ -485,7 +485,7 @@ COPY tests/test_indexer_workspace.py tests/test_indexer_warmup_range.py /opt/glm
 COPY overlay/patch_tool_choice_none.py /opt/glm53/patch_tool_choice_none.py
 COPY overlay/patch_loadclone.py /opt/glm53/patch_loadclone.py
 COPY tests/test_loadclone.py /opt/glm53/test_loadclone.py
-COPY tests/fixtures/loadclone_weight_utils.py.txt tests/fixtures/kpool_tail_seed_kernel-487ecf187.py.txt tests/fixtures/kpool_tail_seed_kernel-db1bfdd.py.txt /opt/glm53/fixtures/
+COPY tests/fixtures/loadclone_weight_utils.py.txt tests/fixtures/kpool_tail_seed_kernel-487ecf187.py.txt tests/fixtures/kpool_tail_seed_kernel-db1bfdd.py.txt tests/fixtures/legacy_scheduler_helpers.py /opt/glm53/fixtures/
 COPY tests/test_tool_choice_none.py /opt/glm53/test_tool_choice_none.py
 COPY overlay/ablit_runtime.py /opt/glm53/ablit_runtime.py
 COPY overlay/patch_ablit.py /opt/glm53/patch_ablit.py

docs/REFERENCE.md

@@ -854,7 +854,7 @@ uses the resolved scheduler LCM for shared checkpoints, not the drafter's
 smaller block, and preserves smaller Mamba private-state boundaries. EAGLE
 back-off applies only when a participating non-SWA group needs it. Alignment
 capacity is bounded by the actual grant so positive sub-block grants progress.
-The launchers apply the chunking overlay after decode-floor v5.
+The launchers apply the chunking overlay after decode-floor (v5 or v7).
 
 The GPU receipts below qualify the retained PR238 implementation. They do
 not automatically qualify the combined loader and fine-grained APC changes.
@@ -868,7 +868,7 @@ one superseded state per concurrent batch. `patch_mamba_align_chunking.py`
 uses the Mamba group's actual block for checkpoint alignment, not the
 drafter's smaller block, and applies EAGLE back-off only when the full-attention
 group needs it. Sub-block token caps still make progress. The launchers apply
-the chunking overlay after decode-floor v5.
+the chunking overlay after decode-floor (v5 or v7).
 
 **Installer compatibility.** The public InstantTensor image carries a legacy
 `glm53-hybrid-apc` coordinator without the current v3 verification form.
@@ -2001,8 +2001,8 @@ After CUDA compile, Python overlay edits (`overlay/exl3.py`, tests) are a cheap
 | `tests/test_draft_kv_compact.py` | CPU geometry/alignment checks and pinned-source allocator, backend rejection, idempotence, two-file preflight, and prefix-cache lookup (coordinator + managers) tests |
 | `overlay/patch_glm_video_placeholders.py` | align video timestamp blocks to encoder `grid_t` |
 | `overlay/patch_suppress_stops_in_reasoning.py` | fail-closed detokenizer guard: client `stop` dormant until `</think>` |
-| `overlay/patch_scheduler_decode_floor.py` | skip / cap / off / `fair` mixed-prefill; v5 fixed-cost step fit + largest step-fitting chunk + bounded contention credit, decode first; versioned installer |
-| `tests/test_scheduler_decode_floor.py` | v5 migration from v1/v2/v3/v4; cost fit, ladder climb-back, prompt probe, async accounting, bounded credit, alignment and actual scheduler budget regressions |
+| `overlay/patch_scheduler_decode_floor.py` | skip / cap / off / `fair` mixed-prefill; v7 (#283): v5 fixed-cost step fit + largest step-fitting chunk + bounded contention credit, decode first, plus KV-blocked admission progress (#246) and priority tiers (#221); authenticated installer migration (#180) |
+| `tests/test_scheduler_decode_floor.py` | v7 migration from pristine/v1/v2/v5/v5+priority, refusal of v3/v4 and drifted installs; cost fit, ladder climb-back, prompt probe, async accounting, bounded credit, alignment and actual scheduler budget regressions |
 | `overlay/patch_xgrammar_termination.py` | source-exact vLLM #52805/#53046 backports; stop at termination and validate post-reasoning speculative drafts before FSM advance |
 | `tests/test_xgrammar_termination.py` | exact two-file patch, idempotence, cross-file fail-closed drift, termination/rollback/reset and post-reasoning draft behavior, launcher wiring |
 | `overlay/patch_cache_reset.py` | mount only the upstream cache-reset dev router (`/reset_prefix_cache` et al., #31) when `GLM53_EXPOSE_CACHE_RESET=1`; runtime-mounted by `start.sh` (`CACHE_RESET_PATCH_HOST`) |
@@ -2128,7 +2128,7 @@ then-unmerged overlay stack per-group retention + fine-grained hits + gate v2
 main carries per-KV-cache-group retention via merged #130, where an empty
 `GLM53_APC_RETENTION_INTERVAL_SWA` inherits the global retention interval instead of
 applying #83's automatic rule. #84's overlay patch and the #80 gate-v2 knobs are not
-in main. Current TP=2 main defaults to `GLM53_MIXED_PREFILL_CHUNK=fair` (v5);
+in main. Current TP=2 main defaults to `GLM53_MIXED_PREFILL_CHUNK=fair` (decode-floor v7);
 the historical gate-v2 measurements below do not qualify that policy.
 
 | ctx 50K per lane (distinct prefixes, verified warm) | ×1 | ×2 | ×4 | ×8 | ×16 |

overlay/patch_mamba_align_chunking.py

@@ -9,7 +9,8 @@
 The decode-floor overlay owns per-request caps; this overlay owns alignment,
 positive-grant progress and checkpoint backoff. Their anchors do not overlap,
 so either application order is supported. Decode-floor versions before v5 are
-rejected because they do not preserve per-request alignment caps.
+rejected because they do not preserve per-request alignment caps; v5 and
+v7 (#283) are accepted.
 
 Pristine sources and exact retained v1 installations migrate atomically.
 Partial or drifted installations fail before writing.
@@ -31,6 +32,8 @@
 LEGACY_MARK = "# [glm53-mamba-align-chunking-v1]"
 DECODE_FLOOR_MARK = "# [glm53-decode-floor"
 DECODE_FLOOR_V5 = "# [glm53-decode-floor:v5]"
+# v7 (#283) keeps v5's per-request alignment-cap hooks byte-identical apart from the marker.
+DECODE_FLOOR_V7 = "# [glm53-decode-floor:v7]"
 
 IMPORT_OLD = """from vllm.v1.kv_cache_interface import KVCacheConfig
 """
@@ -167,7 +170,7 @@ def main() -> int:
     if not P.is_file():
         raise SystemExit(f"missing {P}")
     original = text = P.read_text()
-    if DECODE_FLOOR_MARK in text and DECODE_FLOOR_V5 not in text:
+    if DECODE_FLOOR_MARK in text and DECODE_FLOOR_V5 not in text and DECODE_FLOOR_V7 not in text:
         raise SystemExit(
             f"{P}: patch_scheduler_decode_floor.py older than v5 present; its "
             "per-request cap is not applied to the Mamba alignment"