[BugFix] Align micro-slicing budgets with the prefill chunk grid

[logprobz]

Micro-slicing can divide a usable prefill budget into slices that _mamba_block_aligned_split reduces to zero. With 2048-token scheduler/cache blocks, 256-token recurrent blocks, and three running prefills, a 4096-token budget is split into 1536/1280/1280-token slices. None can advance.

This draft includes the chunk aligner's grid in the allocation quantum. The same budget then supplies two usable 2048-token slices. A mixed-prefill budget such as 2304 is rejected by the existing validation rather than accepted with an incompatible quantum.

Scope and relation to other PRs

This is a standalone 12-line runtime change against integration/glm53-r26-lmcache-expandable-20260905. It does not depend on the cache-checkpoint follow-up and does not change cache retention or state alignment rules.

#647 is closed. Current #648 replaces the micro-slicing path with automatic compute sharing and interleaved lanes. This draft fixes the path still present in the r26 base; it does not duplicate #648. If #648 lands first, maintainers should either retain this policy explicitly or port the allocation invariant to its replacement. The #648 branch has not been GPU-qualified with this patch.

The local LP26 screen found useful short-prompt convoy gains from interleaving, but the tested automatic profiles lost about 40% of median per-request decode compared with LP15. A micro-slicing profile passed the no-decode-loss gate, so preserving a working option is useful while the replacement is qualified. This is workload-specific evidence, not a claim that micro-slicing is universally better.

Behavior

• Include the cache block width when the chunk aligner requires an aligned step.
• Preserve sub-block progress when the global batch budget or long-prefill threshold is itself smaller than one block, matching the existing aligner exception.
• Leave non-Mamba scheduling and global defaults unchanged.
• Reuse existing round-robin allocation and validation. Do not hard-code this station's sequence cap, decode burst, or wait threshold.

The new tests construct real schedulers. They check actual split results for 2048/4096 budgets, reject the incompatible 2304 budget, and preserve both global-budget and long-prefill-cap sub-block exceptions. The three bug assertions fail on the unchanged r26 base.

Validation

CPU command in the LP26 runtime dependency environment, with this source checkout mounted and no GPU devices:

pytest tests/v1/core/test_scheduler.py tests/v1/core/test_micro_slicing.py tests/v1/core/test_compute_fairness.py tests/v1/core/test_mamba_align_chunk_split.py -q

221 passed. No tests were skipped to accommodate this change.

Ruff check, Ruff format, and git diff whitespace checks pass for the changed files.

This automatic-quantum patch has not been installed into production or GPU-qualified. Production instead uses an explicitly compatible 4096 mixed-prefill budget. That complete LP26 profile, including other cache/runtime changes, passed a fresh comparison against LP15: +6.7% aggregate output at 16 clients, +7.8% at 32, +4.4% median request decode, and p95 TTFT within the unchanged 5% limit. It also passed long-context cached retrieval, tools/parser/vision checks, and 64 agent continuation turns. Those results motivate the invariant; they are not a performance claim for this code change.

Keep this draft pending maintainer agreement on #648 policy compatibility and a GPU replay of the final source.

AI assistance: OpenAI Codex prepared this change and ran the recorded checks at the submitter's request. Human line-by-line review is pending before marking the draft ready.

Superseded

Superseded by #669, now merged on dev/jovian-judgement. The coordinated port preserves endpoint checkpoint guards and includes the cache, scheduler, connector and event repairs from this earlier branch. The combined release passed full MTP3 and DFlash2 serving qualification and was promoted, with zero preemptions and all LP26 5% performance gates satisfied. Closing this older proposal to avoid duplicate integration.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Status: superseded by PR #669 and intentionally closed.

PR #669 consolidates the aligned prefill-budget correction from this pull request, preserves the contributing authorship in commit trailers, and was merged into dev/jovian-judgement as 86acae9. Do not merge or cherry-pick PR #656 separately.