r20: simplify prefill compute sharing and balance GLM cache groups

[yatesdr]

Summary

This is the clean follow-up for the exact jovian-judgement-community-20260903-r20 source branch.

• replace the two-engine fairness selector with one optional --prefill-compute-share {FLOAT|auto} control
• remove the unsuccessful micro-slicing implementation and its tuning surface
• preserve stock scheduler behavior when the flag is omitted
• start adaptive mode at 0.4, retaining bounded, pressure-aware adjustment and idle-only runtime switching
• carry the weighted GLM-5.3 split-cache allocator from fix(kv-cache): balance GLM split groups by memory cost #603, which was not included in r20

The scheduler and allocator changes remain separate commits so they can be reviewed or integrated independently.

Scheduler contract

--prefill-compute-share is the complete public control:

• omitted: stock scheduling, with no controller or request-pressure scans
• numeric value strictly between zero and one: fixed weighted compute share
• auto: adaptive weighted compute share, initialized at 0.4

The development GET/POST /prefill_fairness API remains available for idle-only changes without a model reload. Removed selector and micro-slicing fields are rejected instead of silently ignored. Compute-share mode remains incompatible with prefill_schedule_interval > 1 and unsynchronized data parallelism.

This supersedes the r18-oriented composition in #627 for r20 integration while preserving its measured-runtime controller design. It also replaces the r20 copy of #625's selectable-engine surface with the simpler single-control contract.

Weighted GLM cache allocation

The #603 commits search a bounded set of compatible GLM-5.3 split-cache groupings and minimize maximum-request shared-pool memory cost, with fewer groups as the deterministic tie-breaker.

On the r20 DCP4 unit geometry:

Cache dtype	Uniform cost	Weighted cost	Change
fp8_ds_mla	592,846,848 bytes	592,846,848 bytes	unchanged
nvfp4_ds_mla	579,108,864 bytes	457,205,760 bytes	-21.05%

Physical page formats, cache keys, and logical cache geometry are unchanged.

Validation

Exact base: 7015eb6949a93247df02fb6f9101d17c40bd83e8.

CN4 Linux source/unit validation passed 65 focused tests:

• 20 compute-share controller tests
• 24 scheduler/configuration tests, including CLI, external-cache accounting, runtime switching, and DP rejection
• 10 engine timing/runtime API tests
• 5 HTTP API tests
• 6 GLM-5.3 FP8/NVFP4 cache-layout tests

Ruff and git diff --check pass on the touched source.

Known qualification limits

• Full r20 GPU traffic characterization of adaptive mode is still pending.
• The allocator's r20 capacity increase has source/unit validation here; its previously reported live FP8/NVFP4 capacity evidence came from the fix(kv-cache): balance GLM split groups by memory cost #603 combined candidate, not this exact r20 replay.
• This PR does not address the newly reported r20 TP4/DCP2/MTP3 output-corruption investigation; that is being isolated separately.

Related work

• Weighted allocator: Reduce GLM-5.3 split KV pool waste with memory-weighted cache groups #602, fix(kv-cache): balance GLM split groups by memory cost #603
• Fairness consolidation and auto mode: JJ R18: consolidate selectable decode/prefill fairness engines #624, feat(scheduler): add selectable prefill fairness engines #625, [Feature]: Add automatic prefill compute-share tuning #626, feat(scheduler): add automatic prefill compute-share tuning #627

OpenAI Codex assisted with the r20 replay, simplification, conflict resolution, source review, and focused validation. The submitted commits and test evidence were reviewed under maintainer direction.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• dev/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 43936ea1-79b2-4f6e-b3dd-1996a40dd7d3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[yatesdr]

Superseded. The weighted cache allocator is already present in the R26 source line, and the fairness portion has been replaced by the simpler combined scheduler implementation in #648.