Escha-W2 dense + OSCAR INT2 KV + HiCache on stock sglang 0.5.19

[Nakanokensetsu]

Stock v0.5.19 → the configuration this fork actually runs in production:
dense Qwen3.8-27B-Escha-W2 with an OSCAR INT2 KV cache and HiCache (L2,
host RAM) on 2× 12 GB consumer GPUs, 128K context, 210,000-token KV pool.

The base branch escha-0519-base is the unmodified v0.5.19 tag, so this
diff is exactly what the fork adds. The seven commits are separable on
purpose — the four hicache: ones are independent of the INT2 work and two
of them are not specific to this fork at all.

Commits

commit	what
escha: OSCAR INT2 KV cache	the 2-bit KV cache (from upstream PR sgl-project#32129 / FutureMLS-Lab/OSCAR) and the Escha-W2 quantization layers
escha: backport upstream cache fixes	sgl-project#38191, sgl-project#37943, sgl-project#39526 (which also covers sgl-project#39342)
escha: mixed HP+INT2 KV window	carried but never constructed on a hybrid GDN model — kept for the hybrid wiring work
hicache: host KV pool for the INT2 device pool	packed codes and scales/zeros are two tensors; the stock host pool assumes one
hicache: use cudaHostAlloc where cudaHostRegister is not device-mappable	WSL2: a kernel cannot store to cudaHostRegistered memory, with any flag
hicache: read host memory once per TP group	backport of sgl-project#38157 (issue sgl-project#38156), plus a configurable reserve
hicache: verify the mamba transfer geometry	turn an asynchronous illegal memory access into a named assertion

Why the two HiCache fixes exist

Enabling --enable-hierarchical-cache on this stack died with
CUDA error: an illegal memory access was encountered on the first long
request. Two independent causes:

1. The INT2 host pool disagreed with itself about how wide a token is. The
   host stride, element_dim and the device row were 192 B, 512 B and
   128 B. _assert_transfer_geometry() now refuses to start on a mismatch.

2. The mamba (GDN state) backup kernel stores to the host pool through its
   host pointer. On WSL2 that memory is not device-mappable when it comes
   from cudaHostRegister. Isolated with tools/mamba_transfer_repro.py:

   dst on device         -> ok
   dst torch pin_memory  -> ok                    (cudaHostAlloc)
   dst cudaHostRegister  -> illegal memory access (flags 0/1/2/3)
   dst pageable          -> illegal memory access
   

The second one is the reason HiCache looked like it "silently breaks hybrid
GDN models" here; it is unrelated to sgl-project#39830.

Measured (2× RTX 5060 Ti 12 GB, TP2, 32 GB host, WSL2)

HiCache on (--hicache-ratio 1) vs off, same build:

	off	on
cold prefill	1010 tok/s	964 tok/s
decode, bs1	51.5 tok/s	50.7 tok/s
8 concurrent, distinct 18K prefixes, second pass	5.2 s	1.6 s
Test D (96K aggregation/enumeration)	18/20	18/20, identical per question

--hicache-ratio 2 pins 10.3 GB of host memory here (KV 2.58 GB + mamba
2.55 GB, per rank, ×2 ranks). Pinned pages cannot be paged out, so
everything else is pushed to swap: page cache collapses to 589 MB and the
machine swaps continuously while idle. Ratio 1 pins 5.2 GB and is fine.

--hicache-mem-layout layer_first is rejected by MambaPoolHost; use
page_first. L3 storage backends raise NotImplementedError for INT2
because the flat data page carries the packed codes without their
scales/zeros.

Not included

Code comments are in Japanese in the parts written for this fork.

---

CI States

Latest PR Test (Base): ❌ Run #35548496035
Latest PR Test (Extra): ❌ Run #35548495844
Latest PR Test (AMD ROCm 7.2): ❌ Run #35548496059