[JaredforReal]

Overview

FLASHINFER_MLA_SPARSE_SM90 hands pageable CPU tensors to FlashInfer's plan(), which copies them to the GPU with non_blocking=True. The copy is silently synchronous, and with VLLM_GPU_SYNC_CHECK=error (set in the CI image) every GLM-5.3-Flash prefill on H200 fails. This PR passes plan() a pinned copy per call, the same pattern vllm-project#54660 applied to the FlashInfer backend.

Found while running the layer-pruned GLM-5.3-Flash tests of vllm-project#57537 on H200 CI (build 97524): every test that generates died with this error. vllm-project#61399 proposed the same fix and was closed unmerged without discussion; no open PR covers it.

Claims

• _SM90State.plan() no longer performs an implicit GPU<->CPU sync, so FLASHINFER_MLA_SPARSE_SM90 runs under VLLM_GPU_SYNC_CHECK=error.
• The staging buffers stay pageable and are reused as before; only the tensors handed to the wrapper are pinned copies.
• No performance claim.

Validation

New test test_plan_does_not_force_gpu_sync runs the real wrapper's plan() inside a sync-checked region. No H200 was available, so it was run on GB300 with SM90 faked for collection and for FlashInfer's FA3 plan gate (plan() only runs the host-side planner and the staging copy, no kernel):

	result
without the fix	RuntimeError: Tensor.copy_: a non_blocking H2D copy that may be silently synchronous (the CPU tensor is not pinned)
with the fix	passed

The rest of tests/v1/attention/test_flashinfer_mla_sparse_sm90.py passes with the fix (21 passed; the 4 real-kernel tests skip off SM90). In CI, the (H100) V1 Attention job runs this file on SM90, so the new test runs without faking there.

Not done: an end-to-end GLM-5.3-Flash run on H200 with the fix.

Details

Why not pin the reused _qo_cpu / _kv_cpu / _lens_cpu buffers: the next plan() rewrites them on the host while the previous step's async copy may still be reading them. A per-call pin_memory() comes from the caching host allocator, which recycles a block only after the copies that used it complete.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code