feat(qwen): support atomic external QSA checkpoints

[yatesdr]

Summary

Allow Qwen QSA external KV transfer only through connectors that provide atomic request-boundary checkpoints. This enables LMCache restore without permitting an incomplete recurrent state.

Target: integration/karmic-kraken-beta. This includes .lil/changes/vllm-864.json. Runtime PR #69 depends on this change.

Validation

• Rebased onto integration head c62f4b72bd; all changed Python files passed Python 3.12 py_compile, and git diff origin/integration/karmic-kraken-beta..HEAD --check passed. The container publisher's validate_fragment accepted the new fragment.
• Surrogate GPU qualification: the public Qwen r6 test image, on the earlier published KK base, restored a 32K-token LMCache L2 hit after restart with the same first token and zero first-token log-probability delta. On 96 distinct greedy GSM8K items it scored 94/96 at C1 and 93/96 at C8 and C16, with no confirmed repeated-output loops.
• The rebased integration head has not yet had a combined CN4 run; that is the next integration qualification step.

Duplicate check

No open KK integration PR enables Qwen QSA request-boundary KV transfer. Existing recurrent-checkpoint work does not lift Qwen QSA's connector prohibition.

AI assistance: OpenAI Codex helped prepare and rebase this change. Human review and integration qualification remain required before merge.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• dev/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 10a654f1-bc9a-4d69-92d5-2a1dac74e20d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[voipmonitor]

Validated together for Karmic beta on September 23 (vLLM #861–#864 with runtime #68/#69).

Stack. Published beta acb6193c with these overlays: the four vLLM PRs merged onto integration 01f1b874 (which already includes #865), B12X #417, and the runtime from main plus #68 and #69. All four vLLM PRs merge cleanly on the current integration head.

Unit tests. The touched files (test_indexer_dcp_localize, test_prefill_compute_share_scheduler, test_gpu_worker, test_b12x_qsa, kv_connector test_config, test_partial_prefix_cache_primitives, test_cudagraph_manager): 409 pass, 9 skip. The two test_kv_connector[...-2-2-...] cases need four GPUs and pass with them. On the base image, 57 of these fail, including every new test.

Qwen TP4/DCP2 end to end (4× RTX PRO 6000, MTP3, fp8 KV, LMCache disk L2 with O_DIRECT, vision on, PREFILL_COMPUTE_SHARE=auto, MAX_NUM_PREFILL_TOKENS_PER_STEP=3008, your QSA DCP environment):

• 6/6 arithmetic answers correct, and image OCR returned Hello, AI world! exactly.
• Greedy long-context counting with mixed short requests: 21/21 exact at 12k tokens and 4/4 at 30k. The control without short requests was 7/7 exact.
• Restore after a container restart (GPU cache and L1 gone, L2 kept): 8,204 and 32,780-token prompts were fully served from L2, and their 64 greedy tokens were identical.
• A 228,896-token prefill during a running decode stream: the largest decode gap was 286 ms (p99 224 ms). No LMCache store-admission failures appeared in the log.

GLM-5.3 TP4/DCP4/MTP3 end to end (the shared DCP top-k kernel): 6/6 arithmetic answers correct; greedy long-context counting with mixed short requests was 21/21 exact at 12k tokens and 4/4 at 30k.

DCP top-k merge kernel (#861). Old versus new kernel, 16 shapes (DCP 2/4, top-k 512/2048, 1–64 rows): the selected sets are identical everywhere. The old order changed between repeated calls; the new order is stable. The kernel costs 0.6–3.1 µs more per call. With 12 QSA layers in Qwen or about 11 DSA layers in GLM-5.3, that is roughly 35 µs per decode step at most. I widened vllm-861.json to list GLM-5.3 and DeepSeek V4/V4.1, because _merge_dcp_topk and the sparse indexer share this kernel.

Runtime. Both runtime PRs failed only ruff format; I pushed the formatting fixes. #69 also needed a guard, because the canonical and beta images share the recipe. Canonical vLLM does not include #864. The Qwen QSA backend is not chosen through the attention selector, so its supports_kv_connector() = False is never consulted, and DCP>1 would silently lose request-boundary checkpoints. lil-serve now asks the installed vLLM in a GPU-free child and refuses a Qwen DCP>1 external cache without #864. Checked on the canonical image: exit 2 with a clear message, before any service starts. Runtime tests: 287 pass.

Configuration notes, unrelated to these PRs:

• TP4/DCP4 is rejected on our base: "DCP (4) exceeds the maximum supported value (2) for TP4 and 2 model KV heads". Your r6 image must carry more than these PRs.
• Qwen DCP needs a CP_KV_CACHE_INTERLEAVE_SIZE that is divisible by 4 and divides the resolved block size (1424 at TP4/DCP2, so 712 worked). The runtime does not derive it yet.
• Short distinct prompts still write about 167 MB per recurrent checkpoint to L2, as your perf(linear): route prefill-sized batches to DeepGEMM inside the b12x FP8 linear kernel #69 notes say. That L2 admission question is tracked separately.

🤖 Generated with Claude Code