[KV Connector] Opt DecodeBenchConnector into SupportsHMA

[liuzijing2014]

Summary

Opt DecodeBenchConnector into SupportsHMA so decode-only benchmark recipes can run with the hybrid KV cache manager enabled. Without this, users must pass --disable-hybrid-kv-cache-manager, which collapses hybrid-model KV cache groups (SWA / MLA compress=4 / MLA compress=128 / sparse indexer) into a single uniform page size and throws away the compression savings.

Implementation is minimal because this connector is a dummy fill that owns no external per-block state:

• Inherit from SupportsHMA.
• Implement request_finished_all_groups: same cleanup as the single-group request_finished (delegates to connector_scheduler.request_finished()); block_ids is ignored because there is no per-block external state to release.

Not a duplicate

Searched open PRs for DecodeBenchConnector, SupportsHMA, and decode_bench_connector (via the GitHub search API). No PR targets this connector. The closest neighbor, #41644 ("Keep HMA enabled for supported KV connectors"), is orthogonal — it changes the default HMA decision in vllm/config/vllm.py for connectors that already declare SupportsHMA. This PR adds one more connector to that supported set; the two changes compose.

Test plan

Hardware: 1× node with 4× GB200, single-node DEP=4 (DP=4, EP=4). Model: DeepSeek-V4-Flash with --load-format dummy and --tokenizer-mode deepseek_v4. Connector: DecodeBenchConnector, HMA explicitly enabled via --no-disable-hybrid-kv-cache-manager.

Same vllm serve command in both runs:

vllm serve <DSv4-Flash> --port 8000 \
  -dp 4 -ep --data-parallel-size-local 4 \
  --load-format dummy --tokenizer-mode deepseek_v4 \
  --kv-cache-dtype fp8 --max-model-len 8192 \
  --no-disable-hybrid-kv-cache-manager \
  --max-num-seqs 32 --max-num-batched-tokens 32 \
  --block-size 256 --gpu-memory-utilization 0.85 \
  --kv-transfer-config '{"kv_connector":"DecodeBenchConnector","kv_role":"kv_both","kv_connector_extra_config":{"fill_mean":0.015,"fill_std":0.02}}'

Without the PR (only difference: class DecodeBenchConnector(KVConnectorBase_V1) — no SupportsHMA mixin)

Crashes at engine init in KVConnectorFactory.create_connector_v1():

ValueError: Connector DecodeBenchConnector does not support HMA but HMA is enabled.
            Please set `--disable-hybrid-kv-cache-manager`.

All four DP ranks fail identically; the engine cores then exit with RuntimeError: Worker failed with error '...'.

With the PR

Server starts cleanly. HMA is active:

INFO ... [deepseek_v4_attention.py:614] Using DeepSeek's fp8_ds_mla KV cache format.
INFO ... [kv_cache_utils.py:1713] GPU KV cache size: 86,928 tokens     (each of 4 ranks)
INFO ...:     Application startup complete.

Functional sanity check via vllm bench serve (random ISL=4000, OSL=100, 32 prompts, max-concurrency=8):

Successful requests:                     32
Failed requests:                         0
Request throughput (req/s):              4.52
Total token throughput (tok/s):          18545.58
Mean TPOT (ms):                          12.78    P99: 20.53
Mean TTFT (ms):                          486.70   P99: 1456.91

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[gemini-code-assist]

Code Review

This pull request updates the DecodeBenchConnector to support Hierarchical Memory Access (HMA) by inheriting from the SupportsHMA interface and implementing the request_finished_all_groups method for proper cleanup. I have no feedback to provide as there were no review comments to evaluate.