[DSV4] Generalize attention metadata, sparse prefill, and KV pool over compress ratios

[hnyls2002]

Summary

• Make the set of compressed-KV ratios data owned by the KV pool (present_ratios) and give the shared code one accessor per concept keyed by ratio, instead of hard-coding c4 / c128 in field names, branches and PD payload code
• No behavior change for V4: every existing path produces the same tensors and indices (see Verification); the new surface is consumed by dsv4.1: remaining model and runtime integration #38798

Changes

Attention metadata (DSV4AttnMetadata)

• Rename c4_sparse_topk -> index_topk (the indexer's top-k, not a c4 property); add required present_ratios from the pool
• Build compression metadata, FlashMLA schedules, compressor and indexer metadata only for present ratios; CP reindex treats per-ratio fields as optional; copy_ / breakable-CUDA-graph refresh check present_ratios
• Add ratio-keyed sparse_page_indices(r), sparse_topk_lengths(r), sparse_raw_indices(r) and the writer set_sparse_topk(r, ...); call sites and the attention test kit use them, forward dispatches on compress_ratio != 0 (storage stays flat)

Sparse prefill (SparsePrefillChunkCache, combine_topk_swa_indices)

• Kernel takes each row's absolute query_pos instead of assuming rows are the trailing extend tokens, and keeps -1 entries inside the top-k span as -1 instead of shifting them by compressed_base (both identities on V4 inputs)
• build() takes query_lens / query_pos alongside extend_seq_lens (the SWA gather still spans the whole extend)
• Keep per-ratio gather state in CompressedGather, keyed by ratio in cache.compressed (c128 included); both sparse-prefill forwards share cache.layer_inputs(...)

KV pool and PD state

• Add DeepSeekV4TokenToKVPool.present_ratios; CompressStatePool.request_scoped (set by the pool factory) replaces ratio == 128 checks in the state-buffer bookkeeping
• PD prefill/decode call request_state_transfer_indices(req_pool_idx, seq_len) / clear_request_scoped_state(req_pool_idx) on the pool instead of combining an env lookup, get_ring_size(128) and a c128 helper; the index arithmetic moves to deepseek_v4_compress_state.py (c4_state_transfer_indices, request_scoped_state_transfer_indices, CompressStatePool.transfer_indices)
• Set mla_compression_ratios in setup_state_kv_args for both roles
• Add get_swa_key_layout / get_extra_key_layout / get_*_bytes_per_token so the attention kernel views each cache with its own row width (the extra cache previously reused the SWA width)

Removed

• is_dsv4_c128_online_enabled (the pool's online flag is the single source), CompressedGather.page_size (unused), get_dsv4_c4_state_indices / get_dsv4_c128_state_indices in disaggregation/utils.py (moved as above)

Verification

• AST comparison against main per function: only the functions listed above differ
• Differential dump of the V4 paths (metadata init for prefill / decode / trtllm, copy_, refresh, CP reindex; chunk-cache build, c128 / c4 gathers and combines; pool state-buffer bookkeeping and request-state clear; PD request-state indices) between main and this branch: 274 shared entries, 0 mismatches (main-only entries are the renamed c4_sparse_topk)
• Registered tests pass: test_deepseek_v4.py (incl. real sparse-prefill c4 / c128 vs. torch reference), test_q8kv8_sparse_prefill_backend.py, test_disaggregation_wire.py, test_dsv4_c4_state_lifecycle.py, test_dsv4_compressed_pools.py, test_dsv4_unified_fp8_pool.py, test_deepseek_v4_compress_state_runtime_shapes.py
• New: test/registered/kernel/attention/test_combine_topk_swa_indices.py (torch oracle: trailing extend, -1 holes, non-trailing query_pos with cross-chunk offset, SWA-only); metadata contract tests for present_ratios gating, accessor / writer routing and CP reindex; pool request-state transfer tests

---

CI States

Latest PR Test (Base): ✅ Run #35276391750
Latest PR Test (Extra): 🚫 Run #35276390533
Latest PR Test (AMD ROCm 10): ⏳ Run #35276390785