[Feature] Enable GLM-5.3-Flash PD disaggregation on Model Runner V2

[sunbaosong]

What this PR does / why we need it?

Enables GLM-Next (GLM-5.3-Flash) prefill/decode disaggregation on Model Runner V2 with MooncakeConnectorV2, for both equal TP (P=TP4 / D=TP4) and unequal TP (P=TP8 / D=DP2xTP4).

GLM-Next could not run PD on MRV2 because of three structural gaps:

1. Granularity mismatch. MRV2 hands metadata builders kernel-granularity specs (block_size == the C128 SFA kernel size), while the scheduler, the common block tables, and the GLM-Next cache contract all use the logical block from cache_config.block_size. Sizing persistent buffers from the spec overflows them by the ceil-rounding slack (required=1044 vs capacity=1040 on the first forward).
2. GLM-Next layout. The v2 MLA spec re-wrap dropped the model_version / compress_ratio markers the cache-group classification needs; the compressed indexer small page (compress_ratio > 1) is a single-tensor overlay on a shared padded slot, not an MLA latent/rope K/V pair; NoPE MLA layers legitimately carry an empty rope/V component.
3. Unequal TP. Page-size unification is TP-dependent for hybrid models (the Mamba state page scales with heads/TP), so a TP8 producer and a TP4 consumer publish different unified block sizes for byte-identical replicated caches — kernel blocks can no longer be paired index-wise.

Changes:

• v2 attn_utils: propagate model_version / indexes_kv_by_block_stride / compress_ratio / non_causal_multi_token_decode through the MLA spec re-wrap; reshape the compressed indexer small-page layer (compress_ratio > 1) as a single strided tensor overlaid from byte zero of the shared padded slot.
• KPool indexer/tail backends: accept upstream's cache_dtype_str probe kwarg; take the logical block size from cache_config.block_size.
• SFA metadata state: size the persistent block table buffer from the cache config's logical block expansion instead of the spec's.
• Mooncake V2 connector: skip zero-size cache components when collecting register ranges (their degenerate storage corrupts the range arithmetic) and skip zero-length transfer entries the engine rejects with E19999.
• Unequal-TP: transfer replicated MLA/indexer caches as GCD-sized token-segment byte runs (_compute_cross_tp_runs); transfer the indexer tail whole-block instead of through the HND head-sharded path; raise with the layer name on kernel-block-size mismatch.

Does this PR introduce any user-facing change?

Yes — it enables a new deployment topology: GLM-Next hybrid models can now run PD disaggregation on MRV2 with MooncakeConnectorV2 (pull mode), including a TP8 producer feeding a DP2xTP4 consumer. Requires VLLM_USE_V2_MODEL_RUNNER=1 and a kv-transfer-config with MooncakeConnectorV2. No behavior change for existing models and configurations: the new code paths are gated (the strided reshape only triggers for MLA-family specs with compress_ratio > 1; the cross-TP run path only activates when local and remote kernel block sizes differ; the equal-TP block pairing path is unchanged).

How was this patch tested?

• Unit tests: the mooncake_v2 suite passes 150/150, covering cross-TP run splitting/coalescing/computed-offset/bounds, tail replica/whole-block mapping, bucket routing, and per-component address scaling. The full UT sweep over tests/ut/{models,attention,worker,core,kv_offload} produces a failure set identical to upstream main (pre-existing failures only, none introduced by this PR).

• E2E, 10-layer GLM-5.3-Flash debug model: 1P1D TP4/TP4 (producer cards 0-3, consumer cards 4-7, eager), pull-mode KV transfer over the mooncake engine relayed through a load-balance proxy: first-token fidelity 18/18, outputs match the single-instance baseline to the model's own determinism, real per-rank transfer logs, zero recompute fallback.

• E2E, full w8a8 GLM-5.3-Flash: P=TP8 producer, D=DP2xTP4 consumer (FULL_DECODE_ONLY graph mode): zero pull errors, ~18.5 MB transferred per request, TextVQA accuracy consistent with the colocated baseline.

• vLLM main: vllm-project/vllm@84030bb

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces support for GLM-5.3-Flash prefill/decode disaggregation on Model Runner V2 using the MooncakeConnectorV2 engine. It enables efficient KV cache transfers across both equal and unequal tensor parallelism (TP) topologies by implementing robust cross-TP run logic and updating metadata propagation and cache spec handling.

Highlights

• GLM-Next PD Disaggregation Support: Enabled prefill/decode disaggregation on Model Runner V2 using MooncakeConnectorV2 for both equal and unequal tensor parallelism (TP) topologies.
• Cross-TP Transfer Logic: Implemented GCD-based token-segment transfer logic to handle replicated MLA and indexer caches across mismatched TP topologies.
• Metadata and Spec Updates: Updated attn_utils, KPool indexer, and SFA metadata to propagate model versions and utilize correct cache configuration block sizes.
• Connector Robustness: Added handling for zero-size cache components and zero-length transfer entries in the Mooncake V2 connector to improve stability.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

---

Tip

💡 Consider Linking a Related Issue or RFC

Your PR title contains the [Feature] tag, indicating a bug fix or new feature.

Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:

• Fixes #<issue_number>
• Closes #<issue_number>
• Resolves #<issue_number>
• Refs #<rfc_or_issue_number> (for RFCs)

🙏 Thanks for helping us keep the project well-organized!

[gemini-code-assist]

Code Review

Suggested PR Title:

[Attention][Feature] Support cross-TP Mooncake transfers and indexer tail replication

Suggested PR Summary:

### What this PR does / why we need it?
This PR introduces support for cross-TP Mooncake transfers across unequal block grids (e.g., TP4 consumer vs TP8 producer) by splitting and coalescing token-unit runs on GCD boundaries. It also adds support for replicating the GLM-Next indexer tail spec (`AscendIndexerKPoolTailSpec`) across mismatched TP deployments. Additionally, it handles compressed indexer cache reshaping using `torch.as_strided` and updates the relevant unit tests.

Feedback from the review highlights two main improvements:
1. Update the return type annotation of `_build_transfer_block_buckets` in `pull_worker.py` to reflect the new 3-tuple return value.
2. Use `getattr(spec, "non_causal_multi_token_decode", False)` instead of direct attribute access in `attn_utils.py` to prevent potential `AttributeError` across different vLLM versions.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Tested with new and updated unit tests in `tests/ut/kv_offload/mooncake_v2/test_pull_worker.py` covering cross-TP runs, tail spec replication, and transfer bucket routing.

[nwpu-zxr]

I think using a large block size such as 160 or 288 is not well, it's better that separate the padding size to each small blocks with 32 kernel block size indexer cache.