[DCP] Default query replication for GLM sparse attention

[LucasWilkinson]

Summary

• add --dcp-q-replicate and --no-dcp-q-replicate as explicit CLI options;
• preserve VLLM_DCP_Q_REPLICATE as the highest-precedence override;
• enable query replication automatically for glm_moe_dsa models when decode DCP is enabled and PCP is not enabled;
• keep query replication opt-in for other MLA model families.

The automatic default removes GLM's decode query all-gather at the cost of replicating the query projection weights across DCP ranks. Explicit CLI or environment selections still override the model default.

Performance and memory

GLM-5.2-NVFP4, TP=4, DCP=4, 32K input tokens, 128 output tokens, batch 8, two warmups, five measured iterations:

• QREP off: 20.1777 s
• QREP on: 19.7426 s
• delta: -0.4351 s (-2.16%)

With QREP enabled, a fresh DCP backend comparison measured:

DCP backend	Average latency
ag_rs	19.7636 s
a2a	18.0213 s

For the matched 32K benchmark configuration, model-loading memory increased from 110.37 GiB/rank to 114.94 GiB/rank with QREP, approximately +4.57 GiB/rank.

Model evaluation

GLM-5.2-NVFP4, TP=4, DCP=4, GSM8K 5-shot, 300 questions:

• automatic GLM QREP default: 94.0% accuracy, 0% invalid outputs;
• explicit QREP off: 95.0% accuracy, 0% invalid outputs.

The QREP-off evaluation used max_num_seqs=32 because FlashInfer's sparse-MLA workspace overflowed when all prompts were scheduled concurrently. Both runs evaluated the same 300 questions with the same decoding settings. The one-point difference is not significant at this sample size.

AI assistance

AI assistance was used for implementation, tests, benchmarking, profiling analysis, and drafting this description. The human submitter has reviewed.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[robertgshaw2-redhat]

nice find!

[LucasWilkinson]

/run ci

[LucasWilkinson]

/ci run

[github-actions]

✅ Triggered Buildkite CI #84623 for commit 0b7f5309d377.

[mergify]

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[mergify]

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[LucasWilkinson]

/ci run

[github-actions]

✅ Triggered Buildkite CI #84947 for commit 1b33f97a2dab.