[Feature]Enable DispatchGmmCombineDecode when eagle is moe with w8a8 or not moe [RFC: issue 5476]

[wangqiankun13]

What this PR does / why we need it?

Operator DispatchGmmCombineDecode does not support non-W8A8 scenarios and cannot share the same communication domain with Operator Dispatch/Combine.

for instance, when the draft model uses a non-W8A8 MOE architecture while the main model employs a W8A8 MOE architecture.

Therefore days ago, I implemented an interception that unconditionally disables Operator DispatchGmmCombineDecode whenever the speculative mode is EAGLE or EAGLE-3. PR: 5293
However, this approach was not precise enough.
This PR further refines the logic by specifically identifying the draft model's configuration: Operator DispatchGmmCombineDecode will now be disabled only when the draft model uses an MOE architecture and is non-W8A8.

More info about this operator, please refer to RFC: issue #5476

Does this PR introduce any user-facing change?

No

How was this patch tested?

Acc
test qwen3-235b eplb on a single A3 node(ep16),
with dispatch_gmm_combine_decode

nic_name="xxxx"
local_ip="xxx.xxx.xxx.xxx"

export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name

export VLLM_ASCEND_ENABLE_FUSED_MC2=2
echo "VLLM_ASCEND_ENABLE_FUSED_MC2=${VLLM_ASCEND_ENABLE_FUSED_MC2}"

export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_BUFFSIZE=512
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=10
export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOAD
vllm serve /dataset/Qwen3-235B-A22B-Instruct-2507-w8a8-QuaRot/ \
        --served-model-name "qwen" \
        --host 0.0.0.0 \
        --port 8004 \
        --async-scheduling \
        --tensor-parallel-size 4 \
        --data-parallel-size 4 \
        --max-num-seqs 64 \
        --max-model-len 40960 \
        --max-num-batched-tokens 16384 \
        --gpu-memory-utilization 0.9 \
        --enable-expert-parallel \
        --no-enable-prefix-caching \
        --quantization "ascend" \
        --trust-remote-code \
        --speculative_config \
        '{
            "method": "eagle3",
            "model": "/dataset/Qwen3-235B-A22B-Instruct-2507-speculator-eagle3/",
            "num_speculative_tokens": 2
        }' \
        --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
        2>&1 | tee qwen3_235b_eagle3.log

dataset	version	metric	mode	vllm-api-stream-chat
aime2024	604a78	accuracy	gen	80.00

• vLLM version: v0.13.0
• vLLM main: vllm-project/vllm@2f4e654

[gemini-code-assist]

Code Review

This pull request introduces logic to conditionally enable DispatchGmmCombineDecode for speculative decoding, particularly for Eagle models. The change involves refactoring speculative_enable_dispatch_gmm_combine_decode to handle more complex conditions and adding a caching layer.

My main feedback is regarding the implementation of the caching mechanism. It uses a global variable without proper thread-safe access, which can lead to race conditions in a concurrent environment. I've provided a detailed comment on how to address this using a lock to ensure thread safety. Addressing this is critical for the stability of the application.

[gemini-code-assist]

The caching implementation for speculative_enable_dispatch_gmm_combine_decode has a couple of critical issues:

1. Typo: The global variable _DRAFT_ATAPT_FUSED_MC2_MODE_2 seems to have a typo and should likely be _DRAFT_ADAPT_FUSED_MC2_MODE_2.

2. Race Condition: The use of a global variable for caching is not thread-safe. If multiple threads call this function concurrently when the global is None, it can lead to a race condition where the value is computed multiple times, and the global variable is written to without synchronization.

To fix this, I recommend renaming the variable and using a double-checked locking pattern to ensure thread-safe initialization.

First, update the global variable definition around line 72:

from threading import Lock
...
_DRAFT_ADAPT_FUSED_MC2_MODE_2 = None
_DRAFT_ADAPT_FUSED_MC2_MODE_2_LOCK = Lock()

Then, update this function as follows:

def speculative_enable_dispatch_gmm_combine_decode(
        vllm_config: VllmConfig) -> bool:
    global _DRAFT_ADAPT_FUSED_MC2_MODE_2
    # Use double-checked locking for thread-safe initialization.
    if _DRAFT_ADAPT_FUSED_MC2_MODE_2 is None:
        with _DRAFT_ADAPT_FUSED_MC2_MODE_2_LOCK:
            if _DRAFT_ADAPT_FUSED_MC2_MODE_2 is None:
                _DRAFT_ADAPT_FUSED_MC2_MODE_2 = _draft_adapt_fused_mc2_mode_2(vllm_config)
    return _DRAFT_ADAPT_FUSED_MC2_MODE_2

This approach ensures correctness and stability in a multi-threaded environment.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.