[yeahdongcn]

Purpose

Enable the MiniMax H3 fused SwiGLU and modulation paths on MUSA without changing the shared CUDA implementation or assuming that unverified accelerator backends can run the Triton kernels.

This is a review branch in yeahdongcn/vllm-omni. It is intentionally opened as a draft while the final MUSA end-to-end A/B is running. After review, the same branch can be used for an upstream PR.

The branch:

• adds an explicit SiluAndMul.forward_musa() that calls the registered MUSA torch.ops._C.silu_and_mul implementation;
• adds a narrow platform capability for the MiniMax H3 Triton modulation kernels, enabled only for the CUDA and MUSA paths that have evidence;
• keeps unsupported platforms on their existing platform-specific RMSNorm path;
• accepts the row-strided adaln.chunk(6, dim=-1) layout used by the real H3 model while checking platform, device, and unit last-dimension stride;
• uses vLLM's HAS_TRITON import contract so environments without Triton can import the module and use the native fallback;
• passes output row strides to the RMSNorm fusion kernels instead of assuming dense row-major output;
• adds a CPU fallback test with the real strided AdaLN layout.

Related upstream PR: vllm-project#6364. That PR changes forward_cuda; this branch instead keeps the common CUDA method unchanged and provides a separate MUSA dispatch path.

Test Plan

1. Run static formatting, lint, byte-compilation, DCO, and diff checks.
2. Compare native, forced forward_cuda without the device check, explicit forward_musa, and the vLLM-MUSA JIT path on S5000.
3. Validate contiguous and real row-strided AdaLN layouts for all four modulation operations in eager and compiled modes.
4. Run same-host compiled-path MiniMax H3 FL2VA A/B for control, [Perf] Fuse MiniMax H3 SwiGLU activation vllm-project/vllm-omni#6283-only, [Perf] Fuse MiniMax H3 modulation with FP32 accumulation vllm-project/vllm-omni#6281-only, and the combined branch.
5. Run final FL2VA and Ref2VA correctness requests from the release image.

vLLM Version: 0.1.dev1+gee0da84ab.d20260818.empty (vLLM-MUSA 0.1.24)

vLLM-Omni Commit: 4ddb22c4b2ed930d6565b2784736a0836ba78616

Test Result

Completed:

• ruff format / ruff check: passed.
• python -m compileall: passed.
• git diff --check: passed.
• DCO: one signed-off commit.
• Local h3-full image build from the exact v0.24 overlay source: passed, including the dependency-freeze and package-metadata gates.
• S5000 BF16 SwiGLU microbenchmark: the forced no-device-check _C.silu_and_mul path and explicit forward_musa have matching behavior. In eager mode the fused path was 2.4-2.7x faster for the measured H3 shapes; under torch.compile it was approximately neutral for small/medium shapes and about 25% faster at 16,384 rows. The optional MUSA JIT path was not selected because it regressed the small/medium compiled cases.
• S5000 BF16 modulation microbenchmark with hidden size 5,376: the fused combined RMSNorm paths improved compiled latency by about 10.6-27.6% at 4,096-16,384 rows. The simpler gate/scale-shift kernels were roughly neutral to 4.3% faster depending on shape.
• Compiled TP4 FL2VA control, 4-second output, 50 steps, fixed seed: 159.186 s, 159.172 s, and 158.996 s wall time; all outputs are valid 1,344x768 H.264/AAC videos with 107 frames.

Pending before the draft is ready:

• real row-strided AdaLN MUSA microbenchmark after the shared-GPU lease fence clears;
• [Perf] Fuse MiniMax H3 SwiGLU activation vllm-project/vllm-omni#6283-only, [Perf] Fuse MiniMax H3 modulation with FP32 accumulation vllm-project/vllm-omni#6281-only, and combined H3 end-to-end A/B;
• final Ref2VA/FL2VA correctness requests from the pushed image.

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.
(anything written below this line will be removed by GitHub Actions)