feat(moe): Add is_act_and_mul=False support for Triton MoE kernels

[rabi]

Purpose

Add support for non-fused activations (relu2_no_mul, silu_no_mul, gelu_no_mul) in Triton MoE kernels for models like Nemotron-H that use non-SwiGLU activations.

• Add is_act_and_mul flag to FusedMoEQuantConfig
• Implement non-fused activations in modular_kernel.py
• Fix workspace sizes in TritonExperts for is_act_and_mul=False
• Enable on ROCm when AITER is disabled

Test Plan

Add test_triton_moe_no_act_mul.py for CUDA and ROCm

Test Result

Tests pass successfully on local env and would be tested in CI

[gemini-code-assist]

Code Review

This pull request enables MoE models with is_act_and_mul=False to run on ROCm by leveraging Triton kernels. The changes are well-structured, introducing is_act_and_mul to FusedMoEQuantConfig, updating workspace sizing calculations, and adding support for non-fused activations. The inclusion of a new test file for ROCm is a great addition for ensuring correctness. I have one suggestion to enhance the performance of the new activation function implementations by minimizing intermediate tensor allocations.

[mergify]

Hi @rabi, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[tjtanaa]

LGTM

[mgoin]

@tjtanaa @rabi @danielafrimi can we actually revert this PR and land #31528 instead? I feel this fix adding is_act_and_mul to the quant config and the activation implementations are not as nice as the refactor in the other PR