[Refactor] Replace activation: str with MoEActivation enum

[mgoin]

Purpose

We have had activation defined, validated, and passed around as a raw string forever. Now that we have popular models that don't just use the silu default and also need to support non-gated MoEs and their activation functions, we need to have a single source of truth for all the activation functions that exist in vLLM and which MoE kernels support which functions. I want to start with MoE since that is where we have the most divergence in support due to fused kernels.

This PR introduces MoEActivation and aims to replace all usage of activation: str in internal interfaces for MoE. Note that we do keep FusedMoE.__init__ as activation: str in order to avoid changes to the model definitions, however we do validate it with MoEActivation.from_str(activation) within the constructor.

We do still need to keep activation: str in some function signatures to be compatible with torch's custom op registration so we will pass in the MoEActivation as activation.value and convert it back within the custom op with MoEActivation.from_str(activation)

Test Plan

Test Result

Run all tests in CI and discuss with external hardware plugins

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request introduces a great refactoring by defining a MoEActivation enum to replace string-based activation function identifiers. This significantly improves type safety and code clarity across the MoE-related modules. The changes are extensive and have been applied consistently throughout the codebase. I have one piece of feedback regarding a potentially unsafe in-place modification in the new activation utility function.

[mergify]

Hi @mgoin, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[robertgshaw2-redhat]

labor of love

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @mgoin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @mgoin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[bnellnm]

Nice refactor. I added a few comments/questions.

[bnellnm]

Nice!