Flashinfer fused MoE and flashattn v2/v3 all-in-one

[guoqingbao]

No description provided.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4de938bdc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[sempervictus]

@guoqingbao are all the feature flags consolidating to flashinfer or is the thought to have impl determined at compile time and only build what we have to for thetargeg tuple?

[guoqingbao]

@guoqingbao are all the feature flags consolidating to flashinfer or is the thought to have impl determined at compile time and only build what we have to for thetargeg tuple?

Kernels are compiled under specific features (the all-in-one flashattn.rs is used to replace candle-flash-attn, which was controlled by the flash-attn and flash-context features), while flashinfer controls the FlashInfer attention and FlashInfer fused MoE kernel path.