Flashinfer MOE FP8 support for Mistral Large 3.

[dcampora]

Motivation

This PR brings in Flashinfer MOE FP8 support for Mistral Large 3.

It requires an upcoming release of flashinfer to work.

Modifications

Accuracy Tests

Without EP8:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     8|exact_match|↑  |0.9257|±  |0.0072|
|     |       |strict-match    |     8|exact_match|↑  |0.6012|±  |0.0135|

With EP8:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     8|exact_match|↑  |0.9287|±  |0.0071|
|     |       |strict-match    |     8|exact_match|↑  |0.5861|±  |0.0136|

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.
• Work with maintainers to merge your PR. See the PR Merge Process

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[Fridge003]

Hi @dcampora flashinfer has been upgraded. Can you please take a look again

[elvischenv]

Hi @dcampora flashinfer has been upgraded. Can you please take a look again

Hi @Fridge003, this PR depends on flashinfer-ai/flashinfer@36380e2 which is out of 0.6.1 but included in 0.6.2.

[Fridge003]

@dcampora @elvischenv 0.6.2 has already been included on latest main branch

[elvischenv]

Hi @Fridge003, could you help remove "draft" from PR's title and enable CI for this PR? I don't have permission to update the title.

[Fridge003]

/tag-and-rerun-ci