Mistral Large 3 NVFP4 support

[dcampora]

Support Mistral Large 3 NVFP4.

Depends on #14466.

• GSM8K test results:

SGLANG_ENABLE_JIT_DEEPGEMM=0 \
python3 -m sglang.launch_server \
--model mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4 \
--kv-cache-dtype fp8_e4m3 \
--tensor-parallel-size 8 \
--disable-radix-cache \
--stream-interval 20 \
--mem-fraction-static 0.9 \
--attention-backend trtllm_mla \
--model-loader-extra-config '{"enable_multithread_load": true}' \
--max-running-requests 1024 \
--cuda-graph-max-bs 1024 \
--chat-template mistral

lm_eval \
--model local-chat-completions \
--model_args model=mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4,\
base_url=http://127.0.0.1:30000/v1/chat/completions,\
num_concurrent=128,timeout=999999,max_gen_toks=8192 \
--tasks gsm8k \
--batch_size 128 \
--apply_chat_template \
--num_fewshot 8

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     8|exact_match|↑  |0.9249|±  |0.0073|
|     |       |strict-match    |     8|exact_match|↑  |0.7104|±  |0.0125|

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.
• Work with maintainers to merge your PR. See the PR Merge Process

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[JustinTong0323]

/tag-and-rerun-ci

[elvischenv]

Before merging main, the server can be launched and the accuracy is good. After merging, I got lots of rope related issues:

1. [Bugfix] Fix KeyError for Mistral-Large-3 rope_scaling config #14627

`rope_parameters`'s factor field must be a float >= 1, got 36
`rope_parameters`'s beta_fast field must be a float, got 32
`rope_parameters`'s beta_slow field must be a float, got 1

  File "/workspace/sglang/python/sglang/srt/model_loader/__init__.py", line 28, in get_model
    return loader.load_model(
           ^^^^^^^^^^^^^^^^^^
  File "/workspace/sglang/python/sglang/srt/model_loader/loader.py", line 595, in load_model
    model = _initialize_model(
            ^^^^^^^^^^^^^^^^^^
  File "/workspace/sglang/python/sglang/srt/model_loader/loader.py", line 263, in _initialize_model
    return model_class(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/sglang/python/sglang/srt/models/pixtral.py", line 87, in __init__
    self.vision_args = VisionEncoderArgs(**vision_args)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: VisionEncoderArgs.__init__() missing 1 required positional argument: 'rope_theta'

Update: all these issues are caused by transformers update: 8200fb5

[ispobock]

/tag-and-rerun-ci

[Fridge003]

@elvischenv @dcampora
This PR also handled the rope issue. Is it conflicting with your code?
#14745

[elvischenv]

@elvischenv @dcampora This PR also handled the rope issue. Is it conflicting with your code? #14745

@Fridge003 It won't have conflicts, but that fix is quite ugly. Like for ml3, there are lots of sub attributes in rope_scaling:

rope_scaling: {'rope_type': 'yarn', 'mscale_all_dim': 1, 'factor': 36.0, 'original_max_position_embeddings': 8192.0, 'beta_fast': 32.0, 'beta_slow': 1.0}

That fix will pull all these sub attributes to the top level of the config, which is unnecessary and error-prone. Not all attributes are needed to be pulled, it would better just pull rope_theta for resolving that issue. @yhyang201

[Fridge003]

@elvischenv We reverted the transformers version to 4.57 and I removed the logics in model_config.py. Please check whether it works on your side

[Fridge003]

/tag-and-rerun-ci

[ispobock]

/tag-and-rerun-ci