Fix: fix flashmla fp8 kv cache acc error

[FlamingoPg]

Motivation

#13832

Modifications

Accuracy Tests

Fixed.

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.
• Work with maintainers to merge your PR. See the PR Merge Process

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[Fridge003]

@FlamingoPg Please fix lint

[cscyuge]

launch error:

...
  File "/sgl-workspace/sglang/python/sglang/srt/layers/attention/flashmla_backend.py", line 78, in __init__
    self.is_fp8_kvcache = model_runner.kv_cache_dtype.startswith("fp8")
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'torch.dtype' object has no attribute 'startswith'

Tried using model_runner.server_args.kv_cache_dtype.startswith("fp8") but didn't fix #13832.

I think it may because the current deepseek_v2 model doesn’t implement load_kv_cache_scales(), so any quantization_param_path is ignored, and MLATokenToKVPool simply casts the latent KV tensors to fp8 without storing or reusing scale factors. As a result, the KV cache relies on raw FP8 range only, which introduces much larger quantization error than the other models that load and apply layer-wise scales.

[akhoroshev]

#13832 (comment)