Add FP32 dtype support for RoPE - Part1

[jinyouzhi]

Motivation

Fix #13124

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[jinyouzhi]

Thanks for your review and triggering ci @FlamingoPg

[Fridge003]

Hi @jinyouzhi , can you split this PR into two. The first part is the change on sgl-kernel side (which can be merged now I think), and the second part is the change on srt side.

We need to bump sgl-kernel to make the srt change work

[jinyouzhi]

Hi @jinyouzhi , can you split this PR into two. The first part is the change on sgl-kernel side (which can be merged now I think), and the second part is the change on srt side.您好，能否将此 PR 拆分为两部分？第一部分是 sgl-kernel 端的更改（我认为现在可以合并了），第二部分是 srt 端的更改。

We need to bump sgl-kernel to make the srt change work我们需要更新 sgl-kernel 内核才能使 srt 的更改生效。

Sure, great advice! I have followed it.