Skip to content

Aiter fp8 kv cache #13147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
HaiShaw merged 29 commits into from
Dec 9, 2025
Merged

Aiter fp8 kv cache #13147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
bb955d2
[FP8-KV-CACHE] Init vers.
Oct 15, 2025
f558e8b
[FP8 KV-CACHE] Force dtype conversion from fp8 to bflot16 for enable …
Oct 15, 2025
7fc664b
Pass mla_decode_fwd accuracy test
Oct 29, 2025
8245d3a
Fix no scale issue for non-fp8 kv and default to use non-persistent m…
Oct 30, 2025
06b78dc
Add new env vairable to control mla decode persist kernel use or not
Oct 31, 2025
ac6ee4c
MTP fp8-kv accuracy pass
Nov 3, 2025
7be57a8
Fix GPU fault when using persist mla_decode_fwd kernel in MTP
Nov 3, 2025
f3864a9
Code refactor
Nov 4, 2025
5ce2aa1
Refactor code v2
Nov 6, 2025
64ec4f8
Merge branch 'sgl-project:main' into aiter-fp8-kv-cache
kkHuang-amd Nov 12, 2025
8ceda76
Merge branch 'sgl-project:main' into aiter-fp8-kv-cache
kkHuang-amd Nov 12, 2025
7848b53
Code refactor v3
Nov 13, 2025
3a2c5c8
Merge branch 'sgl-project:main' into aiter-fp8-kv-cache
kkHuang-amd Nov 17, 2025
a8463cc
Format code
Nov 17, 2025
82294e9
According to the q type to convert kv cache type for following comput…
Nov 17, 2025
6a3f650
Merge branch 'main' into aiter-fp8-kv-cache
kkHuang-amd Nov 18, 2025
26c12e6
Merge branch 'sgl-project:main' into aiter-fp8-kv-cache
kkHuang-amd Nov 19, 2025
e4d27b5
Fix the MTP accuracy issues with using aiter persisent v2 mla_decode …
Nov 21, 2025
f4b0e59
Merge branch 'sgl-project:main' into aiter-fp8-kv-cache
kkHuang-amd Nov 21, 2025
2d682c1
Change the code for adaptive kv-split mla_decode_forward version
Nov 28, 2025
fe0f700
Fused q quantization, kv cache store into qk_rope_cat
Dec 1, 2025
4cd758b
Fix the run-time error when run the fp8 deepseek-v3 model
Dec 2, 2025
30c7f41
Remove un-neccessary code
Dec 3, 2025
29f40b6
Merge branch 'main' into aiter-fp8-kv-cache
kkHuang-amd Dec 4, 2025
a0f9685
Fix MTP + FP8-KV + DP accuracy issue
Dec 6, 2025
f7492c6
Fix the performance regression without using dp
Dec 7, 2025
3e9a03e
Merge branch 'main' into aiter-fp8-kv-cache
kkHuang-amd Dec 8, 2025
1610fc9
Fix CI error "Command 'cd /root/.aiter/build/pa_ragged_afefebe1d44cff…
Dec 8, 2025
9889990
Add more comment and set SGLANG_AITER_MLA_PERSIST default True
Dec 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 0 additions & 49 deletions Makefile
View file
Open in desktop

This file was deleted.

Loading
Loading