feat: KV cache quantization support in fp8 rollout in GRPO#1212

Merged

terrykong merged 44 commits intoNVIDIA-NeMo:mainfrom

sharonyu-115:kv-cache-fp8

Dec 2, 2025

Commits on Nov 17, 2025

kv-cache: prepare clean commit without excluded files

Shuang Yu
authored and
zpqiu
committed
kv cache fp8 code refine and cleanup

Shuang Yu
authored and
zpqiu
committed
Fix indentiation error. Enable using environment variables to set FP8 max.

Shuang Yu
authored and
zpqiu
committed
Correct typos

Shuang Yu
authored and
zpqiu
committed
Update fp8.py

Shuang Yu
authored and
zpqiu
committed
Remove dapo.py from response_datasets

Shuang Yu
authored and
zpqiu
committed
Update sanity check in grpo.py. Remove redundant code in megatron backend.

Shuang Yu
authored and
zpqiu
committed
Remove redundant comments

Shuang Yu
authored and
zpqiu
committed
Remove _hook_builder

Shuang Yu
authored and
zpqiu
committed
rebase and update refitting process
zpqiu
committed
fix refitting bugs after rebase
zpqiu
committed
Refactor FP8 KV cache scale handling by centralizing vLLM parameter naming and deduplicating conversion logic

Shuang Yu
authored and
zpqiu
committed
lint check
zpqiu
committed
Update to correct BF16 issue with load_weights

Shuang Yu
authored and
zpqiu
committed
WIP: changes before rebase

Shuang Yu
authored and
zpqiu
committed
Code draft to support dynamic kv scales calculation

Shuang Yu
authored and
zpqiu
committed
Refit should only takes care of kv_scales when kv_cache_dtype is fp8 otherwise bf16 won't work

Shuang Yu
authored and
zpqiu
committed
lint check
zpqiu
committed
remove debug prints
zpqiu
committed
make refitting with kv scales cleaner
zpqiu
committed

Commits on Nov 21, 2025

Commits on Nov 24, 2025

Merge branch 'main' into kv-cache-fp8
guyueh1
authored

Commits on Nov 25, 2025

Commits on Dec 1, 2025

update kv_cache_dtype with choices

zpqiu
and
terrykong
authored

Commits on Dec 2, 2025