Skip to content

[feat][DCP] Enable DCP with fp8 MTP - #1781

Merged
yitingw1 merged 2 commits into
mainfrom
yitingw1/enable_DCP_fp8MTP
Aug 7, 2026
Merged

[feat][DCP] Enable DCP with fp8 MTP#1781
yitingw1 merged 2 commits into
mainfrom
yitingw1/enable_DCP_fp8MTP

Conversation

@yitingw1

@yitingw1 yitingw1 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Motivation

DCP + MTP (speculative decode) previously required bf16 KV cache — the round-robin CP (cprr) MLA kernel had no fp8 variant, so ATOM raised NotImplementedError for fp8. The ROCm/aiter#4521 now ships fp8 cp-round-robin kernels (gqa16/gqa32, qseqlen4, plain + lse), so fp8 KV cache can be combined with MTP under DCP — cutting per-GPU KV memory further while keeping speculative decode.

Need ROCm/aiter#4521 merged first.

Test

8*MI355 DeepSeek-R1, -tp8, --method mtp, gsm8k 5-shot (flexible / strict), 0 crash:

KV dtype num_spec dcp gsm8k
fp8 1 8 0.9560 / 0.9545 (full 1319)
fp8 2 8 0.9553 / 0.9560 (full 1319)
fp8 3 8 0.9538 / 0.9507 (full 1319)
fp8 3 4 0.95 / 0.95 (limit 100)
fp8 3 2 0.94 / 0.94 (limit 100)

Submission Checklist

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 1781 --add-label <label>

@zufayu
zufayu requested a review from ZhangLirong-amd August 5, 2026 01:39
@yitingw1
yitingw1 merged commit 0ae0434 into main Aug 7, 2026
47 of 49 checks passed
@yitingw1
yitingw1 deleted the yitingw1/enable_DCP_fp8MTP branch August 7, 2026 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants