Skip to content

[ROCm] Support DeepSeek V4.1 PTPC-FP8 runtime layouts - #57760

Open
wangjiaxin99 wants to merge 1 commit into
vllm-project:mainfrom
wangjiaxin99:jiaxwang/deepseek-v41-ptpc-fp8-runtime
Open

wangjiaxin99 wants to merge 1 commit into
vllm-project:mainfrom
wangjiaxin99:jiaxwang/deepseek-v41-ptpc-fp8-runtime

Conversation

@wangjiaxin99

@wangjiaxin99 wangjiaxin99 commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

This PR adds the remaining runtime support needed to run
amd/DeepSeek-V4.1-Flash-Quark-PTPC-FP8
on ROCm.

This PR is built on top of PR #57071. It relies on PR #57071 for:

  • DeepSeek V4.1 Quark mixed-precision checkpoint loading.
  • Quantization mapper compatibility, including routing engram.embed scale
    tensors into the engram module.
  • OCP-MX/MXFP8 linear support needed by the base DeepSeek V4.1 quantized model.

This PR only adds the remaining PTPC-FP8 runtime layout support that is not part
of PR #57071.

Changes

  • Support PTPC-FP8 wo_a reconstruction in the ROCm sparse MLA path.

    • Accept per-channel scale tensors.
    • Handle Quark-loaded [in, out] wo_a weights before caching the BF16
      layout used by the inverse-RoPE einsum.
    • Preserve the existing MXFP8/block-scale path.
  • Support PTPC-FP8 weight-only engram.embed row-wise scales.

    • Allow 1-D FP32 row-wise scale shards in ParallelEngramEmbedding.
    • Add a Triton lookup branch for row-wise FP32 scales.
    • Preserve the existing E8M0 block-scale path.

Validation

  • Docker image:

    • vllm/vllm-openai-rocm:nightly-eed1f3d0c6043bd494424a22443ee198dd56f657
  • GSM8K full eval:

    • Original DeepSeek V4.1 Flash model:
      • flexible-extract: 0.9295
      • strict-match: 0.9295
    • amd/DeepSeek-V4.1-Flash-Quark-PTPC-FP8:
      • flexible-extract: 0.9295
      • strict-match: 0.9287

Signed-off-by: jiaxwang <jiaxwang@amd.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added deepseek Related to DeepSeek models DSv4.1 Related to DeepSeek-V4.1 models rocm Related to AMD ROCm labels Sep 20, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Sep 20, 2026
@mergify

mergify Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wangjiaxin99.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Oct 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek Related to DeepSeek models DSv4.1 Related to DeepSeek-V4.1 models needs-rebase rocm Related to AMD ROCm

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant