Skip to content

[ROCm][Bugfix] Ignore redundant qzeros in symmetric RDNAHybridW4A16 - #52637

Open
tangzzycc wants to merge 1 commit into
vllm-project:mainfrom
tangzzycc:rocm-rdna-hybrid-symmetric-qzeros
Open

tangzzycc wants to merge 1 commit into
vllm-project:mainfrom
tangzzycc:rocm-rdna-hybrid-symmetric-qzeros

Conversation

@tangzzycc

Copy link
Copy Markdown

Purpose

Symmetric GPTQ uint4b8 uses a fixed zero point of 8, but some checkpoints still contain redundant packed qzeros. RDNAHybridW4A16LinearKernel previously passed this tensor to the kernel, causing startup failures such as:

AssertionError: zp shape mismatch: torch.Size([40, 4352]) vs (34816, 40)

This PR drops redundant qzeros after loading symmetric GPTQ weights and adds regression coverage. The existing asymmetric zero-point path is unchanged.

Duplicate-work check

Open PR searches for RDNAHybridW4A16LinearKernel qzeros and symmetric qzeros ROCm W4A16 found no duplicate. PRs #47770 and #48998 only address TritonW4A16LinearKernel; neither changes the RDNA hybrid kernel.

Test Plan

Component Value
GPU AMD Radeon 8060S (gfx1151)
Memory 128 GB unified memory (96 GiB VRAM)
Operating system Ubuntu 24.04.4 LTS
ROCm 7.2.3

I ran the focused and complete hybrid W4A16 tests, applicable pre-commit hooks, and an end-to-end before/after serving test with the official Qwen/Qwen3.5-27B-GPTQ-Int4 checkpoint.

Test commands
VLLM_ROCM_USE_AITER=0 VLLM_ENABLE_V1_MULTIPROCESSING=0 \
.venv/bin/vllm serve ~/models/Qwen3.5-27B-GPTQ-Int4 \
  --language-model-only --attention-backend ROCM_ATTN \
  --max-model-len 2048 --max-num-seqs 1 \
  --gpu-memory-utilization 0.3 --enforce-eager --port 8001

Test Result

Revision Result
Before this PR Startup failed during the profiling forward pass with the qzeros shape mismatch above
With this PR Engine startup completed and the Chat Completions API generated 32 tokens successfully
  • Focused weight-processing tests: 6 passed, 77 deselected, 14 warnings
  • Complete hybrid W4A16 test file: 83 passed, 14 warnings
  • Applicable pre-commit hooks: passed
  • git diff --check: passed

AI assistance

OpenAI Codex assisted with this change. I reviewed and understand every submitted line and ran the validation reported above.

No documentation update is required because this change does not alter any user-facing API or configuration.


Essential Elements of an Effective PR Description Checklist
  • Purpose and failure mode documented.
  • Duplicate PRs checked.
  • Test commands and results included.
  • End-to-end model serving validated.
  • Documentation impact considered.
  • AI assistance disclosed.

Assisted-by: OpenAI Codex
Signed-off-by: tangzzycc <3081129260@qq.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added rocm Related to AMD ROCm bug Something isn't working labels Aug 17, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Aug 17, 2026
)
w_zp = zp_unpacked.to(c.act_type).contiguous()
self._transform_param(layer, self.w_zp_name, lambda x: w_zp)
elif self.w_zp_name is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be removed in the GPTQ loader instead of the RDNAHybridW4A16 kernel? I guess the same mismatch between c.zero_points and self.w_zp_name might also confuse other kernels.

@mergify

mergify Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @tangzzycc.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working needs-rebase rocm Related to AMD ROCm

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

2 participants