[ROCm] Fix DeepSeek R1/V3 incorrect output in eager mode.#27392
[ROCm] Fix DeepSeek R1/V3 incorrect output in eager mode.#27392Duyi-Wang wants to merge 1 commit intovllm-project:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request addresses an issue where DeepSeek models produce incorrect output on ROCm in eager mode. The fix involves changing how forward_hip calls forward_cuda for rotary embeddings, correctly using the returned values instead of assuming an in-place operation. This change appears correct based on the problem description. I've added one comment to improve code clarity by removing a now-misleading comment.
| # ops.rotary_embedding() is an in-place operation | ||
| # that updates the query and key tensors. |
There was a problem hiding this comment.
These comments state that an internal operation is in-place, which contradicts the FIXME on the next line and the logic of the fix (which treats forward_cuda as not in-place by using its return value). To avoid confusion for future readers and maintainers, it would be better to remove these now-misleading comments.
…roject#19) Signed-off-by: Duyi-Wang <duyi.wang@amd.com>
47bdbdf to
8e17ff2
Compare
|
A similar fix recently got merged: #27373 |
Purpose
DeepSeek R1/V3 models produce incorrect output when running in eager mode, while graph mode works correctly.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.