[Feature] Batch Invariant: Support DeepGEMM and Blackwell #27127
Conversation
Signed-off-by: yewentao256 <[email protected]>
yewentao256
requested review from
mgoin,
robertgshaw2-redhat and
tlrmchlsmth
as code owners
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Signed-off-by: yewentao256 <[email protected]>
Signed-off-by: yewentao256 <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
yewentao256
deleted the
wentao-blackwell-deepgemm-support-for-batch-invariant
branch
| q_input, input_scale = QuantFP8( | ||
| False, | ||
| self.act_q_group_shape, | ||
| column_major_scales=True, | ||
| )(x) |
There was a problem hiding this comment.
I don't think this is the intended way to use QuantFP8. Is there a reason why this cannot be put in the shared block w8a8 utils so it can be reused for other backends like compressed tensors?
…ct#27127) Signed-off-by: yewentao256 <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]> Signed-off-by: Alberto Perdomo <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]> Signed-off-by: 0xrushi <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]> Signed-off-by: 0xrushi <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]>
…ct#27127) Signed-off-by: yewentao256 <[email protected]>
Purpose
Thanks for the previous effort from @bwasti !
Support DeepGEMM and Blackwell architecture
Test
Tested on B200