Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Process separate q/k/v weights in MHA converter #1020

Closed
wants to merge 1 commit into from
Closed

Process separate q/k/v weights in MHA converter #1020

wants to merge 1 commit into from
+66 −2

Commits on Aug 12, 2024

  1. Process separate q/k/v weights in MHA converter 

    Summary:
ATT. The converter was not ready for the `not self._qkv_same_embed_dim` case [here](https://github.com/pytorch/pytorch/blob/80ed3e9ccdaab20814b4156611a19043aaaaef03/torch/nn/modules/activation.py#L1074) with separate q/k/v weights. Here we cover this case.

Intenral:

This causes a failure in the AIT lowering of the IGCTR MC model. See the post: https://fb.workplace.com/groups/gpuinference/permalink/2872581106223872/ .

Differential Revision: D61155566
    @aakhundov @facebook-github-bot
    aakhundov authored and facebook-github-bot committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    b813ac7 View commit details
    Browse the repository at this point in the history