Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed f-string printing of NF4Tensors #297

Merged
merged 2 commits into from
May 30, 2024
Merged

Conversation

awgu
Copy link
Contributor

@awgu awgu commented May 30, 2024

import torch
from torchao.dtypes import NF4Tensor

weight = torch.randn((16, 16), device="cuda", dtype=torch.bfloat16)
block_size, scaler_block_size = 2, 2
nf4_weight = NF4Tensor.from_tensor(weight, block_size, scaler_block_size)
print(f"{nf4_weight}")
File "/data/users/andgu/pytorch/test_nf4.py", line 13, in main
    print(f"{nf4_weight}")
  File "/data/users/andgu/pytorch/torch/_tensor.py", line 986, in __format__
    return handle_torch_function(Tensor.__format__, (self,), self, format_spec)
  File "/data/users/andgu/pytorch/torch/overrides.py", line 1646, in handle_torch_function
    result = torch_func_method(public_api, types, args, kwargs)
  File "/data/users/andgu/ao/torchao/dtypes/nf4tensor.py", line 797, in __torch_function__
    return func(*args, **kwargs)
  File "/data/users/andgu/pytorch/torch/_tensor.py", line 989, in __format__
    return object.__format__(self, format_spec)
  File "/data/users/andgu/ao/torchao/dtypes/nf4tensor.py", line 754, in __str__
    return self.to(torch.float32).__str__()
  File "/data/users/andgu/ao/torchao/dtypes/nf4tensor.py", line 778, in __torch_dispatch__
    return NF4_OPS_TABLE[func](func, args, kwargs)
  File "/data/users/andgu/ao/torchao/dtypes/nf4tensor.py", line 277, in _to_copy
    return args[0][0].get_original_weight().to(args[1]["dtype"]).to(args[1]["device"])
KeyError: 'device'

Note that args[1] only has a "dtype" key, not "device":

args[1]: {'dtype': torch.float32}

Copy link

pytorch-bot bot commented May 30, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/297

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit 613d577 with merge base 38dad9b (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 30, 2024
@msaroufim msaroufim self-requested a review May 30, 2024 15:23
@msaroufim msaroufim merged commit 4c1d568 into pytorch:main May 30, 2024
13 checks passed
dbyoung18 pushed a commit to dbyoung18/ao that referenced this pull request Jul 31, 2024
yanbing-j pushed a commit to yanbing-j/ao that referenced this pull request Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants