Fix dimension issues for int4 weight only quant path

[jerryzh168]

Summary:
Currently the accepted dimension of _quantized_linear is not clear, this PR fixes the issue.

Currently the "tensor_core_tiled" layout tensor does not do repacking in view operation, which is incorrect, this PR removes the view support (which is not needed right now), and restrict the use case to transpose op, and records the transpose status of the tensor instead of doing repacking for performance.

Test Plan:
python test/quantization/test_quant_api.py
python test/integration/test_integration.py

TORCH_LOGS='output_code' python tutorials/quantize_vit/run_vit_b_quant.py

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/330

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Rebase your PRs: Unstable CUDA signal in CI caused by cudnn 9 update

✅ No Failures

As of commit 0732101 with merge base e2196fd ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[HDCharles]

why did you change teh name to transpose change shape? it doesn't have anything to do with the transpose handling, it just changes teh external shape.

[jerryzh168]

why did you change teh name to transpose change shape? it doesn't have anything to do with the transpose handling, it just changes teh external shape.

it's not a general _change_shape, since it only makes sense with transpose dimension I think? otherwise we need to unpack and repack the packed weight

[cpuhrsch]

I'd also like to suggest some unit tests for aqt eventually. They're actually quite easy to write for shape operations, because a dtype has no influence on the change in shape.

That means you can do self.assertEqual(fn(t).shape, fn(to_aqt(t)).shape).

We also have OpInfos and such that could be used here. For example, some tests might easily translate, just with different tolerances due to decreased bit width.

[jerryzh168]

I'd also like to suggest some unit tests for aqt eventually. They're actually quite easy to write for shape operations, because a dtype has no influence on the change in shape.

That means you can do self.assertEqual(fn(t).shape, fn(to_aqt(t)).shape).

We also have OpInfos and such that could be used here. For example, some tests might easily translate, just with different tolerances due to decreased bit width.

yeah sure, I was thinking of adding some tests for aqt afterwards, but I can start with transpose

[jerryzh168]

why did you change teh name to transpose change shape? it doesn't have anything to do with the transpose handling, it just changes teh external shape.

OK removed the function since it's a bit confusing, we can add back later if needed. I understand it's just changing the external shape without touching the internal data representation now.

[msaroufim]

@HDCharles mind reviewing this, one more time?