Fix regression loading dtype

[SunMarc]

What does this PR do?

This PR fixes the regression that @BenjaminBossan found out. It was caused by this PR. Basically, old_param was always going to be None with the current logic. Hence, we don't set the correct dtype to the parameter param = param.to(old_param.dtype). This caused a dtype mismatch with torchao.

[...]
    return torch.nn.functional.linear(input_tensor, weight_tensor, bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: self and mat2 must have the same dtype, but got Float and Half

To reproduce

import torch
from transformers import AutoModelForCausalLM, TorchAoConfig

quantization_config = TorchAoConfig(quant_type="int8_dynamic_activation_int8_weight")
model_id = "facebook/opt-125m"
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=quantization_config)
inputs = torch.arange(10).view(-1, 1)
model(inputs)

I tested that hqq serialization and the new test that I added both passed

[BenjaminBossan]

I don't have enough background to judge if the fix is good, but the test should cover it, as the generate call itself would raise an error without the fix.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

Thanks, noting as it will be in a patch

[ArthurZucker]

😅 I knew it!
It was a bit too specific