Pass HFQuantizer to from_pretrained kwargs

[liamd101]

Feature request

Currently, when loading a model in quantized form, the HFQuantizer is created based on other kwargs passed into the from_pretrained function. See current implementation below:

# modeling_utils::from_pretrained()
    if pre_quantized or quantization_config is not None:
        if pre_quantized:
            config.quantization_config = AutoHfQuantizer.merge_quantization_configs(
                config.quantization_config, quantization_config
            )
        else:
            config.quantization_config = quantization_config
        hf_quantizer = AutoHfQuantizer.from_config(config.quantization_config, pre_quantized=pre_quantized)
    else:
        hf_quantizer = None

This should be a straightforward addition, by adding the following lines:

# modeling_utils::from_pretrained()
    hf_quantizer = kwargs.pop("hf_quantizer", None)
    if hf_quantizer is not None:
        pass
    elif pre_quantized or quantization_config is not None:
        ...

Motivation

This would give users more flexibility, and allow one to easily create and integrate custom implementations of the HFQuantizer class. I am personally working on a project where this change is necessary to work with quantization methods that have not yet been added to the library

Your contribution

I can make a PR and contribution

[amyeroberts]

cc @SunMarc