Implement support for AutoGPTQ for loading GPTQ quantized models.#1668

Merged

oobabooga merged 35 commits into

oobabooga:mainfrom

Jun 2, 2023

Commits on Apr 30, 2023

Implement support for AutoGPTQ for loading GPTQ quantized models.
LaaZa
committed
Disable cpu support for now. Something in the inference path assumes gpu.
LaaZa
committed
Implement offloading and splitting between multiple devices.
LaaZa
committed

Commits on May 1, 2023

Check for quantize_config.json and set wbits and groupsize according to it.
LaaZa
committed
More robust quantize config parsing and setting for act-order or desc_act
LaaZa
committed

Commits on May 2, 2023

Commits on May 3, 2023

Commits on May 5, 2023

Merge branch 'oobabooga:main' into AutoGPTQ
LaaZa
authored

Commits on May 7, 2023

Merge branch 'oobabooga:main' into AutoGPTQ
LaaZa
authored

Commits on May 9, 2023

Commits on May 11, 2023

Merge branch 'oobabooga:main' into AutoGPTQ
LaaZa
authored

Commits on May 12, 2023

Commits on May 13, 2023

Merge branch 'oobabooga:main' into AutoGPTQ
LaaZa
authored

Commits on May 14, 2023

Commits on May 15, 2023

Commits on May 16, 2023

Commits on Jun 2, 2023

Merge branch 'main' into LaaZa-AutoGPTQ
oobabooga
committed