2-bit quantizations#4897

Merged

ikawrakow merged 5 commits intomasterfrom

ik/quantize-iq2

Jan 14, 2024

Commits on Jan 12, 2024

imatrix: load
Iwan Kawrakow
committed
imatrix: WIP
Iwan Kawrakow
committed
imatrix: Add Q2_K quantization
Iwan Kawrakow
committed
imatrix: also guard against Q2_K_S quantization without importance matrix
Iwan Kawrakow
committed
imatrix: guard even more against low-bit quantization misuse
Iwan Kawrakow
committed