Rotation and Permutation for Advanced Outlier Management and Efficient Quantization of LLMs

Welcome to the official code repository for Rotation and Permutation for Advanced Outlier Management and Efficient Quantization of LLMs.

Code Availability

The code will be made available in a short time. Please check back soon or watch this repository to get notified upon release.

Contact

For immediate queries or further information, please open an issue or contact [email protected] or [email protected].

Acknowledgement

This repo is build upon the following projects:

• OmniQuant
• IntactKV
• EAGLE
• FastChat

We thank the authors for their code.

Citation

Please cite our work if you use our code or discuss our findings in your own research:

@article{lin2024rotation,
  title={Rotation and Permutation for Advanced Outlier Management and Efficient Quantization of LLMs},
  author={Lin, Haokun and Xu, Haobo and Wu, Yichen and Cui, Jingzhi and Zhang, Yingtao and Mou, Linzhan and Song, Linqi and Sun, Zhenan and Wei, Ying},
  journal={arXiv preprint arXiv:2406.01721},
  year={2024}
}