对GPTQ模型或AWQ模型进行lora训练 #9101
Description
Reminder
- I have read the above rules and searched the existing issues.
System Info
llamafactoryversion: 0.9.3.dev0- Platform: Linux-6.2.0-36-generic-x86_64-with-glibc2.35
- Python version: 3.11.13
- PyTorch version: 2.7.0+cu126 (GPU)
- Transformers version: 4.52.4
- Datasets version: 3.6.0
- Accelerate version: 1.7.0
- PEFT version: 0.14.0
- TRL version: 0.8.6
- GPU type: NVIDIA GeForce RTX 4080 SUPER
- GPU number: 1
- GPU memory: 15.59GB
- vLLM version: 0.9.1
- Default data directory: detected
Reproduction
我在尝试通过Lora训练Qwen3-0.6B-GPTQ-INT8模型时，出现报错：
ValueError: Target module TritonV2QuantLinear() is not supported. Currently, only the following modules are supported: `torch.nn.Linear`, `torch.nn.Embedding`, `torch.nn.Conv2d`, `torch.nn.Conv3d`, `transformers.pytorch_utils.Conv1D`.
我尝试查找PEFT库的相关文档，并尝试按照其流程修改了源码，但是总是会在get_peft_model（）这个函数出现报错，显示不支持Triton加速的量化模型，我想问一下是无法Lora训练GPTQ和AWQ的模型么？只能训练官方推荐的bitsandbytes模型么？QLora实际也是把原始模型先转换为bnb模型再训练么？
Others
No response