feat: add torchao int4, nf4, int8 LoRA QLoRA support#3417

Open

NanoCode012 wants to merge 18 commits into

mainfrom

feat/torchao-qlora

Commits on May 29, 2026

feat: add torchao's int4, nf4, int8

NanoCode012
authored and
winglian
committed
feat: handle lora kernels compat with torchao

NanoCode012
authored and
winglian
committed
feat: test for config validation and BC for new peft weight dtype

NanoCode012
authored and
winglian
committed
feat(torchao-qlora): harden the feature for production
winglian
committed
feat(torchao-qlora): wire NVFP4 + FP8 weight-only paths
winglian
committed
feat(torchao-qlora): surface mixed-quant conflicts instead of silent overrides
winglian
committed
fix(torchao-qlora): hoist quantized-checkpoint conflict check above the awq/gptq/bnb branch
winglian
committed
refactor(torchao-qlora): structured discriminator under model_quantization_config
winglian
committed
feat(torchao-qlora): drop adapter: qlora; QLoRA is lora + 4-bit base
winglian
committed
feat(torchao-qlora): invert source of truth — model_quantization_config first, load_in_4bit/8bit deprecated
winglian
committed

Commits on May 30, 2026

fix(torchao-qlora): reject qlora+gptq / qlora+load_in_4bit:false; adapt legacy tests
winglian
committed

Commits on May 31, 2026

fix(lora-kernels): treat nn.Parameter as plain Tensor in quant discriminator
winglian
committed