Fail early for unsupported PEFT + Liger Kernel in DPO

[albertvillanova]

Fail early for unsupported PEFT + Liger Kernel in DPO.

This pull request introduces a check to prevent the use of the Liger DPO loss with PEFT models, as this combination is not yet supported. It also adds a corresponding test to ensure that initializing a DPOTrainer with both Liger Kernel and PEFT raises a clear error.

Changes

Validation and error handling improvements:

• Added a check in the DPOTrainer initializer to raise a NotImplementedError if both use_liger_kernel is enabled and the model is a PEFT model, with a clear error message.

Testing enhancements:

• Added a new test (test_init_fails_with_peft_and_liger) in tests/test_dpo_trainer.py to verify that initializing DPOTrainer with both Liger Kernel and PEFT fails as expected, raising the correct error.

---

Note

Low Risk
Low risk: adds an early validation guard and a regression test, without changing supported training flows beyond producing a clearer error sooner for an already-unsupported configuration.

Overview
Prevents unsupported configurations by failing fast when initializing DPOTrainer with use_liger_kernel=True on a PEFT-wrapped model, raising a clear NotImplementedError.

Adds a regression test (test_init_fails_with_peft_and_liger) to ensure the PEFT+Liger combination errors at init time, and removes the now-redundant PEFT check from the Liger loss computation path.

^{Reviewed by Cursor Bugbot for commit 623f168. Bugbot is set up for automated code reviews on this repo. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

maybe we could remove this then?

trl/trl/trainer/dpo_trainer.py

Lines 1101 to 1102 in 62a66af

	if is_peft_model(model):
	raise NotImplementedError("Liger DPO loss is not implemented for PEFT models.")

[albertvillanova]

I thought the same! But codex told me to keep it... I should have followed my intuition.