FIX:ValueError:You passed a PeftModel instance with peft_config

[ved1beta]

Description

ValueError: You passed a PeftModel instance together with a peft_config to the trainer.
for SIMPO
just pass simpo in rl type

Motivation and Context

https://discord.com/channels/1104757954588196865/1502375546409975939/1504310571196219442

test

config runs fine

Summary by CodeRabbit

• Chores
  • Updated reinforcement learning trainer configuration handling for improved compatibility with additional model types.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 01217778-f8c4-4456-8164-286e07b19fc0

📥 Commits

Reviewing files that changed from the base of the PR and between d7cb1c9 and 8e6f812.

📒 Files selected for processing (1)

• src/axolotl/core/builders/rl.py

---

📝 Walkthrough

Walkthrough

HFRLTrainerBuilder.build updates the condition that gates peft_config assignment to exclude SIMPO as an additional RL type. SIMPO now joins GRPO, ORPO, and EBFT as RL types that do not receive peft_config during trainer construction.

Changes

SIMPO RL trainer configuration

Layer / File(s)	Summary
peft_config conditional update src/axolotl/core/builders/rl.py	The condition determining when peft_config is attached to the RL trainer now excludes SIMPO alongside existing exclusions for GRPO, ORPO, and EBFT.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the bug being fixed—a ValueError involving PeftModel and peft_config—and is specific to the actual change, which excludes SIMPO from peft_config attachment in the RL trainer builder.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[NanoCode012]

Nice catch