Merged
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
This reverts commit 077fd59.
danielhanchen
force-pushed
the
dh/recover-2752-vlm-grpo-credit
branch
from
e8214b2 to
f18186a
Compare
Contributor
Author
|
@GAD-cell thanks so much again and apologies on the delay on the PR. We changed some of the design of components in Unsloth, so we had to rebase your PR. We accepted your contribution and appreciate it, although we had to change it quite a bit. Thanks again. |
shibizhao
pushed a commit
to shibizhao/unsloth-npu
that referenced
this pull request
Apr 7, 2026
* Updated rl and rl_replacements * Revert "Updated rl and rl_replacements" This reverts commit 077fd59. --------- Co-authored-by: Sinoué GAD <85933501+GAD-cell@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replacement for #2752 due to Studio rebasing
This replacement preserves the original authored change in repo history and immediately reverts it in the same PR so current main behavior stays unchanged.
This PR aims to add support for VLMs in GRPO, which is currently not supported by HF.
I've implemented a working version that does not yet include VLLM or video input support (mainly due to limited resources for testing video inputs haha).
I added a new variable, use_vision, to the GRPO config. Setting use_vision = True enables vision inputs, while use_vision = False keeps the default GRPO behavior. Default is False.
I also had to change a function in unsloth_zoo.peft_utils (requires_grad_post_hook) to make it work.
I've tested the implementation with Qwen 2.5 VL 7B for 250 steps, and training appears to proceed correctly (see TensorBoard screenshots for reference).