Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Flex.1 Alpha Functionality #1893

Open
wants to merge 7 commits into
base: sd3
Choose a base branch
from

Conversation

stepfunction83
Copy link

Create --bypass_flux_guidance argument and mechanism to allow training of the Flex.1-Alpha model created by Ostris (https://huggingface.co/ostris/Flex.1-alpha). This is an Apache 2.0 model based off of Flux Schnell with the distillation removed and a custom guidance module created, which allows the model to be more easily finetuned.

Initial testing I've done confirms that this is in fact the case and the model is extremely easy to finetune compared to Flux Dev, with a minor deterioration in quality.

The implementation here is based off of this commit of ai-toolkit: ostris/ai-toolkit@8ef07a9

Currently, I have an issue created here in reference to this with some initial testing: #1891

@stepfunction83
Copy link
Author

Apparently this is a lot more difficult than I anticipated since the Flux model is stored differently in sd-scripts to how it is stored in ai-toolkit.

@stepfunction83
Copy link
Author

Apparently this might be a lot simpler than I thought since there is already a Flux.params.guidance toggle to turn off guidance selectively.

@kohya-ss
Copy link
Owner

kohya-ss commented Jan 23, 2025

Thank you for this! Flex.1 Alpha looks promising.
I'm currently working on another slightly larger PR, so please wait a little longer for review and merging.

@stepfunction83
Copy link
Author

stepfunction83 commented Jan 23, 2025

I would definitely love to hear comments or improvements. This is my first PR here and my knowledge of the inner workings of sd-scripts and of coding for neural nets in torch is weak at best, so this is really just me poking around in the dark.

I really just wanted to get something up and running quickly so I could start training :)

@stepfunction83
Copy link
Author

stepfunction83 commented Jan 24, 2025

I can confirm from my testing at least that this does appear to be working correctly. The results from both full finetunes and LoRA seem to be consistent with each other and substantially better than a naive run without the bypass mechanism applied.

kijai added a commit to kijai/ComfyUI-FluxTrainer that referenced this pull request Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants