-
Notifications
You must be signed in to change notification settings - Fork 929
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement Flex.1 Alpha Functionality #1893
base: sd3
Are you sure you want to change the base?
Conversation
Apparently this is a lot more difficult than I anticipated since the Flux model is stored differently in sd-scripts to how it is stored in ai-toolkit. |
Apparently this might be a lot simpler than I thought since there is already a Flux.params.guidance toggle to turn off guidance selectively. |
Thank you for this! Flex.1 Alpha looks promising. |
I would definitely love to hear comments or improvements. This is my first PR here and my knowledge of the inner workings of sd-scripts and of coding for neural nets in torch is weak at best, so this is really just me poking around in the dark. I really just wanted to get something up and running quickly so I could start training :) |
I can confirm from my testing at least that this does appear to be working correctly. The results from both full finetunes and LoRA seem to be consistent with each other and substantially better than a naive run without the bypass mechanism applied. |
Create --bypass_flux_guidance argument and mechanism to allow training of the Flex.1-Alpha model created by Ostris (https://huggingface.co/ostris/Flex.1-alpha). This is an Apache 2.0 model based off of Flux Schnell with the distillation removed and a custom guidance module created, which allows the model to be more easily finetuned.
Initial testing I've done confirms that this is in fact the case and the model is extremely easy to finetune compared to Flux Dev, with a minor deterioration in quality.
The implementation here is based off of this commit of ai-toolkit: ostris/ai-toolkit@8ef07a9
Currently, I have an issue created here in reference to this with some initial testing: #1891