New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rationale behind converting proj_out of FluxSingleTransformerBlock to ConcatLinear #43

Open

vinovo opened this issue Jan 29, 2025 · 0 comments

Labels

question svdquant

vinovo commented Jan 29, 2025 •

edited

Loading

We can see only proj_out of FluxSingleTransformerBlock are converted to ConcatLinear with only a single split [module.proj_out.out_features].

Can anyone help explain the reasoning behind this?

Why we only cares about proj_out of FluxSingleTransformerBlock. Should this operation be performed on all transformer based diffusion models?
Why do we only care to create one single split? What is the benefit of using ConcatLinear?

Thank you in advance.

The text was updated successfully, but these errors were encountered:

lmxyy added question svdquant labels

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment