Train the lora flux with 2 x 4090

[ilkergalipatak]

Hello, would 2 x Nvidia 4090 be enough to train the flux model or would there be a need for vram while training? Can we allocate the required memory size over 2 GPUs?

[bghira]

you can't currently do multi-gpu quantised training due to #644

it'd take a lot more 4090s than that to train it without quantisation. you would need DeepSpeed Zero 2 or 3.

it's not something I'd recommend - a single A40 or A100-40 or A6000 or L40(S) would do better.