Lora Trainer Setup Guides

Lora Trainer Sezup Guides for Nvidia and AMD:

Nvidia Install Guides:

OneTrainer
Kohya_ss
[FluxGym]

AMD Install Guides:

[NVIDIA] OneTrainer

Install OneTrainer on AMD Windows for NVIDIA GPUs:
Preparation:
Make sure your Graphics Card Driver is updated.
You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win
You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/
In the first Python install screen check "Add Python to Path"

Installing OneTrainer:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command:
git clone https://github.com/Nerogar/OneTrainer.git
Press enter and after its done you can close the cmd.
Launch the install.bat, and wait until its done. Then close the cmd.
Launch the update.bat, and wait until its done. Then close the cmd.
Launch the start-ui.bat and its ready to use.

(Source: https://github.com/Nerogar/OneTrainer)

[NVIDIA] Kohya_ss

Install Kohya_ss on AMD Windows for NVIDIA GPUs:
Preparation:
Make sure your Graphics Card Driver is updated.
You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win
You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/
In the first Python install screen check "Add Python to Path"

Installing OneTrainer:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command:
git clone --recursive https://github.com/bmaltais/kohya_ss.git
Press enter and after its done you can close the cmd.
If only Python 3.10.11 64bit is installed, launch the setup.bat, and wait until its done. Then close the cmd.
If multiple Python Versions are installed, launch the setup-3.10.bat, and wait until its done. Then close the cmd.
Launch the gui.bat and its ready to use.

Optional: CUDNN 8.9.6.50
The following steps are optional but will improve the learning speed for owners of NVIDIA 30X0/40X0 GPUs. These steps enable larger training batch sizes and faster training speeds.
Run the setup.bat and select 2. (Optional) Install cudnn files (if you want to use the latest supported cudnn version).

(Source: https://github.com/bmaltais/kohya_ss)

[AMD] OneTrainer with ZLUDA

Install OneTrainer on AMD Windows with Zluda:
Supported GPUs
Requires AMD Adrenalin 24.1.1 Driver or newer.

Preparation:
Make sure your Graphics Card Driver is updated.
You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win
You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/
In the first Python install screen check "Add Python to Path"

Getting the OneTrainer AMD Fork:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda.
Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command:
git clone https://github.com/lshqqytiger/OneTrainer.git && cd OneTrainer && py scripts/install_zluda.py
Press enter and after its done you can close the cmd.
Launch the install.bat, and wait until its done. Then close the cmd.
Launch the update.bat, and wait until its done. Then close the cmd.

Getting Zluda to run:
Important: If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is higher/newer than a RX6800, then:

Download and install AMD HIP SDK 6.2.4 from here
Download the latest Zluda files for ROCm 6 from here
Unzip as Folder and move it onto the C: Drive and rename the Folder to ZLUDA
Then restart your PC and skip to Step 4 below and proceed there with the Guide.

If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is below/older than a RX 6800, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.1
If the files are available for your GPU then you need to download them from here:
Install AMD HIP SDK 6.2.4 from here
Download the latest Zluda files for ROCm 6 from here
Unzip as Folder and move it onto the C: Drive and rename the Folder to ZLUDA
Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder.
There copy and rename the library folder to old_library
Open the .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.
If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder.
Important: Restart your PC after this Step before proceeding at Step 4 (skip 1-3) of the Guide below.

If your GPU is not supported by ROCm 6.2.4 then proceed with the Guide at Step 1 below:
IMPORTANT: If your GPU is higher/newer than a RX6800, then skip Step 3

Download and install AMD HIP SDK 6.1.2 from here: https://www.amd.com/en/developer/resources/rocm-hub/hip-sdk.html
Dont check the Pro Driver.
Download the latest ZLUDA build for ROCm 6.1.2 as zip file from here
Unzip as Folder and move it onto the C: Drive and rename the Folder to ZLUDA
If your GPU is below a RX 6800, you need to do the following steps:
Check your GPUs GFX Version here: GPU GFX List
3.1 Download the corresponding .zip File for your GPUs GFX Version from here
For gfx900/906 - VEGA 56, 64 and Vii its this
3.2 Go into C:\Program Files\AMD\ROCm\6.1\bin\rocblas folder.
There copy and rename the library folder to old_library
3.3 Open the .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.
If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.1\bin\rocblas folder.
Important: Restart your PC after this Step before proceeding.
Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here:
https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows
It should look like this here
Go into the C:\ZLUDA\ Folder.
There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll
Copy all files to OneTrainer/.zluda and overwrite if asked.
Launch the start-ui.bat. It will open up in your default Browser.
Noteworthy: The first Training can take longer as it will first initialize ZLUDA in the background. This can take up from 15 to 40 minutes. First time only.

Attention: In the Training tab set the Attention to Default or SDP instead of xformers as xformers won't work on AMD and you'll get an Error.

For training Loras with OneTrainer checkout this Guide on Civitai.com:
https://civitai.com/articles/4789/lazy-lora-making-with-onetrainer-and-ai-generation

(Source: https://github.com/lshqqytiger/OneTrainer)

[AMD] FluxGym with ZLUDA

Install FluxGym on AMD Windows with Zluda:
Supported GPUs
Requires AMD Adrenalin 24.1.1 Driver or newer.

Preparation:
Make sure your Graphics Card Driver is updated.
You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win
You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/
In the first Python install screen check "Add Python to Path"

Getting FluxGym:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda.
Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command:
git clone https://github.com/cocktailpeanut/fluxgym && cd fluxgym && git clone -b sd3 https://github.com/kohya-ss/sd-scripts
Press enter and after its done you can close the cmd.

Installing/Launching FluxGym:
Now we need to create a Launch File to easily install and start FluxGym:

To create the File first open the link below in a new tab:
https://raw.githubusercontent.com/CS1o/Stable-Diffusion-Info/main/Resources/Start-FluxGym.bat
Then on the Site make a right click and select "Save Site as/under"
Before Saving make sure to set the Filetype to All Files (.) then press Save.
If it's still a .txt file, then you need to Enable the File Extensions in the File Explorer by clicking:
in Win10 View > File Name Extensions
in Win11 View > Show > File Name Extensions
Then rename the File and change the .txt to .bat
Move the Start-FluxGym.bat into the fluxgym Folder.
Launch the Start-FluxGym.bat. It will now install everything needed and when it's done the Webui opens in your default Browser.
Using FluxGym wont work as Zluda is needed, so close everything and proceed with the next steps.

Getting Zluda to run:
Important: If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is higher/newer than a RX6800, then:

Download and install AMD HIP SDK 6.2.4 from here
Download the latest Zluda files for ROCm 6 from here
Unzip as Folder and move it onto the C: Drive and rename the Folder to ZLUDA
Then restart your PC and skip to Step 4 below and proceed there with the Guide.

If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is below/older than a RX 6800, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.1
If the files are available for your GPU then you need to download them from here:
Install AMD HIP SDK 6.2.4 from here
Download the latest Zluda files for ROCm 6 from here
Unzip as Folder and move it onto the C: Drive and rename the Folder to ZLUDA
Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder.
There copy and rename the library folder to old_library
Open the .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.
If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder.
Important: Restart your PC after this Step before proceeding at Step 4 (skip 1-3) of the Guide below.

If your GPU is not supported by ROCm 6.2.4 then proceed with the Guide at Step 1 below:
IMPORTANT: If your GPU is higher/newer than a RX6800, then skip Step 3

Download and install AMD HIP SDK 6.1.2 from here: https://www.amd.com/en/developer/resources/rocm-hub/hip-sdk.html
Dont check the Pro Driver.
Download the latest ZLUDA build for ROCm 6.1.2 as zip file from here
Unzip as Folder and move it onto the C: Drive and rename the Folder to ZLUDA
If your GPU is below a RX 6800, you need to do the following steps:
Check your GPUs GFX Version here: GPU GFX List
3.1 Download the corresponding .zip File for your GPUs GFX Version from here
For gfx900/906 - VEGA 56, 64 and Vii its this
3.2 Go into C:\Program Files\AMD\ROCm\6.1\bin\rocblas folder.
There copy and rename the library folder to old_library
3.3 Open the .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.
If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.1\bin\rocblas folder.
Important: Restart your PC after this Step before proceeding.
Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here:
https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows
It should look like this here
Go into the C:\ZLUDA\ Folder.
There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll
Copy the three renamed files to fluxgym\venv\Lib\site-packages\torch\lib and overwrite if asked.
Now go into the fluxgym\sd-scripts Folder.
There right click the flux_train_network.py and edit it with Notepad or Notepad++.
In the Script at line 7 it looks like this:

import torch
from accelerate import Accelerator

change it to:

import torch
from accelerate import Accelerator
import torch
torch.backends.cudnn.enabled = False
torch.backends.cuda.enable_flash_sdp(False)
torch.backends.cuda.enable_math_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(False)
if hasattr(torch.backends.cuda, "enable_cudnn_sdp"):
    torch.backends.cuda.enable_cudnn_sdp(False)

Save and close the File.

Launch the Start-FluxGym.bat. It will open up in your default Browser.
Noteworthy: The first Training can take longer as it will first initialize ZLUDA in the background. This can take up from 15 to 40 minutes. First time only.

(Source: https://github.com/cocktailpeanut/fluxgym)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly