Code

Deep speed#1139

Code

Merged

kohya-ss merged 22 commits intodevkohya-ss/sd-scripts:devfrom deep-speedkohya-ss/sd-scripts:deep-speedCopy head branch name to clipboardMar 26, 2024

+288-59

Commits

Commits on Feb 3, 2024

• support deepspeed

  BootsofLagrangian

  committed
  dfe08f3

Commits on Feb 5, 2024

• fix offload_optimizer_device typo

  BootsofLagrangian

  committed
  64873c1

• fix vae type error during training sdxl

  BootsofLagrangian

  committed
  2824312

• fix all trainer about vae

  BootsofLagrangian

  committed
  4295f91

• maybe fix branch to run offloading

  BootsofLagrangian

  committed
  3970bf4

• apply offloading method runable for all trainer

  BootsofLagrangian

  committed
  7d2a926

Commits on Feb 7, 2024

• fix full_fp16 compatible and train_step

  BootsofLagrangian

  committed
  6255661

• remove test requirements

  BootsofLagrangian

  committed
  2445a5b

• forgot setting mixed_precision for deepspeed. sorry

  BootsofLagrangian

  committed
  a98feca

Commits on Feb 9, 2024

• the reason not working grad accum steps found. it was becasue of my accelerate settings

  BootsofLagrangian

  committed
  03f0816

Commits on Feb 22, 2024

• refactored codes, some function moved into train_utils.py

  BootsofLagrangian

  committed
  4d5186d

Commits on Feb 27, 2024

• Merge branch 'deep-speed' into deepspeed

  kohya-ss

  authored
  eefb3cc

• Merge pull request #1101 from BootsofLagrangian/deepspeed

  Show description for 0e4a573
  kohya-ss

  authored
  0e4a573

• make deepspeed_utils

  kohya-ss

  committed
  e3ccf8f

Commits on Mar 12, 2024

• Merge branch 'dev' into deep-speed

  kohya-ss

  committed
  97524f1

Commits on Mar 17, 2024

• Merge branch 'dev' into deep-speed

  kohya-ss

  committed
  86e40fa

Commits on Mar 20, 2024

• Merge branch 'dev' into deep-speed

  kohya-ss

  committed
  fbb98f1

• Fix most of ZeRO stage uses optimizer partitioning

  Show description for d945602
  BootsofLagrangian

  committed
  d945602

• Merge pull request #1200 from BootsofLagrangian/deep-speed

  Show description for a35e7bd
  kohya-ss

  authored
  a35e7bd

Commits on Mar 24, 2024

• Merge branch 'dev' into deep-speed

  kohya-ss

  committed
  993b2ab

Commits on Mar 25, 2024

• Merge branch 'dev' into deep-speed

  kohya-ss

  committed
  c24422f

• make each script consistent, fix to work w/o DeepSpeed

  kohya-ss

  committed
  a2b8531