Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption #1228

Merged
merged 14 commits into from
Apr 7, 2024
Merged

Add option to use Scheduled Huber Loss in all training pipelines to improve resilience to data corruption #1228

merged 14 commits into from
Apr 7, 2024

Commits on Mar 31, 2024

  1. add huber loss and huber_c compute to train_util

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    7e346b0 View commit details
    Browse the repository at this point in the history

  2. add reduction modes

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    9a7112a View commit details
    Browse the repository at this point in the history

  3. add huber_c retrieval from timestep getter

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    7da9bcf View commit details
    Browse the repository at this point in the history

  4. move get timesteps and huber to own function

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    81e2618 View commit details
    Browse the repository at this point in the history

  5. add conditional loss to all training scripts

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    029fcf8 View commit details
    Browse the repository at this point in the history

  6. add cond loss to train network

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    117a207 View commit details
    Browse the repository at this point in the history

  7. add (scheduled) huber_loss to args

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    b1085b9 View commit details
    Browse the repository at this point in the history

  8. fixup twice timesteps getting

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    8abc112 View commit details
    Browse the repository at this point in the history

  9. PHL-schedule should depend on noise scheduler's num timesteps

    @kabachuha
    kabachuha committed Mar 31, 2024
    Configuration menu
    Copy the full SHA
    a58f290 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. *2 multiplier to huber loss cause of 1/2 a^2 conv. 

    The Taylor expansion of sqrt near zero gives 1/2 a^2, which differs from a^2 of the standard MSE loss. This change scales them better against one another
    @kabachuha
    kabachuha committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    c6495de View commit details
    Browse the repository at this point in the history

  2. add option for smooth l1 (huber / delta)

    @kabachuha
    kabachuha committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    dd22958 View commit details
    Browse the repository at this point in the history

  3. unify huber scheduling

    @kabachuha
    kabachuha committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    19a834c View commit details
    Browse the repository at this point in the history

  4. add snr huber scheduler

    @kabachuha
    kabachuha committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    47fb1a6 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2024

  1. Merge branch 'dev' into main

    @kohya-ss
    kohya-ss authored Apr 7, 2024
    Configuration menu
    Copy the full SHA
    8ab8e4a View commit details
    Browse the repository at this point in the history