losses.md

Losses for NNs

Composite loss

Alternate between the losses

GradNorm addresses the problem of balancing multiple losses for multi-task learning by learning adjustable weight coefficients.

• https://arxiv.org/abs/1711.02257
• https://github.com/brianlan/pytorch-grad-norm

PCGrad Gradient Surgery for Multi-Task Learning

• https://arxiv.org/abs/2001.06782
• https://github.com/WeiChengTseng/Pytorch-PCGrad