7263 add diffusion loss

[kvttt]

Fixes #7263.

Description

Add diffusion loss. I also made a demo notebook to provide some explanations and analyses of diffusion loss.

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[kvttt]

Hi @ebrahimebrahim, you worked on implementing BendingEnergyLoss earlier and provided an awesome demo notebook. Could you take a look at my demo notebook attached in this PR? It would be great to have you confirm that my analysis of diffusion loss make sense! Thank you!

[ebrahimebrahim]

Happy to take a look! I'll get back to you on this soon

[ebrahimebrahim]

I just had a look. I like your derivation of the scaling law for diffusion loss!
In your result

$$L(\hat{{\bf u}}) = \frac{1}{|D|} \sum_{{\bf x} \in D} \left( \lambda_x^{-1}\lambda \odot \frac{\partial}{\partial x} ({\bf u}({\bf x})) \right)^2 + \left( \lambda_y^{-1} \lambda \odot \frac{\partial}{\partial y} ({\bf u}({\bf x})) \right)^2 + \left( \lambda_z^{-1} \lambda \odot \frac{\partial}{\partial z} ({\bf u}({\bf x})) \right)^2$$

if you assume isotropic scaling $\lambda_x=\lambda_y=\lambda_z$ then there is cancellation and you've shown that $L(\hat{{\bf u}}) \approx L({{\bf u}}) $. That is, there's only a weird scale-dependent effect to worry about when you change your image resolution along different axes differently. The examples in the notebook always do isotropic changes of resolution. This explains why in your plot you see scale-invariance (the eventually-flat line) regardless of whether normalize=True or not:

To really see the benefit of your normalize=True, you'd want to change resolution differently on the x axis versus the y axis.

But yeah I think I agree with your derivation of the scaling law and I like your approach!

[KumoLiu]

/build

[ericspod]

Looks good, I had a few comments with minor changes.

[KumoLiu]

/build

[KumoLiu]

Would be great to add the demo to the tutorial repo and mention it in the docstring.