About the warning for svd AD

[Confusio]

For the SVD AD, there is a warning:

Warning: "svd cotangents sensitive to gauge choice: (|Δgauge| = 1.2888961545189787e-16)"

Is this warning related to the autodifferentiation of the SVD for complex numbers? If we are only concerned with gauge-invariant results, this warning should not pose any issues. Is that correct?

[Jutho]

Yes it comes from the AD applied to the SVD; it tells you that in the backward process, there is component entering the adjoint Jacobian of the SVD process that depends on the gauge. However, that component is only 1.28e-16 in size, so it effectively zero. Therefore, I am surprised that the warning is even printed. What did you set for the tol keyword in the primal algorithm?

[Confusio]

I used the default settings without setting any tol. I am also surprised about the "warning" with the tiny value.

[Jutho]

I was a bit confused; this is coming from the full svd in TensorKit, right? I had KrylovKit in mind. In TensorKit, the current cutoff for printing this warning if no defaults are specified is: eps(eltype(S))^(3 / 4) * S[1, 1] with S[1,1] the largest singular value. I guess that is not a really good default: when S[1,1] is for some reason small, it should still not be expected that the tolerance gets of the order or smaller than machine precision. I will change the default to something like eps(eltype(S))^(3 / 4) * max(S[1,1], one(S[1,1]).

[Confusio]

Yes, it is from full tsvd in TensorKit.

[Jutho]

Ok, this is now changed on master, hopefully this fixes it. I will tag a new release this weekend, but there are some breaking changes.

[Confusio]

The new tol may cause a problem in safe_inv.(Sp' .- Sp, tol)'. Here we need a relative tol. Otherwise, it causes an incorrect result when Sp` is small.