add vecdot, in analogy with vecnorm

[stevengj]

As discussed in #11064, if we have a vecnorm function (computing the Euclidean norm of any iterable container, equivalent to norm(vec(x))), we should really also have a vecdot (computing the Euclidean dot product of any iterable container, equivalent to dot(vec(x),vec(y))).

(If, as @jiahao argued in #7990, we renamed vecnorm to normfro, then for consistency we should use dotfro here. I don't care too much either way as long as we are consistent, but I would point out that if you're worried about the inherent ambiguity of the terms "norm" and "dot product", that would argue against having norm and dot functions at all.)

[johnmyleswhite]

+1

We made dot do this in Optim and annoyed people. Would be great to have a real name for this function.

[toivoh]

I think that for the dot product of eg two matrices, I would really like a function that checks that the dimensions match also. That should also allow a more efficient implementation for matrices without fast linear indexing. Should that be a different function though?

[andreasnoack]

@toivoh Note that for e.g. StridedMatrix, this implementation uses the CartesianIndex_2 iterator. Shouldn't that be efficient?

[mbauman]

Iterating over any abstract array already uses the most efficient form of indexing available by default. For Array (and other LinearFast() arrays) that's linear indexing. For SubArray (and other LinearSlow() arrays) that's cartesian indexing. So the iteration depends upon which StridedMatrix you're using.

Is there any overhead involved in asking:

applicable(length, x) && applicable(length, y) && assert(length(x) == length(y))

?

[timholy]

Hmm, while we get

julia> a = 0x00:0xff
0x00:0xff

julia> sum(a)
32640

we have

julia> dot(a,a)
0x80

Since a dot product is a reduction, it would be nice to include the same widening we use for our reductions.

[marius311]

If I understood the discussion in this PR and the related Issue correctly, then care was taken not to allow dot(Matrix,Matrix) because it might be ambiguous/confusing what that means. However, it looks like currently the definition here does allow dot(Matrix{<:BlasReal},Matrix{<:BlasReal}) which then confusingly does not work on complex matrices. Should that definition perhaps be changed to only work for DenseArray{T,1} instead?

[andreasnoack]

I think that is a good idea.