Add more documentation on array iteration

[timholy]

Following up on suggestion for further documentation in #10858.

[tkelman]

Cool. As an aside, I'm wondering if for the purposes of #8001 it might make sense to add an eachnonzero version of this iterator that's specialized for sparse (or banded, tridiag, etc) matrices, then reductions or other operations that are invariant with respect to implicit zeros can be written in terms of eachnonzero instead of eachindex...

[timholy]

It's an interesting idea, but #6769 made the prospect of asking "what is a nonzero?" absolutely terrifying. Someone braver than me will have to implement it 😄.

[toivoh]

eachstored?

[timholy]

eachstored could work. Upon thinking about it, there doesn't seem to be a generic implementation: each "sparse" matrix type would have to implement its own.

[tkelman]

Really #6769 is more noise than substance. Anyone who actually works with sparse matrices regularly is perfectly comfortable with allowing explicitly stored zeros. A structural/stored nonzero can have a value equal to zero, and it shouldn't change any calculation results if things are implemented correctly.

Upon thinking about it, there doesn't seem to be a generic implementation: each "sparse" matrix type would have to implement its own.

That's exactly the point. Better than re-writing the same messy structure-specific iteration code inside every single method you want to implement for each matrix type. I'd rather write the iteration code once per type and use that everywhere, than have to duplicate it ntypes * nmethods times.

[toivoh]

Still, I'm not sure if you really want to use reading and assignment to A[I] as the idiomatic way to operate on sparse matrices independent of type (where I is aCartesianIndex or some sparse matrix analogue). Would that really be efficient? Which kind of abstraction would we want to support for sparse matrices? What would make for efficient code?

[tkelman]

(We could move this discussion elsewhere? #8001 maybe?) For CSC as an example, the analog to CartesianIndex would need to store column index and nonzero index. Reading from or writing to an existing nonzero can be efficient in that case. The inefficient operations would be inserting a new nonzero that was not previously allocated in the structure, or removing an existing nonzero from the structure. There would be a slight ambiguity in whether A[I] = 0 means "set the existing stored value to zero" or "remove the stored value at index I from the structure and replace it with an implicit zero," but I think for efficiency it would have to mean the former. The latter operation should be given a different name.

[toivoh]

I agree with all of the above, and #8001 seems like a good place to continue the discussion.

[timholy]

Also agreed, @tkelman, including the move of future discussion.