Missing: Array of Parrays to Parray of Arrays

[JordiManyer]

Hi @fverdugo, I think there is a functionality missing that could be useful in some cases. Again, I might be missing something but I haven't found an elegant way of dealing with it.

It involves converting an Array of PArrays to a PArray of Arrays. Consider the following example:

using PartitionedArrays

ranks = with_debug() do distribute
  distribute(LinearIndices((4,)))
end

a = map(r -> r, ranks)
b = map(r -> -r, ranks)

# Array of PArrays
v = [a,b];

# PArray of Arrays
map(ranks) do r
  map(v) do v_i
    # Used to be: PartitionedArrays.get_part(v_i,r)
    # But now is limited to:
    PartitionedArrays.getany(v_i)
  end
end

The current code works fine for MPIArrays, but not for DebugArrays (since getany always returns the first element). The old solution (commented) does not work anymore since there is no way to access a particular element in a DebugArray/MPIArray without getting an error/warning (even if it is safe to do so).

I think the solution to this involves a new function (basically an analog to tuple_of_arrays) that specializes for DebugArray and MPIArray. Would that be something you would like to have in PartitionedArrays?

[fverdugo]

I see 2 possible solutions:

• Map over v and then over ranks if possible in your context
• Implement getindex in MPIArray and return the value only if the given index is the current rank, otherwise error since we would need communication. This would be analogous to the former get_part

Both solutions are compatible

[JordiManyer]

The solution I now have is

function to_parray_of_arrays(a::AbstractArray{<:MPIArray})
  indices = linear_indices(first(a))
  map(indices) do i
    map(a) do aj
      PartitionedArrays.getany(aj)
    end
  end
end

function to_parray_of_arrays(a::AbstractArray{<:DebugArray})
  indices = linear_indices(first(a))
  map(indices) do i
    map(a) do aj
      aj.items[i]
    end
  end
end

My question was if you'd like this implemented in PartitionedArrays (since it is a generic use case) or not.

[fverdugo]

Hi @JordiManyer I would go with this other option:

Implement getindex in MPIArray and return the value only if the given index is the current rank, otherwise error since we would need communication. This would be analogous to the former get_part