-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Add ArrayLike #34196
RFC: Add ArrayLike #34196
Conversation
tv = jl_svec2(tvar("T"), tvar("N")); | ||
jl_abstractarray_type = (jl_unionall_t*) | ||
jl_new_abstracttype((jl_value_t*)jl_symbol("AbstractArray"), core, | ||
jl_any_type, tv)->name->wrapper; | ||
(jl_datatype_t*)jl_apply_type((jl_value_t*)jl_arraylike_type, jl_svec_data(jl_svec1(tv_N)), 1), | ||
tv)->name->wrapper; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wrote this code by guess and check because I don't know C, but it seemed to work
@@ -3396,7 +3396,7 @@ void jl_init_serializer(void) | |||
deser_tag[LAST_TAG+1+i] = (jl_value_t*)vals[i]; | |||
i += 1; | |||
} | |||
assert(LAST_TAG+1+i < 256); | |||
assert(LAST_TAG+1+i < 257); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure it this is valid???
Can you explain why this is better than const AnyArray{N} = AbstractArray{<:Any,N} |
That could also be a workable option. I think that what makes this special is that this is a meaningful subtype relationship: arrays with an eltype are a subtype of arrays |
Understood, but are there methods that would be defined for |
I think the main difference is:
There are methods defined for AbstractArray that make use of an eltype that we wouldn't want to be inherited by ArrayLike objects. |
I understand the difference in the type hierarchy. I am just asking about methods. I am not opposed to inserting something into the type hierarchy, but I'm trying to understand what advantages accrue if we decide to do so. #32940 asserts that this would be useful without really making it clear what problem gets solved that can't be solved by alternative means. I see Jeff wondered the same thing, #32940 (comment). For example, there's always the option to define Base.eltype(::Type{<:MyArrayLikeType}) = error("eltype is undefined for MyArrayLikeType") That would automatically invalidate all methods that call |
I guess that could work. Bottom line, this just seems like a more elegant solution to me. |
Or I guess, to be more specific, that solution would work, but it would require encoding an eltype of Any into the signature of an AbstractArray, but then error when an eltype is requested, which seems paradoxical. |
@timholy I think it depends on what do you mean by "complete"; i.e., if you prefer false-positive dispatch (error when a method should not be called) or false-negative dispatch (the method really needs f(array::AbstractArray) = # code using `eltype`
f(itr) = # code using mutate-or-widen approach (I guess I'm repeating what @bramtayl just said.) As Julia language usually encourages duck-typing, I can see that false-positive dispatch might be preferable. You can also "fix" the code above by using IteratorEltype(::Type{<:AbstractArray}) = EltypeUnknown()
IteratorEltype(::Type{<:AbstractArray{T}}) where {T} = HasEltype() is technically breaking (ATM, Having said that, using a trait may be a better option. For example, |
Well anyway, if people are interested in this I can take the next step of auditing the use of AbstractArray in Base |
Certainly if there are cases that need to be intercepted prior to failing on |
@bramtayl It would be nice if you clarify your view on some challenges with
|
Can someone explain the conceptual difference between leaving the eltype undefined and defining it as |
@martinholters My understanding is that the common expectation is |
@tkf I agree. On the two challenges you mention:
I haven't really gotten far enough along in my thinking to plan out an interface for |
I don't think we can do SubArrayLike <: ArrayLike
ReshapedArrayLike <: ArrayLike
AbstractArray <: ArrayLike
SubArray <: AbstractArray
ReshapedArray <: AbstractArray
SubArrayLikeImpl = Union{SubArray, SubArrayLike}
ReshapedArrayLikeImpl = Union{ReshapedArray, ReshapedArrayLike} where |
Darn you're right. I wonder how much would be lost if all views and reshapes were |
I don't think that's possible as linear algebra functions often need to dispatch on the element type of these arrays. |
:( oh that's too bad
Would it be possible to compute this at compile time? |
If we forget backward compatibility, I think it's possible but it'd still be hard to do it without increase compilation time. I think you need to ask @mbauman to be sure. But, as it'd be a breaking change, I think it's impossible to change this in Julia 1.x. |
I thought that widening type parameters from AbstractArray to ArrayLike would be non-breaking, but it turns out that it can create ambiguity errors. Is there a go-to solution for these kind of problems? |
I think the go-to solution is trait. One idea I had was to use the struct NonIndexable <: IndexStyle end
IndexStyle(::Type) = NonIndexable() As you can safely call f(x::MyArrayLike) = _f(IndexStyle(x), x)
_f(::IndexCartesian, x) = ...
_f(::NonIndexable, x) = ... This way, defining |
Ok, I've audited Base, test, and the standard library for I'm managed to convince myself that this should be non-breaking (again); the only way it could introduce ambiguities in a package is if that package is conducting piracy. Is that correct? I used this overly aggressive strategy not because I think that all these methods should be extended to The next step is to see if I can't get |
I definitely get the need for improvements surrounding non-AbstractArray indexability, but I'm not sure this is it. I've been following this thread closely but I remain skeptical. My reservations include:
|
I guess I assumed without thinking about it too carefully. Can you give some examples?
That's how I'm thinking of it.
I need to add documentation, but yes, that's how I'm thinking of it, with two exceptions:
I'm not sure about the "should" answer (what do you think?). Currently the "is" answer is that the extra functionality provided by AbstractArrays is simply everything without an ArrayLike method in the PR, that is, every function that dispatches on the eltype. But there are two reasons this could change:
|
It just feels far too fiddly If the only difference between When to preserve eltypes is a hard problem — especially in the context of If we're inserting a new supertype here, I'd rather it represent a bigger departure from the requirements of |
That's fair. I'm not particularly wedded to this solution, just looking for a way to get index-able iterators within multiple dispatch. |
If you want to see the kind of things that are possible with this PR, see my fork of MappedArrays https://github.com/bramtayl/MappedArrays.jl which I updated to be ArrayLike |
@mbauman I started to look at the broadcast.jl code to see if I could get it to be struct Broadcasted{N, Style<:Union{Nothing,BroadcastStyle}, Axes, F, Args<:Tuple} <: ArrayLike{N}
f::F
args::Args
axes::Axes
end
Broadcasted(f::F, args::Args, axes=nothing) where {F, Args<:Tuple} =
Broadcasted{length(axes), typeof(combine_styles(args...))}(f, args, axes)
function Broadcasted{Style}(f::F, args::Args, axes=nothing) where {Style, F, Args<:Tuple}
Broadcasted{length(axes), Style, typeof(axes), Core.Typeof(f), Args}(f, args, axes)
end And because |
To me it looks like you had to delete some nice functionality. What's the benefit? |
I did have to delete some stuff, but hopefully some of it can come back:
Are those the things you were referring to? The benefit is that it removes the need for the maybe a bit brittle
If a type don't have a zero defined, and if the array contains a small union of eltypes (e.g. |
Yeah, that function is brittle. See JuliaArrays/MappedArrays.jl#29 for a potential workaround. To me this mostly illustrates why we want traits. There's no doubt that it's nice to be able to wrap a more general object, but when it's wrapping an AbstractArray I want to be able to call I should get back to andyferris/Traitor.jl#8 someday. |
I would be very happy with a trait based solution too! |
Well, I guess I'll close this in favor of a trait based solution, and hope that it comes sooner rather than later. |
This is going to need a bit of explanation.
There is a meaningful idea of an array without a predefined eltype. The most notable examples are generators and broadcasts over arrays. If they could be <: ArrayLike, then they could automatically inherit much of the machinery already constructed for AbstractArrays.
The next step would be to audit all uses of AbstractArray in Base and the stdlibs and decide whether they could be widened to ArrayLike instead.
One way to do this would be simply to replace every occurrence unless the method dispatches on the eltype, i.e. AbstractArray{T} where T. There are two reasons why I didn't do this:
The good news is that this is barely breaking, and I only had to change one test (show_supertypes).
Closes #32940