Skip to content

Commit

Permalink
Add a lot of documentation on the bounds checking hierarchy [ci skip]
Browse files Browse the repository at this point in the history 
This also reorganizes code to make the order follow the hierarchy; makes more sense for the "big picture" documentation to be near the top node.
  • Loading branch information
@timholy
timholy committed Jul 12, 2016
1 parent 33ee846 commit e797ae4
Show file tree
Hide file tree
Showing 4 changed files with 151 additions and 76 deletions.
150 changes: 88 additions & 62 deletions base/abstractarray.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -150,53 +150,30 @@ linearindexing(::LinearFast, ::LinearFast) = LinearFast()
linearindexing(::LinearIndexing, ::LinearIndexing) = LinearSlow()

## Bounds checking ##
@generated function trailingsize{T,N,n}(A::AbstractArray{T,N}, ::Type{Val{n}})
(isa(n, Int) && isa(N, Int)) || error("Must have concrete type")
n > N && return 1
ex = :(size(A, $n))
for m = n+1:N
ex = :($ex * size(A, $m))
end
Expr(:block, Expr(:meta, :inline), ex)
end

# check along a single dimension
"""
checkindex(Bool, inds::UnitRange, index)
# The overall hierarchy is
# `checkbounds(A, I...)` ->
# `checkbounds(Bool, A, I...)` -> either of:
# - `checkbounds_indices(IA, I)` which calls `checkindex(Bool, inds, i)`
# - `checkbounds_logical(A, I)` when `I` is a single logical array
#
# See the "boundscheck" devdocs for more information.
#
# Note this hierarchy has been designed to reduce the likelihood of
# method ambiguities. We try to make `checkbounds` the place to
# specialize on array type, and try to avoid specializations on index
# types; conversely, `checkindex` is intended to be specialized only
# on index type (especially, its last argument).

Return `true` if the given `index` is within the bounds of
`inds`. Custom types that would like to behave as indices for all
arrays can extend this method in order to provide a specialized bounds
checking implementation.
"""
checkindex(::Type{Bool}, inds::AbstractUnitRange, i) = throw(ArgumentError("unable to check bounds for indices of type $(typeof(i))"))
checkindex(::Type{Bool}, inds::AbstractUnitRange, i::Real) = (first(inds) <= i) & (i <= last(inds))
checkindex(::Type{Bool}, inds::AbstractUnitRange, ::Colon) = true
function checkindex(::Type{Bool}, inds::AbstractUnitRange, r::Range)
@_propagate_inbounds_meta
isempty(r) | (checkindex(Bool, inds, first(r)) & checkindex(Bool, inds, last(r)))
end
checkindex{N}(::Type{Bool}, indx::AbstractUnitRange, I::AbstractArray{Bool,N}) = N == 1 && indx == indices1(I)
function checkindex(::Type{Bool}, inds::AbstractUnitRange, I::AbstractArray)
@_inline_meta
b = true
for i in I
b &= checkindex(Bool, inds, i)
end
b
end
checkbounds(Bool, A, I...)
# check all indices/dimensions
Return `true` if the specified indices `I` are in bounds for the given
array `A`. Subtypes of `AbstractArray` should specialize this method
if they need to provide custom bounds checking behaviors; however, in
many cases one can rely on `A`'s indices and `checkindex`.
# To facilitate extension for custom array types without triggering
# ambiguities, limit the number of specializations of checkbounds on
# the types of the indices.
"""
checkbounds(Bool, array, indexes...)
Return `true` if the specified `indexes` are in bounds for the given
`array`. Subtypes of `AbstractArray` should specialize this method if
they need to provide custom bounds checking behaviors.
See also `checkindex`.
"""
function checkbounds(::Type{Bool}, A::AbstractArray, I...)
@_inline_meta
Expand All @@ -207,35 +184,48 @@ function checkbounds(::Type{Bool}, A::AbstractArray, I::AbstractArray{Bool})
checkbounds_logical(A, I)
end

# checkbounds_indices iteratively consumes elements of the
# indices-tuple of an arrray and the indices-tuple supplied by the
# caller. These two tuples are usually consumed in a 1-for-1 fashion,
# i.e.,
#
# checkbounds_indices((R1, R...), (I1, I...)) = checkindex(Bool, R1, I1) &
# checkbounds_indices(R, I)
#
# However, there are two exceptions: linear indexing and CartesianIndex{N}.
"""
checkbounds_indices(IA, I)
checks whether the "requested" indices in the tuple `I` fall within
the bounds of the "permitted" indices specified by the tuple
`IA`. This function recursively consumes elements of these tuples,
usually in a 1-for-1 fashion,
checkbounds_indices((IA1, IA...), (I1, I...)) = checkindex(Bool, IA1, I1) &
checkbounds_indices(IA, I)
Note that `checkindex` is being used to perform the actual
bounds-check for a single dimension of the array.
There are two important exceptions to the 1-1 rule: linear indexing and
CartesianIndex{N}, both of which may "consume" more than one element
of `IA`.
"""
function checkbounds_indices(IA::Tuple, I::Tuple)
@_inline_meta
checkindex(Bool, IA[1], I[1]) & checkbounds_indices(tail(IA), tail(I))
end
checkbounds_indices(::Tuple{}, ::Tuple{}) = true
checkbounds_indices(::Tuple{}, I::Tuple{Any}) = (@_inline_meta; checkindex(Bool, 1:1, I[1]))
function checkbounds_indices(::Tuple{}, I::Tuple)
@_inline_meta
checkindex(Bool, 1:1, I[1]) & checkbounds_indices((), tail(I))
end
function checkbounds_indices(inds::Tuple{Any}, I::Tuple{Any})
@_inline_meta
checkindex(Bool, inds[1], I[1])
end
function checkbounds_indices(inds::Tuple, I::Tuple{Any})
function checkbounds_indices(IA::Tuple{Any}, I::Tuple{Any})
@_inline_meta
checkindex(Bool, 1:prod(map(dimlength, inds)), I[1]) # linear indexing
checkindex(Bool, IA[1], I[1])
end
function checkbounds_indices(inds::Tuple, I::Tuple)
function checkbounds_indices(IA::Tuple, I::Tuple{Any})
@_inline_meta
checkindex(Bool, inds[1], I[1]) & checkbounds_indices(tail(inds), tail(I))
checkindex(Bool, 1:prod(map(dimlength, IA)), I[1]) # linear indexing
end

# Single logical array indexing:
"""
checkbounds_logical(A, I::AbstractArray{Bool})
tests whether the logical array `I` is consistent with the indices of `A`.
"""
checkbounds_logical(A::AbstractArray, I::AbstractArray{Bool}) = indices(A) == indices(I)
checkbounds_logical(A::AbstractArray, I::AbstractVector{Bool}) = length(A) == length(I)
checkbounds_logical(A::AbstractVector, I::AbstractArray{Bool}) = length(A) == length(I)
Expand All @@ -244,9 +234,9 @@ checkbounds_logical(A::AbstractVector, I::AbstractVector{Bool}) = indices(A) ==
throw_boundserror(A, I) = (@_noinline_meta; throw(BoundsError(A, I)))

"""
checkbounds(array, indexes...)
checkbounds(A, I...)
Throw an error if the specified `indexes` are not in bounds for the given `array`.
Throw an error if the specified indices `I` are not in bounds for the given array `A`.
"""
function checkbounds(A::AbstractArray, I...)
@_inline_meta
Expand All @@ -255,6 +245,42 @@ function checkbounds(A::AbstractArray, I...)
end
checkbounds(A::AbstractArray) = checkbounds(A, 1) # 0-d case

@generated function trailingsize{T,N,n}(A::AbstractArray{T,N}, ::Type{Val{n}})
(isa(n, Int) && isa(N, Int)) || error("Must have concrete type")
n > N && return 1
ex = :(size(A, $n))
for m = n+1:N
ex = :($ex * size(A, $m))
end
Expr(:block, Expr(:meta, :inline), ex)
end

# check along a single dimension
"""
checkindex(Bool, inds::AbstractUnitRange, index)
Return `true` if the given `index` is within the bounds of
`inds`. Custom types that would like to behave as indices for all
arrays can extend this method in order to provide a specialized bounds
checking implementation.
"""
checkindex(::Type{Bool}, inds::AbstractUnitRange, i) = throw(ArgumentError("unable to check bounds for indices of type $(typeof(i))"))
checkindex(::Type{Bool}, inds::AbstractUnitRange, i::Real) = (first(inds) <= i) & (i <= last(inds))
checkindex(::Type{Bool}, inds::AbstractUnitRange, ::Colon) = true
function checkindex(::Type{Bool}, inds::AbstractUnitRange, r::Range)
@_propagate_inbounds_meta
isempty(r) | (checkindex(Bool, inds, first(r)) & checkindex(Bool, inds, last(r)))
end
checkindex{N}(::Type{Bool}, indx::AbstractUnitRange, I::AbstractArray{Bool,N}) = N == 1 && indx == indices1(I)
function checkindex(::Type{Bool}, inds::AbstractUnitRange, I::AbstractArray)
@_inline_meta
b = true
for i in I
b &= checkindex(Bool, inds, i)
end
b
end

# See also specializations in multidimensional

## Constructors ##
Expand Down
18 changes: 9 additions & 9 deletions base/multidimensional.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -156,23 +156,23 @@ using .IteratorsMD
## Bounds-checking with CartesianIndex
@inline checkbounds_indices(::Tuple{}, I::Tuple{CartesianIndex,Vararg{Any}}) =
checkbounds_indices((), (I[1].I..., tail(I)...))
@inline checkbounds_indices(inds::Tuple{Any}, I::Tuple{CartesianIndex,Vararg{Any}}) =
checkbounds_indices(inds, (I[1].I..., tail(I)...))
@inline checkbounds_indices(inds::Tuple, I::Tuple{CartesianIndex,Vararg{Any}}) =
checkbounds_indices(inds, (I[1].I..., tail(I)...))
@inline checkbounds_indices(IA::Tuple{Any}, I::Tuple{CartesianIndex,Vararg{Any}}) =
checkbounds_indices(IA, (I[1].I..., tail(I)...))
@inline checkbounds_indices(IA::Tuple, I::Tuple{CartesianIndex,Vararg{Any}}) =
checkbounds_indices(IA, (I[1].I..., tail(I)...))

# Support indexing with an array of CartesianIndex{N}s
# Here we try to consume N of the indices (if there are that many available)
# The first two simply handle ambiguities
@inline function checkbounds_indices{N}(::Tuple{}, I::Tuple{AbstractArray{CartesianIndex{N}},Vararg{Any}})
checkindex(Bool, (), I[1]) & checkbounds_indices((), tail(I))
end
@inline function checkbounds_indices{N}(inds::Tuple{Any}, I::Tuple{AbstractArray{CartesianIndex{N}},Vararg{Any}})
checkindex(Bool, inds, I[1]) & checkbounds_indices((), tail(I))
@inline function checkbounds_indices{N}(IA::Tuple{Any}, I::Tuple{AbstractArray{CartesianIndex{N}},Vararg{Any}})
checkindex(Bool, IA, I[1]) & checkbounds_indices((), tail(I))
end
@inline function checkbounds_indices{N}(inds::Tuple, I::Tuple{AbstractArray{CartesianIndex{N}},Vararg{Any}})
inds1, indsrest = IteratorsMD.split(inds, Val{N})
checkindex(Bool, inds1, I[1]) & checkbounds_indices(indsrest, tail(I))
@inline function checkbounds_indices{N}(IA::Tuple, I::Tuple{AbstractArray{CartesianIndex{N}},Vararg{Any}})
IA1, IArest = IteratorsMD.split(IA, Val{N})
checkindex(Bool, IA1, I[1]) & checkbounds_indices(IArest, tail(I))
end

function checkindex{N}(::Type{Bool}, inds::Tuple, I::AbstractArray{CartesianIndex{N}})
Expand Down
47 changes: 47 additions & 0 deletions doc/devdocs/boundscheck.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,50 @@ instance, the default ``getindex`` methods have the chain
To override the "one layer of inlining" rule, a function may be marked with
``@propagate_inbounds`` to propagate an inbounds context (or out of bounds
context) through one additional layer of inlining.

The bounds checking call hierarchy
----------------------------------

The overall hierarchy is:

| ``checkbounds(A, I...)`` which calls
| ``checkbounds(Bool, A, I...)`` which calls either of:
| ``checkbounds_logical(A, I)`` when ``I`` is a single logical array
| ``checkbounds_indices(indices(A), I)`` otherwise
|
Here ``A`` is the array, and ``I`` contains the "requested" indices.
``indices(A)`` returns a tuple of "permitted" indices of ``A``.

``checkbounds(A, I...)`` throws an error if the indices are invalid,
whereas ``checkbounds(Bool, A, I...)`` returns ``false`` in that
circumstance. ``checkbounds_indices`` discards any information about
the array other than its ``indices`` tuple, and performs a pure
indices-vs-indices comparison: this allows relatively few compiled
methods to serve a huge variety of array types. Indices are specified
as tuples, and are usually compared in a 1-1 fashion with individual
dimensions handled by calling another important function,
``checkindex``: typically,
::
checkbounds_indices((IA1, IA...), (I1, I...)) = checkindex(Bool, IA1, I1) &
checkbounds_indices(IA, I)

so ``checkindex`` checks a single dimension. All of these functions,
including the unexported ``checkbounds_indices`` and
``checkbounds_logical``, have docstrings accessible with ``?`` .

If you have to customize bounds checking for a specific array type,
you should specialize ``checkbounds(Bool, A, I...)``. However, in most
cases you should be able to rely on ``checkbounds_indices`` as long as
you supply useful ``indices`` for your array type.

If you have novel index types, first consider specializing
``checkindex``, which handles a single index for a particular
dimension of an array. If you have a custom multidimensional index
type (similar to ``CartesianIndex``), then you may have to consider
specializing ``checkbounds_indices``.

Note this hierarchy has been designed to reduce the likelihood of
method ambiguities. We try to make ``checkbounds`` the place to
specialize on array type, and try to avoid specializations on index
types; conversely, ``checkindex`` is intended to be specialized only
on index type (especially, the last argument).
12 changes: 7 additions & 5 deletions doc/stdlib/arrays.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -616,19 +616,21 @@ Indexing, Assignment, and Concatenation
Check two array shapes for compatibility, allowing trailing singleton dimensions, and return whichever shape has more dimensions.

.. function:: checkbounds(array, indexes...)
.. function:: checkbounds(A, I...)

.. Docstring generated from Julia source
Throw an error if the specified ``indexes`` are not in bounds for the given ``array``\ .
Throw an error if the specified indices ``I`` are not in bounds for the given array ``A``\ .

.. function:: checkbounds(Bool, array, indexes...)
.. function:: checkbounds(Bool, A, I...)

.. Docstring generated from Julia source
Return ``true`` if the specified ``indexes`` are in bounds for the given ``array``\ . Subtypes of ``AbstractArray`` should specialize this method if they need to provide custom bounds checking behaviors.
Return ``true`` if the specified indices ``I`` are in bounds for the given array ``A``\ . Subtypes of ``AbstractArray`` should specialize this method if they need to provide custom bounds checking behaviors; however, in many cases one can rely on ``A``\ 's indices and ``checkindex``\ .

.. function:: checkindex(Bool, inds::UnitRange, index)
See also ``checkindex``\ .

.. function:: checkindex(Bool, inds::AbstractUnitRange, index)

.. Docstring generated from Julia source
Expand Down

0 comments on commit e797ae4

Please sign in to comment.