diff --git a/src/invwarpedview.jl b/src/invwarpedview.jl index 06995b2..202c399 100644 --- a/src/invwarpedview.jl +++ b/src/invwarpedview.jl @@ -2,10 +2,7 @@ InvWarpedView(img, tinv, [indices]) -> wv Create a view of `img` that lazily transforms any given index `I` -passed to `wv[I]` to correspond to `img[inv(tinv)(I)]`. While -technically this approach is known as backward mode warping, note -that `InvWarpedView` is created by supplying the forward -transformation +passed to `wv[I]` so that `wv[I] == img[inv(tinv)(I)]`. The conceptual difference to [`WarpedView`](@ref) is that `InvWarpedView` is intended to be used when reasoning about the @@ -14,12 +11,10 @@ Furthermore, `InvWarpedView` allows simple nesting of transformations, in which case the transformations will be composed into a single one. -The optional parameter `indices` can be used to specify the -domain of the resulting `wv`. By default the indices are computed -in such a way that `wv` contains all the original pixels in -`img`. +See [`invwarpedview`](@ref) for a convenient constructor of `InvWarpedView`. -see [`invwarpedview`](@ref) for more information. +For detailed explaination of warp, associated arguments and parameters, +please refer to [`warp`](@ref). """ struct InvWarpedView{T,N,A,F,I,FI<:Transformation,E} <: AbstractArray{T,N} inner::WarpedView{T,N,A,F,I,E} @@ -67,28 +62,20 @@ function Base.showarg(io::IO, A::InvWarpedView, toplevel) end """ - invwarpedview(img, tinv, [indices], [degree = Linear()], [fill = NaN]) -> wv + invwarpedview(img, tinv, [indices]; kwargs...) -> wv Create a view of `img` that lazily transforms any given index `I` -passed to `wv[I]` to correspond to `img[inv(tinv)(I)]`. While -technically this approach is known as backward mode warping, note -that `InvWarpedView` is created by supplying the forward -transformation. The given transformation `tinv` must accept a -`SVector` as input and support `inv(tinv)`. A useful package to -create a wide variety of such transformations is -[CoordinateTransformations.jl](https://github.com/FugroRoames/CoordinateTransformations.jl). - -When invoking `wv[I]`, values for `img` must be reconstructed at -arbitrary locations `inv(tinv)(I)`. `InvWarpedView` serves as a -wrapper around [`WarpedView`](@ref) which takes care of -interpolation and extrapolation. The parameters `degree` and -`fill` can be used to specify the b-spline degree and the -extrapolation scheme respectively. - -The optional parameter `indices` can be used to specify the -domain of the resulting `wv`. By default the indices are computed -in such a way that `wv` contains all the original pixels in -`img`. +passed to `wv[I]` so that `wv[I] == img[inv(tinv)(I)]`. + +Except for the lazy evaluation, the following two lines are equivalent: + +```julia +warp(img, inv(tform), [indices]; kwargs...) +invwarpedview(img, tform, [indices]; kwargs...) +``` + +For detailed explaination of warp, associated arguments and parameters, +please refer to [`warp`](@ref). """ function invwarpedview(A::AbstractArray, tinv::Transformation, indices::Tuple=autorange(A, tinv); kwargs...) InvWarpedView(box_extrapolation(A; kwargs...), tinv, indices) diff --git a/src/resizing.jl b/src/resizing.jl index f6b42d5..d8a17b8 100644 --- a/src/resizing.jl +++ b/src/resizing.jl @@ -256,39 +256,66 @@ odims(original, i, short_size::Tuple{}) = axes(original, i) odims(original, i, short_size) = oftype(first(short_size), axes(original, i)) """ - imresize(img, sz) -> imgr - imresize(img, inds) -> imgr - imresize(img; ratio) -> imgr + imresize(img, sz; [method]) -> imgr + imresize(img, inds; [method]) -> imgr + imresize(img; ratio, [method]) -> imgr -Change `img` to be of size `sz` (or to have indices `inds`). If `ratio` is used, then -`sz = ceil(Int, size(img).*ratio)`. This interpolates the values at sub-pixel locations. -If you are shrinking the image, you risk aliasing unless you low-pass filter `img` first. +upsample/downsample the image `img` to a given size `sz` or axes `inds` using interpolations. If +`ratio` is provided, the output size is then `ceil(Int, size(img).*ratio)`. -The keyword `method` takes any InterpolationType from Interpolations.jl or a Degree, -which is used to define a BSpline interpolation of that degree, in order to set -the interpolation method used in the image resizing. +!!! tip + This interpolates the values at sub-pixel locations. If you are shrinking the image, you risk + aliasing unless you low-pass filter `img` first. + +# Arguments + +- `img`: the input image array +- `sz`: the size of output array +- `inds`: the axes of output array + If `inds` is passed, the output array `imgr` will be `OffsetArray`. + +# Parameters + +!!! info + To construct `method`, you may need to load `Interpolations` package first. + +- `ratio`: the upsample/downsample ratio used. + The output size is `ceil(Int, size(img).*ratio)`. If `ratio` is larger than `1`, it is + an upsample operation. Otherwise it is a downsample operation. `ratio` can also be a tuple, + in which case `ratio[i]` specifies the resize ratio at dimension `i`. +- `method::InterpolationType`: + specify the interpolation method used for reconstruction. conveniently, `methold` can + also be a `Degree` type, in which case a `BSpline` object will be created. + For example, `method = Linear()` is equivalent to `method = BSpline(Linear())`. # Examples + ```julia -julia> img = testimage("lena_gray_256") # 256*256 -julia> imresize(img, 128, 128) # 128*128 -julia> imresize(img, 1:128, 1:128) # 128*128 -julia> imresize(img, (128, 128)) # 128*128 -julia> imresize(img, (1:128, 1:128)) # 128*128 -julia> imresize(img, (1:128, )) # 128*256 -julia> imresize(img, 128) # 128*256 -julia> imresize(img, ratio = 0.5) #128*128 -julia> imresize(img, ratio = (2, 1)) # 256*128 -julia> imresize(img, (128,128), method=Linear()) #128*128 -julia> imresize(img, (128,128), method=BSpline(Linear())) #128*128 -julia> imresize(img, (128,128), method=Lanczos4OpenCV()) #128*128 - -σ = map((o,n)->0.75*o/n, size(img), sz) -kern = KernelFactors.gaussian(σ) # from ImageFiltering -imgr = imresize(imfilter(img, kern, NA()), sz) +using ImageTransformations, TestImages, Interpolations + +img = testimage("lighthouse") # 512*768 + +# pass integers as size +imresize(img, 256, 384) # 256*384 +imresize(img, (256, 384)) # 256*384 +imresize(img, 256) # 256*768 + +# pass indices as axes +imresize(img, 1:256, 1:384) # 256*384 +imresize(img, (1:256, 1:384)) # 256*384 +imresize(img, (1:256, )) # 256*768 + +# pass resize ratio +imresize(img, ratio = 0.5) #256*384 +imresize(img, ratio = (2, 1)) # 1024*768 + +# use different interpolation method +imresize(img, (256, 384), method=Linear()) # 256*384 bilinear interpolation +imresize(img, (256, 384), method=Lanczos4OpenCV()) # 256*384 OpenCV-compatible Lanczos 4 interpolation ``` -See also [`restrict`](@ref). +For downsample with `ratio=0.5`, [`restrict`](@ref) is a much faster two-fold implementation that +you can use. """ function imresize(original::AbstractArray{T,0}, new_inds::Tuple{}; kwargs...) where T Tnew = imresize_type(first(original)) diff --git a/src/warp.jl b/src/warp.jl index 5daf247..dd6e5fa 100644 --- a/src/warp.jl +++ b/src/warp.jl @@ -1,94 +1,160 @@ """ warp(img, tform, [indices]; kwargs...) -> imgw -Transform the coordinates of `img`, returning a new `imgw` -satisfying `imgw[I] = img[tform(I)]`. This approach is known as -backward mode warping. The transformation `tform` must accept a -`SVector` as input. A useful package to create a wide variety of -such transformations is -[CoordinateTransformations.jl](https://github.com/FugroRoames/CoordinateTransformations.jl). +Transform the coordinates of `img`, returning a new `imgw` satisfying `imgw[I] = img[tform(I)]`. + +# Output + +The output array `imgw` is an `OffsetArray`. Unless manually specified, `axes(imgw) == axes(img)` +does not hold in general. If you just want a plain array, you can "strip" the custom indices with +`parent(imgw)` or `OffsetArrays.no_offset_view(imgw)`. + +# Arguments + +- `img`: the original image that you need coordinate transformation. +- `tform`: the coordinate transformation function or function-like object, it must accept a + [`SVector`](https://github.com/JuliaArrays/StaticArrays.jl) as input. A useful package to + create a wide variety of such transfomrations is + [CoordinateTransformations.jl](https://github.com/FugroRoames/CoordinateTransformations.jl). +- `indices` (Optional): specifies the output image axes. + By default, the indices are computed in such a way that `imgw` contains all the original pixels + in `img` using [`autorange`](@ref ImageTransformations.autorange). To do this `inv(tform)` has + to be computed. If the given transfomration `tform` does not support `inv` then the parameter + `indices` has to be specified manually. # Parameters +!!! info + To construct `method` and `fillvalue` values, you may need to load `Interpolations` package first. + - `method::Union{Degree, InterpolationType}`: the interpolation method you want to use. By default it is `BSpline(Linear())`. To construct the method instance, one may need to load `Interpolations`. - `fillvalue`: the value that used to fill the new region. The default value is `NaN` if possible, - otherwise is `0`. One can also pass the extrapolation boundary condition: `Flat()`, `Reflect()` and `Periodic()`. - -# Reconstruction scheme - -During warping, values for `img` must be reconstructed at -arbitrary locations `tform(I)` which do not lie on to the lattice -of pixels. How this reconstruction is done depends on the type of -`img` and the optional parameter `degree`. - -When `img` is a plain array, then on-grid b-spline interpolation -will be used. It is possible to configure what degree of b-spline -to use with the parameter `degree`. For example one can use -`degree = Linear()` for linear interpolation, `degree = -Constant()` for nearest neighbor interpolation, or `degree = -Quadratic(Flat())` for quadratic interpolation. - -In the case `tform(I)` maps to indices outside the original -`img`, those locations are set to a value `fill` (which defaults -to `NaN` if the element type supports it, and `0` otherwise). The -parameter `fill` also accepts extrapolation schemes, such as -`Flat()`, `Periodic()` or `Reflect()`. - -For more control over the reconstruction scheme --- and how -beyond-the-edge points are handled --- pass `img` as an -`AbstractInterpolation` or `AbstractExtrapolation` from -[Interpolations.jl](https://github.com/JuliaMath/Interpolations.jl). - -The keyword `method` now also takes any InterpolationType from Interpolations.jl -or a Degree, which is used to define a BSpline interpolation of that degree, in -order to set the interpolation method used. - -# The meaning of the coordinates - -The output array `imgw` has indices that would result from -applying `inv(tform)` to the indices of `img`. This can be very -handy for keeping track of how pixels in `imgw` line up with -pixels in `img`. + otherwise is `0`. One can also pass the extrapolation boundary condition: `Flat()`, `Reflect()` and `Periodic()`. + +# See also + +There're some high-level interfaces of `warp`: + +- image rotation: [`imrotate`](@ref) +- image resize: [`imresize`](@ref) + +There are also lazy version of `warp`: + +- [`WarpedView`](@ref) is almost equivalent to `warp` except that it does not allocate memory. +- [`invwarpedview(img, tform, [indices]; kwargs...)`](@ref ImageTransformations.invwarpedview) + is almost equivalent to `warp(img, inv(tform), [indices]; kwargs...)` except that it does not + allocate memory. + +# Extended help + +## Parameters in detail + +This approach is known as backward mode warping. It is called "backward" because +the internal coordinate transformation is actually an inverse map from `axes(imgr)` to `axes(img)`. + +You can manually specify interpolation behavior by constructing `AbstractExtrapolation` object +and passing it to `warp` as `img`. However, this is usually cumbersome. For this reason, there +are two keywords `method` and `fillvalue` to conveniently construct an `AbstractExtrapolation` +object during `warp`. + +!!! warning + If `img` is an `AbstractExtrapolation`, then additional `method` and `fillvalue` keywords + will be discarded. + +### `method::Union{Degree, InterpolationType}` + +The interpolation method you want to use to reconstruct values in the wrapped image. + +Among those possible `InterpolationType` choice, there are some commonly used methods that you may +have used in other languages: + +- nearest neighbor: `BSpline(Constant())` +- triangle/bilinear: `BSpline(Linear())` +- bicubic: `BSpline(Cubic(Line(OnGrid())))` +- lanczos2: `Lanczos(2)` +- lanczos3: `Lanczos(3)` +- lanczos4: `Lanczos(4)` or `Lanczos4OpenCV()` -If you just want a plain array, you can "strip" the custom -indices with `parent(imgw)`. +When passing a `Degree`, it is expected to be a `BSpline`. For example, `Linear()` is equivalent to +`BSpline(Linear())`. -# Examples: a 2d rotation (see JuliaImages documentation for pictures) +### `fillvalue` +In case `tform(I)` maps to indices outside the original `img`, those locations are set to a value +`fillvalue`. The default fillvalue is `NaN` if the element type of `img` supports it, and `0` +otherwise. + +The parameter `fillvalue` can be either a `Number` or `Colorant`. In this case, it will be +converted to `eltype(imgr)` first. For example, `fillvalue = 1` will be converted to `Gray(1)` which +will fill the outside indices with white pixels. + +Also, `fillvalue` can be extrapolation schemes: `Flat()`, `Periodic()` and `Reflect()`. The best +way to understand these schemes is perhaps try it with small example: + +```jldoctest +using ImageTransformations, TestImages, Interpolations +using OffsetArrays: IdOffsetRange + +img = testimage("lighthouse") + +imgr = imrotate(img, π/4; fillvalue=Flat()) # zero extrapolation slope +imgr = imrotate(img, π/4; fillvalue=Periodic()) # periodic boundary +imgr = imrotate(img, π/4; fillvalue=Reflect()) # mirror boundary + +axes(imgr) + +# output + +(IdOffsetRange(values=-196:709, indices=-196:709), IdOffsetRange(values=-68:837, indices=-68:837)) ``` -julia> using Images, CoordinateTransformations, Rotations, TestImages, OffsetArrays -julia> img = testimage("lighthouse"); +## The meaning of the coordinates + +`imgw` keeps track of the indices that would result from applying `inv(tform)` to the indices of +`img`. This can be very handy for keeping track of how pixels in `imgw` line up with +pixels in `img`. + +```jldoctest +using ImageTransformations, TestImages, Interpolations + +img = testimage("lighthouse") +imgr = imrotate(img, π/4) +imgr_cropped = imrotate(img, π/4, axes(img)) -julia> axes(img) -(Base.OneTo(512),Base.OneTo(768)) +# No need to manually calculate the offsets +imgr[axes(img)...] == imgr_cropped -# Rotate around the center of `img` -julia> tfm = recenter(RotMatrix(-pi/4), center(img)) -AffineMap([0.707107 0.707107; -0.707107 0.707107], [-196.755,293.99]) +# output +true +``` -julia> imgw = warp(img, tfm); +!!! tip + For performance consideration, it's recommended to pass the `inds` positional argument to + `warp` instead of cropping the output with `imgw[inds...]`. -julia> axes(imgw) -(-196:709,-68:837) +# Examples: a 2d rotation -# Alternatively, specify the origin in the image itself -julia> img0 = OffsetArray(img, -30:481, -384:383); # origin near top of image +!!! note + This example only shows how to construct `tform` and calls `warp`. For common usage, it is + recommended to use [`imrotate`](@ref) function directly. -julia> rot = LinearMap(RotMatrix(-pi/4)) -LinearMap([0.707107 -0.707107; 0.707107 0.707107]) +Rotate around the center of `img`: -julia> imgw = warp(img0, rot); +```jldoctest +using ImageTransformations, CoordinateTransformations, Rotations, TestImages, OffsetArrays +img = testimage("lighthouse") # axes (1:512, 1:768) -julia> axes(imgw) -(-293:612,-293:611) +tfm = recenter(RotMatrix(-pi/4), center(img)) +imgw = warp(img, tfm) -julia> imgr = parent(imgw); +axes(imgw) -julia> axes(imgr) -(Base.OneTo(906),Base.OneTo(905)) +# output + +(IdOffsetRange(values=-196:709, indices=-196:709), IdOffsetRange(values=-68:837, indices=-68:837)) ``` + """ function warp(img::AbstractExtrapolation{T}, tform, inds::Tuple = autorange(img, inv(tform))) where T out = similar(Array{T}, inds) @@ -98,6 +164,13 @@ end function warp!(out, img::AbstractExtrapolation, tform) tform = _round(tform) @inbounds for I in CartesianIndices(axes(out)) + # Backward mode: + # 1. get the target index `I` of `out` + # 2. maps _back_ to original index `Ĩ` of `img` + # 3. interpolate/extrapolate the value of `Ĩ` + # 4. this value is then assigned to `out[I]` + # The advantage of backward mode is that all piexels + # in the output image will be iterated once very efficiently. out[I] = _getindex(img, tform(SVector(I.I))) end out @@ -116,17 +189,24 @@ Rotate image `img` by `θ`∈[0,2π) in a clockwise direction around its center # Arguments - `img::AbstractArray`: the original image that you need to rotate. -- `θ::Real`: the rotation angle in clockwise direction. To rotate the image in conter-clockwise - direction, use a negative value instead. To rotate the image by `d` degree, use the formular `θ=d*π/180`. +- `θ::Real`: the rotation angle in clockwise direction. + To rotate the image in conter-clockwise direction, use a negative value instead. + To rotate the image by `d` degree, use the formular `θ=d*π/180`. - `indices` (Optional): specifies the output image axes. By default, rotated image `imgr` will not be cropped, and thus `axes(imgr) == axes(img)` does not hold in general. # Parameters +!!! info + To construct `method` and `fillvalue` values, you may need to load `Interpolations` package first. + - `method::Union{Degree, InterpolationType}`: the interpolation method you want to use. By default it is - `BSpline(Linear())`. To construct the method instance, one may need to load `Interpolations`. + `Linear()`. - `fillvalue`: the value that used to fill the new region. The default value is `NaN` if possible, - otherwise is `0`. One can also pass the extrapolation boundary condition: `Flat()`, `Reflect()` and `Periodic()`. + otherwise is `0`. + +This function is a simple high-level interface to `warp`, for more explaination and details, +please refer to [`warp`](@ref). # Examples @@ -136,6 +216,7 @@ img = testimage("cameraman") # Rotate the image by π/4 in the clockwise direction imgr = imrotate(img, π/4) # output axes (-105:618, -105:618) + # Rotate the image by π/4 in the counter-clockwise direction imgr = imrotate(img, -π/4) # output axes (-105:618, -105:618) @@ -144,7 +225,7 @@ imgr = imrotate(img, -π/4) # output axes (-105:618, -105:618) imgr = imrotate(img, π/4, axes(img)) # output axes (1:512, 1:512) ``` -By default, `imrotate` uses bilinear interpolation with constant fill value. You can, +By default, `imrotate` uses bilinear interpolation with constant fill value (`NaN` or `0`). You can, for example, use the nearest interpolation and fill the new region with white pixels: ```julia @@ -160,8 +241,6 @@ using Interpolations, ImageCore imgr = imrotate(img, π/4, fillvalue = Periodic()) mosaicview([imgr for _ in 1:9]; nrow=3) ``` - -See also [`warp`](@ref). """ function imrotate(img::AbstractArray{T}, θ::Real, inds::Union{Tuple, Nothing} = nothing; kwargs...) where T # TODO: expose rotation center as a keyword diff --git a/src/warpedview.jl b/src/warpedview.jl index c3ff558..d1a055e 100644 --- a/src/warpedview.jl +++ b/src/warpedview.jl @@ -1,18 +1,11 @@ """ - WarpedView(img, tform, [indices]) -> wv + WarpedView(img, tform, [indices]; kwargs...) -> wv Create a view of `img` that lazily transforms any given index `I` -passed to `wv[I]` to correspond to `img[tform(I)]`. This approach -is known as backward mode warping. +passed to `wv[I]` so that `wv[I] == img[tform(I)]`. -The optional parameter `indices` can be used to specify the -domain of the resulting `wv`. By default the indices are computed -in such a way that `wv` contains all the original pixels in -`img`. To do this `inv(tform)` has to be computed. If the given -transformation `tform` does not support `inv`, then the parameter -`indices` has to be specified manually. - -see [`warpedview`](@ref) for more information. +This is the lazy view version of `warp`, please see [`warp`](@ref +for more information. """ struct WarpedView{T,N,A<:AbstractArray,F<:Transformation,I<:Tuple,E<:AbstractExtrapolation} <: AbstractArray{T,N} parent::A