Issues with DIVAnd under Julia 1.0.4 and onwards.

[croachutas]

Okay, I've been using DIVAnd to create a new global atlas of oxygen concentration. A few months back I migrated form the matlab to julia versions. My script runs fine on Julia 1.0.3 but using Julia v1.0.4, v1.1.0 and v1.1.1 (just tried upgrading in the last few days) it instead crashes with a message MethodError: no method matching setindex_shape_check(::Float64, ::Int64). I've tried this on two different machines (one remote VM and a local compute server) with the same results.

The error (shown below) occurs somewhere in DIVAndrun (currently called as: fij, si = DIVAndrun(maski,(pmi,pni,pti),(xj,yj,tj),(xo,yo,to),resid,(lenx,leny,lent),epsilon; velocity=(ui,vi,0*ui), moddim=mdi) )...

Running several of the tests included in DIVAnd (notably DIVAnd_simple_example_big3D.jl) work fine, which would suggest it is either associated with the moddim input (mostly likely given the error complaining about int64s) or possibly the velocity field.

notably, a number of other packages got broken in the Julia v1.0 to v1.1 update...JuliaLang/julia#30374

ERROR: LoadError: MethodError: no method matching setindex_shape_check(::Float64, ::Int64)
Closest candidates are:
  setindex_shape_check(!Matched::AbstractArray{#s72,1} where #s72, ::Integer) at indices.jl:218
  setindex_shape_check(!Matched::AbstractArray{#s72,1} where #s72, ::Integer, !Matched::Integer) at indices.jl:221
  setindex_shape_check(!Matched::AbstractArray{#s72,2} where #s72, ::Integer, !Matched::Integer) at indices.jl:225
  ...
Stacktrace:
 [1] macro expansion at ./multidimensional.jl:694 [inlined]
 [2] _unsafe_setindex!(::IndexLinear, ::Array{Float64,1}, ::Float64, ::Base.LogicalIndex{Int64,BitArray{1}}) at ./multidimensional.jl:689
 [3] _setindex! at ./multidimensional.jl:684 [inlined]
 [4] setindex! at ./abstractarray.jl:1020 [inlined]
 [5] DIVAnd_obs(::DIVAnd.DIVAnd_struct{Float64,Int64,3,SparseArrays.SparseMatrixCSC{Float64,Int64}}, ::Tuple{Array{Float64,3},Array{Float64,3},Array{Float64,3}}, ::Tuple{Array{Float64,1},Array{Float64,1},Array{Float64,1}}, ::Array{Float64,1}, ::LinearAlgebra.Diagonal{Float64,Array{Float64,1}}, ::Array{Float64,2}) at /home/ubuntu/.julia/packages/DIVAnd/2RDHG/src/DIVAnd_obs.jl:44
 [6] #DIVAndrun#265(::Tuple{Array{Float64,3},Array{Float64,3},Array{Float64,3}}, ::Bool, ::Bool, ::Float64, ::Int64, ::Int64, ::Tuple{}, ::Symbol, ::Array{Int64,1}, ::Array{Float64,2}, ::Array{Any,1}, ::Int64, ::typeof(DIVAnd_pc_none), ::getfield(DIVAnd, Symbol("##267#269")), ::Array{Float64,3}, ::Array{Float64,1}, ::Float64, ::Bool, ::Array{Any,1}, ::Float64, ::Tuple{}, ::Tuple{}, ::Int64, ::Tuple{}, ::Tuple{}, ::typeof(DIVAndrun), ::Type, ::BitArray{3}, ::Tuple{Array{Float64,3},Array{Float64,3},Array{Float64,3}}, ::Tuple{Array{Float64,3},Array{Float64,3},Array{Float64,3}}, ::Tuple{Array{Float64,1},Array{Float64,1},Array{Float64,1}}, ::Array{Float64,1}, ::Tuple{Float64,Float64,Float64}, ::Float64) at /home/ubuntu/.julia/packages/DIVAnd/2RDHG/src/DIVAndrun.jl:64
 [7] (::getfield(DIVAnd, Symbol("#kw##DIVAndrun")))(::NamedTuple{(:velocity, :moddim),Tuple{Tuple{Array{Float64,3},Array{Float64,3},Array{Float64,3}},Array{Int64,1}}}, ::typeof(DIVAndrun), ::BitArray{3}, ::Tuple{Array{Float64,3},Array{Float64,3},Array{Float64,3}}, ::Tuple{Array{Float64,3},Array{Float64,3},Array{Float64,3}}, ::Tuple{Array{Float64,1},Array{Float64,1},Array{Float64,1}}, ::Array{Float64,1}, ::Tuple{Float64,Float64,Float64}, ::Float64) at ./none:0
 [8] top-level scope at ./util.jl:156
 [9] include at ./boot.jl:326 [inlined]
 [10] include_relative(::Module, ::String) at ./loading.jl:1038
 [11] include(::Module, ::String) at ./sysimg.jl:29
 [12] exec_options(::Base.JLOptions) at ./client.jl:267
 [13] _start() at ./client.jl:436

[ctroupin]

Hello,
can you indicate which version of DIVAnd you are presently using?

(in the package REPL mode):

(v1.1) pkg> status

Can you also run the tests on DIVAnd?

Pkg.test("DIVAnd")

[croachutas]

I'm using DIVAnd v2.3.1 on Julia v1.1.1 DIVAnd passes all tests.

I think I was still on DIVAnd v2.3.0 on Julia v1.0.3. Again passed all tests.

Also, note that I'm using the Julia pre-compiled linux x86 binaries.

---

Anyway, I've remove the .julia folder form my home directory and reinstalled DIVAnd and all associated packages... And now I'm getting the same problem with Julia v1.0.3...

---

Reviewing the error, it points to the part of DIVAnd_obs that deals with removing NaN values... And I know the input data contains NaNs (issues regarding interpolation near coasts). Adding some extra code to my scripts to remove the NaN values before running DIVAndrun fixes things in both Julia v1.0.3 and v1.1.1.

Still, doesn't explain why I'm getting the error...

[ctroupin]

Could you post your code or a minimum working example (failing in this case) with a subset of the data, so we can try to reproduce the issue on our machines?

I guess you already tried to de-activate the velocity and the modim?

Indeed DIVAnd can deal with NaN's, in fact they are replaced by zeros and a warning is issued, so that should not be an issue.

[Alexander-Barth]

In the last released version, there is indeed a issue when some observations are equal to NaN in DIVAnd.DIVAndrun. It is fixed in the master version which will be released soon.

If you want to try the "master version" of DIVAnd, you can use:

]dev DIVAnd

We only merge changes in the master version when all test pass. So it is reasonable stable.

Thank you for reporting this issue.

[Alexander-Barth]

For your information, I just released a new version of DIVAnd where this problem should be solved.
If this would not be the case, please re-open this issue or file a new issue.
Thanks for your feed-back.