Test fail on 1.2.0-rc

[jarlebring]

Tests failing on the julia 1.2.0 release candidate:

Check when fixed:

• test deflation modes (new)
• test interpolation

Running test deflation (9 / 49)
[ Info: Computing eigenvalue k=1
[ Info: Computing eigenvalue k=2
[ Info: Computing eigenvalue k=3
Deflation modes (new): Error During Test at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/deflation.jl:44
  Got exception outside of a @test
  No convergence: 'Number of iterations exceeded. maxit=30.' eigenvalue approx:-0.06646277598910488 + 0.0im, errmeasure:0.5124720777960479
  Stacktrace:
   [1] #augnewton#12(::Type{DefaultErrmeasure}, ::Float64, ::Int64, ::Float64, ::Array{Float64,1}, ::Array{Float64,1}, ::Int64, ::typeof(backslash_linsolvercreator), ::Float64, ::Int64, ::typeof(augnewton), ::Type{Complex{Float64}}, ::DeflatedGenericNEP) at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/src/method_newton.jl:349
   [2] #augnewton at ./none:0 [inlined]
   [3] #augnewton#11 at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/src/method_newton.jl:273 [inlined]
   [4] (::getfield(NonlinearEigenproblems.NEPSolver, Symbol("#kw##augnewton")))(::NamedTuple{(:v, :λ, :tol),Tuple{Array{Float64,1},Float64,Float64}}, ::typeof(augnewton), ::DeflatedGenericNEP) at ./none:0
   [5] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/deflation.jl:73
   [6] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Test/src/Test.jl:1113
   [7] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/deflation.jl:46
   [8] include at ./boot.jl:328 [inlined]
   [9] include_relative(::Module, ::String) at ./loading.jl:1094
   [10] include(::Module, ::String) at ./Base.jl:31
   [11] include(::String) at ./client.jl:431
   [12] run_all_tests(::String) at ./util.jl:156
   [13] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/runtests.jl:15
   [14] include at ./boot.jl:328 [inlined]
   [15] include_relative(::Module, ::String) at ./loading.jl:1094
   [16] include(::Module, ::String) at ./Base.jl:31
   [17] include(::String) at ./client.jl:431
   [18] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/test/runtests.jl:3
   [19] include at ./boot.jl:328 [inlined]
   [20] include_relative(::Module, ::String) at ./loading.jl:1094
   [21] include(::Module, ::String) at ./Base.jl:31
   [22] include(::String) at ./client.jl:431
   [23] top-level scope at REPL[2]:1
   [24] eval(::Module, ::Any) at ./boot.jl:330
   [25] eval_user_input(::Any, ::REPL.REPLBackend) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/REPL/src/REPL.jl:86
   [26] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/REPL/src/REPL.jl:118 [inlined]
   [27] (::getfield(REPL, Symbol("##26#27")){REPL.REPLBackend})() at ./task.jl:268
  
 18.421221 seconds (48.86 M allocations: 2.413 GiB, 8.34% gc time)
Running test dep_distributed (10 / 49)

iter 6 err:0.09221166165051937 λ=-0.8351242006814612 + 9.376065881700682e-14im
iter 7 err:0.0470030875562586 λ=-0.5205850856233447 + 6.186155722153201e-14im
iter 8 err:0.008078489101207599 λ=-0.3703903337968818 + 8.88412473784992e-15im
iter 9 err:8.407550579517324e-5 λ=-0.35859047993971416 - 4.275319167195343e-15im
iter 10 err:9.268642204288447e-9 λ=-0.35871898625217646 - 4.0906142378585815e-15im
iter 11 err:3.0867757719363646e-16 λ=-0.35871894596863185 - 4.0907450196953015e-15imiter 1 err:0.43623215302547924 λ=-0.75 + 0.0im
iter 2 err:0.10449448816361027 λ=0.7396832358295808 + 0.0im
iter 3 err:0.13774443863222288 λ=7.413041970508159 + 0.0im
iter 4 err:0.12594937536049514 λ=5.510904512232892 + 0.0im
iter 5 err:0.11487700469328763 λ=4.387440348854344 + 0.0im
iter 6 err:0.04461550106787815 λ=4.1196041109446595 + 0.0im
iter 7 err:0.0002386127257451255 λ=4.101540426979313 + 0.0im
iter 8 err:4.73904595803146e-9 λ=4.101471922084464 + 0.0im
iter 9 err:1.6417866787101327e-17 λ=4.101471921490994 + 0.0imiter 1 err:0.44891940655885976 λ=-0.75 + 0.0im
iter 2 err:0.13060581975036242 λ=-47.06990589400386 + 0.0im
iter 3 err:0.1546521425312362 λ=-51.7528610210808 + 0.0im
iter 4 err:0.16453135481631256 λ=-32.40934934578844 + 0.0im
iter 5 err:0.16558649297347458 λ=-11.922433027434579 + 0.0im
iter 6 err:0.10875718235250759 λ=1.7294607557965502 + 0.0im
iter 7 err:0.04784183007647741 λ=2.192717608626996 + 0.0im
iter 8 err:0.15995655600143843 λ=-2.2674749999819133 + 0.0im
iter 9 err:0.14305635460052524 λ=-0.17398194762602115 + 0.0im
iter 10 err:0.13948929206570237 λ=-12.554742831188952 + 0.0im
iter 11 err:0.14310796423122044 λ=-22.961192541580175 + 0.0im
iter 12 err:0.15457035452993634 λ=-18.90717662629944 + 0.0im
iter 13 err:0.14402382569254815 λ=-6.507281911755559 + 0.0im
iter 14 err:0.14243481092134175 λ=322.85706781043615 + 0.0im
iter 15 err:0.04631694088037154 λ=323.632785388941 + 0.0im
iter 16 err:0.01136762567490835 λ=288.3668554857322 + 0.0im
iter 17 err:0.0013583992724845472 λ=272.72225439865 + 0.0im
iter 18 err:3.0257532631691477e-5 λ=270.05276091141246 + 0.0im
iter 19 err:1.0447154567174203e-8 λ=270.01649317154613 + 0.0im
iter 20 err:2.2588167749602545e-15 λ=270.01651218990145 + 0.0im
Random sparse dep: Test Failed at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:42
  Expression: compute_resnorm(pep, λ2, x2) < eps() * 100
   Evaluated: 7.24782847798937e-11 < 2.220446049250313e-14
Stacktrace:
 [1] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:42
 [2] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Test/src/Test.jl:1113
 [3] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:33
 [4] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Test/src/Test.jl:1113
 [5] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:11
Random sparse dep: Test Failed at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:43
  Expression: abs(λ1 - λ2) / abs(λ1) < eps() * 1000
   Evaluated: 64.83405113053738 < 2.220446049250313e-13
Stacktrace:
 [1] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:43
 [2] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Test/src/Test.jl:1113
 [3] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:33
 [4] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Test/src/Test.jl:1113
 [5] top-level scope at /home/jarl/jobb_synced/src/neppack_master2/NonlinearEigenproblems.jl/test/interpolation.jl:11
iter 1 err:0.04137038504793074 λ=1.0 + 0.0im
iter 2 err:0.04622383463479617 λ=-0.43093661068790245 + 0.0im
iter 3 err:0.030833641989311 λ=0.09751327648383945 + 0.0im

[jarlebring]

The deflation test failure seems to be because we get a different matrices from the gallery, so probably not entirely our fault 🎉 :

Old:

julia> versioninfo()
Julia Version 1.1.0
julia> nep=nep_gallery("dep0_sparse",10);
julia> nep.A[1][1,1]
1.4754251427667966

New:

julia> versioninfo()
Julia Version 1.2.0-rc2.0
julia> nep=nep_gallery("dep0_sparse",10);
julia> nep.A[1][1,1]
0.8236475079774124

This seems to be due to JuliaLang/julia#30494

[jarlebring]

The julia developers in julia PR 30494 basically say that one cannot assume that the sequence of numbers generated by sprandn (as well as the other rand-functions) is not stable over julia versions. So we need to change the random examples, to not depend on random numbers. Also, we should never use a random starting vector in the tests (should be mostly be like that already).

[jarlebring]

Decision on meeting 2019-08-28: Do a minimal version of a matrix (+sparse matrix) generator which is stable over versions.

[eringh]

In the JuliaLang issue 27614 it is pretty clearly stated that the RNG-functionallity is not static. However, in the issue 32954 there is an argument to whether or not people need reproducibility.

[eringh]

We could look into Middle Square Weyl Sequence RNG. It seems to not be secure, although that might not be a requirement for us.

[eringh]

Initial implementation in aee82ec.

[jarlebring]

Cool. Can you make a PR so it is easier to read when you want us to start looking at it? It doesn't have to be finished.

Btw. I would recommend 0xFF...F -> typemax(UInt64) for readability.

[eringh]

I think I am getting ready with the generation now.
@jarlebring: Can you take a look at the file GalleryRand.jl in the folder src/gallery_extra, and galleryrand.jl in test?

I might have gone overboard with the typeing. But since the inner most generator is hard-typed (from the algorithm) it somehow escaped out to the whole thing.

[jarlebring]

Can you create a pr so we can have discussion there. It's easier to comment on code etc.