-
-
Notifications
You must be signed in to change notification settings - Fork 230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DifferentialEquations.jl invalidates OrdinaryDiffEq.jl precompile caches for lu-factorization #785
DifferentialEquations.jl invalidates OrdinaryDiffEq.jl precompile caches for lu-factorization #785
Comments
What's up? |
Compiling Julia to use julia> using DifferentialEquations, SnoopCompile
julia> function lorenz(du,u,p,t)
du[1] = 10.0(u[2]-u[1])
du[2] = u[1]*(28.0-u[3]) - u[2]
du[3] = u[1]*u[2] - (8/3)*u[3]
end
lorenz (generic function with 1 method)
julia> u0 = [1.0;0.0;0.0]
3-element Vector{Float64}:
1.0
0.0
0.0
julia> tspan = (0.0,100.0)
(0.0, 100.0)
julia> prob = ODEProblem(lorenz,u0,tspan)
ODEProblem with uType Vector{Float64} and tType Float64. In-place: true
timespan: (0.0, 100.0)
u0: 3-element Vector{Float64}:
1.0
0.0
0.0
julia> alg = Rodas5()
Rodas5{0, true, DefaultLinSolve, Val{:forward}}(DefaultLinSolve(nothing, nothing, nothing))
julia> tinf = @snoopi_deep solve(prob,alg)
InferenceTimingNode: 1.474002/3.069819 on Core.Compiler.Timings.ROOT() with 9 direct children Julia with julia> using DifferentialEquations, SnoopCompile
julia> function lorenz(du,u,p,t)
du[1] = 10.0(u[2]-u[1])
du[2] = u[1]*(28.0-u[3]) - u[2]
du[3] = u[1]*u[2] - (8/3)*u[3]
end
lorenz (generic function with 1 method)
julia> u0 = [1.0;0.0;0.0]
3-element Vector{Float64}:
1.0
0.0
0.0
julia> tspan = (0.0,100.0)
(0.0, 100.0)
julia> prob = ODEProblem(lorenz,u0,tspan)
ODEProblem with uType Vector{Float64} and tType Float64. In-place: true
timespan: (0.0, 100.0)
u0: 3-element Vector{Float64}:
1.0
0.0
0.0
julia> alg = Rodas5()
Rodas5{0, true, DefaultLinSolve, Val{:forward}}(DefaultLinSolve(nothing, nothing, nothing))
julia> tinf = @snoopi_deep solve(prob,alg)
InferenceTimingNode: 4.389920/25.203741 on Core.Compiler.Timings.ROOT() with 7 direct children
julia> using DifferentialEquations, SnoopCompile, DiffEqFlux
julia> function lorenz(du,u,p,t)
du[1] = 10.0(u[2]-u[1])
du[2] = u[1]*(28.0-u[3]) - u[2]
du[3] = u[1]*u[2] - (8/3)*u[3]
end
lorenz (generic function with 1 method)
julia> u0 = [1.0;0.0;0.0]
3-element Vector{Float64}:
1.0
0.0
0.0
julia> tspan = (0.0,100.0)
(0.0, 100.0)
julia> prob = ODEProblem(lorenz,u0,tspan)
ODEProblem with uType Vector{Float64} and tType Float64. In-place: true
timespan: (0.0, 100.0)
u0: 3-element Vector{Float64}:
1.0
0.0
0.0
julia> alg = Rodas5()
Rodas5{0, true, DefaultLinSolve, Val{:forward}}(DefaultLinSolve(nothing, nothing, nothing))
julia> tinf = @snoopi_deep solve(prob,alg)
InferenceTimingNode: 1.507790/3.427428 on Core.Compiler.Timings.ROOT() with 9 direct children But this is still substantially better than |
It may lead to a small performance regression, so where this comes from is something we should look into within the packages. julia> using DifferentialEquations, SnoopCompile
julia> function lorenz(du,u,p,t)
du[1] = 10.0(u[2]-u[1])
du[2] = u[1]*(28.0-u[3]) - u[2]
du[3] = u[1]*u[2] - (8/3)*u[3]
end
lorenz (generic function with 1 method)
julia> u0 = [1.0;0.0;0.0]
3-element Vector{Float64}:
1.0
0.0
0.0
julia> tspan = (0.0,100.0)
(0.0, 100.0)
julia> prob = ODEProblem(lorenz,u0,tspan)
ODEProblem with uType Vector{Float64} and tType Float64. In-place: true
timespan: (0.0, 100.0)
u0: 3-element Vector{Float64}:
1.0
0.0
0.0
julia> alg = Rodas5()
Rodas5{0, true, DefaultLinSolve, Val{:forward}}(DefaultLinSolve(nothing, nothing, nothing))
julia> tinf = @snoopi_deep solve(prob,alg)
InferenceTimingNode: 1.540056/3.249808 on Core.Compiler.Timings.ROOT() with 9 direct children
julia> using BenchmarkTools; @benchmark solve($prob,$alg)
BenchmarkTools.Trial: 1518 samples with 1 evaluation.
Range (min … max): 3.153 ms … 9.154 ms ┊ GC (min … max): 0.00% … 62.66%
Time (median): 3.222 ms ┊ GC (median): 0.00%
Time (mean ± σ): 3.290 ms ± 577.525 μs ┊ GC (mean ± σ): 1.75% ± 6.29%
█
█▅▄▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂ ▂
3.15 ms Histogram: frequency by time 8.67 ms <
Memory estimate: 954.20 KiB, allocs estimate: 10579. Julia 1.6, julia> tinf = @snoopi_deep solve(prob,alg)
InferenceTimingNode: 4.364899/22.365325 on Core.Compiler.Timings.ROOT() with 8 direct children
julia> using BenchmarkTools; @benchmark solve($prob,$alg)
BenchmarkTools.Trial: 1547 samples with 1 evaluation.
Range (min … max): 3.106 ms … 9.695 ms ┊ GC (min … max): 0.00% … 63.91%
Time (median): 3.149 ms ┊ GC (median): 0.00%
Time (mean ± σ): 3.229 ms ± 594.692 μs ┊ GC (mean ± σ): 1.68% ± 6.02%
▅▆▆██▃▁▁▁
▂▃▆▇██████████▆▅▅▄▃▃▃▂▂▂▂▂▂▂▁▂▂▁▁▁▂▂▂▂▃▄▅▅▄▅▄▃▃▃▃▃▂▂▂▂▂▂▁▂▂ ▃
3.11 ms Histogram: frequency by time 3.38 ms <
Memory estimate: 903.67 KiB, allocs estimate: 8962. Julia 1.7, julia> tinf = @snoopi_deep solve(prob,alg)
InferenceTimingNode: 1.579549/3.173668 on Core.Compiler.Timings.ROOT() with 8 direct children
julia> using BenchmarkTools; @benchmark solve($prob,$alg)
BenchmarkTools.Trial: 1569 samples with 1 evaluation.
Range (min … max): 3.087 ms … 10.379 ms ┊ GC (min … max): 0.00% … 68.20%
Time (median): 3.117 ms ┊ GC (median): 0.00%
Time (mean ± σ): 3.185 ms ± 611.371 μs ┊ GC (mean ± σ): 1.71% ± 6.06%
██▄
▃█████▆▅▄▄▃▃▂▂▂▂▃▃▃▃▃▂▂▂▂▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▂ ▃
3.09 ms Histogram: frequency by time 3.54 ms <
Memory estimate: 717.47 KiB, allocs estimate: 10572. Julia 1.7, julia> tinf = @snoopi_deep solve(prob,alg)
InferenceTimingNode: 4.394989/21.127358 on Core.Compiler.Timings.ROOT() with 6 direct children
julia> using BenchmarkTools; @benchmark solve($prob,$alg)
BenchmarkTools.Trial: 1618 samples with 1 evaluation.
Range (min … max): 2.985 ms … 11.468 ms ┊ GC (min … max): 0.00% … 72.16%
Time (median): 3.022 ms ┊ GC (median): 0.00%
Time (mean ± σ): 3.088 ms ± 662.641 μs ┊ GC (mean ± σ): 1.74% ± 5.86%
▅█▇▇▅▅▄▁
▂▂▃▄▆█████████▇▆▆▅▅▄▃▄▃▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▃▃▃▃▃▂▃▃▁▂▂▂▂▂▂▂▂▂ ▃
2.98 ms Histogram: frequency by time 3.19 ms <
Memory estimate: 666.94 KiB, allocs estimate: 8955. |
The AD tooling gets a weird type thing right now that I saw in SciML/OrdinaryDiffEq.jl#1533 . I would re-benchmark after fixing that kind of stuff. It's either that or the instability of the factorization code. |
SciML/OrdinaryDiffEq.jl#1535 + JuliaDiff/SparseDiffTools.jl#167 makes it so that there are only 2 itrigs left: julia> itrigs = inference_triggers(tinf)
2-element Vector{InferenceTrigger}:
Inference triggered to call setindex!(::Vector{Tuple{Float64, Float64, Float64}}, ::Tuple{Bool, Bool, Bool}, ::Int64) from generate_chunked_partials (C:\Users\accou\.julia\dev\SparseDiffTools\src\differentiation\compute_jacobian_ad.jl:75) with specialization SparseDiffTools.generate_chunked_partials(::Vector{Float64}, ::UnitRange{Int64}, ::Val{3})
Inference triggered to call unsafe_copyto!(::Matrix{Float64}, ::Int64, ::Matrix{Float64}, ::Int64, ::Int64) from _copyto_impl! (.\array.jl:331) inlined into OrdinaryDiffEq.calc_W!(::Matrix{Float64}, ::OrdinaryDiffEq.ODEIntegrator{Rodas5{3, true, DefaultLinSolve, Val{:forward}}, true, Vector{Float64}, Nothing, Float64, SciMLBase.NullParameters, Float64, Float64, Float64, Float64, Vector{Vector{Float64}}, ODESolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Rodas5{3, true, DefaultLinSolve, Val{:forward}}, OrdinaryDiffEq.InterpolationData{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.Rosenbrock5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rodas5Tableau{Float64, Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, SciMLBase.NullParameters}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, DefaultLinSolve, SparseDiffTools.ForwardColorJacCache{Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, Float64}, Float64, 3}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, Float64}, Float64, 3}}, Vector{Float64}, Vector{Vector{Tuple{Float64, Float64, Float64}}}, UnitRange{Int64}, Nothing}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, SciMLBase.NullParameters}, Float64}, Float64, 1}}}}, DiffEqBase.DEStats}, ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, OrdinaryDiffEq.Rosenbrock5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rodas5Tableau{Float64, Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, SciMLBase.NullParameters}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, DefaultLinSolve, SparseDiffTools.ForwardColorJacCache{Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, Float64}, Float64, 3}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, Float64}, Float64, 3}}, Vector{Float64}, Vector{Vector{Tuple{Float64, Float64, Float64}}}, UnitRange{Int64}, Nothing}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, SciMLBase.NullParameters}, Float64}, Float64, 1}}}, OrdinaryDiffEq.DEOptions{Float64, Float64, Float64, Float64, PIController{Rational{Int64}}, typeof(DiffEqBase.ODE_DEFAULT_NORM), typeof(LinearAlgebra.opnorm), Nothing, CallbackSet{Tuple{}, Tuple{}}, typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), DataStructures.BinaryHeap{Float64, DataStructures.FasterForward}, DataStructures.BinaryHeap{Float64, DataStructures.FasterForward}, Nothing, Nothing, Int64, Tuple{}, Tuple{}, Tuple{}}, Vector{Float64}, Float64, Nothing, OrdinaryDiffEq.DefaultInit}, ::Nothing, ::OrdinaryDiffEq.Rosenbrock5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, Matrix{Float64}, Matrix{Float64}, OrdinaryDiffEq.Rodas5Tableau{Float64, Float64}, SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, SciMLBase.NullParameters}, SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, DefaultLinSolve, SparseDiffTools.ForwardColorJacCache{Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, Float64}, Float64, 3}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.UJacobianWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Float64, SciMLBase.NullParameters}, Float64}, Float64, 3}}, Vector{Float64}, Vector{Vector{Tuple{Float64, Float64, Float64}}}, UnitRange{Int64}, Nothing}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{SciMLBase.TimeGradientWrapper{ODEFunction{true, typeof(lorenz), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Vector{Float64}, SciMLBase.NullParameters}, Float64}, Float64, 1}}}, ::Float64, ::Bool, ::Bool) (C:\Users\accou\.julia\dev\OrdinaryDiffEq\src\derivative_utils.jl:503) |
With some more fixes, we got to: julia> itrigs = inference_triggers(tinf)
2-element Vector{InferenceTrigger}:
Inference triggered to call setindex!(::Vector{Tuple{Float64, Float64, Float64}}, ::Tuple{Bool, Bool, Bool}, ::Int64) from generate_chunked_partials (C:\Users\accou\.julia\dev\SparseDiffTools\src\differentiation\compute_jacobian_ad.jl:75) with specialization SparseDiffTools.generate_chunked_partials(::Vector{Float64}, ::UnitRange{Int64}, ::Val{3})
Inference triggered to call OrdinaryDiffEq.jacobian2W!(::Matrix{Float64}, ::LinearAlgebra.UniformScaling{Bool}, ::Float64, ::Matrix{Float64}, ::Bool) called from toplevel The first inference trigger should be fixed by JuliaDiff/SparseDiffTools.jl#168, which will need some help. The second looks like SnoopCompile is bugging. @timholy what could cause that? It's definitely not from the top level 😅 |
The first is definitely fixed by JuliaDiff/SparseDiffTools.jl#169 julia> itrigs = inference_triggers(tinf)
1-element Vector{InferenceTrigger}:
Inference triggered to call OrdinaryDiffEq.jacobian2W!(::Matrix{Float64}, ::LinearAlgebra.UniformScaling{Bool}, ::Float64, ::Matrix{Float64}, ::Bool) called from toplevel One inference trigger left, but it's a buggy one 😅 . |
```julia using DifferentialEquations, SnoopCompile function lorenz(du,u,p,t) du[1] = 10.0(u[2]-u[1]) du[2] = u[1]*(28.0-u[3]) - u[2] du[3] = u[1]*u[2] - (8/3)*u[3] end u0 = [1.0;0.0;0.0] tspan = (0.0,100.0) prob = ODEProblem(lorenz,u0,tspan) alg = Rodas5(chunk_size = Val{3}(), linsolve = DiffEqBase.LUFactorize()) tinf = @snoopi_deep solve(prob,alg) ``` Before: After: ```julia # InferenceTimingNode: 1.002721/2.089656 on Core.Compiler.Timings.ROOT() with 3 direct children ``` Fixes SciML/DifferentialEquations.jl#785
```julia using DifferentialEquations, SnoopCompile function lorenz(du,u,p,t) du[1] = 10.0(u[2]-u[1]) du[2] = u[1]*(28.0-u[3]) - u[2] du[3] = u[1]*u[2] - (8/3)*u[3] end u0 = [1.0;0.0;0.0] tspan = (0.0,100.0) prob = ODEProblem(lorenz,u0,tspan) alg = Rodas5(chunk_size = Val{3}(), linsolve = DiffEqBase.LUFactorize()) tinf = @snoopi_deep solve(prob,alg) ``` Before: After: ```julia # InferenceTimingNode: 1.002721/2.089656 on Core.Compiler.Timings.ROOT() with 3 direct children ``` Fixes SciML/DifferentialEquations.jl#785
Solved by removing one line from Symbolics.jl JuliaSymbolics/Symbolics.jl#471 |
And better inferred version:
The text was updated successfully, but these errors were encountered: