OptimizationFunction with AutoForwardDiff doesn't work with sparse ODEProblem
#3376
Labels
bug
Something isn't working
Comments
|
Needs |
|
I'm trying to replicate this in Ubuntu 24.04 and adding |
|
can you show me what you ran? and try v3 |
|
I'm trying to install v3 first, but it says that in order to install it, I need DifferentialEquations v5 or v6... I'll try to downgrade to see if it's fixed |
|
]I ran a fresh install of Julia with only the necessary packages. While trying to install ERROR: Unsatisfiable requirements detected for package RecursiveArrayTools [731186ca]:
RecursiveArrayTools [731186ca] log:
├─possible versions are: 0.16.0 - 3.29.0 or uninstalled
├─restricted by compatibility requirements with LinearSolve [7ed4a6bd] to versions: 3.8.0 - 3.29.0 or uninstalled, leaving only versions: 3.8.0 - 3.29.0
│ └─LinearSolve [7ed4a6bd] log:
│ ├─possible versions are: 0.1.0 - 3.0.0 or uninstalled
│ ├─restricted to versions * by project [43a2c169], leaving only versions: 0.1.0 - 3.0.0
│ │ └─project [43a2c169] log:
│ │ ├─possible versions are: 0.0.0 or uninstalled
│ │ └─project [43a2c169] is fixed to version 0.0.0
│ └─restricted to versions 3 by an explicit requirement, leaving only versions: 3.0.0
└─restricted by compatibility requirements with DifferentialEquations [0c46a032] to versions: 0.16.0 - 2.38.10 — no versions left
└─DifferentialEquations [0c46a032] log:
├─possible versions are: 5.0.0 - 7.15.0 or uninstalled
├─restricted to versions * by project [43a2c169], leaving only versions: 5.0.0 - 7.15.0
│ └─project [43a2c169] log: see above
└─restricted by compatibility requirements with LinearSolve [7ed4a6bd] to versions: 5.0.0 - 6.20.0 or uninstalled, leaving only versions: 5.0.0 - 6.20.0
└─LinearSolve [7ed4a6bd] log: see aboveThe packages that are installed: @v1.11) pkg> status
Status `~/.julia/environments/v1.11/Project.toml`
[336ed68f] CSV v0.10.15
⌃ [13f3f980] CairoMakie v0.12.18
[501788e0] CollocationMethods v0.1.0 `../../../OneDrive/WIP/Repositories/CollocationMethods`
[a93c6f00] DataFrames v1.7.0
[0c46a032] DifferentialEquations v7.16.0
[f6369f11] ForwardDiff v0.10.38
⌃ [e9467ef8] GLMakie v0.10.18
⌅ [7ed4a6bd] LinearSolve v2.39.0
[2fda8390] LsqFit v0.15.0
[65ed5ab3] ModelingBI v0.1.0 `../../../OneDrive/WIP/Repositories/ModelingBI`
[961ee093] ModelingToolkit v9.62.0
[f5f84d89] NumericalTools v0.1.0 `../../../OneDrive/WIP/Repositories/NumericalTools`
[7f7a1694] Optimization v4.1.1
[36348300] OptimizationOptimJL v0.4.1
[1dea7af3] OrdinaryDiffEq v6.91.0
[53ae85a6] SciMLStructures v1.6.1
[e56a9233] Sparspak v0.3.9
[2efcf032] SymbolicIndexingInterface v0.3.37
[3bb67fe8] TranscodingStreams v0.11.3
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated` |
|
Already latest releases should untangle that now. To be clear, this shouldn't be an issue even on the earlier versions, but at least I'm testing on v3 just fine. Let me know if it's not just a versioning thing. I'll be at an Intel CPU in a bit and it could be something BLAS-dependent. |
|
It fixed the ERROR: UndefVarError: `DefaultLinearSolver` not defined in `LinearSolveSparseArraysExt`
Suggestion: check for spelling errors or missing imports.
Stacktrace:
[1] defaultalg(A::SparseArrays.SparseMatrixCSC{ForwardDiff.Dual{ForwardDiff.Tag{…}, Float64, 3}, Int64}, b::Vector{ForwardDiff.Dual{ForwardDiff.Tag{…}, Float64, 3}}, assump::OperatorAssumptions{Bool})
@ LinearSolveSparseArraysExt ~/.julia/packages/LinearSolve/SFMdw/ext/LinearSolveSparseArraysExt.jl:49
[2] init(::LinearProblem{…}, ::Nothing; assumptions::OperatorAssumptions{…}, kwargs::@Kwargs{…})
@ LinearSolve ~/.julia/packages/LinearSolve/SFMdw/src/default.jl:270
[3] init
@ ~/.julia/packages/LinearSolve/SFMdw/src/default.jl:266 [inlined]
[4] alg_cache(alg::Rodas5P{…}, u::Vector{…}, rate_prototype::Vector{…}, ::Type{…}, ::Type{…}, ::Type{…}, uprev::Vector{…}, uprev2::Vector{…}, f::ODEFunction{…}, t::Float64, dt::Float64, reltol::ForwardDiff.Dual{…}, p::MTKParameters{…}, calck::Bool, ::Val{…})
@ OrdinaryDiffEqRosenbrock ~/.julia/packages/OrdinaryDiffEqRosenbrock/bW9xh/src/rosenbrock_caches.jl:780
[5] __init(prob::ODEProblem{…}, alg::Rodas5P{…}, timeseries_init::Tuple{}, ts_init::Tuple{}, ks_init::Tuple{}, recompile::Type{…}; saveat::Vector{…}, tstops::Tuple{}, d_discontinuities::Tuple{}, save_idxs::Nothing, save_everystep::Bool, save_on::Bool, save_start::Bool, save_end::Nothing, callback::Nothing, dense::Bool, calck::Bool, dt::Float64, dtmin::Float64, dtmax::Float64, force_dtmin::Bool, adaptive::Bool, gamma::Rational{…}, abstol::Nothing, reltol::Nothing, qmin::Rational{…}, qmax::Int64, qsteady_min::Int64, qsteady_max::Rational{…}, beta1::Nothing, beta2::Nothing, qoldinit::Rational{…}, controller::Nothing, fullnormalize::Bool, failfactor::Int64, maxiters::Int64, internalnorm::typeof(DiffEqBase.ODE_DEFAULT_NORM), internalopnorm::typeof(LinearAlgebra.opnorm), isoutofdomain::typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), unstable_check::typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), verbose::Bool, timeseries_errors::Bool, dense_errors::Bool, advance_to_tstop::Bool, stop_at_next_tstop::Bool, initialize_save::Bool, progress::Bool, progress_steps::Int64, progress_name::String, progress_message::typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), progress_id::Symbol, userdata::Nothing, allow_extrapolation::Bool, initialize_integrator::Bool, alias::ODEAliasSpecifier, initializealg::OrdinaryDiffEqCore.DefaultInit, kwargs::@Kwargs{})
@ OrdinaryDiffEqCore ~/.julia/packages/OrdinaryDiffEqCore/oa4Vj/src/solve.jl:406
[6] __init (repeats 5 times)
@ ~/.julia/packages/OrdinaryDiffEqCore/oa4Vj/src/solve.jl:11 [inlined]
[7] __solve(::ODEProblem{…}, ::Rodas5P{…}; kwargs::@Kwargs{…})
@ OrdinaryDiffEqCore ~/.julia/packages/OrdinaryDiffEqCore/oa4Vj/src/solve.jl:6
[8] __solve
@ ~/.julia/packages/OrdinaryDiffEqCore/oa4Vj/src/solve.jl:1 [inlined]
[9] #solve_call#35
@ ~/.julia/packages/DiffEqBase/TQ6hN/src/solve.jl:635 [inlined]
[10] solve_up(prob::ODEProblem{…}, sensealg::Nothing, u0::Vector{…}, p::MTKParameters{…}, args::Rodas5P{…}; kwargs::@Kwargs{…})
@ DiffEqBase ~/.julia/packages/DiffEqBase/TQ6hN/src/solve.jl:1124
[11] solve_up
@ ~/.julia/packages/DiffEqBase/TQ6hN/src/solve.jl:1102 [inlined]
[12] solve(prob::ODEProblem{…}, args::Rodas5P{…}; sensealg::Nothing, u0::Nothing, p::Nothing, wrap::Val{…}, kwargs::@Kwargs{…})
@ DiffEqBase ~/.julia/packages/DiffEqBase/TQ6hN/src/solve.jl:1039
[13] (::var"#loss#10"{SymbolicIndexingInterface.ParameterHookWrapper{…}})(x::Vector{ForwardDiff.Dual{…}}, args::Tuple{ODESystem, ODEProblem{…}, Vector{…}, Vector{…}})
@ Main ~/OneDrive/WIP/Immobilized Benzonase/BI-optims.jl:93
[14] #7
@ ~/OneDrive/WIP/Immobilized Benzonase/BI-optims.jl:114 [inlined]
[15] FixTail
@ ~/.julia/packages/DifferentiationInterface/srtnM/src/utils/context.jl:7 [inlined]
[16] vector_mode_dual_eval!
@ ~/.julia/packages/ForwardDiff/UBbGT/src/apiutils.jl:24 [inlined]
[17] vector_mode_gradient!(result::Vector{Float64}, f::DifferentiationInterface.FixTail{var"#7#12"{…}, Tuple{…}}, x::Vector{Float64}, cfg::ForwardDiff.GradientConfig{ForwardDiff.Tag{…}, Float64, 3, Vector{…}})
@ ForwardDiff ~/.julia/packages/ForwardDiff/UBbGT/src/gradient.jl:98
[18] gradient!
@ ~/.julia/packages/ForwardDiff/UBbGT/src/gradient.jl:39 [inlined]
[19] gradient!(f::var"#7#12"{…}, grad::Vector{…}, prep::DifferentiationInterfaceForwardDiffExt.ForwardDiffGradientPrep{…}, backend::AutoForwardDiff{…}, x::Vector{…}, contexts::DifferentiationInterface.Constant{…})
@ DifferentiationInterfaceForwardDiffExt ~/.julia/packages/DifferentiationInterface/srtnM/ext/DifferentiationInterfaceForwardDiffExt/onearg.jl:362
[20] (::OptimizationBase.var"#grad#16"{SciMLBase.NullParameters, OptimizationFunction{…}, AutoForwardDiff{…}})(res::Vector{Float64}, θ::Vector{Float64})
@ OptimizationBase ~/.julia/packages/OptimizationBase/gvXsf/src/OptimizationDIExt.jl:28
[21] (::OptimizationOptimJL.var"#19#23"{OptimizationCache{…}, OptimizationOptimJL.var"#18#22"{…}})(G::Vector{Float64}, θ::Vector{Float64})
@ OptimizationOptimJL ~/.julia/packages/OptimizationOptimJL/e3bUa/src/OptimizationOptimJL.jl:285
[22] value_gradient!!(obj::OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, x::Vector{Float64})
@ NLSolversBase ~/.julia/packages/NLSolversBase/kavn7/src/interface.jl:82
[23] value_gradient!!(bw::Optim.BarrierWrapper{OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, Optim.BoxBarrier{Vector{Float64}, Vector{Float64}}, Float64, Float64, Vector{Float64}}, x::Vector{Float64})
@ Optim ~/.julia/packages/Optim/HvjCd/src/multivariate/solvers/constrained/fminbox.jl:81
[24] initial_state(method::BFGS{…}, options::Optim.Options{…}, d::Optim.BarrierWrapper{…}, initial_x::Vector{…})
@ Optim ~/.julia/packages/Optim/HvjCd/src/multivariate/solvers/first_order/bfgs.jl:94
[25] optimize(df::OnceDifferentiable{…}, l::Vector{…}, u::Vector{…}, initial_x::Vector{…}, F::Fminbox{…}, options::Optim.Options{…})
@ Optim ~/.julia/packages/Optim/HvjCd/src/multivariate/solvers/constrained/fminbox.jl:322
[26] __solve(cache::OptimizationCache{OptimizationFunction{…}, OptimizationBase.ReInitCache{…}, Vector{…}, Vector{…}, Nothing, Nothing, Nothing, Fminbox{…}, Bool, var"#track_results#11", Nothing})
@ OptimizationOptimJL ~/.julia/packages/OptimizationOptimJL/e3bUa/src/OptimizationOptimJL.jl:312
[27] solve!(cache::OptimizationCache{OptimizationFunction{…}, OptimizationBase.ReInitCache{…}, Vector{…}, Vector{…}, Nothing, Nothing, Nothing, Fminbox{…}, Bool, var"#track_results#11", Nothing})
@ SciMLBase ~/.julia/packages/SciMLBase/YTOjh/src/solve.jl:187
[28] solve(::OptimizationProblem{…}, ::BFGS{…}; kwargs::@Kwargs{…})
@ SciMLBase ~/.julia/packages/SciMLBase/YTOjh/src/solve.jl:95
[29] CST_ED_optim_prob(linsolver::AutoForwardDiff{nothing, Nothing}; isplot::Bool)
@ Main ~/OneDrive/WIP/Immobilized Benzonase/BI-optims.jl:119
[30] CST_ED_optim_prob(linsolver::AutoForwardDiff{nothing, Nothing})
@ Main ~/OneDrive/WIP/Immobilized Benzonase/BI-optims.jl:65
[31] top-level scope
@ REPL[12]:1
Some type information was truncated. Use `show(err)` to see complete types.Do I need to add any extra argument? Output of Julia Version 1.11.3
Commit d63adeda50d (2025-01-21 19:42 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 4 × Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, cascadelake)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) |
|
after new updates, the old error message about the ERROR: MethodError: no method matching LinearSolveSparseArraysExt.KLU.KLUFactorization(::SparseArrays.SparseMatrixCSC{ForwardDiff.Dual{ForwardDiff.Tag{…}, Float64, 3}, Int64})
The type `LinearSolveSparseArraysExt.KLU.KLUFactorization` exists, but no method is defined for this combination of argument types when trying to construct it.
Closest candidates are:
LinearSolveSparseArraysExt.KLU.KLUFactorization(::Any, ::Any, ::Any, ::Any)
@ LinearSolveSparseArraysExt ~/.julia/packages/LinearSolve/QuCP8/src/KLU/klu.jl:140
LinearSolveSparseArraysExt.KLU.KLUFactorization(::SparseArrays.SparseMatrixCSC{Tv, Ti}) where {Tv<:Union{Float64, ComplexF64}, Ti<:Union{Int32, Int64}}
@ LinearSolveSparseArraysExt ~/.julia/packages/LinearSolve/QuCP8/src/KLU/klu.jl:174Packages installed: julia> using Pkg; Pkg.status()
Status `~/.julia/environments/v1.11/Project.toml`
[6e4b80f9] BenchmarkTools v1.6.0
[336ed68f] CSV v0.10.15
⌃ [13f3f980] CairoMakie v0.12.18
[501788e0] CollocationMethods v0.1.0 `../../../OneDrive/WIP/Repositories/CollocationMethods`
[a93c6f00] DataFrames v1.7.0
[0c46a032] DifferentialEquations v7.16.0
[7da242da] Enzyme v0.13.30
[f6369f11] ForwardDiff v0.10.38
⌃ [e9467ef8] GLMakie v0.10.18
[929cbde3] LLVM v9.2.0
[7ed4a6bd] LinearSolve v3.3.1
[2fda8390] LsqFit v0.15.0
[65ed5ab3] ModelingBI v0.1.0 `../../../OneDrive/WIP/Repositories/ModelingBI`
[961ee093] ModelingToolkit v9.64.0
[f5f84d89] NumericalTools v0.1.0 `../../../OneDrive/WIP/Repositories/NumericalTools`
[7f7a1694] Optimization v4.1.1
[36348300] OptimizationOptimJL v0.4.1
[1dea7af3] OrdinaryDiffEq v6.91.0
[37e2e3b7] ReverseDiff v1.15.3
[295af30f] Revise v3.7.2
[1ed8b502] SciMLSensitivity v7.74.0
[53ae85a6] SciMLStructures v1.6.1
[e56a9233] Sparspak v0.3.9
[2efcf032] SymbolicIndexingInterface v0.3.37
[3bb67fe8] TranscodingStreams v0.11.3
⌃ [e88e6eb3] Zygote v0.6.75
Info Packages marked with ⌃ have new versions available and may be upgradable. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug 🐞
When trying to optimize an
ODEProblemwithsparse=true, the KLUFactorization gives an error with sparse matrices. Withsparse=falsethe optimization runs as expected.Minimal Reproducible Example 👇
Error & Stacktrace⚠️
Environment (please complete the following information):
using Pkg; Pkg.status()using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)versioninfo()Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: