Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large memory usage proportional to time span #285

Open
ggggggggg opened this issue Mar 6, 2024 · 0 comments
Open

Large memory usage proportional to time span #285

ggggggggg opened this issue Mar 6, 2024 · 0 comments
Labels

Comments

@ggggggggg
Copy link

ggggggggg commented Mar 6, 2024

Describe the bug 🐞
Unexpected large memory usage proportional to tspan[2]-tspan[1] when solving a Delay Differential Equation with an oscillatory driving function. This might not be a bug, and may actually be a feature request to specify a maximum time lag I promise to never look beyond to limit the memory used by the history function.

Expected behavior
I expect memory usage to scale close to the amount of data to be saved. In this case for each time point it saves two state variables and time, at 8 bytes each, I expect ~24 bytes per sample. Instead sol.interp.f stores ~14000 bytes/sample. The amount required by sol.interp.f scales proportional to tspan[2]-tspan[1]. Clearly some amount of memory is required to enable the history function, but this seems excessive. In this MRE the single constant lag is less than 1 part in 10,000 of tspan[2]-tspan[1], so I expect that the memory usage required for the history function should not need to grow when tspan[2]-tspan[1] is increased.

The MRE includes a delay diff eq setup, and some code to calculate the bytes/sample used by various fields of the solution object sol. I want to solve for much longer values of tspan[2]-tspan[1] and the memory usage is the primary limiting factor. The dominant contribution by at least a factor of 100 is the field sol.interp.f. I believe this field is part of the implementation of the history function for the DDE.

Minimal Reproducible Example 👇

using DifferentialEquations

function qw_dde(du,u,h,params,t)
    a,b,c,d,rt,τ = params

    hId,hIu = h(params,t-τ)              
    hdId,hdIu = h(params,t-τ,Val{1})     

    du[1] = ( 4.28e-7*(-2.51e10)*sin(2.51e10*t) + c*(hIu - u[1]) + b*hdIu ) * a
    du[2] = hdId + (rt-1.0)*(u[2] + hId)/(d)
end

const tspan = (0.0, 1e-6)
const τ = 5.6e-11
const params = (0.66, -0.5, 4.4e11, 4.0e-12, 7.8e-6, τ)
h(params,t) = zeros(2);
h(params,t,::Type{Val{1}}) = zeros(2);
u0 = [0.0,0.0];
alg = MethodOfSteps(Tsit5());
saveat = LinRange(tspan[1], tspan[2], 5000)
prob = DDEProblem(qw_dde,u0,h,tspan,params; saveat=saveat, constant_lags=[τ]);


# Run simulation:
println("solve start")
@time sol = solve(prob,alg,reltol=1e-3,abstol=1e-9,maxiters=1e9);
println("solve done")


# investigate space usage
function pretty_print2(d::Dict, pre=1)
    # https://stackoverflow.com/questions/48195775/how-to-pretty-print-nested-dicts-in-julia
    todo = Vector{Tuple}()
    for (k,v) in d
        if typeof(v) <: Dict
            push!(todo, (k,v))
        else
            println(join(fill(" ", pre)) * "$(repr(k)) => $(repr(v))")
        end
    end

    for (k,d) in todo
        s = "$(repr(k)) => "
        println(join(fill(" ", pre)) * s)
        pretty_print2(d, pre+1+length(s))
    end
    nothing
end
# define a version of summarysize that doesn't choke on Vector{Vector{Vector{Float64}}}
# and returns a dict so we can see size by property
vec_safe_summary_size(x::Vector) = sum(vec_safe_summary_size(z) for z in x)
vec_safe_summary_size(x) = Base.summarysize(x)
# vec_safe_summary_size(sol::DelayDiffEq.HistoryODEIntegrator) = Dict(prop => vec_safe_summary_size(getproperty(sol, prop)) for prop in propertynames(sol))
# vec_safe_summary_size(sol::DelayDiffEq.HistoryFunction) = Dict(prop => vec_safe_summary_size(getproperty(sol, prop)) for prop in propertynames(sol))
# vec_safe_summary_size(sol::DelayDiffEq.ODEFunctionWrapper) = Dict(prop => vec_safe_summary_size(getproperty(sol, prop)) for prop in propertynames(sol))
vec_safe_summary_size(sol::OrdinaryDiffEq.InterpolationData) = Dict(prop => vec_safe_summary_size(getproperty(sol, prop)) for prop in propertynames(sol))
vec_safe_summary_size(sol::ODESolution) = Dict(prop => vec_safe_summary_size(getproperty(sol, prop)) for prop in propertynames(sol))
v = vec_safe_summary_size(sol)
calc_per_point(x, n) = x/n
calc_per_point(d::Dict, n) = Dict(key=>calc_per_point(val, n) for (key, val) in pairs(d))
println("bytes/point by property of sol")
pretty_print2(calc_per_point(v, length(sol)))
@show length(sol);
@show tspan;




Error & Stacktrace ⚠️

Environment (please complete the following information):

  • Output of using Pkg; Pkg.status()
julia> using Pkg; Pkg.status()
Status `C:\Users\oneilg\AppData\Local\Temp\jl_tkEYFY\Project.toml`
  [0c46a032] DifferentialEquations v7.13.0
  • Output of using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)
julia> using Pkg; Pkg.status(; mode = PKGMODE_MANIFEST)
Status `C:\Users\oneilg\AppData\Local\Temp\jl_tkEYFY\Manifest.toml`
  [47edcb42] ADTypes v0.2.6
  [79e6a3ab] Adapt v4.0.2
⌅ [ec485272] ArnoldiMethod v0.2.0
  [4fba245c] ArrayInterface v7.8.0
  [4c555306] ArrayLayouts v1.6.1
  [aae01518] BandedMatrices v1.5.0
  [6e4b80f9] BenchmarkTools v1.5.0
  [62783981] BitTwiddlingConvenienceFunctions v0.1.5
  [764a87c0] BoundaryValueDiffEq v5.6.3
  [fa961155] CEnum v0.5.0
  [2a0fbf3d] CPUSummary v0.2.4
  [49dc2e85] Calculus v0.5.1
  [d360d2e6] ChainRulesCore v1.23.0
  [fb6a15b2] CloseOpenIntervals v0.1.12
  [523fee87] CodecBzip2 v0.8.2
  [944b1d66] CodecZlib v0.7.4
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v4.14.0
  [2569d6c7] ConcreteStructs v0.2.3
  [187b0558] ConstructionBase v1.5.4
  [adafc99b] CpuId v0.3.1
  [9a962f9c] DataAPI v1.16.0
  [864edb3b] DataStructures v0.18.18
  [e2d170a0] DataValueInterfaces v1.0.0
  [bcd4f6db] DelayDiffEq v5.47.1
  [2b5f629d] DiffEqBase v6.148.0
  [459566f4] DiffEqCallbacks v3.3.0
  [77a26b50] DiffEqNoiseProcess v5.21.0
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [0c46a032] DifferentialEquations v7.13.0
  [b4f34e82] Distances v0.10.11
  [31c24e10] Distributions v0.25.107
  [ffbed154] DocStringExtensions v0.9.3
  [fa6b7ba4] DualNumbers v0.6.8
  [4e289a0a] EnumX v1.0.4
⌃ [f151be2c] EnzymeCore v0.6.5
  [d4d017d3] ExponentialUtilities v1.26.1
  [e2ba6199] ExprTools v0.1.10
  [9d29842c] FastAlmostBandedMatrices v0.1.0
  [7034ab61] FastBroadcast v0.2.8
  [9aa1b823] FastClosures v0.3.2
  [29a986be] FastLapackInterface v2.0.2
  [1a297f60] FillArrays v1.9.3
  [6a86dc24] FiniteDiff v2.22.0
  [f6369f11] ForwardDiff v0.10.36
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [d9f16b24] Functors v0.4.7
  [46192b85] GPUArraysCore v0.1.6
  [c145ed77] GenericSchur v0.5.3
  [86223c79] Graphs v1.9.0
  [3e5b6fbb] HostCPUFeatures v0.1.16
  [34004b35] HypergeometricFunctions v0.3.23
  [615f187c] IfElse v0.1.1
  [d25df0c9] Inflate v0.1.4
  [92d709cd] IrrationalConstants v0.2.2
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.5.0
  [682c06a0] JSON v0.21.4
  [ccbc3e58] JumpProcesses v9.10.1
  [ef3ab10e] KLU v0.6.0
  [ba0b0d4f] Krylov v0.9.5
  [10f19ff3] LayoutPointers v0.1.15
  [5078a376] LazyArrays v1.8.3
  [2d8b4e74] LevyArea v1.0.0
  [d3d80556] LineSearches v7.2.0
  [7ed4a6bd] LinearSolve v2.27.0
  [2ab3a3ac] LogExpFunctions v0.3.27
  [bdcacae8] LoopVectorization v0.12.166
  [1914dd2f] MacroTools v0.5.13
  [d125e4d3] ManualMemory v0.1.8
  [b8f27783] MathOptInterface v1.27.0
  [a3b82374] MatrixFactorizations v2.1.0
  [bb5d69b7] MaybeInplace v0.1.1
  [e1d29d7a] Missings v1.1.0
  [46d2c3a1] MuladdMacro v0.2.4
  [d8a4904e] MutableArithmetics v1.4.1
  [d41bc354] NLSolversBase v7.8.3
  [2774e3e8] NLsolve v4.5.1
  [77ba4419] NaNMath v1.0.2
  [8913a72c] NonlinearSolve v3.8.0
  [6fe1bfb0] OffsetArrays v1.13.0
  [429524aa] Optim v1.9.2
  [bac558e1] OrderedCollections v1.6.3
  [1dea7af3] OrdinaryDiffEq v6.74.0
  [90014a1f] PDMats v0.11.31
  [65ce6f38] PackageExtensionCompat v1.0.2
  [d96e819e] Parameters v0.12.3
  [69de0a69] Parsers v2.8.1
  [e409e4f3] PoissonRandom v0.4.4
  [f517fe37] Polyester v0.7.9
  [1d0040c9] PolyesterWeave v0.2.1
  [85a6dd25] PositiveFactorizations v0.2.4
  [d236fae5] PreallocationTools v0.4.20
  [aea7be01] PrecompileTools v1.2.0
  [21216c6a] Preferences v1.4.3
  [1fd47b50] QuadGK v2.9.4
  [74087812] Random123 v1.7.0
  [e6cf234a] RandomNumbers v1.5.3
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.10.1
  [f2c3362d] RecursiveFactorization v0.2.21
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [ae5879a3] ResettableStacks v1.1.1
  [79098fc4] Rmath v0.7.1
  [7e49a35a] RuntimeGeneratedFunctions v0.5.12
  [94e857df] SIMDTypes v0.1.0
  [476501e8] SLEEFPirates v0.6.42
  [0bca4576] SciMLBase v2.29.0
  [c0aeaf25] SciMLOperators v0.3.8
  [efcf1570] Setfield v1.1.1
  [727e6d20] SimpleNonlinearSolve v1.5.0
  [699a6c99] SimpleTraits v0.9.4
  [ce78b400] SimpleUnPack v1.1.0
  [a2af1166] SortingAlgorithms v1.2.1
  [47a9eef4] SparseDiffTools v2.17.0
  [e56a9233] Sparspak v0.3.9
  [276daf66] SpecialFunctions v2.3.1
  [aedffcd0] Static v0.8.10
  [0d7ed370] StaticArrayInterface v1.5.0
  [90137ffa] StaticArrays v1.9.3
  [1e83bf80] StaticArraysCore v1.4.2
  [82ae8749] StatsAPI v1.7.0
  [2913bbd2] StatsBase v0.34.2
  [4c63d2b9] StatsFuns v1.3.1
  [9672c7b4] SteadyStateDiffEq v2.1.0
  [789caeaf] StochasticDiffEq v6.65.1
  [7792a7ef] StrideArraysCore v0.5.2
  [c3572dad] Sundials v4.24.0
  [2efcf032] SymbolicIndexingInterface v0.3.8
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.11.1
  [8290d209] ThreadingUtilities v0.5.2
  [a759f4b9] TimerOutputs v0.5.23
  [3bb67fe8] TranscodingStreams v0.10.4
  [d5829a12] TriangularSolve v0.1.20
  [410a4b4d] Tricks v0.1.8
  [781d530d] TruncatedStacktraces v1.4.0
  [3a884ed6] UnPack v1.0.2
  [3d5dd08c] VectorizationBase v0.21.65
  [19fa3120] VertexSafeGraphs v0.2.0
  [6e34b625] Bzip2_jll v1.0.8+1
  [1d5cc7b8] IntelOpenMP_jll v2024.0.2+0
  [856f044c] MKL_jll v2024.0.0+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [f50d1b31] Rmath_jll v0.4.0+0
⌅ [fb77eaff] Sundials_jll v5.2.2+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [4af54fe1] LazyArtifacts
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.10.0
  [de0858da] Printf
  [9abbd945] Profile
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays v1.10.0
  [10745b16] Statistics v1.10.0
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.1.0+0
  [deac9b47] LibCURL_jll v8.4.0+0
  [e37daf67] LibGit2_jll v1.6.4+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.2+1
  [14a3606d] MozillaCACerts_jll v2023.1.10
  [4536629a] OpenBLAS_jll v0.3.23+4
  [05823500] OpenLibm_jll v0.8.1+2
  [bea87d4a] SuiteSparse_jll v7.2.1+1
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.8.0+1
  [8e850ede] nghttp2_jll v1.52.0+1
  [3f19e933] p7zip_jll v17.4.0+2
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`
  • Output of versioninfo()
julia> versioninfo()
Julia Version 1.10.2
Commit bd47eca2c8 (2024-03-01 10:14 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 8 × Intel(R) Xeon(R) W-2123 CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake-avx512)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)

Additional context

Add any other context about the problem here.

@ggggggggg ggggggggg added the bug label Mar 6, 2024
@ggggggggg ggggggggg changed the title Large memory usage proportional in to time span Large memory usage proportional to time span Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant