Generated code memory leaks

[tmptrash]

Hello guys!

I found a generated code related memory leak issue. I wrote a post about it on julia-lang users group, but it looks like everyone have no time for that. I emailed to @JeffBezanson about this issue and he knows about it. I have no possibility to fix this by myself, because i have no such experience in C. So, i decided to create this issue to track it somehow.

So, here is a problem:

function leak()
    for i=1:100000
        t = Task(eval(:(function() produce() end)))
        consume(t)
        try
          Base.throwto(t, null)
        end
    end
    gc()
end

Every call of leak() eats ~30mb of memory on my PC. This problem exists on both Linux and Windows platforms.

julia> versioninfo()
Julia Version 0.4.2
Commit bb73f34 (2015-12-06 21:47 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4700HQ CPU @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (NO_LAPACK NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: liblapack.so.3
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

[ViralBShah]

Thanks for filing - so that this can be tracked.

[yuyichao]

Is this just because we don't free jit code?

[vtjnash]

We don't free the results of eval because the memory cost is lower than the computational cost (in the Julia cost model where functions -- even closures -- are statically defined). Clearly, the eval is trivially unnecessary here, so this won't be fixed unless someone finds a real use case.

[tmptrash]

• I have use case in my app. This example (above) - is simplified version of it. In two words, i have to generate a lot of code from the string, which is modified all the time. So in my case eval(parse("...")) are called all the time. This is something like self modified application. It's used in evolution biology research.
• I also have a case when i modify AST and call eval() after that many times. I was wondering to have this feature (AST modification) in Julia, by the way :) In this case memory leak also appears. So, after 20min of working, my app eats 6Gb of memory :)

Is it possible to solve this?

[JeffBezanson]

Reopened. I think this is a real issue that I'd like to fix eventually.

[tmptrash]

This is great news :) For us this is a real stopper. Thanks Jeff.

[afbarnard]

Is it generated code that is causing the maxrss to monotonically increase with every set of tests in the entire suite? If so, this issue affects my ability to run all the tests on my laptop which has 4GB. Right now, I can only run the tests single-threaded (make testall1) because running the tests multi-threaded (make testall) runs out of memory 2 times. (2 test workers are terminated.) Plus, generated code taking up so much memory is not in concert with my intuition about running tests independently. Perhaps I should open an issue? (Note I have been using the release-0.4 branch, not master. See #13719 for backstory.)

[JeffBezanson]

Yes, that's probably a significant part of the problem. Also ref #14626

[JeffBezanson]

We could also perhaps restart workers more frequently, e.g. every few test files, to use less persistent memory.

[tkelman]

There's an environment variable you can set to do just that. Have to look at runtests to check exactly how it's spelled.

[yuyichao]

We already have JULIA_TEST_MAXRSS_MB

[robertfeldt]

There are also use cases from genetic programming and other code generation/synthesis situations when one wants to compile and run a large number of programs in order to then select some subset of them.

See discussion here:
https://discourse.julialang.org/t/is-mem-of-compiled-evaled-functions-garbage-collected/2231

Also see the (closed) issue here:
#20755 (comment)

[Nosferican]

Status of this?

[freddycct]

My issue #37560 was closed. So I am posting my MWE here. I used Flux/Zygote with pmap.

using Distributed
addprocs(4)

@everywhere mutable struct A
    a::Float32
end

@everywhere function genprog(n, p::A)
    map(1:n) do i
        y = rand()
        mdname = gensym()
        expr = :(module $mdname
            f(x) = 2*x + $y + $p.a
            end
        )
        m = eval(expr)
        Base.invokelatest(m.f, p.a)
    end
end

function main()
    i = 0
    x = A(rand())
    while true
        println("epoch $(i)")
        @everywhere GC.gc()
        
        tasks = rand(1:100, 100)
        _, timeTaken, _, _, _ = @timed let x=x
            pmap(tasks) do n
                genprog(n, x)
            end
        end
        @show timeTaken
        x.a = rand()
        i += 1
    end
end

main()

[aeisman]

In another use case, I have been having the same problem with a combination of Distributed and RCall.jl. It appears that repeated uses of RCall are causing a similar memory leak in my case up to 1TB of combined RAM and VRAM usage.

[schlichtanders]

Just closed my fresh issue as a duplicate of this one. Repeatedly creating closures.

• I run into the problem while using Pluto:
• in certain cases Pluto falls back to mere eval of the code and when done repeatedly, the memory stacks up

Here my minimal reproducible example

for _ in 1:50
    @eval function myfunc() end
    GC.gc(true); GC.gc(false)
    Core.println(Base.gc_live_bytes() / 2^20)
end

[vchuravy]

GC for code is going to be rather challenging. Besides the mechanics of being able to free the memory one must be able to prove that the code has become unreachable.

While Julia's world-age mechanism might be able to be reused for that, we also have an intrinsic invokeinworld that potentially makes everything reachable.

[schlichtanders]

Okay, I see, that is why even the function with the same name cannot "overwrite" itself, because of different world ages...
at least not in an easy automated way.

Is there a manual way to completely cleanup such functions? (let's assume we know their name)

[vchuravy]

Not currently, it would require re-engineering parts of the JIT compiler to separate functions into individual JITDylib (either per world, or per function compilation) and then expose a mechanism to evict specific JITDylibs.

The only current way is to restart your Julia session ;)

[vinhpb]

Hi,
I have an evolutionary algorithm searching for solutions in form of functions, in which I use the package RuntimeGeneratedFunctions.jl to generate the functions. I see that as my code runs the memory use increases gradually over time until my system crashes. Does that also sound like memory leak?
I thought the package is built in a way that allows GC to collect functions once they are out of scope (as mentioned here SciML/RuntimeGeneratedFunctions.jl#7), so I am confused if memory leak actually happens. I would really appreciate it if someone can explain it to me, since I am completely new to this area of code generating. :)
This is how my code looks like:

function main(...)
 ...
 while iterate > 0
   Threads.@threads  for i in n_threads
      expr = ...  % Calling the function to generate an expr
      fitness = eval_solution(expr, data, eval_genfunc)
      ...
   end
   ...
   iterate -= 1
 end
 ...
end

function eval_solution(expr, data, eval_genfunc) 
      f = expr
      f1 = @RuntimeGeneratedFunction(f)
      fitness = evaluate_genfunc(f1, data)
      return fitness
 end

function eval_genfunc(f1, data) 
      parameters = f1(data)
      score = g(parameters)  % g performs a simulation with given parameters and extracts some information from there as the score
      return score
end

[chriselrod]

I don't believe JITed functions can be freed (aside from exiting the process).
One trick I've used to deal with memory leaks in the past is to using Distributed, and do the work in another process. You can manually rmproc and replace with a new addprocs periodically. Cumbersome, but better than crashing your system.

[robertfeldt]

Yes, I had similar goals but at least concluded back then (a few years ago) that generated functions are not GC'ed so cannot use for genetic programming type of algorithms easily.

[vchuravy]

It should be possible to eventually GC native code, but doing so is hard, and the use-case is limited.

For genetic programming it might be better to use https://github.com/JuliaDebug/JuliaInterpreter.jl

[chriselrod]

For genetic programming it might be better to use https://github.com/JuliaDebug/JuliaInterpreter.jl

It might also be easy to write a custom "interpreter" if the functions have a limited enough set of behaviors. E.g., a vector for storing temporaries, and a while loop with if/else to branch quickly on an enum of a limited possible number of functions you may call (assuming it is limited), with some indexes for which temporary to use as arguments, storing the temporary in the vector.
How much further/less you optimize it depends on just how limited the set of behaviors your function may have.

[vinhpb]

Thanks for all the tips, guys! I really appreciate it. I will consider which one is the most suitable for my application and try it out.
@vchuravy: Just for curiosity, can you tell me a bit what would it take to GC native code?