-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
-d:useMalloc for the default GC #15394
Comments
That seems to be more work than moving to 1.4's |
Even with 1.4 and --gc:orc, there are situations where resources can leak: #13289. |
@pmetras wdym? with ORC no memory leaks should happen, otherwise it's a bug in ARC/ORC itself or in code which used manual memory allocation or with custom destructors (if they're not written correctly) |
@Yardanico , I agree that ORC solves memory leaks. I just want to say that there are other types of leaks, like listed by mratsim (sockets, file descriptors, futures, timers, BD cursors, OS locks, etc.) and that ORC does not help with these types of leaks. |
@pmetras but it can and that's one of the reasons why ARC/ORC was created in the first place. You can create a custom destructor for any types including sockets, file descriptors, futures, locks, etc |
@pmetras just for a simple example (compile with arc/orc for reliable results, or add GC_fullCollect() after main with refc) type
MyFile = object
f: File
proc `=destroy`(myf: var MyFile) =
echo "destroying"
myf.f.close() # close already does the isNil check for us
proc main =
let myf = MyFile(f: open("a.nim"))
echo myf.f.readAll()
main() I use a wrapper object MyFile here because you can't create destructors for pointers (and File is a "ptr CFile"). So as you can see it's not even hard |
@Yardanico, OK I understand how it can work: I'll have to encapsulate resource handles into custom types with a finalizer. I had an example working too but did not post it. Perhaps the |
The litmus test here is that not even the compiler works with orc/arc by default - Nimbus is quite a bit a larger project that is expected to run over long periods of time without restard, and that that has been tuned and optimized to the current GC - introducing a new GC, a new memory layout for core types like string and seq etc is a major risk given that it will bring new bugs, require new workarounds and require reoptimizing the memory profile from scratch - it's not something we can do frivolously. We're pretty much aware of how orc / arc works but it's not what we need in this particular case. |
At Status, we have been fighting against resource leaks for months (sockets, file descriptors, futures, timers and most importantly memory)
make witti
status-im/nimbus-eth2#1123We built several tools to help us track the issues
However we are still leaking. As a workaround we are currently advising people to restart every 6 hours or so but for production we need to remove all possible source of leaks so that user can run that application for months without restart.
The computational part of our application is relatively easy to debug for leaks but the async/IO/networking part as been leaking resources quite often via closure iterators/futures due to unattended cancellation/expiration.
This is incredibly hard to debug.
We would like to have
-d:useMalloc
available for the default GC, backported to the 1.2.x branch so that we can use conventional C tools like Valgrind to detect those memory leaks and also run memory leak detection in dedicated CI.Furthermore, we are currently investigating an issue with misreported memory accounting by Nim GC, which makes it even harder to debug with Nim standard tools (memory fragmentation?).
The text was updated successfully, but these errors were encountered: