Skip to content
This repository has been archived by the owner on Jun 1, 2023. It is now read-only.

add Internals::gc() #336

Open
rurban opened this issue Oct 22, 2017 · 7 comments
Open

add Internals::gc() #336

rurban opened this issue Oct 22, 2017 · 7 comments

Comments

@rurban
Copy link
Member

rurban commented Oct 22, 2017

Scan the arenas and free empty ones.

The heads (PL_arenaroot), the bodies (PL_body_arenas and the PL_body_roots[type] free-list),
The opslab's (CvSTART of all CVs) and ops are correctly freed via cv_undef, see -DS.
Check on del_SV if an arena is free, and free its memory then.
Add a malloc_trim(0) call to actually force free() to return memory pages to return to the OS, so that the rss is improved. glibc only.

Furthermore check the implementation of the free-list (body_roots), which seems to support only one (just a ptr, not a linked list as the adesc->arena or HE chain), and check for a free arena on del_SV.
Similarily the obslab impl. also has an OP overhead of 2 ptrs (next and slab owner), whilst it could be easier implemented as sized arenas as with the body roots.
The SV arenas should really be simplified to allocate into one sized arena slot, (no seperation of head and body) but this would make upgrades with references harder, and we don't have small forwards.

For now see feature/gh336-gc, destruction became unstable with it.

@rurban rurban self-assigned this Oct 22, 2017
@EvanCarroll
Copy link

See you removed the label, I was watching closely. Is Perl to brittle to get this change in? Got me looking at cperl though. Thought you were going to solve the world's problems single handedly. Be interested in hearing a report-back on the StackOverflow on the progress and the difficulties.

@rurban
Copy link
Member Author

rurban commented Oct 31, 2017

I'll write a blog post and update the SO answer. Still trying, but no success with RSS or VM sizeonly with Massif. Even if aggressively freeing 4K pages

@EvanCarroll
Copy link

If anyone can do it and get bragging rights on Perl core, it's you. No idea what's going on, but you'll have a steep claim if you can use a function in POSIX.pm and not permanently grow perl by an order of megabytes RSS. =) That'd be a super tangible one for cperl.

@rurban
Copy link
Member Author

rurban commented Nov 1, 2017

The POSIX import problem is known for ages. Leon works on POSIX2 I guess, which doesn't import every existing function. Nobody should use POSIX; but only import the needed functions.
Mark Overmeer worked on POSIX::1003, and I worked on C11.

Regarding automatically calling DynaLoader::dl_unload_file for deleted packages in Symbol::delete_package or Class::Unload::unload: @DynaLoader::dl_modules holds all loaded modules. Just check if it's in this array and unload it then. I'll add this logic to Symbol

@EvanCarroll
Copy link

EvanCarroll commented Nov 1, 2017

Yea, I found dl_unload_file it was in the original question. It's great, but the gains for using aren't really there. Nor with Symbol::delete_packages -- at least for this challenge. If these were effective at shrinking the RSS though it'd be pretty great. That's why this patch is exciting. Then what I would do is fork DynaLoader to create a so_name=>fh tracking object, so those things that used it could be unloaded, and wire it up with XSLoader and we could have short-lived transient XS stuff without the need for forking.

rurban added a commit that referenced this issue Nov 2, 2018
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Nov 25, 2018
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Mar 18, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Apr 1, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Apr 5, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Apr 5, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Apr 30, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jun 12, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jun 24, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jun 26, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jun 27, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jul 1, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jul 2, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jul 2, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jul 3, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Aug 25, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Dec 17, 2019
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jan 19, 2020
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
rurban added a commit that referenced this issue Jan 19, 2020
free empty SV arenas and empty body roots.
Note that is usually doesn't reclaim system memory, very often
it even increases memory usage.

opslab arenas should be deleted by cv_undef() already.

See GH #336
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants