Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset leak issue with at least Tabular #2949

Closed
muellerzr opened this issue Nov 7, 2020 · 1 comment
Closed

Dataset leak issue with at least Tabular #2949

muellerzr opened this issue Nov 7, 2020 · 1 comment

Comments

@muellerzr
Copy link
Contributor

muellerzr commented Nov 7, 2020

For the last day @jph00 and I have been fighting references to datasets inside of tabular and around fastai in general. We fixed two, referenced here and here and this issue partially originated from this flag: #2945

Through the use of the pympler library (docs here ) I was able to narrow down the fact that we have two extraneous DataFrames still being exported via our load_learner. It's a bit more of a complex problem so here is a google colab notebook recreating what we've done:

https://gist.github.com/muellerzr/c126e75b0265f88c3baa1ea50a4dfe6b

@muellerzr muellerzr changed the title Memory leak issue with at least Tabular Dataset leak issue with at least Tabular Nov 7, 2020
@jph00
Copy link
Member

jph00 commented Nov 7, 2020

Should be fixed now.

@jph00 jph00 closed this as completed Nov 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants