Fix file-handle resource leaks #9309
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Open resources (ordinary file, HDF5, tar, urlopen response, etc.) were often not closed properly. When an exception is raised the resource might not be closed. This may eg. result in depleting the number of file handles for the process. Where possible use context managers (`with open(...) as f`) that close the resource even after catching an exception. Still, HDF5Matrix, CSVLogger and keras.preprocess.image.load_img() are difficult to fix without changing their API. In those places resource ownership is not properly defined.
fchollet
approved these changes
Feb 4, 2018
ahundt
added a commit
to ahundt/keras
that referenced
this pull request
Feb 16, 2018
* 'master' of github.com:fchollet/keras: (57 commits) Minor README edit Speed up Travis tests (keras-team#9386) fix typo (keras-team#9391) Fix style issue in docstring Prepare 2.1.4 release. Fix activity regularizer + model composition test Corrected copyright years (keras-team#9375) Change default interpolation from nearest to bilinear. (keras-team#8849) a capsule cnn on cifar-10 (keras-team#9193) Enable us to use sklearn to do cv for functional api (keras-team#9320) Add support for stateful metrics. (keras-team#9253) The type of list keys was float (keras-team#9324) Fix mnist sklearn wrapper example (keras-team#9317) keras-team#9287 Fix most of the file-handle resource leaks. (keras-team#9309) Pass current learning rate to schedule() in LearningRateScheduler (keras-team#8865) Simplify with from six.moves import input (keras-team#9216) fixed RemoteMonitor: Json to handle np.float32 and np.int32 types (keras-team#9261) Update tweet length from 140 to 280 in docs Add `depthconv_conv2d` tests (keras-team#9225) Remove `force` option in progbar ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #9287: Resource leak of file handles
Open resources (ordinary file, HDF5, tar, urlopen response, etc.) were often not closed properly. When an exception is raised the resource might not be closed. This may eg. result in depleting the number of file handles for the process.
Where possible use context managers (
with open(...) as f) that close the resource even after catching an exception.Still,
HDF5Matrix,CSVLoggerandkeras.preprocess.image.load_img()are difficult to fix without changing their API. In those places resource ownership is not properly defined. See comment for HDF5Matrix.PIL images returned by
load_img()should be closed by the user of that function. Possibly using it's context manager:However, I'm not 100% sure if this can be disturbed by operations such as
img.convert(),img.resize()done inload_img().