-
Notifications
You must be signed in to change notification settings - Fork 96
fix: Memory leak #282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
fix: Memory leak #282
Changes from 31 commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
e2566e8
test: Add reproducing test
eddiebergman bbc4161
fix: Make sampling neighbors form uniform Int stable
eddiebergman c31e3d6
fix: Memory leak with UniformIntegerHyperparameter
eddiebergman 5aa9aab
fix: Memory issues with Normal and Beta dists
eddiebergman 7afcaea
chore: Update flake8
eddiebergman 91d590b
fix: flake8 version compatible with Python 3.7
eddiebergman 57b4182
fix: Name generators properly
eddiebergman 9f0b18d
fix: Test numbers
eddiebergman 3a8286d
doc: typo fixes
eddiebergman c36d292
perf: Generate all possible neighbors at once
eddiebergman 367e508
test: Add test for center_range and arange_chunked
eddiebergman c5b3281
perf: Call transform on np vector from rvs
eddiebergman e1fac33
perf: Use numpy `.astype(int)` instead of `int`
eddiebergman 564c0a4
doc: Document how to get flamegraphs for optimizing
eddiebergman c5d2090
fix: Allow for negatives in arange_chunked again
eddiebergman 5f51df1
fix: Change build back to raw Extensions
eddiebergman 29dd3db
build: Properly set compiler_directives
eddiebergman 555c5da
ci: Update makefile with helpful commands
eddiebergman d833df9
ci: Fix docs to install build
eddiebergman 0a7aafb
perf: cython optimizations
eddiebergman e24dd25
perf: Fix possible memory leak with UniformIntegerHyperparam
eddiebergman d52f28c
fix: Duplicates as `list` instead of set
eddiebergman 94b4d9c
fix: Convert to `long long` vector
eddiebergman e6d3222
perf: Revert clip to truncnorm
eddiebergman 395ea60
test: Test values not match implementation
eddiebergman 1f69f51
Intermediate commit
mfeurer ba2a174
INtermediate commit 2
mfeurer dd5bd75
Update neighborhood generation for UniformIntegerHyperparameter
mfeurer c4e7f42
Update tests
mfeurer 2a647a3
Make the benchmark sampling script more robust
mfeurer e0bef0e
Revert small change in util function
mfeurer 1702342
Improve readability
mfeurer File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -36,6 +36,7 @@ jobs: | |
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| pip install build | ||
| pip install ".[dev]" | ||
|
|
||
| - name: Make docs | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,3 @@ | ||
| # cython: language_level=3 | ||
|
|
||
| from collections import deque | ||
|
|
||
| import numpy as np | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,3 @@ | ||
| # cython: language_level=3 | ||
|
|
||
| import numpy as np | ||
| cimport numpy as np | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,3 @@ | ||
| # cython: language_level=3 | ||
|
|
||
| import numpy as np | ||
| cimport numpy as np | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| from typing import Iterator | ||
|
|
||
| from more_itertools import roundrobin | ||
| import numpy as np | ||
|
|
||
|
|
||
| def center_range( | ||
| center: int, | ||
| low: int, | ||
| high: int, | ||
| step: int = 1, | ||
| ) -> Iterator[int]: | ||
| """Get a range centered around a value. | ||
|
|
||
| >>> list(center_range(5, 0, 10)) | ||
| [4, 6, 3, 7, 2, 8, 1, 9, 0, 10] | ||
|
|
||
| Parameters | ||
| ---------- | ||
| center: int | ||
| The center of the range | ||
|
|
||
| low: int | ||
| The low end of the range | ||
|
|
||
| high: int | ||
| The high end of the range | ||
|
|
||
| step: int = 1 | ||
| The step size | ||
|
|
||
| Returns | ||
| ------- | ||
| Iterator[int] | ||
| """ | ||
| assert low <= center <= high | ||
| above_center = range(center + step, high + 1, step) | ||
| below_center = range(center - step, low - 1, -step) | ||
| yield from roundrobin(below_center, above_center) | ||
|
|
||
|
|
||
| def arange_chunked( | ||
| start: int, | ||
| stop: int, | ||
| step: int = 1, | ||
| *, | ||
| chunk_size: int, | ||
| ) -> Iterator[np.ndarray]: | ||
| """Get np.arange in a chunked fashion. | ||
|
|
||
| >>> list(arange_chunked(0, 10, 3)) | ||
| [array([0, 1, 2]), array([3, 4, 5]), array([6, 7, 8]), array([9])] | ||
|
|
||
| Parameters | ||
| ---------- | ||
| start: int | ||
| The start of the range | ||
|
|
||
| stop: int | ||
| The stop of the range | ||
|
|
||
| chunk_size: int | ||
| The size of the chunks | ||
|
|
||
| step: int = 1 | ||
| The step size | ||
|
|
||
| Returns | ||
| ------- | ||
| Iterator[np.ndarray] | ||
| """ | ||
| assert step > 0 | ||
mfeurer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| assert chunk_size > 0 | ||
| assert start < stop | ||
| n_items = int(np.ceil((stop - start) / step)) | ||
| n_chunks = int(np.ceil(n_items / chunk_size)) | ||
|
|
||
| for chunk in range(0, n_chunks): | ||
| chunk_start = start + (chunk * chunk_size) | ||
| chunk_stop = min(chunk_start + chunk_size, stop) | ||
| yield np.arange(chunk_start, chunk_stop, step) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,3 @@ | ||
| # cython: language_level=3 | ||
| from typing import Union | ||
| import numpy as np | ||
| cimport numpy as np | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.