-
Notifications
You must be signed in to change notification settings - Fork 530
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FIX: loadpkl failed when pklz file contained versioning info #3017
Conversation
loadpkl called pickle.load at file position 0, which would fail if the first line of the file was json versioning info (as when using savepkl with versioning=True). This was fixed by only seeking to position 0 if no versioning information is found.
nipype/tests/test_utils.py
Outdated
assert outobj == testobj | ||
|
||
|
||
def test_pickle_versioning(tmp_path): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about parameterizing this into a single test?
@pytest.mark.parametrize("save_ver", (True, False))
@pytest.mark.parametrize("load_ver", (True, False))
def test_pickle(tmp_path, save_ver, load_ver):
testobj = 'iamateststr'
pickle_fname = str(tmp_path / 'testpickle.pklz')
utils.filemanip.savepkl(pickle_fname, testobj, versioning=save_ver)
outobj = utils.filemanip.loadpkl(pickle_fname, versioning=load_ver)
assert outobj == testobj
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks reasonable to me. I left a comment about the test. I think from a quick read that all four combinations of read/write ought to work.
I actually just removed the versioning argument from loadpkl. All it was doing was altering the error reporting, and it seemed unnecessary (although maybe it was there for a reason?). |
Looks like it was added in #2626, and what it seems to be doing is guarding against accessing the undefined We can open an issue to review that, though, if you think it will help with clarity. |
This reverts commit 56ab01e.
Ok I reverted it. However, I don't think it guards against that any more. I'll open up an issue. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Locally, the tests fail without the fix, and pass with it, so this is a good regression test. A little surprised that nothing broke before...
👍 to merge when CI passes.
Summary
loadpkl called pickle.load at file position 0, which would fail if the first line of the file was json versioning info (as when using savepkl with versioning=True). This was fixed by only seeking to position 0 if no versioning information is found.
Acknowledgment