Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: loadpkl failed when pklz file contained versioning info #3017

Merged
merged 6 commits into from
Sep 6, 2019

Conversation

stilley2
Copy link
Contributor

@stilley2 stilley2 commented Sep 5, 2019

Summary

loadpkl called pickle.load at file position 0, which would fail if the first line of the file was json versioning info (as when using savepkl with versioning=True). This was fixed by only seeking to position 0 if no versioning information is found.

Acknowledgment

  • (Mandatory) I acknowledge that this contribution will be available under the Apache 2 license.

loadpkl called pickle.load at file position 0, which would fail
if the first line of the file was json versioning info (as when
using savepkl with versioning=True). This was fixed by only
seeking to position 0 if no versioning information is found.
assert outobj == testobj


def test_pickle_versioning(tmp_path):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about parameterizing this into a single test?

@pytest.mark.parametrize("save_ver", (True, False))
@pytest.mark.parametrize("load_ver", (True, False))
def test_pickle(tmp_path, save_ver, load_ver):
    testobj = 'iamateststr'
    pickle_fname = str(tmp_path / 'testpickle.pklz')
    utils.filemanip.savepkl(pickle_fname, testobj, versioning=save_ver)
    outobj = utils.filemanip.loadpkl(pickle_fname, versioning=load_ver)
    assert outobj == testobj

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.

Copy link
Member

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me. I left a comment about the test. I think from a quick read that all four combinations of read/write ought to work.

@effigies
Copy link
Member

effigies commented Sep 5, 2019

@oesteban I think this issue was introduced in the previous version (#2985). I would recommend this to go into the 1.2.2 release, if it's ready in time.

@stilley2
Copy link
Contributor Author

stilley2 commented Sep 5, 2019

I actually just removed the versioning argument from loadpkl. All it was doing was altering the error reporting, and it seemed unnecessary (although maybe it was there for a reason?).

@effigies
Copy link
Member

effigies commented Sep 5, 2019

Looks like it was added in #2626, and what it seems to be doing is guarding against accessing the undefined pkl_metadata in the next couple lines. I would leave it for now, just because there doesn't seem to be a big gain in removing these lines, and there will be a semantic change.

We can open an issue to review that, though, if you think it will help with clarity.

@stilley2
Copy link
Contributor Author

stilley2 commented Sep 5, 2019

Ok I reverted it. However, I don't think it guards against that any more. I'll open up an issue.

nipype/tests/test_utils.py Outdated Show resolved Hide resolved
Copy link
Member

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Locally, the tests fail without the fix, and pass with it, so this is a good regression test. A little surprised that nothing broke before...

👍 to merge when CI passes.

@effigies effigies added this to the 1.2.2 milestone Sep 5, 2019
@effigies effigies merged commit 64c7b24 into nipy:master Sep 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants