Skip to content

Conversation

@devin-petersohn
Copy link
Contributor

Moving pandas register into default register.

@devin-petersohn devin-petersohn changed the title [ARROW-1971] Add pandas serialization to the default [ARROW-1971] [Python] Add pandas serialization to the default Jan 6, 2018
@devin-petersohn devin-petersohn changed the title [ARROW-1971] [Python] Add pandas serialization to the default ARROW-1971: [Python] Add pandas serialization to the default Jan 6, 2018
@devin-petersohn devin-petersohn force-pushed the jira/1971_pandas_serialization branch from 83425c9 to e0d4de6 Compare January 6, 2018 22:03
Registering pandas in default

Formatting
@devin-petersohn devin-petersohn force-pushed the jira/1971_pandas_serialization branch from e0d4de6 to 2ed3137 Compare January 6, 2018 22:10
@robertnishihara
Copy link
Contributor

@wesm Is there any reason to use _register_pandas_arrow_handlers over _register_custom_pandas_handlers? Why not always use _register_custom_pandas_handlers? Is there a difference in generality?

@wesm
Copy link
Member

wesm commented Jan 6, 2018

I'd be perfectly fine with the faster pandas serialization code path. I guess the one thing separating the pandas_serialization_context from the default one would be the handling of NumPy object arrays?

@robertnishihara
Copy link
Contributor

robertnishihara commented Jan 6, 2018

Is that related to the choice of _register_pandas_arrow_handlers versus _register_custom_pandas_handlers? Both of those can be used with both of the custom numpy object array serializers, right?

EDIT: I think I see what you mean, yes, if we get rid of _register_pandas_arrow_handlers, then the only difference would be the handling of numpy object arrays.

This seems like the obvious thing to do unless there is some drawback with _register_custom_pandas_handlers.

Copy link
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thanks @devin-petersohn!

@robertnishihara
Copy link
Contributor

Yes, thanks @devin-petersohn! This PR looks good to me.

@wesm, is speed the only difference between the two implementations? Just curious if there are edge cases.

@wesm wesm closed this in b49e8f3 Jan 10, 2018
@wesm
Copy link
Member

wesm commented Jan 11, 2018

I think the new version will have marginally better support for "weird pandas data". If you run into issues please open issues so we can investigate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants