-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Read numpy nans/datetimes #741
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timkpaine
reviewed
Sep 24, 2019
timkpaine
requested changes
Sep 24, 2019
timkpaine
changed the title
Correctly parse dataframes, streamline datetime processing
Read numpy nans
Sep 24, 2019
Codecov Report
@@ Coverage Diff @@
## master #741 +/- ##
==========================================
+ Coverage 92.3% 93.36% +1.06%
==========================================
Files 61 61
Lines 2703 2970 +267
==========================================
+ Hits 2495 2773 +278
+ Misses 208 197 -11
Continue to review full report at Codecov.
|
…das.df updates, move all date parsing to python
…s, i.e. int-float and float-int
timkpaine
approved these changes
Sep 25, 2019
Thanks for the PR! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR improves the Python API's ability to parse Pandas DataFrames, ensuring compatibility with
numpy.nan
andnumpy.datetime64
.numpy.nan
will now have their types inferred asfloat
, asnumpy.nan
is a float type. If the column contains a value (i.e.[nan, nan, 1]
) later on, and the value is not offloat
orinteger
type, then the column will be promoted to a string, allnumpy.nan
s will be converted toNone
, and subsequent writes will be converted to string.numpy.nan
serializes toNone
when exported from Perspective.numpy.datetime64
correctly returns Unix timestamps for use within the C++ engine.