ARROW-6556: [Python] Handle future removal of pandas SparseDataFrame #5377
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #5377 +/- ##
==========================================
- Coverage 69.35% 65.56% -3.8%
==========================================
Files 745 497 -248
Lines 87819 68976 -18843
Branches 1437 0 -1437
==========================================
- Hits 60905 45222 -15683
+ Misses 26552 23754 -2798
+ Partials 362 0 -362
Continue to review full report at Codecov.
|
|
Thanks Joris. I was just going to file a JIRA :) Does this change your opinion on removing SparseDataFrame in 1.0? The check in |
|
@TomAugspurger I don't think we should be making accommodations for users to stay on old versions of pyarrow at this point |
pprudhvi
pushed a commit
to pprudhvi/arrow
that referenced
this pull request
Sep 16, 2019
https://issues.apache.org/jira/browse/ARROW-6556 The plan in pandas is to remove SparseDataFrame/Series in pandas 1.0. By making sure our code works with that, we can ensure that a pandas release does not break the pyarrow release that is at that moment the latest stable release. (and this also makes it easier for me to develop on master branches of both together) This isn't merged yet in pandas, so we can also wait until that is done to merge this PR. I was just trying out some things in pandas and saw that the pyarrow feather tests were failing when we remove those classes. Closes apache#5377 from jorisvandenbossche/ARROW-6556-pandas-sparse and squashes the following commits: 705f0a7 <Joris Van den Bossche> ARROW-6556: Handle future removal of pandas SparseDataFrame Authored-by: Joris Van den Bossche <[email protected]> Signed-off-by: Wes McKinney <[email protected]>
wesm
pushed a commit
that referenced
this pull request
Sep 19, 2019
Follow-up on https://issues.apache.org/jira/browse/ARROW-6556 / #5377. We were a bit fast with merging the other PR, as pandas changed the removal slightly (it added back a dummy SparseDataFrame object which triggers a warning upon access, but this means that `hasattr(pd, 'SparseDataFrame)` or try/except on import is not a good check to know if the actual class still exists or not). Using pytests warnings filter I now ensure no such warning is raised in our test suite. Closes #5438 from jorisvandenbossche/ARROW-6556-fix-warning and squashes the following commits: 06a0738 <Joris Van den Bossche> ARROW-6556: Fix warning for pandas SparseDataFrame removal Authored-by: Joris Van den Bossche <[email protected]> Signed-off-by: Wes McKinney <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/ARROW-6556
The plan in pandas is to remove SparseDataFrame/Series in pandas 1.0. By making sure our code works with that, we can ensure that a pandas release does not break the pyarrow release that is at that moment the latest stable release.
(and this also makes it easier for me to develop on master branches of both together)
This isn't merged yet in pandas, so we can also wait until that is done to merge this PR.
I was just trying out some things in pandas and saw that the pyarrow feather tests were failing when we remove those classes.