Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support pd.DataFrame and pd.Series in return type hints #1577

Merged
merged 1 commit into from
Jun 16, 2020

Conversation

HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Jun 12, 2020

This PR proposes to support the type hints with pandas Series and pandas DataFrame directly.

import pandas as pd
import databricks.koalas as ks
def transform(pdf) -> pd.DataFrame[int]:
    # pdf is actually a pandas DataFrame.
    return pdf + 1

ks.range(10).apply_batch(transform)
     c0
0     1
1     2
2     3
3     4
4     5
5     6
6     7
7     8
8     9
9    10

Note that:

  • The type hints with Pandas only works with Python 3.7+.

  • The previous way with Koalas DataFrame type hint will still work but will be deprecated once we deprecate Python 3.5 and 3.6.

    import databricks.koalas as ks
    def transform(pdf) -> ks.DataFrame[int]:
        return pdf + 1
    
    ks.range(10).apply_batch(transform)

@HyukjinKwon HyukjinKwon force-pushed the support-pandas-typehints branch 4 times, most recently from a36fa8c to 7a70174 Compare June 12, 2020 13:33
@codecov-commenter
Copy link

codecov-commenter commented Jun 12, 2020

Codecov Report

Merging #1577 into master will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1577      +/-   ##
==========================================
+ Coverage   94.55%   94.57%   +0.01%     
==========================================
  Files          38       38              
  Lines        8766     8792      +26     
==========================================
+ Hits         8289     8315      +26     
  Misses        477      477              
Impacted Files Coverage Δ
databricks/koalas/__init__.py 95.16% <100.00%> (+1.04%) ⬆️
databricks/koalas/frame.py 96.82% <100.00%> (+<0.01%) ⬆️
databricks/koalas/groupby.py 90.53% <100.00%> (+0.01%) ⬆️
databricks/koalas/indexes.py 96.86% <100.00%> (ø)
databricks/koalas/indexing.py 92.25% <100.00%> (ø)
databricks/koalas/internal.py 97.56% <100.00%> (+<0.01%) ⬆️
databricks/koalas/namespace.py 86.09% <100.00%> (ø)
databricks/koalas/series.py 97.63% <100.00%> (+0.01%) ⬆️
databricks/koalas/typedef/string_typehints.py 100.00% <100.00%> (ø)
databricks/koalas/typedef/typehints.py 88.00% <100.00%> (+0.63%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a6cf5a...be9a840. Read the comment docs.

@HyukjinKwon HyukjinKwon force-pushed the support-pandas-typehints branch 3 times, most recently from fde1cf2 to 756c5c8 Compare June 12, 2020 14:41
@HyukjinKwon HyukjinKwon requested a review from ueshin June 12, 2020 14:52
@HyukjinKwon HyukjinKwon force-pushed the support-pandas-typehints branch 4 times, most recently from f0a329f to fb1a8c0 Compare June 12, 2020 15:20
@HyukjinKwon HyukjinKwon force-pushed the support-pandas-typehints branch 2 times, most recently from aab1c1b to 3ad4732 Compare June 14, 2020 06:30
@HyukjinKwon HyukjinKwon force-pushed the support-pandas-typehints branch from 3ad4732 to be9a840 Compare June 14, 2020 08:34
Copy link
Collaborator

@ueshin ueshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@ueshin
Copy link
Collaborator

ueshin commented Jun 16, 2020

Thanks! merging.

@ueshin ueshin merged commit ff3a68e into databricks:master Jun 16, 2020
@HyukjinKwon HyukjinKwon deleted the support-pandas-typehints branch September 11, 2020 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants