[SPARK-21658][SQL][PYSPARK] Add default None for value in na.replace in PySpark

[chihhanyu]

What changes were proposed in this pull request?

JIRA issue: https://issues.apache.org/jira/browse/SPARK-21658

Add default None for value in na.replace since Dataframe.replace and DataframeNaFunctions.replace are alias.

The default values are the same now.

>>> df = sqlContext.createDataFrame([('Alice', 10, 80.0)])
>>> df.replace({"Alice": "a"}).first()
Row(_1=u'a', _2=10, _3=80.0)
>>> df.na.replace({"Alice": "a"}).first()
Row(_1=u'a', _2=10, _3=80.0)

How was this patch tested?

Existing tests.

cc @viirya

[HyukjinKwon]

Could we add the example in the doctest (under 1362L) so that this can be tested and shown in the documentation?

[HyukjinKwon]

ok to test

[SparkQA]

Test build #80469 has finished for PR 18895 at commit 8af1e15.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

@byakuinss Please add a doc test in DataFrame.replace. There is an example df4.na.replace('Alice', None).show(). We want to make sure it works with default value. Thanks.

[SparkQA]

Test build #80490 has finished for PR 18895 at commit d72c231.

• This patch fails Python style tests.
• This patch does not merge cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #80530 has finished for PR 18895 at commit 5d8d4b6.

• This patch fails Python style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

I think it is close to get merged. Would you address the comments @byakuinss ?

[HyukjinKwon]

looks trailing white spaces should be removed. Could we remove these?

[chihhanyu]

Thanks for your reminding! I'll remove them.

[HyukjinKwon]

I think it is okay to leave this line as was.

[chihhanyu]

Got it, I'll change them back.

[SparkQA]

Test build #80632 has finished for PR 18895 at commit abdef40.

• This patch fails Python style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

LGTM

[SparkQA]

Test build #80633 has finished for PR 18895 at commit d07d49a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

LGTM too.

[HyukjinKwon]

Merged to master.

[HyukjinKwon]

@byakuinss, BTW, do you mind if I ask your JIRA id? I want to assign this to you as you resolved this but I can't find the ID..

[chihhanyu]

@HyukjinKwon
Oh, do you mean my jira full name? It's Chin Han Yu.

[HyukjinKwon]

Hm.. weird. I can't search your account on JIRA ...

[HyukjinKwon]

Can you maybe leave any comment saying.. like .. "here is my JIRA account." in https://issues.apache.org/jira/browse/SPARK-21658 if you don't mind?

[chihhanyu]

Okay, I leave a comment in the issue page.

[viirya]

Thanks! @HyukjinKwon

[HyukjinKwon]

Guys, I am sorry but I think I have to revert this by few explicit objections, for now.
Maybe, I am now rushing to revert it but please don't mind because we are close to 2.3.0.

Please join in the discussion about the original implementation details in #16793.

Thanks.