[SPARK-20566][SQL] ColumnVector should support appendFloats for array

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to add a missing appendFloats API for array into ColumnVector class. For double type, there is appendDoubles for array here.

How was this patch tested?

Pass the Jenkins with a newly added test case.

[dongjoon-hyun]

Hi, @nongli .
Could you review this PR?

[SparkQA]

Test build #76394 has finished for PR 17836 at commit a03f927.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[kiszk]

LGTM if tests pass
cc: @sameeragarwal @cloud-fan

[dongjoon-hyun]

Thank you for review, @kiszk

[SparkQA]

Test build #76404 has finished for PR 17836 at commit d979d0f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[hvanhovell]

cc @michal-databricks

[cloud-fan]

LGTM

[dongjoon-hyun]

Thank you for review, @hvanhovell and @cloud-fan .

[viirya]

Do we really use this API and also appendDoubles? I scan the codes but didn't find anywhere they are used.

[dongjoon-hyun]

Yep. That's the reason why this is missing until now. We need to use them in order to ColumnarBatch independently from Parquet.

[michal-databricks]

Not sure if intentional, but the added test ("Float APIs") does not cover the new function (it uses putType API mostly). Actually as far as I can see the "appendTypes" APIs are not used at all in the code. The singular versions are also not covered by the tests but are used by toBatch (which in turn is used only in testing).

[viirya]

Regarding the added test, do we really test the added appendFloats?

[dongjoon-hyun]

Thank you, @michal-databricks and @viirya .
It was my mistake. I thought I did all. :(
I'll add that correctly, too.

[dongjoon-hyun]

I updated the test cases to include appendFloat/appendFloats/appendDouble/appendDoubles.

[kiszk]

Good catch. Is is better to apply these changes to test cases for other types (e.g. null, boolean, byte, short, int, and long)?

[SparkQA]

Test build #76411 has finished for PR 17836 at commit 6cec5da.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Sure, @kiszk . I'll update the other test suite to improve coverage.

[SparkQA]

Test build #76438 has finished for PR 17836 at commit 2b58c9f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[kiszk]

Thanks, LGTM

[cloud-fan]

thanks, merging to master!

[dongjoon-hyun]

Thank you for review and merging again, @kiszk and @cloud-fan .

[dongjoon-hyun]

Hi, @cloud-fan .
Apache Spark 2.2 seems to be passed now.
Do you think it is possible to backport this into branch-2.2 for 2.2.1?

[dongjoon-hyun]

It's cherry-pickable into branch-2.2.

[cloud-fan]

to be safe, can we create a new PR for 2.2? thanks

[dongjoon-hyun]

Sure!