Skip to content

[SPARK-33040][R][ML] Add SparkR wrapper for vector_to_array#29917

Closed
zero323 wants to merge 4 commits intoapache:masterfrom
zero323:SPARK-33040
Closed

[SPARK-33040][R][ML] Add SparkR wrapper for vector_to_array#29917
zero323 wants to merge 4 commits intoapache:masterfrom
zero323:SPARK-33040

Conversation

@zero323
Copy link
Copy Markdown
Member

@zero323 zero323 commented Sep 30, 2020 •
edited
Loading

What changes were proposed in this pull request?

Add SparkR wrapper for o.a.s.ml.functions.vector_to_array

Why are the changes needed?

  • Currently ML vectors, including predictions, are almost inaccessible to R users. That's is a serious loss of functionality.
  • Feature parity.

Does this PR introduce any user-facing change?

Yes, new R function is added.

How was this patch tested?

  • New unit tests.
  • Manual verification.

@zero323 zero323 changed the title Add SparkR wrapper for vector_to_array [SPARK-33040][R][ML] Add SparkR wrapper for vector_to_array Sep 30, 2020
@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Test build #129287 has finished for PR 29917 at commit 27d1ae2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33903/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33903/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Test build #129288 has finished for PR 29917 at commit c506165.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33904/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33904/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Test build #129291 has finished for PR 29917 at commit b4697bc.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Sep 30, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33907/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 1, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33907/

@zero323 zero323 marked this pull request as ready for review October 1, 2020 05:59
@HyukjinKwon
Copy link
Copy Markdown
Member

HyukjinKwon commented Oct 2, 2020

cc @WeichenXu123 and @mengxr.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 2, 2020

Test build #129346 has finished for PR 29917 at commit fedf073.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 2, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33959/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 2, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33959/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Test build #129375 has finished for PR 29917 at commit 81bb258.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33983/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33983/

@zero323
Copy link
Copy Markdown
Member Author

zero323 commented Oct 3, 2020

Retest this please.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Test build #129376 has finished for PR 29917 at commit 81bb258.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33984/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33984/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Test build #129384 has finished for PR 29917 at commit 28aa2c2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33992/

@SparkQA
Copy link
Copy Markdown

SparkQA commented Oct 3, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33992/

@HyukjinKwon
Copy link
Copy Markdown
Member

HyukjinKwon commented Oct 5, 2020

Merged to master.

HyukjinKwon
HyukjinKwon reviewed Oct 5, 2020
View reviewed changes
Comment thread R/pkg/R/functions.R
HyukjinKwon
HyukjinKwon reviewed Oct 5, 2020
View reviewed changes
Comment thread R/pkg/R/functions.R
@zero323 zero323 mentioned this pull request Oct 5, 2020
Closed
@zero323 zero323 deleted the SPARK-33040 branch October 5, 2020 05:57
HyukjinKwon pushed a commit that referenced this pull request Oct 5, 2020
@zero323 @HyukjinKwon
[SPARK-33040][FOLLOW-UP][R] Reorder argument choices and add examples
24f890e 
### What changes were proposed in this pull request?

- Reorder choices of `dtype` to match Scala defaults.
- Add example to ml_functions.

### Why are the changes needed?

As requested:

- #29917 (review)
- #29917 (review)

### Does this PR introduce _any_ user-facing change?

No (changes to newly added component).

### How was this patch tested?

Existing tests.

Closes #29944 from zero323/SPARK-33040-FOLLOW-UP.

Authored-by: zero323 <mszymkiewicz@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
@HyukjinKwon HyukjinKwon mentioned this pull request Nov 2, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HyukjinKwon HyukjinKwon HyukjinKwon approved these changes

Assignees

No one assigned

Labels

R SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zero323 @SparkQA @HyukjinKwon