[SPARK-15957] [ML] RFormula supports forcing to index label #13675
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Test build #60538 has finished for PR 13675 at commit
|
|
Test build #60598 has finished for PR 13675 at commit
|
Contributor
Author
felixcheung
reviewed
Oct 7, 2016
jkbradley
reviewed
Oct 8, 2016
Member
There was a problem hiding this comment.
Rename to "forceIndexLabel" since we can still index String labels when indexLabel=false
|
Test build #3301 has finished for PR 13675 at commit
|
yanboliang
force-pushed
the
spark-15957
branch
from
a7015ef to
3a8660f
Compare
|
Test build #66661 has finished for PR 13675 at commit
|
|
Test build #66662 has finished for PR 13675 at commit
|
jkbradley
reviewed
Oct 10, 2016
| * Usually we index label only when it is string type. | ||
| * If the formula was used by classification algorithms, | ||
| * we can force to index label even it is numeric type by setting this param with true. | ||
| * Default: false. |
Member
|
I just noticed that last item, but otherwise, this looks ready to me. Thanks! |
|
Test build #66693 has finished for PR 13675 at commit
|
Member
|
Does this affect R code - could we add some R tests for this? |
Contributor
Author
|
@felixcheung This PR does not affect R code, I will send another PR to fix issues like SPARK-15153 which need to add some R tests. |
Contributor
Author
|
I'll merge this into master, thanks for review! @jkbradley @felixcheung |
ghost
pushed a commit
to dbtsai/spark
that referenced
this pull request
Oct 14, 2016
…ceIndexLabel. ## What changes were proposed in this pull request? Follow-up work of apache#13675, add Python API for ```RFormula forceIndexLabel```. ## How was this patch tested? Unit test. Author: Yanbo Liang <[email protected]> Closes apache#15430 from yanboliang/spark-15957-python.
robert3005
pushed a commit
to palantir/spark
that referenced
this pull request
Nov 1, 2016
…ceIndexLabel. ## What changes were proposed in this pull request? Follow-up work of apache#13675, add Python API for ```RFormula forceIndexLabel```. ## How was this patch tested? Unit test. Author: Yanbo Liang <[email protected]> Closes apache#15430 from yanboliang/spark-15957-python.
uzadude
pushed a commit
to uzadude/spark
that referenced
this pull request
Jan 27, 2017
## What changes were proposed in this pull request? ```RFormula``` will index label only when it is string type currently. If the label is numeric type and we use ```RFormula``` to present a classification model, there is no label attributes in label column metadata. The label attributes are useful when making prediction for classification, so we can force to index label by ```StringIndexer``` whether it is numeric or string type for classification. Then SparkR wrappers can extract label attributes from label column metadata successfully. This feature can help us to fix bug similar with [SPARK-15153](https://issues.apache.org/jira/browse/SPARK-15153). For regression, we will still to keep label as numeric type. In this PR, we add a param ```indexLabel``` to control whether to force to index label for ```RFormula```. ## How was this patch tested? Unit tests. Author: Yanbo Liang <[email protected]> Closes apache#13675 from yanboliang/spark-15957.
uzadude
pushed a commit
to uzadude/spark
that referenced
this pull request
Jan 27, 2017
…ceIndexLabel. ## What changes were proposed in this pull request? Follow-up work of apache#13675, add Python API for ```RFormula forceIndexLabel```. ## How was this patch tested? Unit test. Author: Yanbo Liang <[email protected]> Closes apache#15430 from yanboliang/spark-15957-python.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
RFormulawill index label only when it is string type currently. If the label is numeric type and we useRFormulato present a classification model, there is no label attributes in label column metadata. The label attributes are useful when making prediction for classification, so we can force to index label byStringIndexerwhether it is numeric or string type for classification. Then SparkR wrappers can extract label attributes from label column metadata successfully. This feature can help us to fix bug similar with SPARK-15153.For regression, we will still to keep label as numeric type.
In this PR, we add a param
indexLabelto control whether to force to index label forRFormula.How was this patch tested?
Unit tests.