[SPARK-14516][FOLLOWUP] Adding ClusteringEvaluator to examples #19676
Conversation
|
Test build #83500 has finished for PR 19676 at commit
|
|
sorry for pinging you, what do you think about adding |
|
It's good to have this, sorry for late response, I will make a pass tomorrow. Thanks. |
yanboliang
left a comment
yanboliang
left a comment
There was a problem hiding this comment.
LGTM except one minor comment. Thanks.
| // Evaluate clustering by computing Silhouette score | ||
| ClusteringEvaluator evaluator = new ClusteringEvaluator() | ||
| .setFeaturesCol("features") | ||
| .setPredictionCol("prediction") |
There was a problem hiding this comment.
We use default values here, so it's not necessary to set them explicitly. We should keep examples as simple as possible. Thanks.
|
Test build #84681 has finished for PR 19676 at commit
|
| ClusteringEvaluator evaluator = new ClusteringEvaluator(); | ||
|
|
||
| double silhouette = evaluator.evaluate(predictions); | ||
| System.out.println("Silhouette with squared euclidean distance = " + silhouette); |
There was a problem hiding this comment.
euclidean -> Euclidean, but not important to change unless you're touching the code again anyway
There was a problem hiding this comment.
Thanks, I don't think I am changing the code again, but I can fix this grammatical error if you want.
|
Merged to master |
4 participants
What changes were proposed in this pull request?
In SPARK-14516 we have introduced ClusteringEvaluator, but we didn't put any reference in the documentation and the examples were still relying on the sum of squared errors to show a way to evaluate the clustering model.
The PR adds the ClusteringEvaluator in the examples.
How was this patch tested?
Manual runs of the examples.