-
Notifications
You must be signed in to change notification settings - Fork 29k
[MINOR] Remove extra anonymous closure within functional transformations #12382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
cc @rxin |
|
LGTM pending tests. |
|
Test build #55793 has finished for PR 12382 at commit
|
|
retest this please |
| val words = lines.flatMap(_.split(" ")) | ||
| val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) | ||
| wordCounts.foreachRDD((rdd: RDD[(String, Int)], time: Time) => { | ||
| wordCounts.foreachRDD { (rdd: RDD[(String, Int)], time: Time) => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While we're here, can this just be (rdd, time) =>? and likewise in the following file, maybe others
|
Test build #55802 has finished for PR 12382 at commit
|
|
Merged to master |
|
Maybe we should add a scalastyle rule for this |
|
@andrewor14 Hm.. I tried to add a rule but I realised (or I think) it cannot find perfectly by a static way. One of the problems I met was below: For example, there is a class graph.aggregateMessages(ctx => { ctx.sendToSrc(1); ctx.sendToDst(1) }, _ + _,
TripletFields.None)
}In this case, this should not be the error case since In order to check this, it should be able to check Maybe I am not smart enough but this was just my conclusion. |
|
@andrewor14 @srowen Do you mind If I open another (minor) PR for some more corrections? While trying to test regular expressions, I found some more things to correct about this and about @srowen comment, which are pretty many. +If you think it is able to add a rule, then I won't. |
|
If there's a built-in scalastyle rule for this, let's enable it, but otherwise not sure it's worth building custom code to enforce it. If you've found ways to look for more instances of this and found more classes of this problem, OK open a new PR. |
…riting apache#14323 [MINOR] Remove extra anonymous closure within functional transformations apache#12382 just JdbcUtils.scala
What changes were proposed in this pull request?
This PR removes extra anonymous closure within functional transformations.
For example,
.map(item => { ... })which can be just simply as below:
.map { item => ... }How was this patch tested?
Related unit tests and
sbt scalastyle.