[SPARK-20875] Spark should print the log when the directory has been deleted#18102
[SPARK-20875] Spark should print the log when the directory has been deleted#18102liu-zhaokun wants to merge 6 commits intoapache:masterfrom
Conversation
|
I would doubt it will be too verbose to print out the path. |
|
@jerryshao |
|
Agree -- logging at INFO level is an overkill. At the very least it should be at the TRACE level. |
|
@sameeragarwal |
|
@jerryshao |
|
Looks like it will only print out the path of directory, not file, is that on purpose? |
|
@jerryshao |
|
@jerryshao |
srowen
left a comment
srowen
left a comment
There was a problem hiding this comment.
Ok but you invoke string interpolation but then don't use it. The string message needs to be fixed.
|
@srowen |
| for (child <- listFilesSafely(file)) { | ||
| try { | ||
| deleteRecursively(child) | ||
| logTrace(file.getAbsolutePath + " has been deleted") |
There was a problem hiding this comment.
I mean, why not s"${file.getAbsolutePath} has been deleted"? it's actually important for a log trace statement
|
@srowen |
|
I found there are some mistakes.So I modify the PR again.Please review it. |
| } finally { | ||
| if (!file.delete()) { | ||
| if (file.delete()) { | ||
| logTrace(s"${file.getAbsolutePath} has been deleted") |
There was a problem hiding this comment.
I guess this should have the double spaced indentation.
|
@HyukjinKwon |
|
Test build #3759 has finished for PR 18102 at commit
|
|
@srowen |
|
@HyukjinKwon |
|
Test build #3765 has finished for PR 18102 at commit
|
|
Merged to master |
6 participants
https://issues.apache.org/jira/browse/SPARK-20875
When the "deleteRecursively" method is invoked,spark doesn't print any log if the path was deleted.For example,spark only print "Removing directory" when the worker began cleaning spark.work.dir,but didn't print any log about "the path has been delete".So, I can't judge whether the path was deleted form the worker's logfile,If there is any accidents about Linux.