[SPARK-12053][CORE] EventLoggingListener.getLogPath needs 4 parameters#10044
[SPARK-12053][CORE] EventLoggingListener.getLogPath needs 4 parameters#10044chutium wants to merge 2 commits intoapache:masterfrom
Conversation
|
Ah, good catch. While you're there, can you change the fourth parameter to explicitly have a name? e.g.: getLogPath(foo, bar, baz, compressionCodecName = codec) There should have been a warning on the output about that missing name, although it's pretty easy to ignore warnings... :-/ |
|
ha, yes, it happens, okey, i will add some parameter names |
|
LGTM good catch |
|
Test build #46897 has finished for PR 10044 at commit
|
|
Test build #46900 has finished for PR 10044 at commit
|
|
LGTM, merging into master, branch-1.6, branch-1.5 and branch-1.4. Thanks @chutium ! |
```EventLoggingListener.getLogPath``` needs 4 input arguments: https://github.com/apache/spark/blob/v1.6.0-preview2/core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala#L276-L280 the 3rd parameter should be appAttemptId, 4th parameter is codec... Author: Teng Qiu <teng.qiu@gmail.com> Closes #10044 from chutium/SPARK-12053. (cherry picked from commit a8ceec5) Signed-off-by: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
```EventLoggingListener.getLogPath``` needs 4 input arguments: https://github.com/apache/spark/blob/v1.6.0-preview2/core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala#L276-L280 the 3rd parameter should be appAttemptId, 4th parameter is codec... Author: Teng Qiu <teng.qiu@gmail.com> Closes #10044 from chutium/SPARK-12053. (cherry picked from commit a8ceec5) Signed-off-by: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
|
I noticed we cannot merge this patch without modifying manually so I don't merge this into branch-1.4 for now. Please feel free to open another PR for branch-1.4. |
|
thanks for merging, i will check this for branch-1.4 |
|
@sarutak it seems branch-1.4 was correct with the 3rd parameter |
|
it was caused by this commit about half year ago: 3bee0f1#diff-29dffdccd5a7f4c8b496c293e87c8668L771 hope no other small hidden bugs there... maybe worth a double check again |
|
Yeah, I also noticed that branch-1.4 has no issue discussed in this PR. |
EventLoggingListener.getLogPathneeds 4 input arguments:https://github.com/apache/spark/blob/v1.6.0-preview2/core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala#L276-L280
the 3rd parameter should be appAttemptId, 4th parameter is codec...