[SPARK-20945] Fix TID key not found in TaskSchedulerImpl#18171
[SPARK-20945] Fix TID key not found in TaskSchedulerImpl#18171liupc wants to merge 3 commits intoapache:masterfrom
Conversation
|
Hi @liupc, Thanks for the JIRA and PR. Could you edit the PR's title to incorporate the JIRA number? e.g.
Also, could you re-target this PR to be against the |
liupc
force-pushed
the
Fix-tid-key-not-found-in-TaskSchedulerImpl
branch
from
d6831fb to
1e35879
Compare
liupc
changed the title
| backend.killTask(tid, execId, interruptThread, reason = "stage cancelled") | ||
| if (taskIdToExecutorId.contains(tid)) { | ||
| val execId = taskIdToExecutorId(tid) | ||
| backend.killTask(tid, execId, interruptThread) |
There was a problem hiding this comment.
I think you lost the value of "reason" here.
It's not a big deal but you could also write:
taskIdToExecutorId.get(tid).foreach(execId => backend.killTask(tid, execId, interruptThread, reason = "stage cancelled"))
There was a problem hiding this comment.
Looks good! I will update soon, thanks!
|
Test build #3775 has finished for PR 18171 at commit
|
|
Merged to master |
|
Why this is not merged into 2.2 ? |
5 participants
What changes were proposed in this pull request?
This pull request fix the TaskScheulerImpl bug in some condition.
Detail see:
https://issues.apache.org/jira/browse/SPARK-20945
(Please fill in changes proposed in this fix)
How was this patch tested?
manual tests
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.