[SPARK-20713][Spark Core] Convert CommitDenied to TaskKilled.

[nlyu]

What changes were proposed in this pull request?

In executor, toTaskFailedReason is converted to toTaskCommitDeniedReason to avoid the inconsistency of taskState. In JobProgressListener, add case TaskCommitDenied so that now the stage killed number is been incremented other than failed number.
This pull request is picked up from: #18070 using commit: ff93ade
The case match for TaskCommitDenied is added incrementing the correct num of killed after pull/18070.

How was this patch tested?

Run a normal speculative job and check the Stage UI page, should have no failed displayed.

[nlyu]

@tgravescs Hi Tom i am Louis Lyu, here is the PR

[tgravescs]

ok to test

[tgravescs]

give this a variable name like case commitDenied: TaskCommitDenied(jobID, partitionID, attemptNumber) that way you can use the variable commitDenied below in the toErrorString

[nlyu]

okay , working on it

[tgravescs]

same here use a variable

[tgravescs]

use a variable

[SparkQA]

Test build #80164 has finished for PR 18819 at commit 499f1e0.

• This patch fails SparkR unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #80165 has finished for PR 18819 at commit f975922.

• This patch fails SparkR unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #80179 has started for PR 18819 at commit 2d5ccb4.

[SparkQA]

Test build #80184 has finished for PR 18819 at commit f975922.

• This patch fails from timeout after a configured wait of `250m`.
• This patch merges cleanly.
• This patch adds no public classes.

[tgravescs]

test this please

[SparkQA]

Test build #80210 has finished for PR 18819 at commit f975922.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[tgravescs]

+1

[nlyu]

@tgravescs Hi Tom a quick question.
So for the UI data table covert JIRA, is there a corresponding JIRA number in the community for that? Like SPARK-20XXX.

[tgravescs]

@nlyu no please file one