Skip to content

Commit cf7ee17

Browse files
jerryshaoMarcelo Vanzin
authored andcommitted
[SPARK-23147][UI] Fix task page table IndexOutOfBound Exception
## What changes were proposed in this pull request? Stage's task page table will throw an exception when there's no complete tasks. Furthermore, because the `dataSize` doesn't take running tasks into account, so sometimes UI cannot show the running tasks. Besides table will only be displayed when first task is finished according to the default sortColumn("index"). ![screen shot 2018-01-18 at 8 50 08 pm](https://user-images.githubusercontent.com/850797/35100052-470b4cae-fc95-11e7-96a2-ad9636e732b3.png) To reproduce this issue, user could try `sc.parallelize(1 to 20, 20).map { i => Thread.sleep(10000); i }.collect()` or `sc.parallelize(1 to 20, 20).map { i => Thread.sleep((20 - i) * 1000); i }.collect` to reproduce the above issue. Here propose a solution to fix it. Not sure if it is a right fix, please help to review. ## How was this patch tested? Manual test. Author: jerryshao <[email protected]> Closes #20315 from jerryshao/SPARK-23147.
1 parent 5063b74 commit cf7ee17

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -676,7 +676,7 @@ private[ui] class TaskDataSource(
676676

677677
private var _tasksToShow: Seq[TaskData] = null
678678

679-
override def dataSize: Int = stage.numCompleteTasks + stage.numFailedTasks + stage.numKilledTasks
679+
override def dataSize: Int = stage.numTasks
680680

681681
override def sliceData(from: Int, to: Int): Seq[TaskData] = {
682682
if (_tasksToShow == null) {

0 commit comments

Comments
 (0)