Skip to content

[SPARK-24851][UI] Map a Stage ID to it's Associated Job ID #21809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pgandhi999 wants to merge 9 commits into from
Closed

[SPARK-24851][UI] Map a Stage ID to it's Associated Job ID #21809

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
7be0520
[SPARK-24851] : Map a Stage ID to it's Associated Job ID in UI
Jul 18, 2018
3151a62
[SPARK-24851] : Using both stage id as well as the attempt id to fetc…
Jul 23, 2018
a50e8b1
[SPARK-24851] : Avoiding extra store lookup and adding href links
Jul 25, 2018
d57e6dc
[SPARK-24851] : Fixing Scalastyle Tests
Jul 25, 2018
3a06b87
[SPARK-24851] : Adding exception handling part to avoid NoSuchElement…
Jul 26, 2018
52b08b2
[SPARK-24851] : Addressing reviews July 27
Jul 27, 2018
9eff537
[SPARK-24851] : Addressing Reviews August 9
Aug 9, 2018
3b73840
[SPARK-24851] : Addressing reviews September 28, 2018
Sep 28, 2018
0be099a
[SPARK-24851] : Addressing Reviews October 5, 2018
Oct 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 19 additions & 6 deletions core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
package org.apache.spark.ui.jobs

import java.net.URLEncoder
import java.util.Date
import java.util.{Date, NoSuchElementException}
import java.util.concurrent.TimeUnit
import javax.servlet.http.HttpServletRequest

Expand Down Expand Up @@ -105,16 +105,29 @@ private[ui] class StagePage(parent: StagesTab, store: AppStatusStore) extends We
val stageAttemptId = parameterAttempt.toInt

val stageHeader = s"Details for Stage $stageId (Attempt $stageAttemptId)"
val stageDataWrapper = parent.store.stageAttempt(stageId, stageAttemptId, details = false)
val stageData = parent.store
.asOption(stageDataWrapper.info)
.getOrElse {
var stageDataWrapper: StageDataWrapper = null
try {
Copy link
Contributor

@tgravescs tgravescs Aug 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could use options here instead of try and null checks to make this cleaner

Copy link
Author

@pgandhi999 pgandhi999 Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks!

stageDataWrapper = parent.store.stageAttempt(stageId, stageAttemptId, details = false)
} catch {
case e: NoSuchElementException => e.getMessage
}
var stageData: StageData = null
if (stageDataWrapper != null) {
stageData = parent.store
.asOption(stageDataWrapper.info)
.get
} else {
stageData = {
Copy link
Member

@gengliangwang gengliangwang Jul 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this code branch is unreachable.

Copy link
Author

@pgandhi999 pgandhi999 Jul 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unreachable by the IDE but during runtime, the code does work, I have confirmed that.

val content =
<div id="no-info">
<p>No information to display for Stage {stageId} (Attempt {stageAttemptId})</p>
<p>No information to display for Stage
{stageId}
(Attempt
{stageAttemptId})</p>
</div>
return UIUtils.headerSparkPage(request, stageHeader, content, parent)
}
}

val stageJobIds = stageDataWrapper.jobIds.toSeq

Expand Down