Skip to content

[SPARK-24506][UI] Add UI filters to tabs added after binding #21523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

[SPARK-24506][UI] Add UI filters to tabs added after binding #21523

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
086a4f6
[SPARK-24506][UI] Add UI filters also to thriftserver tab
mgaido91 Jun 10, 2018
0ac1848
address comments
mgaido91 Jun 11, 2018
08cd576
remove unneeded changes
mgaido91 Jun 11, 2018
02a9f4e
address comments
mgaido91 Jun 12, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,6 @@ class HistoryServer(
assert(serverInfo.isDefined, "HistoryServer must be bound before attaching SparkUIs")
handlers.synchronized {
ui.getHandlers.foreach(attachHandler)
addFilters(ui.getHandlers, conf)
}
}

Expand Down
2 changes: 1 addition & 1 deletion core/src/main/scala/org/apache/spark/ui/JettyUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ private[spark] object JettyUtils extends Logging {
filters.foreach {
case filter : String =>
if (!filter.isEmpty) {
logInfo("Adding filter: " + filter)
logInfo(s"Adding filter $filter to ${handlers.map(_.getContextPath).mkString(", ")}.")
val holder : FilterHolder = new FilterHolder()
holder.setClassName(filter)
// Get any parameters for each filter
Expand Down
7 changes: 6 additions & 1 deletion core/src/main/scala/org/apache/spark/ui/WebUI.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,12 @@ private[spark] abstract class WebUI(
/** Attach a handler to this UI. */
def attachHandler(handler: ServletContextHandler) {
handlers += handler
serverInfo.foreach(_.addHandler(handler))
serverInfo.foreach { sInfo =>
Copy link
Contributor

@vanzin vanzin Jun 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a slight preference for doing this in the ServerInfo class, so that we're sure we're covering all paths.

I have a WIP patch for another thing where I did that, something like this:

@@ -507,17 +517,19 @@ private[spark] case class ServerInfo(
     server: Server,
     boundPort: Int,
     securePort: Option[Int],
+    private val conf: SparkConf,
     private val rootHandler: ContextHandlerCollection) {
 
-  def addHandler(handler: ContextHandler): Unit = {
+  def addHandler(handler: ServletContextHandler): Unit = {
     handler.setVirtualHosts(JettyUtils.toVirtualHosts(JettyUtils.SPARK_CONNECTOR_NAME))
+    JettyUtils.addFilters(Seq(handler), conf)
     rootHandler.addHandler(handler)
     if (!handler.isStarted()) {
       handler.start()
     }
   }

It also avoids a race where an already started handler would be added before filters are installed. Extremely unlikely that would be a problem, but well.

Copy link
Contributor Author

@mgaido91 mgaido91 Jun 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, @vanzin , thanks for checking this. Do you have this WIP as a PR? If so, should I close this or updating according to your suggestion?

Copy link
Contributor

@vanzin vanzin Jun 12, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still working on it since it touches a bunch of other things, not just this. This is fine as is, it's better to have a more targeted fix here and leave my (future) PR just as refactoring.

sInfo.addHandler(handler)
// If the UI has already been bound, we need to add the filters to the newly attached
// handlers. Otherwise, they will be attached when binding.
addFilters(Seq(handler), conf)
}
}

/** Detach a handler from this UI. */
Expand Down