[SC-4878] Hacky fix for log viewer's "load more" / "load previous" links

## What changes were proposed in this pull request?

This patch attempts to fix the broken "load more" / "load previous" links in the Spark web UI's log viewer page. In a nutshell, the problem is that some of the links and URLs that power this page are constructed in Javascript and thus are not visible to SparkUIProxyServlet's URL-rewriting code, causing them to break.

A partial fix, implemented by me in 600abffb19a2cb5feccae9d78666f958e30b7da2, is to change the URLs to use the relative path `../log/*` instead of `/log`. The reason that this work is that the log viewer Javascript is only served from the master / worker web UIs at `uiroot/logPage`, so the `../` takes us back to the UI root and the `/log` re-descends back into the endpoint for loading more logs. By using relative paths here we ensure that the browser issues the right requests from inside of the IFrame.

That earlier patch of mine fixed the load more link when navigating to the log viewer page from the driver UI's executor page links, but it turns out that the load more links are still broken when navigating to that same log viewer page from the worker web UI or master web UI. It turns out that this is caused by a subtle difference in behavior due to _how_ those pages are linked to:

- The driver UI generates links with a trailing slash after the `logPage` part, e.g.  `logPage/?appId=%s&executorId=%s&logType=stdout`.
- The master and worker UIs, however, omit the trailing slash and generate URLs like `logPage?appId=%s&executorId=%s&logType=stdout`; these URLs don't work with the `../` trick because they cause the generated URL / request to go up one level too high and miss the SparkContextID parameter, resulting in a broken URL which returns an error response and causes a Javascript error.

The new fix, implemented here, is still somewhat hacky: I just updated the URL generation code to always append that trailing slash. This risks breakage in case the open-source UI changes significantly in the future, but we're already vulnerable to such breakage and the more comprehensive fix will involve either extensive upstream changes to how we generate URLs in the UI (e.g. always fetching the base URL from an attribute of a HTML element on the page and using that in the JS). (Also, before anyone suggests this, the HTML5 `<base>` tag is not a good idea here since it risks serious breakage of anchor links and third-party JS).

## How was this patch tested?

Manually on Dogfood via the following steps:

- From the cluster page, navigate to the driver UI:
  - Click executors tab, then for the first executor:
    - Click 'stderr', verify that log viewer works. Whether this works or not can be verified by clicking on "load more": it should either display more output or should put up blue informational text saying that no more output is available.
    - Go back, click 'stdout', test that page.
- From the cluster page, click on Spark master UI link.
  - Click on application details page for the only running application.
    - Test stdout and stderr pages for the first executor.
- From the cluster page, click on Spark worker UI link:
  - Test sdtout and stderr pages for the first executor.

Author: Josh Rosen <joshrosendatabricks.com>

(cherry picked from commit dfd994f7121517c50d5926ccd0715843b0d548b7)
Signed-off-by: Herman van Hovell <hvanhovelldatabricks.com>

## Conflicts:
	core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala

Author: Josh Rosen <[email protected]>

Closes apache#122 from hvanhovell/SC-4878.

Author: JoshRosen

core/src/main/resources/org/apache/spark/ui/static/log-view.js

@@ -57,7 +57,7 @@ function loadMore() {
 
   $.ajax({
     type: "GET",
-    url: "/log" + baseParams + "&offset=" + offset + "&byteLength=" + moreByteLength,
+    url: "../log" + baseParams + "&offset=" + offset + "&byteLength=" + moreByteLength,
     success: function (data) {
       var oldHeight = $(".log-content")[0].scrollHeight;
       var newlineIndex = data.indexOf('\n');
@@ -83,14 +83,14 @@ function loadMore() {
 function loadNew() {
   $.ajax({
     type: "GET",
-    url: "/log" + baseParams + "&byteLength=0",
+    url: "../log" + baseParams + "&byteLength=0",
     success: function (data) {
       var dataInfo = data.substring(0, data.indexOf('\n')).match(/\d+/g);
       var newDataLen = dataInfo[2] - totalLogLength;
       if (newDataLen != 0) {
         $.ajax({
           type: "GET",
-          url: "/log" + baseParams + "&byteLength=" + newDataLen,
+          url: "../log" + baseParams + "&byteLength=" + newDataLen,
           success: function (data) {
             var newlineIndex = data.indexOf('\n');
             var dataInfo = data.substring(0, newlineIndex).match(/\d+/g);

core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala

@@ -124,9 +124,11 @@ private[ui] class ApplicationPage(parent: MasterWebUI) extends WebUIPage("app")
       <td>{executor.memory}</td>
       <td>{executor.state}</td>
       <td>
-        <a href={"%s/logPage?appId=%s&executorId=%s&logType=stdout"
+        <!-- Note: It's very important that the slash after logPage is kept here.
+             Omitting it will break the load more JS on the log viewer page. -->
+        <a href={"%s/logPage/?appId=%s&executorId=%s&logType=stdout"
           .format(workerUrlRef, executor.application.id, executor.id)}>stdout</a>
-        <a href={"%s/logPage?appId=%s&executorId=%s&logType=stderr"
+        <a href={"%s/logPage/?appId=%s&executorId=%s&logType=stderr"
           .format(workerUrlRef, executor.application.id, executor.id)}>stderr</a>
       </td>
     </tr>

core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala

@@ -155,6 +155,8 @@ private[deploy] class ExecutorRunner(
       builder.environment.put("SPARK_LAUNCH_WITH_SCALA", "0")
 
       // Add webUI log urls
+      // Note: It's very important that the slash after logPage is kept here.
+      // Omitting it will break the load more JS on the log viewer page.
       val baseUrl =
         if (conf.getBoolean("spark.ui.reverseProxy", false)) {
           s"/proxy/$workerId/logPage/?appId=$appId&executorId=$execId&logType="

core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala

@@ -117,9 +117,11 @@ private[ui] class WorkerPage(parent: WorkerWebUI) extends WebUIPage("") {
         </ul>
       </td>
       <td>
-     <a href={"logPage?appId=%s&executorId=%s&logType=stdout"
+     <!-- Note: It's very important that the slash after logPage is kept here.
+          Omitting it will break the load more JS on the log viewer page. -->
+     <a href={"logPage/?appId=%s&executorId=%s&logType=stdout"
         .format(executor.appId, executor.execId)}>stdout</a>
-     <a href={"logPage?appId=%s&executorId=%s&logType=stderr"
+     <a href={"logPage/?appId=%s&executorId=%s&logType=stderr"
         .format(executor.appId, executor.execId)}>stderr</a>
       </td>
     </tr>
@@ -138,8 +140,10 @@ private[ui] class WorkerPage(parent: WorkerWebUI) extends WebUIPage("") {
         {Utils.megabytesToString(driver.driverDesc.mem)}
       </td>
       <td>
-        <a href={s"logPage?driverId=${driver.driverId}&logType=stdout"}>stdout</a>
-        <a href={s"logPage?driverId=${driver.driverId}&logType=stderr"}>stderr</a>
+        <!-- Note: It's very important that the slash after logPage is kept here.
+             Omitting it will break the load more JS on the log viewer page. -->
+        <a href={s"logPage/?driverId=${driver.driverId}&logType=stdout"}>stdout</a>
+        <a href={s"logPage/?driverId=${driver.driverId}&logType=stderr"}>stderr</a>
       </td>
       <td>
         {driver.finalException.getOrElse("")}