[SPARK-29456][WebUI] Improve tooltip for Session Statistics Table column in JDBC/ODBC Server Tab #26138

PavithraRamachandran · 2019-10-16T09:31:42Z

What changes were proposed in this pull request?
Some of the columns of JDBC/ODBC tab Session info in Web UI are hard to understand.

Add tool tip for Start time, finish time , Duration and Total Execution

Why are the changes needed?
To improve the understanding of the WebUI

Does this PR introduce any user-facing change?
No

How was this patch tested?
manual test

wangyum · 2019-10-16T13:21:07Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+    "Session end time, after closing the session."
+
+  val THRIFT_SESSION_DURATION =
+    "Active duration of the session."


Please add tooltips for all columns. You can refer to our web-ui documentation:

spark/docs/web-ui.md

Lines 416 to 447 in d334fee

The second section contains information about active and finished sessions.

* **User** and **IP** of the connection.

* **Session id** link to access to session info.

* **Start time**, **finish time** and **duration** of the session.

* **Total execute** is the number of operations submitted in this session.

<p style="text-align: center;">

<img src="img/JDBCServer2.png" title="JDBC/ODBC sessions" alt="JDBC/ODBC sessions">

</p>

The third section has the SQL statistics of the submitted operations.

* **User** that submit the operation.

* **Job id** link to [jobs tab](web-ui.html#jobs-tab).

* **Group id** of the query that group all jobs together. An application can cancel all running jobs using this group id.

* **Start time** of the operation.

* **Finish time** of the execution, before fetching the results.

* **Close time** of the operation after fetching the results.

* **Execution time** is the difference between finish time and start time.

* **Duration time** is the difference between close time and start time.

* **Statement** is the operation being executed.

* **State** of the process.

* _Started_, first state, when the process begins.

* _Compiled_, execution plan generated.

* _Failed_, final state when the execution failed or finished with error.

* _Canceled_, final state when the execution is canceled.

* _Finished_ processing and waiting to fetch results.

* _Closed_, final state when client closed the statement.

* **Detail** of the execution plan with parsed logical plan, analyzed logical plan, optimized logical plan and physical plan or errors in the the SQL statement.

<p style="text-align: center;">

<img src="img/JDBCServer3.png" title="JDBC/ODBC SQL Statistics" alt="JDBC/ODBC SQL Statistics">

</p>

@wangyum in the same page statistics table does not have tooltip for all the column. Should we address that too?

@wangyum i have added tooltip for all the columns in session table. Could you review?

I don't think we need tooltips for every single column. "IP of the session" does not add anything over the column name. We should only add tooltips where they add something meaningful, like, clarifying for duration fields what the start and end time are exactly.

Duration tooltips can basically specify one useful thing: what is the start and end time? this isn't telling the user anything more right now

wangyum · 2019-10-24T05:58:08Z

ok to test

SparkQA · 2019-10-24T06:06:04Z

Test build #112588 has finished for PR 26138 at commit 029ec03.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-10-24T08:43:11Z

Test build #112597 has finished for PR 26138 at commit 35e6918.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-11-04T11:33:26Z

Test build #113208 has finished for PR 26138 at commit 0028ccd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

Same here, these are basically repeating the title and I don't think we need to clutter the UI with that kind of popup tooltip

PavithraRamachandran · 2019-11-04T15:50:37Z

@srowen do you suggest that we have tooltips only for columns like
Start time, finish time , Duration and Total Execution
and remove
User, IP, Session ID - which are self explanatory?

srowen · 2019-11-13T14:06:29Z

...thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPage.scala

-            </a>
-          </th>
+          if (tooltip.nonEmpty) {
+            <th width={colWidthAttr}>


Can we avoid duplicating the th and a elements in both branches?

srowen · 2019-11-13T14:06:38Z

...thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPage.scala

-          </th>
+          if (tooltip.nonEmpty) {
+            <th width={colWidthAttr}>
+              <a href={headerLink}>


srowen · 2019-11-13T14:06:56Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+    "Number of operations submitted in this session."
+
+  val THRIFT_SESSION_START_TIME =
+    "Session Start time,on launching the session."


Nit: space after ,

srowen · 2019-11-13T14:07:34Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+    "Session Start time,on launching the session."
+
+  val THRIFT_SESSION_FINISH_TIME =
+    "Session end time, after closing the session."


How about "Time that the session was closed"

srowen · 2019-11-13T14:07:46Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+    "Number of operations submitted in this session."
+
+  val THRIFT_SESSION_START_TIME =
+    "Session Start time,on launching the session."


"Time that the session was started"?

srowen · 2019-11-13T14:08:14Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+    "Session end time, after closing the session."
+
+  val THRIFT_SESSION_DURATION =
+    "Active duration of the session."


Duration tooltips can basically specify one useful thing: what is the start and end time? this isn't telling the user anything more right now

SparkQA · 2019-11-13T14:09:47Z

Test build #113708 has finished for PR 26138 at commit 33c0a4e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-11-16T11:37:24Z

Test build #113925 has finished for PR 26138 at commit 49df510.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-11-16T14:43:06Z

Test build #113934 has finished for PR 26138 at commit 064ab01.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

PavithraRamachandran · 2019-11-16T15:04:37Z

@srowen i have changed according to the review comments. Could you review?

srowen · 2019-11-16T16:11:15Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+
+  val THRIFT_SESSION_DURATION =
+    "Total duration the session has been active, from the time session" +
+      " was lauched till it was closed."


lauched till -> launched until
But should we say 'started' instead anyway, to be consistent with the above?

srowen · 2019-11-16T16:11:28Z

...thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPage.scala

+            <th width={colWidthAttr}>
+              <a href={headerLink}>
+                {
+                    if (tooltip.nonEmpty) {


The indents are too deep and a little inconsistent - just use 2 space indents

PavithraRamachandran · 2019-11-16T16:24:23Z

@srowen i have reworked on the review comments. Could you review?

srowen · 2019-11-16T16:28:46Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+    "Time that the session was closed."
+
+  val THRIFT_SESSION_DURATION =
+    "Total duration the session has been active, from the time session" +


What does it show while it's active, time from start until now?

Maybe finally: Elapsed time since session start, or until closed if the session was closed.

srowen · 2019-11-16T16:29:10Z

sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ToolTips.scala

+    "Number of operations submitted in this session."
+
+  val THRIFT_SESSION_START_TIME =
+    "Session Start time, on launching the session."


"Time the session was started." But I'd just delete this and the next tooltip. It doesn't add much.

srowen · 2019-11-16T16:30:15Z

...thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPage.scala

+                      {header}
+                    </span>
+                  } else {
+                      {header}


Nit: unindent

SparkQA · 2019-11-16T16:48:21Z

Test build #113938 has finished for PR 26138 at commit 839dd12.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

PavithraRamachandran · 2019-11-17T12:31:21Z

@srowen I have reworked. kindly verify the changes.

SparkQA · 2019-11-17T12:50:26Z

Test build #113957 has finished for PR 26138 at commit 09ee0d4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen · 2019-11-17T13:04:47Z

Merged to master

wangyum reviewed Oct 16, 2019

View reviewed changes

dongjoon-hyun added the WEB UI label Oct 16, 2019

shahidki31 mentioned this pull request Oct 25, 2019

[SPARK-29599][WEBUI] Support pagination for session table in JDBC/ODBC Tab #26253

Closed

PavithraRamachandran force-pushed the JDBC_tooltip branch from 35e6918 to 0028ccd Compare November 4, 2019 11:03

srowen reviewed Nov 4, 2019

View reviewed changes

srowen requested changes Nov 13, 2019

View reviewed changes

PavithraRamachandran force-pushed the JDBC_tooltip branch from 49df510 to 064ab01 Compare November 16, 2019 14:17

srowen requested changes Nov 16, 2019

View reviewed changes

tooltip for sessions in JDBC /ODBC page

09ee0d4

PavithraRamachandran force-pushed the JDBC_tooltip branch from 839dd12 to 09ee0d4 Compare November 17, 2019 12:26

srowen closed this in a9959be Nov 17, 2019

	The second section contains information about active and finished sessions.
	* User and IP of the connection.
	* Session id link to access to session info.
	* Start time, finish time and duration of the session.
	* Total execute is the number of operations submitted in this session.

	<p style="text-align: center;">
	<img src="img/JDBCServer2.png" title="JDBC/ODBC sessions" alt="JDBC/ODBC sessions">
	</p>

	The third section has the SQL statistics of the submitted operations.
	* User that submit the operation.
	* Job id link to [jobs tab](web-ui.html#jobs-tab).
	* Group id of the query that group all jobs together. An application can cancel all running jobs using this group id.
	* Start time of the operation.
	* Finish time of the execution, before fetching the results.
	* Close time of the operation after fetching the results.
	* Execution time is the difference between finish time and start time.
	* Duration time is the difference between close time and start time.
	* Statement is the operation being executed.
	* State of the process.
	* _Started_, first state, when the process begins.
	* _Compiled_, execution plan generated.
	* _Failed_, final state when the execution failed or finished with error.
	* _Canceled_, final state when the execution is canceled.
	* _Finished_ processing and waiting to fetch results.
	* _Closed_, final state when client closed the statement.
	* Detail of the execution plan with parsed logical plan, analyzed logical plan, optimized logical plan and physical plan or errors in the the SQL statement.

	<p style="text-align: center;">
	<img src="img/JDBCServer3.png" title="JDBC/ODBC SQL Statistics" alt="JDBC/ODBC SQL Statistics">
	</p>

[SPARK-29456][WebUI] Improve tooltip for Session Statistics Table column in JDBC/ODBC Server Tab #26138

[SPARK-29456][WebUI] Improve tooltip for Session Statistics Table column in JDBC/ODBC Server Tab #26138

Uh oh!

Conversation

PavithraRamachandran commented Oct 16, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangyum commented Oct 24, 2019

Uh oh!

SparkQA commented Oct 24, 2019

Uh oh!

SparkQA commented Oct 24, 2019

Uh oh!

SparkQA commented Nov 4, 2019

Uh oh!

srowen left a comment

Choose a reason for hiding this comment

Uh oh!

PavithraRamachandran commented Nov 4, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Nov 13, 2019

Uh oh!

SparkQA commented Nov 16, 2019

Uh oh!

SparkQA commented Nov 16, 2019

Uh oh!

PavithraRamachandran commented Nov 16, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PavithraRamachandran commented Nov 16, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Nov 16, 2019

Uh oh!

PavithraRamachandran commented Nov 17, 2019

Uh oh!

SparkQA commented Nov 17, 2019

Uh oh!

srowen commented Nov 17, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants