diff --git a/docs/img/AllJobsPageDetail1.png b/docs/img/AllJobsPageDetail1.png new file mode 100644 index 0000000000000..c76d2fdefb1aa Binary files /dev/null and b/docs/img/AllJobsPageDetail1.png differ diff --git a/docs/img/AllJobsPageDetail2.png b/docs/img/AllJobsPageDetail2.png new file mode 100644 index 0000000000000..b7203b2e66586 Binary files /dev/null and b/docs/img/AllJobsPageDetail2.png differ diff --git a/docs/img/AllJobsPageDetail3.png b/docs/img/AllJobsPageDetail3.png new file mode 100644 index 0000000000000..75b7caec119b2 Binary files /dev/null and b/docs/img/AllJobsPageDetail3.png differ diff --git a/docs/img/JobPageDetail1.png b/docs/img/JobPageDetail1.png new file mode 100644 index 0000000000000..1ee741d1f09d0 Binary files /dev/null and b/docs/img/JobPageDetail1.png differ diff --git a/docs/img/JobPageDetail2.png b/docs/img/JobPageDetail2.png new file mode 100644 index 0000000000000..ab6d7bdf15ec2 Binary files /dev/null and b/docs/img/JobPageDetail2.png differ diff --git a/docs/img/JobPageDetail3.png b/docs/img/JobPageDetail3.png new file mode 100644 index 0000000000000..9f691e4ed2b6b Binary files /dev/null and b/docs/img/JobPageDetail3.png differ diff --git a/docs/web-ui.md b/docs/web-ui.md index a15c114efc502..d6ee8eb707007 100644 --- a/docs/web-ui.md +++ b/docs/web-ui.md @@ -35,6 +35,64 @@ progress of all jobs and the overall event timeline. When you click on a job on page, you see the details page for that job. The details page further shows the event timeline, DAG visualization, and all stages of the job. +The information that is displayed in this section is +* User: Current Spark user +* Total uptime: Time since Spark application started +* Scheduling mode: See [job scheduling](job-scheduling.html#configuring-pool-properties) +* Number of jobs per status: Active, Completed, Failed + +

+ Basic info +

+ +* Event timeline: Displays in chronological order the events related to the executors (added, removed) and the jobs + +

+ Event timeline +

+ +* Details of jobs grouped by status: Displays detailed information of the jobs including Job ID, description (with a link to detailed job page), submitted time, duration, stages summary and tasks progress bar + +

+ Details of jobs grouped by status +

+ + +When you click on a specific job, you can see the detailed information of this job. + +### Jobs detail + +This page displays the details of a specific job identified by its job ID. +* Job Status: (running, succeeded, failed) +* Number of stages per status (active, pending, completed, skipped, failed) +* Associated SQL Query: Link to the sql tab for this job +* Event timeline: Displays in chronological order the events related to the executors (added, removed) and the stages of the job + +

+ Event timeline +

+ +* DAG visualization: Visual representation of the directed acyclic graph of this job where vertices represent the RDDs or DataFrames and the edges represent an operation to be applied on RDD. + +

+ DAG +

+ +* List of stages (grouped by state active, pending, completed, skipped, and failed) + * Stage ID + * Description of the stage + * Submitted timestamp + * Duration of the stage + * Tasks progress bar + * Input: Bytes read from storage in this stage + * Output: Bytes written in storage in this stage + * Shuffle read: Total shuffle bytes and records read, includes both data read locally and data read from remote executors + * Shuffle write: Bytes and records written to disk in order to be read by a shuffle in a future stage + +

+ DAG +

+ ## Stages Tab The Stages tab displays a summary page that shows the current state of all stages of all jobs in the Spark application, and, when you click on a stage, a details page for that stage. The details @@ -117,3 +175,4 @@ illustrate how Spark parses, analyzes, optimizes and performs the query. The web UI includes a Streaming tab if the application uses Spark streaming. This tab displays scheduling delay and processing time for each micro-batch in the data stream, which can be useful for troubleshooting the streaming application. +