[SPARK-25170][DOC] Add list and short description of Spark Executor Task Metrics to the documentation.#22397
[SPARK-25170][DOC] Add list and short description of Spark Executor Task Metrics to the documentation.#22397LucaCanali wants to merge 2 commits intoapache:masterfrom
Conversation
|
LGTM cc @jiangxb1987 @srowen |
srowen
left a comment
There was a problem hiding this comment.
The docs looks OK. I guess these weren't documented before? I spot-checked the units on a few metrics and they're right. I guess it's a little unfortunate there are some ns and ms times in here, but not a problem.
|
Thanks @srowen for reviewing this. The metrics are commented in the source code of TaskMetrics class, I took most of the descriptions from there, adding some additional explanations where needed. Indeed I agree that the fact that time units are not uniform is a bit incovenient. |
|
Test build #4340 has finished for PR 22397 at commit
|
…ask Metrics to the documentation. ## What changes were proposed in this pull request? Add description of Executor Task Metrics to the documentation. Closes #22397 from LucaCanali/docMonitoringTaskMetrics. Authored-by: LucaCanali <luca.canali@cern.ch> Signed-off-by: Sean Owen <sean.owen@databricks.com> (cherry picked from commit 45c4ebc) Signed-off-by: Sean Owen <sean.owen@databricks.com>
|
Merged to master/2.4 |
|
Yes, time unit looks confusing, but not a problem. In general, due to APIs, elapsed time is |
What changes were proposed in this pull request?
Add description of Executor Task Metrics to the documentation.