[SPARK-32026][CORE][TEST] Add PrometheusServletSuite #28865

erenavsarogullari · 2020-06-19T00:42:25Z

What changes were proposed in this pull request?

This PR aims to add PrometheusServletSuite.

Why are the changes needed?

This improves the test coverage.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the newly added test suite.

erenavsarogullari · 2020-06-19T00:43:18Z

cc @dongjoon-hyun

SparkQA · 2020-06-19T00:52:11Z

Test build #124244 has finished for PR 28865 at commit 7f31529.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun

Thank you for contribution, @erenavsarogullari .

However, Prometheus Servlet is designed to be consistent with the existing Spark 2.4 JMX Sink + Prometheus JMX Exporter format. Also, inside Spark 3.0, the format will follow Spark 3.0 JMX Sink + Prometheus JMX Exporter style mostly. Could you compare your output result with JMX Sink + Prometheus Exporter result of Spark 3.0 first?

For the metric name, we know that's not proper Prometheus. So, we recommend to use spark.metrics.namespace and writeStream.queryName().

It's described in original JIRAs, but the following presentation slides for Spark+AI Summit 2020 will be helpful for you if you are interested in this area.

https://speakerdeck.com/dongjoon/native-support-of-prometheus-monitoring-in-apache-spark-3

erenavsarogullari · 2020-06-24T03:31:10Z

Hi @dongjoon-hyun,

Thanks for the review and details.

Prometheus format seems following JMX-Exporter and also introducing type attribute(e.g: type="gauges"). I think we would like to continue preserving the same format, right?

On the other hand, what about Unit Tests for PrometheusServlet in the light of this PR coverage? I think they can still be useful.

Jmx:

metrics_local_1592959391297_driver_BlockManager_memory_memUsed_MB_Value{instance="172.16.0.122:8088",job="mysparkapp"} 0

Prometheus:

metrics_local_1592960380968_driver_BlockManager_memory_memUsed_MB_Value{instance="172.16.0.122:4041",job="spark-driver",type="gauges"} 0

dongjoon-hyun · 2020-06-27T17:17:27Z

No. Please test on Apache Spark 3.0.0. DropWizard 4 itself introduced type attribute(e.g: type="gauges") and PrometheusSevlet just follows that inevitably.

Prometheus format seems following JMX-Exporter and also introducing type attribute(e.g: type="gauges"). I think we would like to continue preserving the same format, right?

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala

dongjoon-hyun

Let's focus on adding a unit test, @erenavsarogullari . Also, please update JIRA and PR title~ Thank you for your contribution, @erenavsarogullari .

dongjoon-hyun · 2020-06-30T00:14:03Z

Retest this please

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala

dongjoon-hyun · 2020-06-30T00:21:50Z

Also, please update the PR description consistently together. Thanks!

dongjoon-hyun · 2020-06-30T02:40:43Z

Retest this please.

erenavsarogullari · 2020-06-30T03:08:58Z

Thanks @dongjoon-hyun for the review. All comments are addressed. I think it is ready to go.
Also, we plan to use Prometheus + Grafana with proposed format change (through custom release). It may still be useful for the long-term if we plan to revisit the current format in the future.

SparkQA · 2020-06-30T16:46:53Z

Test build #124665 has finished for PR 28865 at commit 5491ffb.

This patch fails to generate documentation.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2020-06-30T17:35:13Z

Sure, you can do that in the custom release.

Also, we plan to use Prometheus + Grafana with proposed format change (through custom release).

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala

SparkQA · 2020-06-30T19:14:32Z

Test build #124689 has finished for PR 28865 at commit 6dd5837.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

erenavsarogullari · 2020-06-30T23:15:06Z

Retest this please

SparkQA · 2020-06-30T23:21:34Z

Test build #124698 has finished for PR 28865 at commit 6dd5837.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

erenavsarogullari · 2020-07-01T17:12:09Z

Retest this please

SparkQA · 2020-07-01T17:21:52Z

Test build #124802 has finished for PR 28865 at commit 6dd5837.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

erenavsarogullari · 2020-07-01T17:30:18Z

Failure seems irrelevant

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:3.0.0-M1:install (default-cli) on project spark-parent_2.12: ArtifactInstallerException: Failed to install metadata org.apache.spark:spark-parent_2.12/maven-metadata.xml: Could not parse metadata /home/jenkins/.m2/repository/org/apache/spark/spark-parent_2.12/maven-metadata-local.xml: in epilog non whitespace content is not allowed but got t (position: END_TAG seen ...</metadata>\nt... @26:2) -> [Help 1]

dongjoon-hyun · 2020-07-01T18:20:05Z

Ya. I'll test it manually, @erenavsarogullari . Thanks for updating.

dongjoon-hyun

+1, LGTM. Thank you, @erenavsarogullari .
I tested this new test suite locally. Merged to master branch for Apache Spark 3.1 on December 2020.

erenavsarogullari · 2020-07-01T18:52:06Z

Thanks @dongjoon-hyun for follow-up and merge.

Support consistency on Prometheus driver and executor metrics

7f31529

probot-autolabeler bot added the CORE label Jun 19, 2020

erenavsarogullari changed the title ~~[SPARK-32026][CORE]Support consistency on Prometheus driver and executor metrics format~~ [SPARK-32026][CORE] Support consistency on Prometheus driver and executor metrics format Jun 19, 2020

dongjoon-hyun requested changes Jun 19, 2020

View reviewed changes

dongjoon-hyun reviewed Jun 27, 2020

View reviewed changes

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala Show resolved Hide resolved

dongjoon-hyun requested changes Jun 27, 2020

View reviewed changes

erenavsarogullari changed the title ~~[SPARK-32026][CORE] Support consistency on Prometheus driver and executor metrics format~~ [SPARK-32026][TEST] Add PrometheusServlet Unit Test coverage Jun 29, 2020

erenavsarogullari added 3 commits June 29, 2020 16:21

Address review comments

1274371

Fixing scalastyle issue

87bca97

Fixing scalastyle issue

83d746b

dongjoon-hyun reviewed Jun 30, 2020

View reviewed changes

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala Outdated Show resolved Hide resolved

dongjoon-hyun reviewed Jun 30, 2020

View reviewed changes

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala Outdated Show resolved Hide resolved

dongjoon-hyun reviewed Jun 30, 2020

View reviewed changes

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala Outdated Show resolved Hide resolved

dongjoon-hyun reviewed Jun 30, 2020

View reviewed changes

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala Outdated Show resolved Hide resolved

dongjoon-hyun changed the title ~~[SPARK-32026][TEST] Add PrometheusServlet Unit Test coverage~~ [SPARK-32026][TEST] Add PrometheusServletSuite Jun 30, 2020

dongjoon-hyun changed the title ~~[SPARK-32026][TEST] Add PrometheusServletSuite~~ [SPARK-32026][CORE][TEST] Add PrometheusServletSuite Jun 30, 2020

dongjoon-hyun added the TESTS label Jun 30, 2020

erenavsarogullari added 2 commits June 29, 2020 18:16

Addressing review comments

bf78996

Fixing scalastyle issue

5491ffb

dongjoon-hyun reviewed Jun 30, 2020

View reviewed changes

core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala Outdated Show resolved Hide resolved

Refactoring PrometheusServlet creation

6dd5837

dongjoon-hyun approved these changes Jul 1, 2020

View reviewed changes

dongjoon-hyun closed this in f843a5b Jul 1, 2020

[SPARK-32026][CORE][TEST] Add PrometheusServletSuite #28865

[SPARK-32026][CORE][TEST] Add PrometheusServletSuite #28865

Uh oh!

Conversation

erenavsarogullari commented Jun 19, 2020 • edited by dongjoon-hyun Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

erenavsarogullari commented Jun 19, 2020

Uh oh!

SparkQA commented Jun 19, 2020

Uh oh!

dongjoon-hyun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

erenavsarogullari commented Jun 24, 2020

Uh oh!

dongjoon-hyun commented Jun 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Jun 30, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dongjoon-hyun commented Jun 30, 2020

Uh oh!

dongjoon-hyun commented Jun 30, 2020

Uh oh!

erenavsarogullari commented Jun 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Jun 30, 2020

Uh oh!

dongjoon-hyun commented Jun 30, 2020

Uh oh!

Uh oh!

SparkQA commented Jun 30, 2020

Uh oh!

erenavsarogullari commented Jun 30, 2020

Uh oh!

SparkQA commented Jun 30, 2020

Uh oh!

erenavsarogullari commented Jul 1, 2020

Uh oh!

SparkQA commented Jul 1, 2020

Uh oh!

erenavsarogullari commented Jul 1, 2020

Uh oh!

dongjoon-hyun commented Jul 1, 2020

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

erenavsarogullari commented Jul 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

erenavsarogullari commented Jun 19, 2020 •

edited by dongjoon-hyun

Loading

dongjoon-hyun left a comment •

edited

Loading

dongjoon-hyun commented Jun 27, 2020 •

edited

Loading

erenavsarogullari commented Jun 30, 2020 •

edited

Loading

erenavsarogullari commented Jul 1, 2020 •

edited

Loading