-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-39944][BUILD] Upgrade dropwizard metrics to 4.2.10 #37372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
dongjoon-hyun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, @LuciferYang . Please update the doc together.
$ git grep 4.2.0 docs/monitoring.md
docs/monitoring.md:Executor memory metrics are also exposed via the Spark metrics system based on the [Dropwizard metrics library](http://metrics.dropwizard.io/4.2.0).
docs/monitoring.md:[Dropwizard Metrics Library](http://metrics.dropwizard.io/4.2.0).
docs/monitoring.md:see [Dropwizard library documentation for details](https://metrics.dropwizard.io/4.2.0/getting-started.html).
docs/monitoring.md: [Dropwizard/Codahale Metric Sets for JVM instrumentation](https://metrics.dropwizard.io/4.2.0/manual/jvm.html)
|
@dongjoon-hyun There should be no new page to update. https://metrics.dropwizard.io/4.2.0/ seems to be the latest, there is no In addition, do we need to change |
|
Oh, got it. It's too bad for us because we cannot have a matched doc inevitably. For |
OK, will update this later |
done |
|
I think the error is unrelated, but can we trigger tests again here? |
|
Sparkr test failed, and it should not pass all now, master has the same problem https://github.com/apache/spark/runs/7643052356?check_suite_focus=true |
|
Yes, #37399 is trying to fix the SparkR failure . |
|
thanks @dongjoon-hyun @HyukjinKwon @srowen |
### What changes were proposed in this pull request? Bump Dropwizard version from 3.2.6 to 4.2.25. Meanwhile, introduce `metrics_jvm_thread_peak_count_Value` and `metrics_jvm_thread_total_started_count_Value` in `celeborn-jvm-dashboard.json`. ### Why are the changes needed? Dropwizard metrics has released v4.2.25 including some bugfixes and improvements including: * [JVM] Fix maximum/total memory calculation: dropwizard/metrics#3125 * [Thread] Add peak and total started thread count to `ThreadStatesGaugeSet`: dropwizard/metrics#1601 Meanwhile, Ratis version has upgraded to 3.0.1 which has no compatibility problem with Dropwizard 4.2.25. Backport: - apache/spark#26332 - apache/spark#29426 - apache/spark#37372 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manual test. Closes #2540 from SteNicholas/CELEBORN-1389. Authored-by: SteNicholas <[email protected]> Signed-off-by: mingji <[email protected]>
### What changes were proposed in this pull request? Bump Dropwizard version from 3.2.6 to 4.2.25. Meanwhile, introduce `metrics_jvm_thread_peak_count_Value` and `metrics_jvm_thread_total_started_count_Value` in `celeborn-jvm-dashboard.json`. ### Why are the changes needed? Dropwizard metrics has released v4.2.25 including some bugfixes and improvements including: * [JVM] Fix maximum/total memory calculation: dropwizard/metrics#3125 * [Thread] Add peak and total started thread count to `ThreadStatesGaugeSet`: dropwizard/metrics#1601 Meanwhile, Ratis version has upgraded to 3.0.1 which has no compatibility problem with Dropwizard 4.2.25. Backport: - apache/spark#26332 - apache/spark#29426 - apache/spark#37372 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manual test. Closes #2540 from SteNicholas/CELEBORN-1389. Authored-by: SteNicholas <[email protected]> Signed-off-by: mingji <[email protected]> (cherry picked from commit 4fc42d7) Signed-off-by: mingji <[email protected]>

What changes were proposed in this pull request?
This pr upgrade dropwizard metrics from 4.2.7 to 4.2.10 and changes the links http://metrics.dropwizard.io/4.2.0 in
docs/monitoring.mdto https://metrics.dropwizard.io/4.2.0Why are the changes needed?
There are 3 versions after 4.2.7, the release notes as follows:
The new version brings a new API for more type safe of registering gauges(dropwizard/metrics#2642)
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass GitHub Actions