Skip to content

[SPARK-28475][CORE] Add regex MetricFilter to GraphiteSink #25232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nkarpov wants to merge 5 commits into from
Closed

[SPARK-28475][CORE] Add regex MetricFilter to GraphiteSink #25232

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
eaf37c8
Add regex filter to GraphiteSink
nkarpov Jul 22, 2019
d0543a0
readding removed empty line separating spark & 3rd party libs
nkarpov Jul 22, 2019
4314fa7
adding tests
nkarpov Jul 23, 2019
72754cd
added better regex test
nkarpov Jul 24, 2019
768777a
add new config to doc & template
nkarpov Jul 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions core/src/main/scala/org/apache/spark/metrics/sink/GraphiteSink.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,8 @@ package org.apache.spark.metrics.sink
import java.util.{Locale, Properties}
import java.util.concurrent.TimeUnit

import com.codahale.metrics.MetricRegistry
import com.codahale.metrics.{Metric, MetricFilter, MetricRegistry}
import com.codahale.metrics.graphite.{Graphite, GraphiteReporter, GraphiteUDP}

@dongjoon-hyun dongjoon-hyun Jul 22, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This empty line is one of the required Apache Spark coding styles.

[error] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/main/scala/org/apache/spark/metrics/sink/GraphiteSink.scala:25:0: There should at least one a single empty line separating groups 3rdParty and spark.

You can run the code style checker by the following.

$ dev/scalastyle

import org.apache.spark.SecurityManager
import org.apache.spark.metrics.MetricsSystem

Expand All @@ -38,6 +37,7 @@ private[spark] class GraphiteSink(val property: Properties, val registry: Metric
val GRAPHITE_KEY_UNIT = "unit"
val GRAPHITE_KEY_PREFIX = "prefix"
val GRAPHITE_KEY_PROTOCOL = "protocol"
val GRAPHITE_KEY_REGEX = "regex"

def propertyToOption(prop: String): Option[String] = Option(property.getProperty(prop))

Expand Down Expand Up @@ -72,10 +72,20 @@ private[spark] class GraphiteSink(val property: Properties, val registry: Metric
case Some(p) => throw new Exception(s"Invalid Graphite protocol: $p")
}

val filter = propertyToOption(GRAPHITE_KEY_REGEX) match {
case Some(pattern) => new MetricFilter() {
override def matches(name: String, metric: Metric): Boolean = {
pattern.r.findFirstMatchIn(name).isDefined
}
}
case None => MetricFilter.ALL
}

val reporter: GraphiteReporter = GraphiteReporter.forRegistry(registry)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.convertRatesTo(TimeUnit.SECONDS)
.prefixedWith(prefix)
.filter(filter)
.build(graphite)

override def start() {
Expand Down