Skip to content

Commit

Permalink
Finished the disable feature, updated documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
brianhks committed Dec 17, 2019
1 parent 8eb0af2 commit 1b40fb4
Show file tree
Hide file tree
Showing 15 changed files with 220 additions and 49 deletions.
48 changes: 44 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This library is still in development, keep checking back as progress is moving q
* report a metric to more than one timeseries backend
* have an application report to something besides prometheus

on an application already deployed in production? Then this library is for your
All of the above on an application already deployed in production? Then this library is for your
(or the developers that wrote the application)

## Philosophy of using Metrics4j
Expand Down Expand Up @@ -167,6 +167,7 @@ The top level of the configuration file looks like this
metrics4j: {
sources: {
#Defines sources and associates sinks, collectors, formatters and triggers with them
#Sources are typically places in code that reports metrics.
}
sinks: {
Expand All @@ -178,7 +179,7 @@ metrics4j: {
}
formatters: {
#Can reformat metric names and tags when reporting
#Can reformat metric names when reporting
}
triggers: {
Expand Down Expand Up @@ -265,6 +266,10 @@ that is convenient for you.
For each source (ie. reportSize()) you can define a collector, a trigger, a formatter and
one or more sinks.

In order to get a metric to report you must define a `_collector`, a `_sink` and a `_trigger`
reference. The values of `_collector` and `_sink` can be either a string value or a list
of strings if you want to reference more than one at the same level.

###### Overrides

Just a quick note on overriding values using Hocon. Lets say in the above example I want
Expand Down Expand Up @@ -351,12 +356,30 @@ metrics4j: {
_dump_file: "dump_sources.conf"
}
```
Start the application and let it run for a bit and then shut it down. Metrics4j
will dump out all the sources it saw while running.

##### Disabling sources

If a source hasn't been configured with a collector it will not report but sometimes
it is easier to disable portions of the source tree. You can disable any part of the
source tree by adding a `_disable: true` at the level you wish to disable. Disabled
sources can be overridden by adding `_disable: false` further down the tree.

### Sinks
A sink defines a destination to send the metrics to.
A sink defines a destination to send the metrics to. The following are built in
sinks.

#### Slf4JMetricSink
Reports metrics to an Slf4j logger. The log-level attribute controls the log level (DEBUG, WARN, INFO, etc).
```hocon
sinks: {
slf4j: {
_class: "org.kairosdb.metrics4j.sinks.Slf4JMetricSink"
log-level: INFO
}
}
```

#### TelnetSink

Expand Down Expand Up @@ -428,6 +451,22 @@ triggers: {
}
}
```
#### KairosSink

The Kairos sink is a separate jar that needs to be placed in the classpath along
with its dependencies.

```hocon
kairos: {
_class: "org.kairosdb.metrics4jplugin.kairosdb.KairosSink"
host-url: "http://192.168.1.55:8080"
#telnet-host: "192.168.1.55"
#telnet-port: 4242
}
```

Depending on which properties are set the sink will send either http or telnet using
the kairosdb client.

#### TimescaleDBSink

Expand All @@ -437,7 +476,8 @@ TODO: https://docs.timescale.com/latest/using-timescaledb/writing-data
A collector defines how to collect values from a source. For reportSize() I could
use a LongCounter or a LongGauge. When looking for a collector for a source metrics4j
will match the type so if you define both a LongCounter and a DoubleCounter it will
know to grab the LongCounter as it inherits from LongCollector.
know to grab the LongCounter as it inherits from LongCollector. The following
collectors are all in the `org.kairosdb.metrics4j.collectors` package.

#### DoubleCounter
Counts up double values to be reported. The counter can be reset when values are reported
Expand Down
4 changes: 2 additions & 2 deletions metrics4j/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
<parent>
<groupId>org.kairosdb</groupId>
<artifactId>metrics4j-all</artifactId>
<version>1.0</version>
<version>1.0.1</version>
</parent>

<name>metrics4j-all</name>
<name>metrics4j</name>
<description>Java library for reporting metrics to any timeseries database.</description>
<url>https://github.com/kairosdb/metrics4j</url>

Expand Down
10 changes: 0 additions & 10 deletions metrics4j/src/main/java/org/kairosdb/metrics4j/Main.java

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ private static TagKey buildTagKey(Map<String, String> tags)
return builder.build();
}

private static void addSource(String className, String methodName, Map<String, String> tags, String help, MetricCollector collector)
public static void addSource(String className, String methodName, Map<String, String> tags, String help, MetricCollector collector)
{
ArgKey key = new LambdaArgKey(className, methodName);
MetricConfig metricConfig = getMetricConfig();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import com.typesafe.config.ConfigFactory;
import com.typesafe.config.ConfigRenderOptions;
import com.typesafe.config.ConfigValue;
import com.typesafe.config.ConfigValueType;
import org.kairosdb.metrics4j.MetricsContext;
import org.kairosdb.metrics4j.internal.ArgKey;
import org.kairosdb.metrics4j.internal.BeanInjector;
Expand Down Expand Up @@ -45,7 +46,7 @@ public class MetricConfig
private final Map<List<String>, Map<String, String>> m_mappedTags;
private final Map<List<String>, Map<String, String>> m_mappedProps;
private final Map<List<String>, String> m_mappedMetricNames;
private final Set<List<String>> m_disabledPaths;
private final Map<List<String>, Boolean> m_disabledPaths;

private final MetricsContextImpl m_context;
private final List<Closeable> m_closeables;
Expand Down Expand Up @@ -224,15 +225,44 @@ private void parseSources(Config root)
}
else if (internalProp.equals("_sink"))
{
//need to map to a list of sinks as there can be more than one
String ref = root.getString(combinePath(path, i));
//sink can be either a single string or a list of string
String sinkPath = combinePath(path, i);

ConfigValueType sinkValueType = root.getValue(sinkPath).valueType();
if (sinkValueType == ConfigValueType.STRING)
{
String ref = root.getString(sinkPath);
m_context.addSinkToPath(ref, createList(path, i-1));
}
else if (sinkValueType == ConfigValueType.LIST)
{
List<String> sinkList = root.getStringList(sinkPath);
for (String sink : sinkList)
{
m_context.addSinkToPath(sink, createList(path, i-1));
}
}

m_context.addSinkToPath(ref, createList(path, i-1));
}
else if (internalProp.equals("_collector"))
{
String ref = root.getString(combinePath(path, i));
m_context.addCollectorToPath(ref, createList(path, i-1));
//collector can be either a single string or a list of string
String collectorPath = combinePath(path, i);

ConfigValueType collectorValueType = root.getValue(collectorPath).valueType();
if (collectorValueType == ConfigValueType.STRING)
{
String ref = root.getString(collectorPath);
m_context.addCollectorToPath(ref, createList(path, i-1));
}
else if (collectorValueType == ConfigValueType.LIST)
{
List<String> collectorList = root.getStringList(collectorPath);
for (String collector : collectorList)
{
m_context.addCollectorToPath(collector, createList(path, i-1));
}
}
}
else if (internalProp.equals("_formatter"))
{
Expand Down Expand Up @@ -264,7 +294,8 @@ else if (internalProp.equals("_prop"))
}
else if (internalProp.equals("_disabled"))
{
m_disabledPaths.add(createList(path, i - 1));
Boolean value = (Boolean)entry.getValue().unwrapped();
m_disabledPaths.put(createList(path, i - 1), value);
}
else
{
Expand Down Expand Up @@ -331,7 +362,7 @@ public MetricConfig(MetricsContextImpl context)
m_mappedTags = new HashMap<>();
m_mappedProps = new HashMap<>();
m_mappedMetricNames = new HashMap<>();
m_disabledPaths = new HashSet<>();
m_disabledPaths = new HashMap<>();


Runtime.getRuntime().addShutdownHook(new Thread(new Runnable()
Expand Down Expand Up @@ -413,7 +444,19 @@ public void setProperties(Properties properties)

public boolean isDisabled(ArgKey argKey)
{
return m_disabledPaths.contains(argKey.getConfigPath());
List<String> configPath = argKey.getConfigPath();
for (int i = configPath.size(); i >= 0; i--)
{
List<String> searchPath = new ArrayList<>(configPath.subList(0, i));

Boolean disabled = m_disabledPaths.get(searchPath);
if (disabled != null)
{
return disabled;
}
}

return false;
}

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
package org.kairosdb.metrics4j.internal;

import lombok.ToString;
import org.kairosdb.metrics4j.collectors.MetricCollector;

import java.util.ArrayList;
import java.util.List;

@ToString
public class CustomArgKey implements ArgKey
{
private final MetricCollector m_collector;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
package org.kairosdb.metrics4j.internal;

import lombok.ToString;

import java.util.ArrayList;
import java.util.List;

@ToString
public class LambdaArgKey implements ArgKey
{
private final String m_className;
Expand All @@ -24,6 +27,9 @@ public List<String> getConfigPath()
ret.add(s);
}

if (m_methodName != null)
ret.add(m_methodName);

return ret;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import java.util.Map;
import java.util.Objects;

@ToString
public class MethodArgKey implements ArgKey
{
private final Method m_method;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CopyOnWriteArraySet;

/**
build this class and then add it to the trigger
Expand All @@ -24,8 +26,8 @@ public TriggerMetricCollection(Trigger trigger)
{
m_trigger = trigger;
m_trigger.setMetricCollection(this);
m_collectors = new ArrayList<>();
m_sinkQueues = new HashSet<>();
m_collectors = new CopyOnWriteArrayList<>();
m_sinkQueues = new CopyOnWriteArraySet<>();
}

public Trigger getTrigger()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@
import org.kairosdb.metrics4j.collectors.DoubleCounter;
import org.kairosdb.metrics4j.collectors.LongCounter;
import org.kairosdb.metrics4j.formatters.DefaultFormatter;
import org.kairosdb.metrics4j.internal.ArgKey;
import org.kairosdb.metrics4j.internal.CustomArgKey;
import org.kairosdb.metrics4j.internal.FormattedMetric;
import org.kairosdb.metrics4j.internal.LambdaArgKey;
import org.kairosdb.metrics4j.internal.MetricsContextImpl;
import org.kairosdb.metrics4j.internal.ReportedMetricImpl;
import org.kairosdb.metrics4j.reporting.LongValue;
Expand Down Expand Up @@ -151,5 +154,26 @@ public void test_appendSourceName()
assertThat(newPath).containsExactly("org", "kairosdb", "test", "MyClass", "myMethod");
}

@Test
public void testDisabledConfiguration()
{
MetricConfig metricConfig = MetricConfig.parseConfig("test_disabled.conf", "Not_there");

ArgKey key = new LambdaArgKey("java.lang.ClassLoading", "LoadedClassCount");
assertThat(metricConfig.isDisabled(key)).isEqualTo(true);

key = new LambdaArgKey("java.lang.GarbageCollector", "CollectionTime");
assertThat(metricConfig.isDisabled(key)).isEqualTo(false);

key = new LambdaArgKey("java.lang.MemoryPool", "UsageThresholdCount");
assertThat(metricConfig.isDisabled(key)).isEqualTo(false);

key = new LambdaArgKey("java.nio.BufferPool", "Count");
assertThat(metricConfig.isDisabled(key)).isEqualTo(false);

key = new LambdaArgKey("org.kairosdb.jmxreporter.JMXReporter", "something");
assertThat(metricConfig.isDisabled(key)).isEqualTo(false);
}


}
Loading

0 comments on commit 1b40fb4

Please sign in to comment.