diff --git a/INSTALL.md b/INSTALL.md index dda1ccf6e7..70aba7dba1 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -45,12 +45,12 @@ You can link OpenTelemetry C++ SDK with libraries provided in ### Building as standalone CMake Project -1. Getting the opentelementry-cpp source: +1. Getting the opentelementry-cpp source with its submodules: ```console # Change to the directory where you want to create the code repository $ cd ~ - $ mkdir source && cd source && git clone --recursive https://github.com/open-telemetry/opentelemetry-cpp + $ mkdir source && cd source && git clone --recurse-submodules https://github.com/open-telemetry/opentelemetry-cpp Cloning into 'opentelemetry-cpp'... ... Resolving deltas: 100% (3225/3225), done. diff --git a/docs/cpp-metrics-api-design.md b/docs/cpp-metrics-api-design.md deleted file mode 100644 index 08c8a981e6..0000000000 --- a/docs/cpp-metrics-api-design.md +++ /dev/null @@ -1,615 +0,0 @@ -# Metrics API Design - -This document outlines a proposed implementation of the OpenTelemetry Metrics -API in C++. The design conforms to the current versions of the [Metrics API -Specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/api.md) -though it is currently under development and subject to change. - -The design supports a minimal implementation for the library to be used by an -application. However, without the reference SDK or another implementation, no -metric data will be collected. - -## Use Cases - -A *metric* is some raw measurement about a service, captured at run-time. -Logically, the moment of capturing one of these measurements is known as a -*metric event* which consists not only of the measurement itself, but the time -that it was captured as well as contextual annotations which tie it to the event -being measured. Users can inject instruments which facilitate the collection of -these measurements into their services or systems which may be running locally, -in containers, or on distributed platforms. The data collected are then used by -monitoring and alerting systems to provide statistical performance data. - -Monitoring and alerting systems commonly use the data provided through metric -events, after applying various aggregations and converting into various -exposition formats. However, we find that there are many other uses for metric -events, such as to record aggregated or raw measurements in tracing and logging -systems. For this reason, OpenTelemetry requires a separation of the API from -the SDK, so that different SDKs can be configured at run time. - -Various instruments also allow for more optimized capture of certain types of -measurements. `Counter` instruments, for example, are monotonic and can -therefore be used to capture rate information. Other potential uses for the -`Counter` include tracking the number of bytes received, requests completed, -accounts created, etc. - -A `ValueRecorder` is commonly used to capture latency measurements. Latency -measurements are not additive in the sense that there is little need to know the -latency-sum of all processed requests. We use a `ValueRecorder` instrument to -capture latency measurements typically because we are interested in knowing -mean, median, and other summary statistics about individual events. - -`Observers` are a good choice in situations where a measurement is expensive to -compute, such that it would be wasteful to compute on every request. For -example, a system call is needed to capture process CPU usage, therefore it -should be done periodically, not on each request. - -## Design Tenets - -* Reliability - * The Metrics API and SDK should be “reliable,” meaning that metrics data will - always be accounted for. It will get back to the user or an error will be - logged. Reliability also entails that the end-user application will never - be blocked. Error handling will therefore not interfere with the execution - of the instrumented program. - * Thread Safety - * As with the Tracer API and SDK, thread safety is not guaranteed on all - functions and will be explicitly mentioned in documentation for functions - that support concurrent calling. Generally, the goal is to lock functions - which change the state of library objects (incrementing the value of a - Counter or adding a new Observer for example) or access global memory. As - a performance consideration, the library strives to hold locks for as - short a duration as possible to avoid lock contention concerns. Calls to - create instrumentation may not be thread-safe as this is expected to occur - during initialization of the program. -* Scalability - * As OpenTelemetry is a distributed tracing system, it must be able to operate - on sizeable systems with predictable overhead growth. A key requirement of - this is that the library does not consume unbounded memory resource. -* Security - * Currently security is not a key consideration but may be addressed at a - later date. - -## **Meter Interface (`MeterProvider` Class)** - -The singleton global `MeterProvider` can be used to obtain a global Meter by -calling `global.GetMeter(name,version)` which calls `GetMeter()` on the -initialized global `MeterProvider` - -**Global Meter Provider:** - -The API should support a global `MeterProvider`. When a global instance is -supported, the API must ensure that `Meter` instances derived from the global -`MeterProvider` are initialized after the global SDK implementation is first -initialized. - -A `MeterProvider` interface must support a -`global.SetMeterProvider(MeterProvider)` function which installs the SDK -implementation of the `MeterProvider` into the API - -**Obtaining a Meter from MeterProvider:** - -**`GetMeter(name, version)` method must be supported** - -* Expects 2 string arguments: - * name (required): identifies the instrumentation scope. - * version (optional): specifies the version of the instrumenting library (the - library injecting OpenTelemetry calls into the code) - -```cpp -# meter_provider.h -class Provider -{ -public: - /* - * Get Meter Provider - * - * Returns the singleton MeterProvider. By default, a no-op MeterProvider - * is returned. This will never return a nullptr MeterProvider. - * - */ - static nostd::shared_ptr GetMeterProvider(); - { - GetProvider(); - } - - /* - * Set Meter Provider - * - * Changes the singleton MeterProvider. - * - * Arguments: - * newMeterProvider, the MeterProvider instance to be set as the new global - * provider. - */ - static void SetMeterProvider(nostd::shared_ptr newMeterProvider); - -private: - /* - * Get Provider - * - * Returns a no-op MeterProvider. - * - */ - static nostd::shared_ptr &GetProvider() noexcept - { - return DefaultMeterProvider(); - } - -}; -``` - -```cpp -# meter_provider.h -class MeterProvider -{ -public: - /* - * Get Meter - * - * Gets or creates a named meter instance. - * - * Arguments: - * library_name, the name of the instrumenting library. - * library_version, the version of the instrumenting library (OPTIONAL). - */ - nostd::shared_ptr GetMeter(nostd::string_view library_name, - nostd::string_view library_version = "") - -}; -``` - -Using this MeterProvider, users can obtain new Meters through the GetMeter -function. - -## Metric Instruments (`Meter` Class) - -**Metric Events:** - -This interface consists of a set of **instrument constructors**, and a -**facility for capturing batches of measurements.** - -``` cpp -# meter.h -class Meter { -public: - -/////////////////////////Metric Instrument Constructors//////////////////////////// - - /* - * New Counter - * - * Function that creates and returns a Counter metric instruent - * - * Arguments: - * name, the name of the metric instrument (must conform to the above syntax). - * description, a brief, readable description of the metric instrument. - * unit, the unit of metric values following the UCUM convention - * (https://unitsofmeasure.org/ucum.html). - * enabled, a boolean that turns on or off collection. - * - */ - virtual nostd::shared_ptr> - NewShortCounter(nostd::string_view name, - nostd::string_view description, - nostd::string_view unit, - nostd::string_view enabled) = 0; - - virtual nostd::shared_ptr> - NewIntCounter(nostd::string_view name, - nostd::string_view description, - nostd::string_view unit, - nostd::string_view enabled) = 0; - - virtual nostd::shared_ptr> - NewFloatCounter(nostd::string_view name, - nostd::string_view description, - nostd::string_view unit, - nostd::string_view enabled) = 0; - - virtual nostd::shared_ptr> - NewDoubleCounter(nostd::string_view name, - nostd::string_view description, - nostd::string_view unit, - nostd::string_view enabled) = 0; - - -//////////////////////////////////////////////////////////////////////////////////// -// // -// Repeat above functions for short, int, float, and // -// double type versions of all 6 metric instruments. // -// // -//////////////////////////////////////////////////////////////////////////////////// - /* - * RecordBatch - * - * Allows the functionality of acting upon multiple metrics with the same set - * of labels with a single API call. Implementations should find bound metric - * instruments that match the key-value pairs in the labels. - * - * Arguments: - * labels, labels associated with all measurements in the batch. - * instruments, a span of pointers to instruments to record to. - * values, a synchronized span of values to record to those instruments. - * - */ - virtual void RecordIntBatch(nostd::KeyValueIterable labels, - nostd::span>> instruments, - nostd::span values) noexcept; - - - /* - * Overloaded RecordBatch function which takes initializer lists of pairs. - * Provided to improve ease-of-use of the BatchRecord function. - * - */ - template::value, int> = 0> - void RecordBatch(std::initializer_list> labels, - std::initializer_list>, - int>> values) - { - // Translate parameters - // return RecordIntBatch(@ translated_params ); - } - -private: - MeterProvider meterProvider_; - InstrumentationInfo instrumentationInfo_; -} -``` - -### Meter API Class Design Considerations - -According to the specification, both signed integer and floating point value -types must be supported. This implementation will use short, int, float, and -double types. Different constructors are used for the different metric -instruments and even for different value types due to C++ being a strongly typed -language. This is similar to Java’s implementation of the meter class. Python -gets around this by passing the value type and metric type to a single function -called `create_metric`. - -## Instrument Types (`Metric` Class) - -Metric instruments capture raw measurements of designated quantities in -instrumented applications. All measurements captured by the Metrics API are -associated with the instrument which collected that measurement. These -instruments are also templated allowing users to decide which data type to -capture. This enhances user control over the memory used by their instrument -set and provides greater precision when necessary. - -### Metric Instrument Data Model - -Each instrument must have enough information to meaningfully attach its measured -values with a process in the instrumented application. As such, metric -instruments contain the following information: - -* name (string) — Identifier for this metric instrument. -* description (string) — Short description of what this instrument is capturing. -* value_type (string or enum) — Determines whether the value tracked is an int64 - or double. -* meter (Meter) — The Meter instance from which this instrument was derived. -* label_keys (KeyValueIterable) — A nostd class acting as a map from - nostd::string_view to nostd::string_view -* enabled (boolean) — Determines whether the instrument is currently collecting - data. -* bound_instruments (key value container) — Contains the bound instruments - derived from this instrument. - -Metric instruments are created through instances of the `Meter` class and each -type of instrument can be described with the following properties: - -* Synchronicity: A synchronous instrument is called by the user in a - distributed - [Context](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/context/context.md) - (i.e., Span context, Correlation context) and is updated once per request. An - asynchronous instrument is called by the SDK once per collection interval and - only one value from the interval is kept. -* Additivity: An additive instrument is one that records additive measurements, - meaning the final sum of updates is the only useful value. Non-additive - instruments should be used when the intent is to capture information about the - distribution of values. -* Monotonicity: A monotonic instrument is an additive instrument, where the - progression of each sum is non-decreasing. Monotonic instruments are useful - for monitoring rate information. - -The following instrument types will be supported: - - -### Metric Event Data Model - -Each measurement taken by a Metric instrument is a Metric event which must -contain the following information: - -* timestamp (implicit) — System time when measurement was captured. -* instrument definition(strings) — Name of instrument, kind, description, and - unit of measure -* label set (key value pairs) — Labels associated with the capture, described - further below. -* resources associated with the SDK at startup - -**Label Set:** - -A key:value mapping of some kind MUST be supported as annotation each metric -event. Labels must be represented the same way throughout the API (i.e. using -the same idiomatic data structure) and duplicates are dealt with by taking the -last value mapping. - -To maintain ABI stability, we have chosen to implement this as a -KeyValueIterable type. However, due to performance concerns, we may convert to a -std::string internally. - -**Calling Conventions:** - -Metric instruments must support bound instrument calling where the labels for -each capture remain the same. After a call to `instrument.Bind(labels)` , all -subsequent calls to `instrument.add()` will include the labels implicitly in -their capture. - -Direct calling must also be supported. The user can specify labels with the -capture rather than binding beforehand by including the labels in the update -call: `instrument.Add(x, labels)`. - -MUST support `RecordBatch` calling (where a single set of labels is applied to -several metric instruments). - -```cpp -# metric.h - -/* - * Enum classes to hold the various types of Metric Instruments and their - * bound complements. - */ -enum class MetricKind -{ - Counter, - UpDownCounter, - ValueRecorder, - SumObserver, - UpDownSumObserver, - ValueObserver, -}; - - /* - * Instrument - * - * Base class for all metric types. - * - * Also known as metric instrument. This is the class that is used to - * represent a metric that is to be continuously recorded and tracked. Each - * metric has a set of bound metrics that are created from the metric. See - * `BoundSychnronousInstrument` for information on bound metric instruments. - */ -class Instrument { -public: - // Note that Instruments should be created using the Meter class. - // Please refer to meter.h for documentation. - Instrument() = default; - - /** - * Base class constructor for all other instrument types. Whether or not - * an instrument is synchronous or bound, it requires a name, description, - * unit, and enabled flag. - * - * @param name is the identifier of the instrumenting library - * @param description explains what the metric captures - * @param unit specified the data type held in the instrument - * @param enabled determines if the metric is currently capturing data - * @return Instrument type with the specified attributes - */ - Instrument(nostd::string_view name, nostd::string_view description, nostd::string_view unit, bool enabled); - - // Returns true if the instrument is enabled and collecting data - bool IsEnabled(); - - // Return the instrument name - nostd::string_view GetName(); - - // Return the instrument description - nostd::string_view GetDescription(); - - // Return the insrument's units of measurement - nostd::string_view GetUnits(); - - // Return the kind of the instrument e.g. Counter - InstrumentKind GetKind(); -}; -``` - -```cpp -template -class SynchronousInstrument: public Instrument { -public: - SynchronousInstrument() = default; - - SynchronousInstrument(nostd::string_view name, nostd::string_view description, nostd::string_view unit, - bool enabled); - - /** - * Returns a Bound Instrument associated with the specified labels. - * Multiples requests with the same set of labels may return the same - * Bound Instrument instance. - * - * It is recommended that callers keep a reference to the Bound Instrument instead of always - * calling this method for every operation. - * - * @param labels the set of labels, as key-value pairs. - * @return a Bound Instrument - */ - BoundSynchronousInstrument bind(nostd::KeyValueIterable labels); - - /** - * Records a single synchronous metric event. - * Since this is an unbound synchronous instrument, labels are required in * metric capture calls. - * - * - * @param labels the set of labels, as key-value pairs. - * @param value the numerical representation of the metric being captured - * @return void - */ - void update(T value, nostd::KeyValueIterable labels); //add or record - -}; - -template -class BoundSynchronousInstrument: public Instrument { -public: - BoundSynchronousInstrument() = default; - - // Will also call the processor to acquire the correct aggregator for this instrument - BoundSynchronousInstrument(nostd::string_view name, nostd::string_view description, nostd::string_view unit, - bool enabled); - - /** - * Frees the resources associated with this Bound Instrument. - * The Metric from which this instrument was created is not impacted. - * - * @param none - * @return void - */ - void unbind(); - - /** - * Records a single synchronous metric event. //Call to aggregator - * Since this is a bound synchronous instrument, labels are notrequired in * metric capture calls. - * - * @param value the numerical representation of the metric being captured - * @return void - */ - void update(T value); //add or record - -}; - -template -class AsynchronousInstrument: public Instrument{ -public: - AsynchronousInstrument(nostd::string_view name, nostd::string_view description, nostd::string_view unit, - bool enabled, void (*callback)(ObserverResult)); - - /** - * Captures data by activating the callback function associated with the - * instrument and storing its return value. Callbacks for asynchronous - * instruments are defined during construction. - * - * @param none - * @return none - */ - void observe(); - - /** - * Captures data from the stored callback function. The callback itself - * makes use of the instrument's observe function to take capture - * responsibilities out of the user's hands. - * - * @param none - * @return none - */ - void run(); - -private: - - // Callback function which takes a pointer to an Asynchronous instrument (this) - // type which is stored in an observer result type and returns nothing. - // The function calls the instrument's observe method. - void (*callback_)(ObserverResult); -}; -``` - -The Counter below is an example of one Metric instrument. It is important to -note that in the Counter’s add function, it binds the labels to the instrument -before calling add, then unbinds. Therefore all interactions with the -aggregator take place through bound instruments and by extension, the -BaseBoundInstrument Class. - -```cpp -template -class BoundCounter: public BoundSynchronousInstrument{ //override bind? -public: - BoundCounter() = default; - BoundCounter(nostd::string_view name, nostd::string_view description, nostd::string_view unit, bool enabled); - - /* - * Add adds the value to the counter's sum. The labels are already linked - * to the instrument and are not specified. - * - * @param value the numerical representation of the metric being captured - * @param labels the set of labels, as key-value pairs - */ - void add(T value, nostd::KeyValueIterable labels); - - void unbind(); -}; - -template -class Counter: public SynchronousInstrument{ -public: - Counter() = default; - Counter(nostd::string_view name, nostd::string_view description, nostd::string_view unit, bool enabled); - - /* - * Bind creates a bound instrument for this counter. The labels are - * associated with values recorded via subsequent calls to Record. - * - * @param labels the set of labels, as key-value pairs. - * @return a BoundIntCounter tied to the specified labels - */ - BoundCounter bind(nostd::KeyValueIterable labels); - - /* - * Add adds the value to the counter's sum by sending to aggregator. The labels should contain - * the keys and values to be associated with this value. Counters only - * accept positive valued updates. - * - * @param value the numerical representation of the metric being captured - * @param labels the set of labels, as key-value pairs - */ - void add(T value, nostd::KeyValueIterable labels); -}; - -template -class ValueObserver: public AsynchronousInstrument{ -public: - /* - * Add adds the value to the counter's sum. The labels should contain - * the keys and values to be associated with this value. Counters only - * accept positive valued updates. - * - * @param value the numerical representation of the metric being captured - * @param labels the set of labels, as key-value pairs - */ - void observe(T value, KeyValueIterable &labels) override; -} -``` - -```cpp -// The above Counter and BoundCounter are examples of 1 metric instrument. -// The remaining 5 will also be implemented in a similar fashion. -class UpDownCounter: public SynchronousInstrument; -class BoundUpDownCounter: public BoundSynchronousInstrument; -class ValueRecorder: public SynchronousInstrument; -class BoundValueRecorder: public BoundSynchronousInstrument; -class SumObserver: public AsynchronousInstrument; -class BoundSumObserver: public AsynchronousInstrument; -class UpDownSumObserver: public AsynchronousInstrument; -class BoundUpDownSumObserver: public AsynchronousInstrument; -class ValueObserver: public AsynchronousInstrument; -class BoundValueObserver: public AsynchronousInstrument; -``` - -### Metric Class Design Considerations - -OpenTelemetry requires several types of metric instruments with very similar -core usage, but slightly different tracking schemes. As such, a base Metric -class defines the necessary functions for each instrument leaving the -implementation for the specific instrument type. Each instrument then inherits -from this base class making the necessary modifications. In order to facilitate -efficient aggregation of labeled data, a complementary BoundInstrument class is -included which attaches the same set of labels to each capture. Knowing that -all data in an instrument has the same labels enhances the efficiency of any -post-collection calculations as there is no need for filtering or separation. -In the above code examples, a Counter instrument is shown but all 6 mandated by -the specification will be supported. - -A base BoundInstrument class also serves as the foundation for more specific -bound instruments. It also facilitates the practice of reference counting which -can determine when an instrument is unused and can improve memory optimization -as inactive bound instruments can be removed for performance. diff --git a/docs/cpp-metrics-sdk-design.md b/docs/cpp-metrics-sdk-design.md deleted file mode 100644 index 5ff355046d..0000000000 --- a/docs/cpp-metrics-sdk-design.md +++ /dev/null @@ -1,817 +0,0 @@ -# Metrics SDK Design - -## Design Tenets - -* Reliability - * The Metrics API and SDK should be “reliable,” meaning that metrics data will - always be accounted for. It will get back to the user or an error will be - logged. Reliability also entails that the end-user application will never - be blocked. Error handling will therefore not interfere with the execution - of the instrumented program. The library may “fail fast” during the - initialization or configuration path however. - * Thread Safety - * As with the Tracer API and SDK, thread safety is not guaranteed on all - functions and will be explicitly mentioned in documentation for functions - that support concurrent calling. Generally, the goal is to lock functions - which change the state of library objects (incrementing the value of a - Counter or adding a new Observer for example) or access global memory. As - a performance consideration, the library strives to hold locks for as - short a duration as possible to avoid lock contention concerns. Calls to - create instrumentation may not be thread-safe as this is expected to occur - during initialization of the program. -* Scalability - * As OpenTelemetry is a distributed tracing system, it must be able to operate - on sizeable systems with predictable overhead growth. A key requirement of - this is that the library does not consume unbounded memory resource. -* Security - * Currently security is not a key consideration but may be addressed at a - later date. - -## SDK Data Path Diagram - - - -This is the control path our implementation of the metrics SDK will follow. -There are five main components: The controller, accumulator, aggregators, -processor, and exporter. Each of these components will be further elaborated on. - -## API Class Implementations - -### MeterProvider Class - -The singleton global `MeterProvider` can be used to obtain a global Meter by -calling `global.GetMeter(name,version)` which calls `GetMeter()` on the -initialized global `MeterProvider`. - -**Global Meter Provider:** - -The API should support a global `MeterProvider`. When a global instance is -supported, the API must ensure that `Meter` instances derived from the global -`MeterProvider` are initialized after the global SDK implementation is first -initialized. - -A `MeterProvider` interface must support a -`global.SetMeterProvider(MeterProvider)` function which installs the SDK -implementation of the `MeterProvider` into the API. - -**Obtaining a Meter from MeterProvider:** - -**`GetMeter(name, version)` method must be supported** - -* Expects 2 string arguments: - * name (required): identifies the instrumentation scope. - * version (optional): specifies the version of the instrumenting library (the - library injecting OpenTelemetry calls into the code). - -#### Implementation - -The Provider class offers static functions to both get and set the global -MeterProvider. Once a user sets the MeterProvider, it will replace the default -No-op implementation stored as a private variable and persist for the remainder -of the program’s execution. This pattern imitates the TracerProvider used in the -Tracing portion of this SDK. - -```cpp -# meter_provider.cc -class MeterProvider -{ -public: - /* - * Get Meter - * - * Gets or creates a named meter instance. - * - * Arguments: - * library_name, the name of the instrumenting library. - * library_version, the version of the instrumenting library (OPTIONAL). - */ - nostd::shared_ptr GetMeter(nostd::string_view library_name, - nostd::string_view library_version = "") { - - // Create an InstrumentationInfo object which holds the library name and version. - // Call the Meter constructor with InstrumentationInfo. - InstrumentationInfo instrumentationInfo; - instrumentationInfo.SetName(library_name); - if library_version: - instrumentationInfo.SetVersion(library_version); - return nostd::shared_ptr(Meter(this, instrumentationInfo)); - } - -}; -``` - -### Meter Class - -**Metric Events:** - -Metric instruments are primarily defined by their name. Names MUST conform to -the following syntax: - -* Non-empty string -* case-insensitive -* first character non-numeric, non-space, non-punctuation -* subsequent characters alphanumeric, ‘_’, ‘.’ , and ‘-’ - -`Meter` instances MUST return an error when multiple instruments with the same -name are registered - -**The meter implementation will throw an illegal argument exception if the -user-passed `name` for a metric instrument either conflicts with the name of -another metric instrument created from the same `meter` or violates the `name` -syntax outlined above.** - -Each distinctly named Meter (i.e. Meters derived from different instrumentation -libraries) MUST create a new namespace for metric instruments descending from -them. Thus, the same instrument name can be used in an application provided -they come from different Meter instances. - -**In order to achieve this, each instance of the `Meter` class will have a -container storing all metric instruments that were created using that meter. -This way, metric instruments created from different instantiations of the -`Meter` class will never be compared to one another and will never result in an -error.** - -**Implementation:** - -```cpp -# meter.h / meter.cc -class Meter : public API::Meter { -public: - /* - * Constructor for Meter class - * - * Arguments: - * MeterProvider, the MeterProvider object that spawned this Meter. - * InstrumentationInfo, the name of the instrumentation scope and, optionally, - * the version. - * - */ - explicit Meter(MeterProvider meterProvider, - InstrumentationInfo instrumentationInfo) { - meterProvider_(meterProvider); - instrumentationInfo_(instrumentationInfo); - } - -/////////////////////////Metric Instrument Constructors//////////////////////////// - - /* - * New Int Counter - * - * Function that creates and returns a Counter metric instrument with value - * type int. - * - * Arguments: - * name, the name of the metric instrument (must conform to the above syntax). - * description, a brief, readable description of the metric instrument. - * unit, the unit of metric values following the UCUM convention - * (https://unitsofmeasure.org/ucum.html). - * - */ - nostd::shared_ptr> NewIntCounter(nostd::string_view name, - nostd::string_view description, - nostd::string_view unit, - nostd::string_view enabled) { - auto intCounter = Counter(name, description, unit, enabled); - ptr = shared_ptr>(intCounter) - int_metrics_.insert(name, ptr); - return ptr; - } - - /* - * New float Counter - * - * Function that creates and returns a Counter metric instrument with value - * type float. - * - * Arguments: - * name, the name of the metric instrument (must conform to the above syntax). - * description, a brief, readable description of the metric instrument. - * unit, the unit of metric values following the UCUM convention - * (https://unitsofmeasure.org/ucum.html). - * - */ - nostd::unique_ptr> NewFloatCounter(nostd::string_view name, - nostd::string_view description, - nostd::string_view unit, - nostd::string_view enabled) { - auto floatCounter = Counter(name, description, unit, enabled); - ptr = unique_ptr>(floatCounter) - float_metrics_.insert(name, ptr); - return ptr; - } - -//////////////////////////////////////////////////////////////////////////////////// -// // -// Repeat above two functions for all // -// six (five other) metric instruments // -// of types short, int, float, and double. // -// // -//////////////////////////////////////////////////////////////////////////////////// - -private: - /* - * Collect (THREADSAFE) - * - * Checkpoints all metric instruments created from this meter and returns a - * vector of records containing the name, labels, and values of each instrument. - * This function also removes instruments that have not received updates in the - * last collection period. - * - */ - std::vector Collect() { - std::vector records; - metrics_lock_.lock(); - for instr in ALL_metrics_: - if instr is not enabled: - continue - else: - for bound_instr in instr.BoundInstruments: - records.push_back(Record(instr->name, instr->description, - bound_instr->labels, - bound_instr->GetAggregator()->Checkpoint()); - metrics_lock_.unlock(); - return records; - } - - /* - * Record Batch - * - * Allows the functionality of acting upon multiple metrics with the same set - * of labels with a single API call. Implementations should find bound metric - * instruments that match the key-value pairs in the labels. - * - * Arguments: - * labels, labels associated with all measurements in the batch. - * records, a KeyValueIterable containing metric instrument names such as - * "IntCounter" or "DoubleSumObserver" and the corresponding value - * to record for that metric. - * - */ - void RecordBatch(nostd::string_view labels, - nostd::KeyValueIterable values) { - for instr in metrics: - instr.bind(labels) // Bind the instrument to the label set - instr.record(values.GetValue(instr.type)) // Record the corresponding value - // to the instrument. - } - - std::map>> short_metrics_; - std::map>> int_metrics_; - std::map>> float_metrics_; - std::map>> double_metrics_; - - std::map>> short_observers_; - std::map>> int_observers_; - std::map>> float_observers_; - std::map>> double_observers_; - - std::mutex metrics_lock_; - unique_ptr meterProvider_; - InstrumentationInfo instrumentationInfo_; -}; -``` - -```cpp -# record.h -/* - * This class is used to pass checkpointed values from the Meter - * class, to the processor, to the exporter. This class is not - * templated but instead uses variants in order to avoid needing - * to template the exporters. - * - */ -class Record -{ -public: - explicit Record(std::string name, std::string description, - metrics_api::BoundInstrumentKind instrumentKind, - std::string labels, - nostd::variant, Aggregator, Aggregator, Aggregator> agg) - { - name_ = name; - description_ = description; - instrumentKind_ = instrumentKind; - labels_ = labels; - aggregator_ = aggregator; - } - - string GetName() {return name_;} - - string GetDescription() {return description_;} - - BoundInstrumentKind GetInstrumentKind() {return instrumentKind_;} - - string GetLabels() {return labels_;} - - nostd::variant, Aggregator, Aggregator, Aggregator> GetAggregator() {return aggregator_;} - -private: - string name_; - string description_; - BoundInstrumentKind instrumentKind_; - string labels_; - nostd::variant, Aggregator, Aggregator, Aggregator> aggregator_; -}; -``` - -Metric instruments created from this Meter class will be stored in a map (or -another, similar container [needs to be nostd]) called “metrics.” This is -identical to the Python implementation and makes sense because the SDK -implementation of the `Meter` class should have a function titled -`collect_all()` that collects metrics for every instrument created from this -meter. In contrast, Java’s implementation has a `MeterSharedState` class that -contains a registry (hash map) of all metric instruments spawned from this -meter. However, since each `Meter` has its own unique instruments it is easier -to store the instruments in the meter itself. - -The SDK implementation of the `Meter` class will contain a function called -`collect_all()` that will collect the measurements from each metric stored in -the `metrics` container. The implementation of this class acts as the -accumulator in the SDK specification. - -**Pros of this implementation:** - -* Different constructors and overloaded template calls to those constructors for - the various metric instruments allows us to forego much of the code - duplication involved in supporting various types. -* Storing the metric instruments created from this meter directly in the meter - object itself allows us to implement the collect_all method without creating a - new class that contains the meter state and instrument registry. - -**Cons of this implementation:** - -* Different constructors for the different metric instruments means less - duplicated code but still a lot. -* Storing the metric instruments in the Meter class means that if we have - multiple meters, metric instruments are stored in various objects. Using an - instrument registry that maps meters to metric instruments resolves this. - However, we have designed our SDK to only support one Meter instance. -* Storing 8 maps in the meter class is costly. However, we believe that this is - ok because these maps will only need to be created once, at the instantiation - of the meter class. **We believe that these maps will not slow down the - pipeline in any meaningful way** - -**The SDK implementation of the `Meter` class will act as the Accumulator -mentioned in the SDK specification.** - -## **Metric Instrument Class** - -Metric instruments capture raw measurements of designated quantities in -instrumented applications. All measurements captured by the Metrics API are -associated with the instrument which collected that measurement. - -### Metric Instrument Data Model - -Each instrument must have enough information to meaningfully attach its measured -values with a process in the instrumented application. As such, metric -instruments contain the following fields - -* name (string) — Identifier for this metric instrument. -* description (string) — Short description of what this instrument is capturing. -* value_type (string or enum) — Determines whether the value tracked is an int64 - or double. -* meter (Meter) — The Meter instance from which this instrument was derived. -* label_keys (KeyValueIterable) — A nostd class acting as map from - nostd::string_view to nostd::string_view. -* enabled (boolean) — Determines whether the instrument is currently collecting - data. -* bound_instruments (key value container) — Contains the bound instruments - derived from this instrument. - -### Metric Event Data Model - -Each measurement taken by a Metric instrument is a Metric event which must -contain the following information: - -* timestamp (implicit) — System time when measurement was captured. -* instrument definition(strings) — Name of instrument, kind, description, and - unit of measure -* label set (key value pairs) — Labels associated with the capture, described - further below. -* resources associated with the SDK at startup - -**Label Set:** - -A key:value mapping of some kind MUST be supported as annotation each metric -event. Labels must be represented the same way throughout the API (i.e. using -the same idiomatic data structure) and duplicates are dealt with by taking the -last value mapping. - -Due to the requirement to maintain ABI stability we have chosen to implement -labels as type KeyValueIterable. Though, due to performance reasons, we may -convert to std::string internally. - -**Implementation:** - -A base Metric class defines the constructor and binding functions which each -metric instrument will need. Once an instrument is bound, it becomes a -BoundInstrument which extends the BaseBoundInstrument class. The -BaseBoundInstrument is what communicates with the aggregator and performs the -actual updating of values. An enum helps to organize the numerous types of -metric instruments that will be supported. - -The only addition to the SDK metric instrument classes from their API -counterparts is the function GetRecords() and the private variables -std::map to hold bound instruments and -`Aggregator` to hold the instrument's aggregator. - -**For more information about the structure of metric instruments, refer to the -Metrics API Design document.** - -## Metrics SDK Data Path Implementation - -Note: these requirements come from a specification currently under development. -Changes and feedback are in [PR -347](https://github.com/open-telemetry/opentelemetry-specification/pull/347) -and the current document is linked -[here](https://github.com/open-telemetry/opentelemetry-specification/blob/64bbb0c611d849b90916005d7714fa2a7132d0bf/specification/metrics/sdk.md). - - - -### Accumulator - -The Accumulator is responsible for computing aggregation over a fixed unit of -time. It essentially takes a set of captures and turns them into a quantity that -can be collected and used for meaningful analysis by maintaining aggregators for -each active instrument and each distinct label set. For example, the aggregator -for a counter must combine multiple calls to Add(increment) into a single sum. - -Accumulators MUST support a `Checkpoint()` operation which saves a snapshot of -the current state for collection and a `Merge()` operation which combines the -state from multiple aggregators into one. - -Calls to the Accumulator's `Collect()` sweep through metric instruments with -un-exported updates, checkpoints their aggregators, and submits them to the -processor/exporter. This and all other accumulator operations should be -extremely efficient and follow the shortest code path possible. - -Design choice: We have chosen to implement the Accumulator as the SDK -implementation of the Meter interface shown above. - -### Aggregator - -The term *aggregator* refers to an implementation that can combine multiple -metric updates into a single, combined state for a specific function. -Aggregators MUST support `Update()`, `Checkpoint()`, and `Merge()` operations. -`Update()` is called directly from the Metric instrument in response to a metric -event, and may be called concurrently. The `Checkpoint()` operation is called -to atomically save a snapshot of the Aggregator. The `Merge()` operation -supports dimensionality reduction by combining state from multiple aggregators -into a single Aggregator state. - -The SDK must include the Counter aggregator which maintains a sum and the gauge -aggregator which maintains last value and timestamp. In addition, the SDK should -include MinMaxSumCount, Sketch, Histogram, and Exact aggregators All operations -should be atomic in languages that support them. - -```cpp -# aggregator.cc -class Aggregator { -public: - explicit Aggregator() { - self.current_ = nullptr - self.checkpoint_ = nullptr - } - - /* - * Update - * - * Updates the current value with the new value. - * - * Arguments: - * value, the new value to update the instrument with. - * - */ - virtual void Update( value); - - /* - * Checkpoint - * - * Stores a snapshot of the current value. - * - */ - virtual void Checkpoint(); - - /* - * Merge - * - * Combines two aggregator values. Update to most recent time stamp. - * - * Arguments: - * other, the aggregator whose value to merge. - * - */ - virtual void Merge(Aggregator other); - - /* - * Getters for various aggregator specific fields - */ - virtual std::vector get_value() {return current_;} - virtual std::vector get_checkpoint() {return checkpoint_;} - virtual core::SystemTimeStamp get_timestamp() {return last_update_timestamp_;} - -private: - std::vector current_; - std::vector checkpoint_; - core::Systemtimestamp last_update_timestamp_; -}; -``` - -```cpp -# counter_aggregator.cc -template -class CounterAggregator : public Aggregator { -public: - explicit CounterAggregator(): current(0), checkpoint(0), - last_update_timestamp(nullptr){} - - void Update(T value) { - // thread lock - // current += value - this->last_update_timestamp = time_ns() - } - - void Checkpoint() { - // thread lock - this->checkpoint = this->current - this->current = 0 - } - - void Merge(CounterAggregator* other) { - // thread lock - // combine checkpoints - // update timestamp to now - } -}; -``` - -This Counter is an example Aggregator. We plan on implementing all the -Aggregators in the specification: Counter, Gauge, MinMaxSumCount, Sketch, -Histogram, and Exact. - -### Processor - -The Processor SHOULD act as the primary source of configuration for exporting -metrics from the SDK. The two kinds of configuration are: - -1. Given a metric instrument, choose which concrete aggregator type to apply for - in-process aggregation. -2. Given a metric instrument, choose which dimensions to export by (i.e., the - "grouping" function). - -During the collection pass, the Processor receives a full set of check-pointed -aggregators corresponding to each (Instrument, LabelSet) pair with an active -record managed by the Accumulator. According to its own configuration, the -Processor at this point determines which dimensions to aggregate for export; it -computes a checkpoint of (possibly) reduced-dimension export records ready for -export. It can be thought of as the business logic or processing phase in the -pipeline. - -Change of dimensions: The user-facing metric API allows users to supply -LabelSets containing an unlimited number of labels for any metric update. Some -metric exporters will restrict the set of labels when exporting metric data, -either to reduce cost or because of system-imposed requirements. A *change of -dimensions* maps input LabelSets with potentially many labels into a LabelSet -with a fixed set of label keys. A change of dimensions eliminates labels with -keys not in the output LabelSet and fills in empty values for label keys that -are not in the input LabelSet. This can be used for different filtering options, -rate limiting, and alternate aggregation schemes. Additionally, it will be used -to prevent unbounded memory growth through capping collected data. The community -is still deciding exactly how metrics data will be pruned and this document will -be updated when a decision is made. - -The following is a pseudo code implementation of a ‘simple’ Processor. - -Note: Josh MacDonald is working on implementing a [‘basic’ -Processor](https://github.com/jmacd/opentelemetry-go/blob/jmacd/mexport/sdk/metric/processor/simple/simple.go) -which allows for further Configuration that lines up with the specification in -Go. He will be finishing the implementation and updating the specification -within the next few weeks. - -Design choice: We recommend that we implement the ‘simple’ Processor first as -apart of the MVP and then will also implement the ‘basic’ Processor later on. -Josh recommended having both for doing different processes. - -```cpp -#processor.cc -class Processor { -public: - - explicit Processor(Bool stateful) { - // stateful determines whether the processor computes deltas or lifetime changes - // in metric values - stateful_ = stateful; - } - - /* - * Process - * - * This function chooses which dimensions to aggregate for export. In the - * reference implementation, the UngroupedProcessor does not process records - * and simple passes them along to the next step. - * - * Arguments: - * record, a record containing data collected from the active Accumulator - * in this data pipeline - */ - void Process(Record record); - - /* - * Checkpoint - * - * This function computes a new (possibly dimension-reduced) checkpoint set of - * all instruments in the meter passed to process. - * - */ - Collection Checkpoint(); - - /* - * Finished Collection - * - * Signals to the intergrator that a collection has been completed and - * can now be sent for export. - * - */ - Error FinishedCollection(); - - /* - * Aggregator For - * - * Returns the correct aggregator type for a given metric instrument. Used in - * the instrument constructor to select which aggregator to use. - * - * Arguments: - * kind, the instrument type asking to be assigned an aggregator - * - */ - Aggregator AggregatorFor(MetricKind kind); - -private: - Bool stateful_; - Batch batch_; -}; -``` - -### Controller - -Controllers generally are responsible for binding the Accumulator, the -Processor, and the Exporter. The controller initiates the collection and export -pipeline and manages all the moving parts within it. It also governs the flow of -data through the SDK components. Users interface with the controller to begin -collection process. - -Once the decision has been made to export, the controller must call `Collect()` -on the Accumulator, then read the checkpoint from the Processor, then invoke the -Exporter. - -Java’s IntervalMetricReader class acts as a parallel to the controller. The user -gets an instance of this class, sets the configuration options (like the tick -rate) and then the controller takes care of the collection and exporting of -metric data from instruments the user defines. - -There are two different controllers: Push and Pull. The “Push” Controller will -establish a periodic timer to regularly collect and export metrics. A “Pull” -Controller will await a pull request before initiating metric collection. - -We recommend implementing the PushController as the initial implementation of -the Controller. This Controller is the base controller in the specification. We -may also implement the PullController if we have the time to do it. - -```cpp -#push_controller.cc -class PushController { - - explicit PushController(Meter meter, Exporter exporter, - int period, int timeout) { - meter_ = meter; - exporter_ = exporter(); - period_ = period; - timeout_ = timeout; - provider_ = NewMeterProvider(accumulator); - } - - /* - * Provider - * - * Returns the MeterProvider stored by this controller. - * - */ - MeterProvider Provider { - return this.provider_; - } - - /* - * Start (THREAD SAFE) - * - * Begins a ticker that periodically collects and exports metrics with a - * configured interval. - * - */ - void Start(); - - /* - * Stop (THREAD SAFE) - * - * Waits for collection interval to end then exports metrics one last time - * before returning. - * - */ - void Stop(); - - /* - * Run - * - * Used to wait between collection intervals. - * - */ - void run(); - - /* - * Tick (THREAD SAFE) - * - * Called at regular intervals, this function collects all values from the - * member variable meter_, then sends them to the processor_ for - * processing. After the records have been processed they are sent to the - * exporter_ to be exported. - * - */ - void tick(); - -private: - mutex lock_; - Meter meter_; - MeterProvider provider_; - Exporter exporter_; - int period_; - int timeout_; -}; -``` - -### Exporter - -The exporter SHOULD be called with a checkpoint of finished (possibly -dimensionally reduced) export records. Most configuration decisions have been -made before the exporter is invoked, including which instruments are enabled, -which concrete aggregator types to use, and which dimensions to aggregate by. - -There is very little left for the exporter to do other than format the metric -updates into the desired format and send them on their way. - -Design choice: Our idea is to take the simple trace example -[OStreamSpanExporter](https://github.com/open-telemetry/opentelemetry-cpp/blob/main/examples/simple/main.cc) -and add Metric functionality to it. This will allow us to verify that what we -are implementing in the API and SDK works as intended. The exporter will go -through the different metric instruments and print the value stored in their -aggregators to stdout, **for simplicity only Sum is shown here, but all -aggregators will be implemented**. - -```cpp -# stdout_exporter.cc -class StdoutExporter: public exporter { - /* - * Export - * - * For each checkpoint in checkpointSet, print the value found in their - * aggregator to stdout. Returns an ExportResult enum error code. - * - * Arguments, - * checkpointSet, the set of checkpoints to be exported to stdout. - * - */ - ExportResult Export(CheckpointSet checkpointSet) noexcept; - - /* - * Shutdown - * - * Shuts down the channel and cleans up resources as required. - * - */ - bool Shutdown(); -}; -``` - -```cpp -enum class ExportResult { - kSuccess, - kFailure, -}; -``` - -## Test Strategy / Plan - -Since there is a specification we will be following, we will not have to write -out user stories for testing. We will generally only be writing functional unit -tests for this project. The C++ Open Telemetry repository uses -[Googletest](https://github.com/google/googletest) because it provides test -coverage reports, also allows us to easily integrate code coverage tools such as -[codecov.io](http://codecov.io/) with the project. A required coverage target of -90% will help to ensure that our code is fully tested. - -An open-source header-only testing framework called -[Catch2](https://github.com/catchorg/Catch2) is an alternate option which would -satisfy our testing needs. It is easy to use, supports behavior driven -development, and does not need to be embedded in the project as source files to -operate (unlike Googletest). Code coverage would still be possible using this -testing framework but would require us to integrate additional tools. This -framework may be preferred as an agnostic replacement for Googletest and is -widely used in open source projects.