diff --git a/CHANGELOG.md b/CHANGELOG.md index b040c61a9d3..f40ac8223e3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,11 @@ release. ### Logs +- Remove implementation detail from Logs Bridge API. + ([#3884](https://github.com/open-telemetry/opentelemetry-specification/pull/3884)) +- Clarify that logs attributes are a superset of standard attributes. + ([#3852](https://github.com/open-telemetry/opentelemetry-specification/pull/3852)) + ### Resource ### OpenTelemetry Protocol @@ -30,6 +35,9 @@ release. ### Common +- Tighten stability requirements for well-known attribute values. + ([#3879](https://github.com/open-telemetry/opentelemetry-specification/pull/3879)) + ### Supplementary Guidelines ## v1.30.0 (2024-02-15) diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md index 10570f92fc6..a20644d5c73 100644 --- a/spec-compliance-matrix.md +++ b/spec-compliance-matrix.md @@ -199,19 +199,19 @@ formats is required. Implementing more than one format is optional. | The metrics Exporter `ForceFlush` can inform the caller whether it succeeded, failed or timed out. | | | + | + | - | | + | + | | + | + | | | The metrics Exporter provides a `shutdown` function. | | + | + | + | - | | + | + | + | + | + | | | The metrics Exporter `shutdown` function do not block indefinitely. | | + | + | + | - | | + | | | + | + | | -| The metrics SDK samples `Exemplar`s from measurements. | | | + | | - | | - | | | | - | | -| Exemplar sampling can be disabled. | | | - | | - | | - | | | | - | | -| The metrics SDK supports SDK-wide exemplar filter configuration | | | + | | - | | - | | | | - | | -| The metrics SDK supports `TraceBased` exemplar filter | | | + | | - | | - | | | | - | | -| The metrics SDK supports `AlwaysOn` exemplar filter | | | + | | - | | - | | | | - | | -| The metrics SDK supports `AlwaysOff` exemplar filter | | | + | | - | | - | | | | - | | -| Exemplars retain any attributes available in the measurement that are not preserved by aggregation or view configuration. | | | + | | - | | - | | | | - | | -| Exemplars contain the associated trace id and span id of the active span in the Context when the measurement was taken. | | | + | | - | | - | | | | - | | -| Exemplars contain the timestamp when the measurement was taken. | | | + | | - | | - | | | | - | | -| The metrics SDK provides an `ExemplarReservoir` interface or extension point. | X | | - | | - | | - | + | | | - | | -| An `ExemplarReservoir` has an `offer` method with access to the measurement value, attributes, `Context` and timestamp. | X | | - | | - | | - | + | | | - | | -| The metrics SDK provides a `SimpleFixedSizeExemplarReservoir` that is used by default for all aggregations except `ExplicitBucketHistogram`. | | | + | | - | | - | + | | | - | | -| The metrics SDK provides an `AlignedHistogramBucketExemplarReservoir` that is used by default for `ExplicitBucketHistogram` aggregation. | | | + | | - | | - | | | | - | | +| The metrics SDK samples `Exemplar`s from measurements. | | | + | | - | | + | | | | - | | +| Exemplar sampling can be disabled. | | | - | | - | | + | | | | - | | +| The metrics SDK supports SDK-wide exemplar filter configuration | | | + | | - | | + | | | | - | | +| The metrics SDK supports `TraceBased` exemplar filter | | | + | | - | | + | | | | - | | +| The metrics SDK supports `AlwaysOn` exemplar filter | | | + | | - | | + | | | | - | | +| The metrics SDK supports `AlwaysOff` exemplar filter | | | + | | - | | + | | | | - | | +| Exemplars retain any attributes available in the measurement that are not preserved by aggregation or view configuration. | | | + | | - | | + | | | | - | | +| Exemplars contain the associated trace id and span id of the active span in the Context when the measurement was taken. | | | + | | - | | + | | | | - | | +| Exemplars contain the timestamp when the measurement was taken. | | | + | | - | | + | | | | - | | +| The metrics SDK provides an `ExemplarReservoir` interface or extension point. | X | | - | | - | | + | + | | | - | | +| An `ExemplarReservoir` has an `offer` method with access to the measurement value, attributes, `Context` and timestamp. | X | | - | | - | | + | + | | | - | | +| The metrics SDK provides a `SimpleFixedSizeExemplarReservoir` that is used by default for all aggregations except `ExplicitBucketHistogram`. | | | + | | - | | + | + | | | - | | +| The metrics SDK provides an `AlignedHistogramBucketExemplarReservoir` that is used by default for `ExplicitBucketHistogram` aggregation. | | | + | | - | | + | | | | - | | | A metric Producer accepts an optional metric Filter | | | | | | | - | | | | | | | The metric Reader implementation supports registering metric Filter and passing them its registered metric Producers | | | | | | | - | | | | | | | The metric SDK's metric Producer implementations uses the metric Filter | | | | | | | - | | | | | | diff --git a/specification/logs/bridge-api.md b/specification/logs/bridge-api.md index a612f275a8c..610fa4862c7 100644 --- a/specification/logs/bridge-api.md +++ b/specification/logs/bridge-api.md @@ -46,9 +46,6 @@ graph TD `Logger`s can be accessed with a `LoggerProvider`. -In implementations of the API, the `LoggerProvider` is expected to be the stateful -object that holds any configuration. - Normally, the `LoggerProvider` is expected to be accessed from a central place. Thus, the API SHOULD provide a way to set/register and access a global default `LoggerProvider`. diff --git a/specification/logs/data-model.md b/specification/logs/data-model.md index 4da9becbddc..8bace296920 100644 --- a/specification/logs/data-model.md +++ b/specification/logs/data-model.md @@ -442,13 +442,15 @@ is optional. ### Field: `Attributes` -Type: `map`. +Type: [`map`](#type-mapstring-any). Description: Additional information about the specific event occurrence. Unlike the `Resource` field, which is fixed for a particular source, `Attributes` can vary for each occurrence of the event coming from the same source. Can contain -information about the request context (other than TraceId/SpanId). SHOULD follow -OpenTelemetry [semantic conventions for attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md). +information about the request context (other than [Trace Context Fields](#trace-context-fields)). +The log attribute model MUST support [`any` type](#type-any), +a superset of [standard Attribute](../common/README.md#attribute), +to preserve the semantics of structured attributes emitted by the applications. This field is optional. #### Errors and Exceptions diff --git a/specification/metrics/data-model.md b/specification/metrics/data-model.md index a7d15bfb9cf..a66a754e1a5 100644 --- a/specification/metrics/data-model.md +++ b/specification/metrics/data-model.md @@ -23,7 +23,7 @@ aliases: [/docs/reference/specification/metrics/datamodel] + [OpenTelemetry Protocol data model: Producer recommendations](#opentelemetry-protocol-data-model-producer-recommendations) + [OpenTelemetry Protocol data model: Consumer recommendations](#opentelemetry-protocol-data-model-consumer-recommendations) + [Point kinds](#point-kinds) -- [Metric points](#metric-points) +- [Metric Points](#metric-points) * [Sums](#sums) * [Gauge](#gauge) * [Histogram](#histogram) @@ -381,10 +381,15 @@ designed for compatibility with existing metric formats. - [Summary](#summary-legacy) -## Metric points +## Metric Points **Status**: [Stable](../document-status.md) +Metric points are the basic building blocks of metrics. Depending on the [point +kind](#point-kinds), a metric point may have different fields. The following +sections describe the fields for each point kind and how these points form +metrics. + ### Sums [Sum](https://github.com/open-telemetry/opentelemetry-proto/blob/v0.9.0/opentelemetry/proto/metrics/v1/metrics.proto#L230)s diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index 9e9365a5c4f..b049a733d59 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -733,8 +733,9 @@ of metrics across successive collections. SDKs SHOULD support being configured with a cardinality limit. The number of unique combinations of attributes is called cardinality. For a given metric, the -cardinality limit is a hard limit on the number of metric points that can be -collected during a collection cycle. +cardinality limit is a hard limit on the number of [Metric +Points](./data-model.md#metric-points) that can be collected during a collection +cycle. #### Configuration @@ -1383,12 +1384,12 @@ A Push Metric Exporter MUST support the following functions: ##### Export(batch) -Exports a batch of [Metric points](./data-model.md#metric-points). Protocol +Exports a batch of [Metric Points](./data-model.md#metric-points). Protocol exporters that will implement this function are typically expected to serialize and transmit the data to the destination. The SDK MUST provide a way for the exporter to get the [Meter](./api.md#meter) -information (e.g. name, version, etc.) associated with each `Metric point`. +information (e.g. name, version, etc.) associated with each `Metric Point`. `Export` will never be called concurrently for the same exporter instance. `Export` can be called again only after the current call returns. @@ -1403,10 +1404,10 @@ are being sent to. **Parameters:** -`batch` - a batch of `Metric point`s. The exact data type of the batch is -language specific, typically it is some kind of list. The exact type of `Metric -point` is language specific, and is typically optimized for high performance. -Here are some examples: +`batch` - a batch of [Metric Points](./data-model.md#metric-points). The exact +data type of the batch is language specific, typically it is some kind of list. +The exact type of `Metric Point` is language specific, and is typically +optimized for high performance. Here are some examples: ```text +--------+ +--------+ +--------+ @@ -1422,7 +1423,7 @@ Batch: | Metric | | Metric | ... | Metric | +--> timestamps, attributes, value (or buckets), exemplars, ... ``` -Refer to the [Metric points](./data-model.md#metric-points) section from the +Refer to the [Metric Points](./data-model.md#metric-points) section from the Metrics Data Model specification for more details. Note: it is highly recommended that implementors design the `Metric` data type @@ -1562,12 +1563,12 @@ A `MetricProducer` MUST support the following functions: #### Produce batch `Produce` provides metrics from the MetricProducer to the caller. `Produce` -MUST return a batch of [Metric points](./data-model.md#metric-points), filtered by the optional +MUST return a batch of [Metric Points](./data-model.md#metric-points), filtered by the optional `metricFilter` parameter. Implementation SHOULD use the filter as early as possible to gain as much performance gain possible (memory allocation, internal metric fetching, etc). -If the batch of [Metric points](./data-model.md#metric-points) includes +If the batch of [Metric Points](./data-model.md#metric-points) includes resource information, `Produce` SHOULD require a resource as a parameter. `Produce` does not have any other required parameters, however, [OpenTelemetry SDK](../overview.md#sdk) authors MAY choose to add required or optional @@ -1578,7 +1579,7 @@ failed or timed out. When the `Produce` operation fails, the `MetricProducer` MAY return successfully collected results and a failed reasons list to the caller. -If a batch of [Metric points](./data-model.md#metric-points) can include +If a batch of [Metric Points](./data-model.md#metric-points) can include [`InstrumentationScope`](../glossary.md#instrumentation-scope) information, `Produce` SHOULD include a single InstrumentationScope which identifies the `MetricProducer`. @@ -1593,7 +1594,7 @@ If a batch of [Metric points](./data-model.md#metric-points) can include `MetricFilter` defines the interface which enables the [MetricReader](#metricreader)'s registered [MetricProducers](#metricproducer) or the SDK's [MetricProducer](#metricproducer) to filter aggregated data points -([Metric points](./data-model.md#metric-points)) inside its `Produce` operation. +([Metric Points](./data-model.md#metric-points)) inside its `Produce` operation. The filtering is done at the [MetricProducer](#metricproducer) for performance reasons. The `MetricFilter` allows filtering an entire metric stream - dropping or allowing all its attribute sets - diff --git a/specification/overview.md b/specification/overview.md index ca6ef858fdb..c8e2a302db1 100644 --- a/specification/overview.md +++ b/specification/overview.md @@ -22,10 +22,9 @@ weight: 1 * [Links between spans](#links-between-spans) - [Metric Signal](#metric-signal) * [Recording raw measurements](#recording-raw-measurements) - + [Measure](#measure) - + [Measurement](#measurement) - * [Recording metrics with predefined aggregation](#recording-metrics-with-predefined-aggregation) + + [Instruments](#instruments) * [Metrics data model and SDK](#metrics-data-model-and-sdk) + + [Views](#views) - [Log Signal](#log-signal) * [Data model](#data-model) - [Baggage Signal](#baggage-signal) @@ -218,65 +217,53 @@ scenarios. ## Metric Signal -OpenTelemetry allows to record raw measurements or metrics with predefined -aggregation and a [set of attributes](./common/README.md#attribute). +OpenTelemetry allows recording raw measurements or metrics with predefined +aggregations and a [set of attributes](common/README.md#attribute). -Recording raw measurements using OpenTelemetry API allows to defer to end-user -the decision on what aggregation algorithm should be applied for this metric as -well as defining attributes (dimensions). It will be used in client libraries like -gRPC to record raw measurements "server_latency" or "received_bytes". So end -user will decide what type of aggregated values should be collected out of these -raw measurements. It may be simple average or elaborate histogram calculation. - -Recording of metrics with the pre-defined aggregation using OpenTelemetry API is -not less important. It allows to collect values like cpu and memory usage, or -simple metrics like "queue length". +Using the OpenTelemetry API to record raw measurements gives end-users the +flexibility to choose the aggregation algorithm for a given metric. This functionality +is particularly useful in client libraries such as gRPC, where it enables the +recording of raw measurements like "server_latency" or "received_bytes." End-users +then have the autonomy to decide on the aggregation method for these raw measurements, +options for which range from straightforward averages to more complex histogram calculations. ### Recording raw measurements -The main classes used to record raw measurements are `Measure` and -`Measurement`. List of `Measurement`s alongside the additional context can be -recorded using OpenTelemetry API. So user may define to aggregate those -`Measurement`s and use the context passed alongside to define additional -attributes of the resulting metric. - -#### Measure - -`Measure` describes the type of the individual values recorded by a library. It -defines a contract between the library exposing the measurements and an -application that will aggregate those individual measurements into a `Metric`. -`Measure` is identified by name, description and a unit of values. - -#### Measurement - -`Measurement` describes a single value to be collected for a `Measure`. -`Measurement` is an empty interface in API surface. This interface is defined in -SDK. +The primary components involved in recording raw measurements using the OpenTelemetry +API are `Measurement`, `Instrument` and `Meter`. A `Meter` is obtained from a +`MeterProvider` and used to create an `Instrument`, which is then responsible for capturing +[measurements](metrics/api.md#measurement). -### Recording metrics with predefined aggregation - -The base class for all types of pre-aggregated metrics is called `Metric`. It -defines basic metric properties like a name and attributes. Classes inheriting from -the `Metric` define their aggregation type as well as a structure of individual -measurements or Points. API defines the following types of pre-aggregated -metrics: +``` ++------------------+ +| MeterProvider | +-----------------+ +--------------+ +| Meter A | Measurements... | | Metrics... | | +| Instrument X +-----------------> In-memory state +-------------> MetricReader | +| Instrument Y | | | | | +| Meter B | +-----------------+ +--------------+ +| Instrument Z | +| ... | +-----------------+ +--------------+ +| ... | Measurements... | | Metrics... | | +| ... +-----------------> In-memory state +-------------> MetricReader | +| ... | | | | | +| ... | +-----------------+ +--------------+ ++------------------+ +``` -- Counter metric to report instantaneous measurement. Counter values can go - up or stay the same, but can never go down. Counter values cannot be - negative. There are two types of counter metric values - `double` and `long`. -- Gauge metric to report instantaneous measurement of a numeric value. Gauges can - go both up and down. The gauges values can be negative. There are two types of - gauge metric values - `double` and `long`. +#### Instruments -API allows to construct the `Metric` of a chosen type. SDK defines the way to -query the current value of a `Metric` to be exported. +[Instruments](metrics/api.md#instrument) are used to report `Measurement`s, and are identified +by a name, kind, description and a unit of values. -Every type of a `Metric` has it's API to record values to be aggregated. API -supports both - push and pull model of setting the `Metric` value. +There are several types of metric instruments for specific use cases, such as counters for +incrementing values, gauges for capturing current values, and histograms for capturing +distributions of measurements. Instruments can be synchronous, meaning that they are invoked +inline by application logic, or asynchronous where the user registers a callback +function that is invoked on demand by the SDK. ### Metrics data model and SDK -Metrics data model is [specified here](metrics/data-model.md) and is based on +The Metrics data model is [specified here](metrics/data-model.md) and is based on [metrics.proto](https://github.com/open-telemetry/opentelemetry-proto/blob/master/opentelemetry/proto/metrics/v1/metrics.proto). This data model defines three semantics: An Event model used by the API, an in-flight data model used by the SDK and OTLP, and a TimeSeries model which @@ -284,7 +271,7 @@ denotes how exporters should interpret the in-flight model. Different exporters have different capabilities (e.g. which data types are supported) and different constraints (e.g. which characters are allowed in attribute -keys). Metrics is intended to be a superset of what's possible, not a lowest +keys). Metrics is intended to be a superset of what's possible, not the lowest common denominator that's supported everywhere. All exporters consume data from Metrics Data Model via a Metric Producer interface defined in OpenTelemetry SDK. @@ -297,6 +284,13 @@ from the backend. See [Metrics Data Model Specification](metrics/data-model.md) for more information. +#### Views + +[Views](metrics/sdk.md#view) are configurations that specify how the data from an `Instrument` should be processed, +aggregated, and exported. They can be applied globally through the `MeterProvider` or more +specifically at the `Meter` level. A `View` allows the customization of metric data beyond the default +collection behavior, enabling specific aggregations, transformations, and filtering of metrics. + ## Log Signal ### Data model diff --git a/specification/versioning-and-stability.md b/specification/versioning-and-stability.md index b2632067aad..07a545afb5c 100644 --- a/specification/versioning-and-stability.md +++ b/specification/versioning-and-stability.md @@ -183,6 +183,7 @@ Semantic Conventions defines the set of fields in the OTLP data model: - Attribute keys - provided to [get a tracer](trace/api.md#get-a-tracer) - provided to [get a meter](metrics/api.md#get-a-meter) + - Attribute values that are defined in a list of well-known values. - [Trace](trace/api.md) - The following data on [span](trace/api.md#span): - The span name @@ -190,9 +191,13 @@ Semantic Conventions defines the set of fields in the OTLP data model: - The attribute keys provided to the span - Whether these attributes must be provided at span start time, due to sampling concerns. + - The attribute values provided to the span that are defined in a list of + well-known values. - The following data provided on [span events](trace/api.md#add-events) - The event name - The attribute keys provided for the event + - The attribute values provided for the event that are defined in a list of + well-known values. - [Metrics](metrics/api.md) - The following portions of a Metric (passed when constructing [an instrument](metrics/api.md#instrument)): @@ -207,8 +212,12 @@ Semantic Conventions defines the set of fields in the OTLP data model: both synchronous and asynchronous instruments. - These exist on `NumberDataPoint`, `HistogramDataPoint`, `ExponentialHistogramDataPoint` and `SummaryDataPoint`. + - The attribute values on any `*DataPoint` that are defined in a list of + well-known values. - [Log Records](logs/data-model.md#log-and-event-record-definition) - The attribute keys provided on the LogRecord + - The attribute values provided on the LogRecord that are defined in a list + of well-known values. - For log records that are [Log Events](logs/event-api.md) - The following data provided to [emit event](logs/event-api.md#emit-event): - The event name (the value of the `event.name` attribute) @@ -217,10 +226,9 @@ Things not listed in the above are not expected to remain stable via semantic convention and are allowed (or expected) to change. A few examples: - The values of attributes - - Specifically for `enums` the list of allowed values is expected to change - overtime. - - Even for `enums` that limit allowed values to semconv, some may need to - updated values in the future. Tooling should expect unknown values. + - An exception are existing values in lists of well-known values. However, + new values can be added to such lists. Consumers should expect unknown + values. - The links attached to a span - The recorded measurement type (float or integer) of a metric is not enforced and allowed to change. - The description of a metric instrument.