enhancement(observability): Standardize buffer size metric names

changelog.d/buffer_size_metrics.deprecation.md

@@ -0,0 +1,10 @@
+Buffers now emit metric names for sizes that better follow the metric naming standard specification
+while keeping the old related gauges available for a transition period. Operators should update
+dashboards/alerts to the new variants as the legacy names are now deprecated.
+
+* `buffer_max_size_bytes` deprecates `buffer_max_byte_size`
+* `buffer_max_size_events` deprecates `buffer_max_event_size`
+* `buffer_size_bytes` deprecates `buffer_byte_size`
+* `buffer_size_events` deprecates `buffer_events`
+
+authors: bruceg

docs/DEPRECATION.md

@@ -68,8 +68,8 @@ When possible, Vector will error at start-up when a removed configuration option
 
 When introducing a deprecation into Vector, the pull request introducing the deprecation should:
 
-- Add a note to the Deprecations section of the upgrade guide for the next release with a description and
-  directions for transitioning if applicable.
+- Add a note to the Deprecations section of the upgrade guide in `website/content/en/highlights` for
+  the next release with a description and directions for transitioning if applicable.
 - Copy the same note from the previous step, to a changelog fragment, with type="deprecation". See the changelog
   fragment [README.md](../changelog.d/README.md) for details.
 - Add a deprecation note to the docs. Typically, this means adding `deprecation: "description of the deprecation"`
@@ -80,7 +80,7 @@ When introducing a deprecation into Vector, the pull request introducing the dep
   the new name will be appended with the text `(formerly OldName)`.
 - Add a log message to Vector that is logged at the `WARN` level starting with the word `DEPRECATION` if Vector detects
   the deprecated configuration or feature being used (when possible).
-- Add the deprecation to [DEPRECATIONS.md](DEPRECATIONS.md) to track migration (if applicable) and removal
+- Add the deprecation to [docs/DEPRECATIONS.md](../docs/DEPRECATIONS.md) to track migration (if applicable) and removal
 
 When removing a deprecation in a subsequent release, the pull request should:
 
@@ -90,4 +90,4 @@ When removing a deprecation in a subsequent release, the pull request should:
   for transitioning if applicable.
 - Copy the same note from the previous step, to a changelog fragment, with type="breaking". See the changelog
   fragment [README.md](../changelog.d/README.md) for details.
-- Remove the deprecation from [DEPRECATIONS.md](DEPRECATIONS.md)
+- Remove the deprecation from [docs/DEPRECATIONS.md](../docs/DEPRECATIONS.md)

docs/DEPRECATIONS.md

@@ -15,6 +15,7 @@ For example:
 ## To be deprecated
 
 - `v0.50.0` | `http-server-encoding` | The `encoding` field will be removed. Use `decoding` and `framing` instead.
+- `v0.53.0` | `buffer-bytes-events-metrics` | The `buffer_byte_size` and `buffer_events` gauges are deprecated in favor of the `buffer_size_bytes`/`buffer_size_events` metrics described in `docs/specs/buffer.md`.
 
 ## To be migrated
 

docs/specs/buffer.md

@@ -42,8 +42,8 @@ _All buffers_ MUST emit a `BufferCreated` event upon creation. To avoid stale me
   - `max_size_bytes` - the max size of the buffer in bytes if relevant
   - `max_size_events` - the max size of the buffer in number of events if relevant
 - Metric
-  - MUST emit the `buffer_max_event_size` gauge (in-memory buffers) if the defined `max_size_events` value is present
-  - MUST emit the `buffer_max_byte_size` gauge (disk buffers) if the defined `max_size_bytes` value is present
+  - MUST emit the `buffer_max_size_events` gauge (in-memory buffers) if the defined `max_size_events` value is present, and emit `buffer_max_event_size` for backward compatibility
+  - MUST emit the `buffer_max_size_bytes` gauge (disk buffers) if the defined `max_size_bytes` value is present, and emit `buffer_max_byte_size` for backward compatibility
 
 #### BufferEventsReceived
 
@@ -58,8 +58,8 @@ _All buffers_ MUST emit a `BufferEventsReceived` event:
 - Metric
   - MUST increment the `buffer_received_events_total` counter by the defined `count`
   - MUST increment the `buffer_received_bytes_total` counter by the defined `byte_size`
-  - MUST increment the `buffer_events` gauge by the defined `count`
-  - MUST increment the `buffer_byte_size` gauge by the defined `byte_size`
+  - MUST increment the `buffer_size_events` gauge by the defined `count`, and emit `buffer_events` for backward compatibility
+  - MUST increment the `buffer_size_bytes` gauge by the defined `byte_size`, and emit `buffer_byte_size` for backward compatibility
 
 #### BufferEventsSent
 
@@ -71,8 +71,8 @@ _All buffers_ MUST emit a `BufferEventsSent` event after sending one or more Vec
 - Metric
   - MUST increment the `buffer_sent_events_total` counter by the defined `count`
   - MUST increment the `buffer_sent_bytes_total` counter by the defined `byte_size`
-  - MUST decrement the `buffer_events` gauge by the defined `count`
-  - MUST decrement the `buffer_byte_size` gauge by the defined `byte_size`
+  - MUST decrement the `buffer_size_events` gauge by the defined `count`, and emit `buffer_events` for backward compatibility
+  - MUST decrement the `buffer_size_bytes` gauge by the defined `byte_size`, and emit `buffer_byte_size` for backward compatibility
 
 #### BufferError
 

lib/vector-buffers/src/internal_events.rs

@@ -18,19 +18,34 @@ pub struct BufferCreated {
 impl InternalEvent for BufferCreated {
     #[expect(clippy::cast_precision_loss)]
     fn emit(self) {
+        let stage = self.idx.to_string();
         if self.max_size_events != 0 {
+            gauge!(
+                "buffer_max_size_events",
+                "buffer_id" => self.buffer_id.clone(),
+                "stage" => stage.clone(),
+            )
+            .set(self.max_size_events as f64);
+            // DEPRECATED: buffer-bytes-events-metrics
             gauge!(
                 "buffer_max_event_size",
                 "buffer_id" => self.buffer_id.clone(),
-                "stage" => self.idx.to_string(),
+                "stage" => stage.clone(),
             )
             .set(self.max_size_events as f64);
         }
         if self.max_size_bytes != 0 {
+            gauge!(
+                "buffer_max_size_bytes",
+                "buffer_id" => self.buffer_id.clone(),
+                "stage" => stage.clone(),
+            )
+            .set(self.max_size_bytes as f64);
+            // DEPRECATED: buffer-bytes-events-metrics
             gauge!(
                 "buffer_max_byte_size",
                 "buffer_id" => self.buffer_id,
-                "stage" => self.idx.to_string(),
+                "stage" => stage,
             )
             .set(self.max_size_bytes as f64);
         }
@@ -63,12 +78,26 @@ impl InternalEvent for BufferEventsReceived {
             "stage" => self.idx.to_string()
         )
         .increment(self.byte_size);
+        // DEPRECATED: buffer-bytes-events-metrics
         gauge!(
             "buffer_events",
             "buffer_id" => self.buffer_id.clone(),
             "stage" => self.idx.to_string()
         )
         .set(self.total_count as f64);
+        gauge!(
+            "buffer_size_events",
+            "buffer_id" => self.buffer_id.clone(),
+            "stage" => self.idx.to_string()
+        )
+        .set(self.total_count as f64);
+        gauge!(
+            "buffer_size_bytes",
+            "buffer_id" => self.buffer_id.clone(),
+            "stage" => self.idx.to_string()
+        )
+        .set(self.total_byte_size as f64);
+        // DEPRECATED: buffer-bytes-events-metrics
         gauge!(
             "buffer_byte_size",
             "buffer_id" => self.buffer_id,
@@ -103,12 +132,26 @@ impl InternalEvent for BufferEventsSent {
             "stage" => self.idx.to_string()
         )
         .increment(self.byte_size);
+        // DEPRECATED: buffer-bytes-events-metrics
         gauge!(
             "buffer_events",
             "buffer_id" => self.buffer_id.clone(),
             "stage" => self.idx.to_string()
         )
         .set(self.total_count as f64);
+        gauge!(
+            "buffer_size_events",
+            "buffer_id" => self.buffer_id.clone(),
+            "stage" => self.idx.to_string()
+        )
+        .set(self.total_count as f64);
+        gauge!(
+            "buffer_size_bytes",
+            "buffer_id" => self.buffer_id.clone(),
+            "stage" => self.idx.to_string()
+        )
+        .set(self.total_byte_size as f64);
+        // DEPRECATED: buffer-bytes-events-metrics
         gauge!(
             "buffer_byte_size",
             "buffer_id" => self.buffer_id,
@@ -170,12 +213,26 @@ impl InternalEvent for BufferEventsDropped {
             "intentional" => intentional_str,
         )
         .increment(self.byte_size);
+        // DEPRECATED: buffer-bytes-events-metrics
         gauge!(
             "buffer_events",
             "buffer_id" => self.buffer_id.clone(),
             "stage" => self.idx.to_string()
         )
         .set(self.total_count as f64);
+        gauge!(
+            "buffer_size_events",
+            "buffer_id" => self.buffer_id.clone(),
+            "stage" => self.idx.to_string()
+        )
+        .set(self.total_count as f64);
+        gauge!(
+            "buffer_size_bytes",
+            "buffer_id" => self.buffer_id.clone(),
+            "stage" => self.idx.to_string()
+        )
+        .set(self.total_byte_size as f64);
+        // DEPRECATED: buffer-bytes-events-metrics
         gauge!(
             "buffer_byte_size",
             "buffer_id" => self.buffer_id,

lib/vector-buffers/src/topology/channel/limited_queue.rs

@@ -121,6 +121,8 @@ struct Metrics {
     // field, so we need to suppress the warning here.
     #[expect(dead_code)]
     max_gauge: Gauge,
+    #[expect(dead_code)]
+    legacy_max_gauge: Gauge,
     #[cfg(test)]
     recorded_values: Arc<Mutex<Vec<usize>>>,
 }
@@ -133,11 +135,18 @@ impl Metrics {
         ewma_alpha: Option<f64>,
     ) -> Self {
         let ChannelMetricMetadata { prefix, output } = metadata;
-        let (gauge_suffix, max_value) = match limit {
-            MemoryBufferSize::MaxEvents(max_events) => ("_max_event_size", max_events.get() as f64),
-            MemoryBufferSize::MaxSize(max_bytes) => ("_max_byte_size", max_bytes.get() as f64),
+        let (legacy_suffix, gauge_suffix, max_value) = match limit {
+            MemoryBufferSize::MaxEvents(max_events) => (
+                "_max_event_size",
+                "_max_size_events",
+                max_events.get() as f64,
+            ),
+            MemoryBufferSize::MaxSize(max_bytes) => {
+                ("_max_byte_size", "_max_size_bytes", max_bytes.get() as f64)
+            }
         };
         let max_gauge_name = format!("{prefix}{gauge_suffix}");
+        let legacy_max_gauge_name = format!("{prefix}{legacy_suffix}");
         let histogram_name = format!("{prefix}_utilization");
         let gauge_name = format!("{prefix}_utilization_level");
         let mean_name = format!("{prefix}_utilization_mean");
@@ -147,24 +156,32 @@ impl Metrics {
         if let Some(label_value) = output {
             let max_gauge = gauge!(max_gauge_name, "output" => label_value.clone());
             max_gauge.set(max_value);
+            // DEPRECATED: buffer-bytes-events-metrics
+            let legacy_max_gauge = gauge!(legacy_max_gauge_name, "output" => label_value.clone());
+            legacy_max_gauge.set(max_value);
             Self {
                 histogram: histogram!(histogram_name, "output" => label_value.clone()),
                 gauge: gauge!(gauge_name, "output" => label_value.clone()),
                 mean_gauge: gauge!(mean_name, "output" => label_value.clone()),
                 max_gauge,
                 ewma,
+                legacy_max_gauge,
                 #[cfg(test)]
                 recorded_values,
             }
         } else {
             let max_gauge = gauge!(max_gauge_name);
             max_gauge.set(max_value);
+            // DEPRECATED: buffer-bytes-events-metrics
+            let legacy_max_gauge = gauge!(legacy_max_gauge_name);
+            legacy_max_gauge.set(max_value);
             Self {
                 histogram: histogram!(histogram_name),
                 gauge: gauge!(gauge_name),
                 mean_gauge: gauge!(mean_name),
                 max_gauge,
                 ewma,
+                legacy_max_gauge,
                 #[cfg(test)]
                 recorded_values,
             }

lib/vector-core/src/source_sender/tests.rs

@@ -266,7 +266,7 @@ async fn emits_buffer_utilization_histogram_on_send_and_receive() {
         .into_iter()
         .filter(|metric| metric.name().starts_with("source_buffer_"))
         .collect();
-    assert_eq!(metrics.len(), 4, "expected 4 utilization metrics");
+    assert_eq!(metrics.len(), 5, "expected 5 utilization metrics");
 
     let find_metric = |name: &str| {
         metrics
@@ -290,4 +290,10 @@ async fn emits_buffer_utilization_histogram_on_send_and_receive() {
         panic!("source_buffer_max_event_size should be a gauge");
     };
     assert_eq!(*value, buffer_size as f64);
+
+    let metric = find_metric("source_buffer_max_size_events");
+    let MetricValue::Gauge { value } = metric.value() else {
+        panic!("source_buffer_max_size_events should be a gauge");
+    };
+    assert_eq!(*value, buffer_size as f64);
 }

src/test_util/components.rs

@@ -66,11 +66,12 @@ pub const HTTP_SINK_TAGS: [&str; 2] = ["endpoint", "protocol"];
 pub const AWS_SINK_TAGS: [&str; 2] = ["protocol", "region"];
 
 /// The set of suffixes that define the source/transform buffer metric family.
-const BUFFER_METRIC_SUFFIXES: [&str; 3] = [
-    // While hypothetically possible, the `max_byte_size` metric is never actually emitted, because
-    // both sources and transforms limit their buffers by event count. If we ever allow
-    // configuration by byte size, we will need to account for this in these tests.
+const BUFFER_METRIC_SUFFIXES: [&str; 4] = [
+    // While hypothetically possible, the `max_byte_size`/`max_size_bytes` metrics are never
+    // actually emitted, because both sources and transforms limit their buffers by event count. If
+    // we ever allow configuration by byte size, we will need to account for this in these tests.
     "max_event_size",
+    "max_size_events",
     "utilization",
     "utilization_level",
 ];

website/cue/reference/components.cue

@@ -469,8 +469,12 @@ components: {
 	}
 
 	#MetricOutput: [Name=string]: {
-		description:        string
-		relevant_when?:     string
+		description:    string
+		relevant_when?: string
+		deprecated:     bool | *false
+		if deprecated {
+			deprecated_message?: string
+		}
 		tags?:              #MetricTags
 		name?:              Name
 		type?:              #MetricType

website/cue/reference/components/sinks.cue

@@ -663,6 +663,8 @@ components: sinks: [Name=string]: {
 	telemetry: metrics: {
 		buffer_byte_size:                     components.sources.internal_metrics.output.metrics.buffer_byte_size
 		buffer_discarded_events_total:        components.sources.internal_metrics.output.metrics.buffer_discarded_events_total
+		buffer_size_bytes:                    components.sources.internal_metrics.output.metrics.buffer_size_bytes
+		buffer_size_events:                   components.sources.internal_metrics.output.metrics.buffer_size_events
 		buffer_events:                        components.sources.internal_metrics.output.metrics.buffer_events
 		buffer_received_events_total:         components.sources.internal_metrics.output.metrics.buffer_received_events_total
 		buffer_received_event_bytes_total:    components.sources.internal_metrics.output.metrics.buffer_received_event_bytes_total

website/cue/reference/components/sources.cue

@@ -419,6 +419,8 @@ components: sources: [Name=string]: {
 		source_lag_time_seconds:              components.sources.internal_metrics.output.metrics.source_lag_time_seconds
 		source_buffer_max_byte_size:          components.sources.internal_metrics.output.metrics.source_buffer_max_byte_size
 		source_buffer_max_event_size:         components.sources.internal_metrics.output.metrics.source_buffer_max_event_size
+		source_buffer_max_size_bytes:         components.sources.internal_metrics.output.metrics.source_buffer_max_size_bytes
+		source_buffer_max_size_events:        components.sources.internal_metrics.output.metrics.source_buffer_max_size_events
 		source_buffer_utilization:            components.sources.internal_metrics.output.metrics.source_buffer_utilization
 		source_buffer_utilization_level:      components.sources.internal_metrics.output.metrics.source_buffer_utilization_level
 		source_buffer_utilization_mean:       components.sources.internal_metrics.output.metrics.source_buffer_utilization_mean