Update general event guidance and policies to allow complex attributes on events only

.chloggen/complex-attributes-on-events.yaml

@@ -0,0 +1,7 @@
+change_type: enhancement
+component: events
+note: >
+  Update general event guidance to allow complex attributes on events and use them
+  instead of the body fields.
+
+issues: [1651, 1669]

.vscode/settings.json

@@ -1,14 +1,5 @@
 {
-    "rewrap.wrappingColumn": 80,
     "editor.rulers": [80],
-    "markdownlint.config": {
-        "MD004": false,
-        "MD013": false,
-        "MD024": {"allow_different_nesting": true},
-        "MD029": {"style": "ordered"},
-        "MD033": false,
-        "MD040": false,
-    },
     "yaml.schemas": {
         "https://raw.githubusercontent.com/open-telemetry/weaver/v0.15.0/schemas/semconv.schema.json": [
             "model/**/*.yaml"

docs/general/events.md

@@ -17,74 +17,29 @@ network connection severed, etc.
 
 In OpenTelemetry, Events are implemented as a specific type of [`LogRecord`](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.45.0/specification/logs/api.md) that conforms to the conventions included here.
 
-In addition to a required name, an Event may contain a _payload_ (body) of any type permitted
-by the [LogRecord body](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model.md#field-body).
-In its implementation, the Event _payload_ (body) will constitute the `Body` of the `LogRecord`.
-Like all other OpenTelemetry signals, an Event has optional attribute metadata that helps describe
-the event context.
-
-Over time, some Events will be specified by OpenTelemetry and will have documented payload structure,
-field semantics, and stability and requirement levels. Other events may be user-defined and carry
-bespoke user semantics. When an Event name exists in the semantic conventions, its _payload_
-structure and semantics will also be defined.
+OpenTelemetry Semantic Conventions that define events SHOULD document the event name along
+with attributes and the type of the body if any.
 
 ## General event semantics
 
 * An event MUST have an [Event name property](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.45.0/specification/logs/data-model.md#field-eventname)
-  that uniquely identifies the event. Event names are subject to the [Naming guidelines](/docs/general/naming.md).
-* Event MAY have [standard](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.45.0/specification/common#attribute)
-  attributes that provide additional context about the event.
-* It MAY contain a _payload_ (body) that describes the specific details of the
-  named event.
-* The event name uniquely identifies event structure / type of the _payload_ (body)
-  and the set of attributes.
-* The _payload_ (body) MAY contain any type supported by the OpenTelemetry data
-  model for the log [body](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model.md#field-body)
-  and the semantic conventions will define the expected structure of the _payload_
-  (body) for the event.
-* The _payload_ (body) SHOULD be used to represent the structure of the event.
-
-Recommendations for defining events:
-
-* Use the _payload_ (body) to represent the details of the event instead of a
-  collection of [standard](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.45.0/specification/common#attribute)
-  attributes.
-* Events SHOULD be generated / produced / recorded using the
-    [Logs API](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.45.0/specification/logs/api.md#emit-a-logrecord)
-    to ensure that the event is created using the configured SDK instance.
-  * The Emit Event API is not yet available in all OpenTelemetry SDKs. Check [spec-compliance matrix](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.45.0/spec-compliance-matrix.md#logs) to see the implementation status in corresponding language.
-* It's NOT RECOMMENDED to prefix the _payload_ (body) _fields_ with the `EventName` to
-  avoid redundancy and to keep the event definition clean.
-* The events SHOULD document their semantic conventions including event name,
-  attributes, and the payload.
-
-Recommendations on using attributes vs. body fields:
-
-* If the field should be comparable across events with different `EventName` (or between an event and other telemetry items),
-  it should be an attribute.
-* If the field is specific to the event itself, then it should be a body field.
-* Body fields that belong to events with different event names are not comparable.
-  For example, body field `id` on event `my_company.order_submitted` is semantically different from
-  field `id` on an event with name `session.start`.
+  that uniquely identifies the event. Event names are subject to the [Naming guidelines](/docs/general/naming.md). Event name SHOULD be of a low-cardinality.
+* Events MAY have [attributes](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.45.0/specification/logs/data-model.md#field-attributes)
+  that provide additional context about the event.
+* Events MAY contain a [body](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.45.0/specification/logs/data-model.md#field-body) of type [`any`](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.45.0/specification/logs/data-model.md#type-any).
+* The event name uniquely identifies event structure: the set of attributes and
+  the type of the body.
+* Events MAY have [severity number](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.45.0/specification/logs/data-model.md#field-severitynumber).
 
-### Event payload (body)
+It's RECOMMENDED to use the collection of attributes to represent the details of
+the event.
 
-* Common attribute naming conventions and [registry](../registry/attributes/README.md)
-  requirements don't apply to event payload fields.
-* The definition for OpenTelemetry defined events supports describing
-  individual _fields_ (Body Fields)
-  * The _fields_ are unique to the named event ([EventName](https://github.com/open-telemetry/opentelemetry-specification/tree/v1.45.0/specification/logs/data-model.md#field-eventname)) and different events
-    may use the same _field_ name to represent different data, due to the unique
-    nature of the event.
+<!-- Body use cases are not clear - see https://github.com/open-telemetry/semantic-conventions/issues/1651 for the context.-->
 
 ## External event compatibility
 
-When recording events from an existing system as OpenTelemetry Events, it is
-possible that the existing system does not have the equivalent of a name or
-requires multiple fields to identify the structure of the events. In such cases,
-OpenTelemetry recommends using a combination of one or more fields as the name
-such that the name identifies the event structurally. It is also recommended that
-the event names have low-cardinality, so care must be taken to use fields
-that identify the class of Events but not the instance of the Event.
+When recording events from an existing system as OpenTelemetry Events, the system
+may lack a single name field or require multiple fields to identify the event.
+In such cases, use a combination of fields to create a low-cardinality event name.
 
 [DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status

docs/general/how-to-define-semantic-conventions.md

@@ -84,7 +84,15 @@ When defining a new attribute:
       rather than combining them into a single array, as they represent distinct concepts.
   - Use the template type to define attributes with dynamic names (only the last segment of the name should be dynamic).
     This is useful for capturing user-defined key-value pairs, such as HTTP headers.
-  - Represent complex values as a set of flat attributes. <!-- This may change, check out https://github.com/open-telemetry/semantic-conventions/issues/1669 to monitor the progress -->
+  - Represent complex values as a set of flat attributes whenever possible.
+    - Complex or structured attributes (not listed in the
+      [set of standard attributes](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.44.0/specification/common/README.md#attribute))
+      are supported on events and logs only. <!-- This may change, check out https://github.com/open-telemetry/opentelemetry-specification/pull/4485 to monitor the progress -->
+
+      Semantic convention authors should assume that backends do not index individual properties of complex attributes,
+      that querying or aggregating on such properties is inefficient and complicated,
+      and that reporting complex attributes carries higher performance overhead.
+
 - Define new attributes with `development` stability.
 - Provide realistic examples
 - Avoid defining attributes with potentially unbounded values, such as strings longer than

policies/attribute_types.rego

@@ -0,0 +1,24 @@
+package after_resolution
+import rego.v1
+
+# checks that complex attributes are not used in any groups except events
+deny contains group_stability_violation(description, group.id, attr.name) if {
+    group := input.groups[_]
+    group.type != "event"
+    group.type != "attribute_group"
+
+    attr := group.attributes[_]
+    attr.type in ["any", "template[any]"]
+
+    description := sprintf("Attribute '%s' has type '%s' and is referenced on group '%s' of type '%s'. Complex attributes are only allowed on events.", [attr.name, attr.type, group.id, group.type])
+}
+
+attribute_type_violation(description, group, attr) = violation if {
+    violation := {
+        "id": description,
+        "type": "semconv_attribute",
+        "category": "attribute_type_violation",
+        "attr": attr,
+        "group": group,
+    }
+}

policies_test/attribute_types_test.rego

@@ -0,0 +1,37 @@
+package after_resolution
+import future.keywords
+
+test_fails_on_complex_attribute if {
+    attribute_types := ["any", "template[any]"]
+    group_types = ["span", "entity", "resource", "metric"]
+    every attribute_type in attribute_types {
+        every group_type in group_types {
+            count(deny) == 1 with input as {"groups": [{ "id": concat(".", [group_type, "attr"]),
+                                                "type": group_type,
+                                                "stability": "development",
+                                                "attributes": [{
+                                                    "name": "test.any",
+                                                    "stability": "development",
+                                                    "type": attribute_type
+                                                }]}]}
+        }
+    }
+}
+
+
+test_pass_on_complex_attribute if {
+    attribute_types := ["any", "template[any]"]
+    group_types = ["attribute_group", "event"]
+    every attribute_type in attribute_types {
+        every group_type in group_types {
+            count(deny) == 0 with input as {"groups": [{ "id": concat(".", [group_type, "attr"]),
+                                                "type": group_type,
+                                                "stability": "development",
+                                                "attributes": [{
+                                                    "name": "test.any",
+                                                    "stability": "development",
+                                                    "type": attribute_type
+                                                }]}]}
+        }
+    }
+}