From 9efeb892f1fcbe8e9c21157236afbb2c9a39d24e Mon Sep 17 00:00:00 2001 From: Dmitry Anoshin Date: Wed, 20 Aug 2025 22:01:14 -0700 Subject: [PATCH 1/4] [entities] Specifying entity information via an environment variable Add a document on how to inject entity information into workloads using new environment variable OTEL_ENTITIES. --- .../sdk-environment-variables.md | 1 + specification/entities/README.md | 7 + specification/entities/entity-propagation.md | 222 ++++++++++++++++++ 3 files changed, 230 insertions(+) create mode 100644 specification/entities/entity-propagation.md diff --git a/specification/configuration/sdk-environment-variables.md b/specification/configuration/sdk-environment-variables.md index b7daca0a284..b63edb8df7f 100644 --- a/specification/configuration/sdk-environment-variables.md +++ b/specification/configuration/sdk-environment-variables.md @@ -172,6 +172,7 @@ representing it MUST be `"none"`. | Name | Description | Default | Type | Notes | |--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | OTEL_SDK_DISABLED | Disable the SDK for all signals | false | [Boolean][] | If "true", a no-op SDK implementation will be used for all telemetry signals. Any other value or absence of the variable will have no effect and the SDK will remain enabled. This setting has no effect on propagators configured through the OTEL_PROPAGATORS variable. | +| OTEL_ENTITIES | Entity information to be associated with the resource | | [String][] | See [Entities SDK](../entities/entity-propagation.md#specifying-entity-information-via-an-environment-variable) for more details. | | OTEL_RESOURCE_ATTRIBUTES | Key-value pairs to be used as resource attributes | See [Resource semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/resource/README.md#semantic-attributes-with-dedicated-environment-variable) for details. | [String][] | See [Resource SDK](../resource/sdk.md#specifying-resource-information-via-an-environment-variable) for more details. | | OTEL_SERVICE_NAME | Sets the value of the [`service.name`](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/resource/README.md#service) resource attribute | | [String][] | If `service.name` is also provided in `OTEL_RESOURCE_ATTRIBUTES`, then `OTEL_SERVICE_NAME` takes precedence. | | OTEL_LOG_LEVEL | Log level used by the [SDK internal logger](../error-handling.md#self-diagnostics) | "info" | [Enum][] | | diff --git a/specification/entities/README.md b/specification/entities/README.md index be0fc3b2a7c..aea7b06d150 100644 --- a/specification/entities/README.md +++ b/specification/entities/README.md @@ -23,6 +23,13 @@ path_base_for_github_subdir: Entity represents an object of interest associated with produced telemetry: traces, metrics, logs, profiles etc. +Entities can be instantiated through two complementary approaches: + +1. **Pull-based Model**: Entities discover themselves from the current environment by examining the runtime context, system properties, metadata services, and other available sources of entity information. + +2. **Push-based Model**: Entities are explicitly provided from external sources, typically through environment variables or configuration. This approach allows entity information to be propagated across process boundaries. See [Entity Propagation](./entity-propagation.md) for more details. + ## Specifications - [Data Model](./data-model.md) +- [Entity Propagation](./entity-propagation.md) - Describes the push-based model for entity instantiation diff --git a/specification/entities/entity-propagation.md b/specification/entities/entity-propagation.md new file mode 100644 index 00000000000..8d953b244ae --- /dev/null +++ b/specification/entities/entity-propagation.md @@ -0,0 +1,222 @@ + + +# Entity Propagation + +**Status**: [Development](../document-status.md) + +
+Table of Contents + + + +- [Overview](#overview) +- [Specifying entity information via an environment variable](#specifying-entity-information-via-an-environment-variable) + * [Format Specification](#format-specification) + * [Grammar](#grammar) + * [Examples](#examples) + * [Parsing Algorithm](#parsing-algorithm) + * [Character Encoding](#character-encoding) + * [Validation Requirements](#validation-requirements) + * [Error Handling](#error-handling) + * [Environment Variable Conflict Resolution](#environment-variable-conflict-resolution) +- [EntityDetector](#entitydetector) + + + +
+ +## Overview + +Entity propagation provides a mechanism to pass entity information across +process boundaries using environment variables. This allows entities to be +shared between parent and child processes, similar to how trace context is +propagated in distributed systems. This approach is particularly useful when +the outside process knows more about the child entity than the child process +can discover on its own. + +Common scenarios where entity propagation is beneficial include: + +- Container orchestration systems where the orchestrator knows container metadata +- CI/CD pipelines where the build system knows job and environment details +- Batch processing systems where the scheduler knows task context +- Command-line tools spawned with specific entity context + +Environment variables provide a reliable, cross-platform mechanism for this +propagation since they are automatically inherited by child processes and +available during process initialization. + +## Specifying entity information via an environment variable + +To enable standardized entity propagation across OpenTelemetry implementations, +this specification defines the `OTEL_ENTITIES` environment variable format and +processing requirements. + +The SDK SHOULD provide an `EntityDetector` which will use the `OTEL_ENTITIES` environment +variable to discover and associate defined entities with the resource. + +The `OTEL_ENTITIES` environment variable contains a list of entities in a +compact format designed for human readability and concise representation. + +### Format Specification + +Each entity follows this structure: + +``` +type{id_key1=id_value1,id_key2=id_value2}[desc_key1=desc_value1,desc_key2=desc_value2]@schema_url +``` + +Where: + +- `type` is the entity type (required, e.g., "service", "host", "container") +- `{...}` contains identifying attributes as comma-separated key=value pairs (required, at least one pair) +- `[...]` contains descriptive attributes as comma-separated key=value pairs (optional) +- `@schema_url` specifies the Schema URL for the entity (optional) + +Multiple entities are separated by semicolons (`;`). + +### Grammar + +``` +entities := entity (";" entity)* +entity := type id_attrs desc_attrs? schema_url? | "" +type := [a-zA-Z][a-zA-Z0-9._-]* +id_attrs := "{" key_value_list "}" +desc_attrs := "[" key_value_list "]" +schema_url := "@" url_string +key_value_list := key_value ("," key_value)* +key_value := key "=" value +key := [a-zA-Z][a-zA-Z0-9._-]* +value := [^{}[\]@;,=]* +url_string := [^;]* +``` + +### Examples + +```bash +# Single service entity +OTEL_ENTITIES="service{service.name=my-app,service.instance.id=instance-1}[service.version=1.0.0]" + +# Multiple entities with schema URL +OTEL_ENTITIES="service{service.name=my-app,service.instance.id=instance-1}[service.version=1.0.0]@https://opentelemetry.io/schemas/1.21.0;host{host.id=host-123}[host.name=web-server-01]" + +# Kubernetes pod entity +OTEL_ENTITIES="k8s.pod{k8s.pod.uid=pod-abc123}[k8s.pod.name=my-pod,k8s.pod.label.app=my-app]" + +# Container with host (minimal descriptive attributes) +OTEL_ENTITIES="container{container.id=cont-456};host{host.id=host-789}[host.name=docker-host]" + +# Minimal entity (only required fields) +OTEL_ENTITIES="service{service.name=minimal-app}" + +# Empty strings are allowed (leading, trailing, and consecutive semicolons are ignored) +OTEL_ENTITIES=";service{service.name=app1};;host{host.id=host-123};" +``` + +### Parsing Algorithm + +1. Split the input string by semicolons (`;`) to get individual entity definitions +2. For each entity definition: + a. Skip if the entity definition is empty (allows consecutive semicolons and leading/trailing semicolons) + b. Extract the entity type (everything before the first `{`) + c. Extract identifying attributes from `{...}` block + d. Extract descriptive attributes from `[...]` block (if present) + e. Extract schema URL from `@...` portion (if present) +3. Parse key-value lists using comma (`,`) as separator and equals (`=`) for assignment +4. Validate that each entity has a non-empty type and at least one identifying attribute +5. Create entity objects and associate them with the resource + +### Character Encoding + +All attribute values MUST be considered strings and characters outside the +`baggage-octet` range MUST be percent-encoded following the [W3C Baggage](https://www.w3.org/TR/baggage/#header-content) specification. + +The reserved characters `{}[]@;,=` MUST be percent-encoded when they appear literally in attribute values: + +- `{` → `%7B` +- `}` → `%7D` +- `[` → `%5B` +- `]` → `%5D` +- `@` → `%40` +- `;` → `%3B` +- `,` → `%2C` +- `=` → `%3D` + +**Example:** + +```bash +# Entity with reserved characters in attribute values +OTEL_ENTITIES="service{service.name=my%2Capp,service.instance.id=inst-1}[config=key%3Dvalue%5Bprod%5D]" +# Resolves to: service.name="my,app", config="key=value[prod]" +``` + +### Validation Requirements + +- Entity type MUST NOT be empty and MUST match the pattern `[a-zA-Z][a-zA-Z0-9._-]*` +- At least one identifying attribute MUST be present in the `{...}` block +- Attribute keys MUST NOT be empty and SHOULD follow OpenTelemetry semantic conventions +- Schema URL, if present, MUST be a valid URI +- Entity types SHOULD follow existing OpenTelemetry entity naming conventions (e.g., "service", "host", "container", "k8s.pod") + +### Error Handling + +The SDK SHOULD be resilient to malformed input and follow these error handling rules: + +1. **Invalid syntax**: If the environment variable contains invalid syntax, the SDK SHOULD log a warning and ignore the malformed portions while processing valid parts + + Example: `OTEL_ENTITIES="service{service.name=app1};invalid{syntax;service{service.name=app2}"` processes the first valid entity and skips the malformed part + +2. **Missing required fields**: If an entity is missing required fields (type or identifying attributes), the SDK SHOULD log a warning and skip that entity + + Example: `OTEL_ENTITIES="service{};host{host.id=123}"` skips the service entity (no identifying attributes) and processes the host entity + +3. **Duplicate entities**: If multiple entities of the same type with identical identifying attributes are defined, the SDK SHOULD use the last occurrence and SHOULD log a warning + + Example: `OTEL_ENTITIES="service{service.name=app1}[version=1.0];service{service.name=app1}[version=2.0]"` uses `version=2.0` + +4. **Schema URL validation**: If a schema URL is present but invalid, the SDK SHOULD log a warning and ignore the URL while processing the entity + + Example: `OTEL_ENTITIES="service{service.name=app1}@invalid-url"` processes the entity but ignores the invalid URL + +5. **Conflicting identifying attributes**: If two entities of the same type define different values for the same identifying attribute key, the SDK SHOULD treat them as separate entities. If this results in ambiguous entity identification, the SDK SHOULD log a warning and preserve only the last entity + + Example: `OTEL_ENTITIES="service{service.name=app1};service{service.name=app2}"` creates only service.name=app2 entity + +6. **Conflicting descriptive attributes**: If two entities define different values for the same descriptive attribute key, the SDK SHOULD use the value from the last entity definition and SHOULD log a warning. The conflicting attributes SHOULD NOT be recorded for entities other than the last one + + Example: `OTEL_ENTITIES="service{service.name=app1}[version=1.0];service{service.name=app2}[version=2.0]"` results in app1 service without version attribute and app2 service with `version=2.0` + +### Environment Variable Conflict Resolution + +When multiple environment variables define overlapping configuration, a warning should be logged and the following precedence order applied (highest to lowest precedence): + +1. **Programmatic configuration**: Entities configured via SDK API take highest precedence and override all environment variable configurations +2. **OTEL_ENTITIES**: Entity definitions from this environment variable override resource attributes and other OTEL_* environment variables for the same attribute keys +3. **OTEL_SERVICE_NAME**: Takes precedence over `service.name` in `OTEL_RESOURCE_ATTRIBUTES` but is overridden by `service.name` in `OTEL_ENTITIES` +4. **Other specific OTEL_* variables**: Individual environment variables (when they exist) take precedence over equivalent attributes in `OTEL_RESOURCE_ATTRIBUTES` +5. **OTEL_RESOURCE_ATTRIBUTES**: Lowest precedence for overlapping attribute keys + +**Precedence rules within OTEL_ENTITIES:** + +- When the same entity type appears multiple times with identical identifying attributes, the last occurrence takes precedence + +**Example of conflict resolution:** + +```bash +# These environment variables: +OTEL_SERVICE_NAME="old-service-name" +OTEL_RESOURCE_ATTRIBUTES="service.name=resource-service,host.name=resource-host,custom.attr=resource-value" +OTEL_ENTITIES="service{service.name=entity-service,service.instance.id=inst-1}[service.version=1.0.0];host{host.id=host-123}[host.name=entity-host]" + +# Result in: +# - Service entity with identifying attributes: {service.name=entity-service, service.instance.id=inst-1} (from OTEL_ENTITIES, overrides others) +# and descriptive attributes: {service.version=1.0.0} +# - Host entity with identifying attributes: {host.id=host-123} +# and descriptive attributes: {host.name=entity-host} (from OTEL_ENTITIES, overrides OTEL_RESOURCE_ATTRIBUTES) +# - Resource: remaining attributes from OTEL_RESOURCE_ATTRIBUTES that don't conflict: {custom.attr=resource-value} +``` + +## EntityDetector + +TODO: fill out From 2649534bc82bae5e2ca90d63fa4f8a28f28a0fbc Mon Sep 17 00:00:00 2001 From: Dmitry Anoshin Date: Thu, 4 Sep 2025 08:13:26 -0700 Subject: [PATCH 2/4] review feedback --- specification/entities/entity-propagation.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/specification/entities/entity-propagation.md b/specification/entities/entity-propagation.md index 8d953b244ae..bab86acd558 100644 --- a/specification/entities/entity-propagation.md +++ b/specification/entities/entity-propagation.md @@ -21,7 +21,7 @@ linkTitle: Entity Propagation * [Validation Requirements](#validation-requirements) * [Error Handling](#error-handling) * [Environment Variable Conflict Resolution](#environment-variable-conflict-resolution) -- [EntityDetector](#entitydetector) +- [EnvEntityDetector](#Enventitydetector) @@ -53,8 +53,9 @@ To enable standardized entity propagation across OpenTelemetry implementations, this specification defines the `OTEL_ENTITIES` environment variable format and processing requirements. -The SDK SHOULD provide an `EntityDetector` which will use the `OTEL_ENTITIES` environment -variable to discover and associate defined entities with the resource. +The SDK that has access to environment variables MUST provide +an `EnvEntityDetector` which will use the `OTEL_ENTITIES` environment variable +to discover and associate defined entities with the resource. The `OTEL_ENTITIES` environment variable contains a list of entities in a compact format designed for human readability and concise representation. @@ -217,6 +218,6 @@ OTEL_ENTITIES="service{service.name=entity-service,service.instance.id=inst-1}[s # - Resource: remaining attributes from OTEL_RESOURCE_ATTRIBUTES that don't conflict: {custom.attr=resource-value} ``` -## EntityDetector +## EnvEntityDetector TODO: fill out From c941e30aff534c539d515ba265d3aef8bed6035a Mon Sep 17 00:00:00 2001 From: Dmitry Anoshin Date: Thu, 4 Sep 2025 18:25:51 -0700 Subject: [PATCH 3/4] PR review feedback --- specification/entities/entity-propagation.md | 33 +------------------- 1 file changed, 1 insertion(+), 32 deletions(-) diff --git a/specification/entities/entity-propagation.md b/specification/entities/entity-propagation.md index bab86acd558..7178a684d4f 100644 --- a/specification/entities/entity-propagation.md +++ b/specification/entities/entity-propagation.md @@ -20,7 +20,6 @@ linkTitle: Entity Propagation * [Character Encoding](#character-encoding) * [Validation Requirements](#validation-requirements) * [Error Handling](#error-handling) - * [Environment Variable Conflict Resolution](#environment-variable-conflict-resolution) - [EnvEntityDetector](#Enventitydetector) @@ -180,7 +179,7 @@ The SDK SHOULD be resilient to malformed input and follow these error handling r Example: `OTEL_ENTITIES="service{service.name=app1}@invalid-url"` processes the entity but ignores the invalid URL -5. **Conflicting identifying attributes**: If two entities of the same type define different values for the same identifying attribute key, the SDK SHOULD treat them as separate entities. If this results in ambiguous entity identification, the SDK SHOULD log a warning and preserve only the last entity +5. **Conflicting identifying attributes**: If two entities of the same type define different values for the same identifying attribute key, the SDK SHOULD log a warning and preserve only the last entity Example: `OTEL_ENTITIES="service{service.name=app1};service{service.name=app2}"` creates only service.name=app2 entity @@ -188,36 +187,6 @@ The SDK SHOULD be resilient to malformed input and follow these error handling r Example: `OTEL_ENTITIES="service{service.name=app1}[version=1.0];service{service.name=app2}[version=2.0]"` results in app1 service without version attribute and app2 service with `version=2.0` -### Environment Variable Conflict Resolution - -When multiple environment variables define overlapping configuration, a warning should be logged and the following precedence order applied (highest to lowest precedence): - -1. **Programmatic configuration**: Entities configured via SDK API take highest precedence and override all environment variable configurations -2. **OTEL_ENTITIES**: Entity definitions from this environment variable override resource attributes and other OTEL_* environment variables for the same attribute keys -3. **OTEL_SERVICE_NAME**: Takes precedence over `service.name` in `OTEL_RESOURCE_ATTRIBUTES` but is overridden by `service.name` in `OTEL_ENTITIES` -4. **Other specific OTEL_* variables**: Individual environment variables (when they exist) take precedence over equivalent attributes in `OTEL_RESOURCE_ATTRIBUTES` -5. **OTEL_RESOURCE_ATTRIBUTES**: Lowest precedence for overlapping attribute keys - -**Precedence rules within OTEL_ENTITIES:** - -- When the same entity type appears multiple times with identical identifying attributes, the last occurrence takes precedence - -**Example of conflict resolution:** - -```bash -# These environment variables: -OTEL_SERVICE_NAME="old-service-name" -OTEL_RESOURCE_ATTRIBUTES="service.name=resource-service,host.name=resource-host,custom.attr=resource-value" -OTEL_ENTITIES="service{service.name=entity-service,service.instance.id=inst-1}[service.version=1.0.0];host{host.id=host-123}[host.name=entity-host]" - -# Result in: -# - Service entity with identifying attributes: {service.name=entity-service, service.instance.id=inst-1} (from OTEL_ENTITIES, overrides others) -# and descriptive attributes: {service.version=1.0.0} -# - Host entity with identifying attributes: {host.id=host-123} -# and descriptive attributes: {host.name=entity-host} (from OTEL_ENTITIES, overrides OTEL_RESOURCE_ATTRIBUTES) -# - Resource: remaining attributes from OTEL_RESOURCE_ATTRIBUTES that don't conflict: {custom.attr=resource-value} -``` - ## EnvEntityDetector TODO: fill out From adff6136ba2c8c500dc1a06aa742153d91c1cd7e Mon Sep 17 00:00:00 2001 From: Dmitry Anoshin Date: Thu, 4 Sep 2025 18:52:51 -0700 Subject: [PATCH 4/4] make markdown-toc --- specification/entities/entity-propagation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/entities/entity-propagation.md b/specification/entities/entity-propagation.md index 7178a684d4f..e3863169f60 100644 --- a/specification/entities/entity-propagation.md +++ b/specification/entities/entity-propagation.md @@ -20,7 +20,7 @@ linkTitle: Entity Propagation * [Character Encoding](#character-encoding) * [Validation Requirements](#validation-requirements) * [Error Handling](#error-handling) -- [EnvEntityDetector](#Enventitydetector) +- [EnvEntityDetector](#enventitydetector)