Analyze the overlap between OpenTelemetry tracing attributes and ECS attributes

[trask]

HTTP

• Generalize the http.url, http.scheme, and http.target attributes to url.* (aligning with ECS) opentelemetry-specification#3181
• Rename http.* attributes to align with ECS opentelemetry-specification#3182
• Rename net peer/host attributes to align with ECS opentelemetry-specification#3199
• Rename net.protocol.*, net.sock.family and net.transport attributes to align with ECS opentelemetry-specification#3371

Exceptions

• Rename exception.* to error.* (aligning with ECS) opentelemetry-specification#3198

General remote service attributes

@AlexanderWert to confirm that ECS doesn't have an equivalent to peer.service.

General identity attributes

OpenTelemetry	ECS
enduser.id	user.id
enduser.role, enduser.scope	user.roles (?)

General thread attributes

OpenTelemetry	ECS
thread.id	process.thread.id
thread.name	process.thread.name

Source code attributes

OpenTelemetry	ECS
code.function	log.origin.function
code.namespace
code.filepath	log.origin.file.name
code.lineno	log.origin.file.line
code.column

Messaging

This is also discussed/tracked at open-telemetry/opentelemetry-specification#3196.

Only impacted by network attribute changes above (#3199 and open-telemetry/opentelemetry-specification#3371).

(ECS doesn't have any messaging-specific attributes, @AlexanderWert to confirm)

Could also be impacted by conflict with source.* and destination.* namespace, see open-telemetry/opentelemetry-specification#3407

Database

Only impacted by network attribute changes above (#3199 and open-telemetry/opentelemetry-specification#3371).

(ECS doesn't have any database-specific attributes, @AlexanderWert to confirm).

RPC

Only impacted by network attribute changes above (#3199 and open-telemetry/opentelemetry-specification#3371).

(ECS doesn't have any RPC-specific attributes, @AlexanderWert to confirm).

FaaS tracing and resources

OpenTelemetry	ECS
faas.coldstart	faas.coldstart
faas.invocation_id	faas.execution
cloud.resource_id	faas.id
faas.name	faas.name
	faas.trigger.request_id
faas.trigger	faas.trigger.type
faas.version	faas.version

CloudEvents

ECS doesn't have any cloud event specific attributes (@AlexanderWert to confirm)

Feature Flags

ECS doesn't have any feature flag specific attributes (@AlexanderWert to confirm)

[Oberon00]

I want to call out two differences for FaaS that were just very recently introduced in OTel:

• faas.id vs cloud.resource_id: This was just recently renamed from the ECS name to the different name at Expand scope of faas.id to cloud.resource_id opentelemetry-specification#3188
• faas.execution vs. faas.invocation_id: The same in Rename faas.execution to faas.invocation_id and change units for faas.max_memory opentelemetry-specification#3209
• (Interestingly, ECS has a separate faas.trigger.request_id (CC @damemi @jsuereth), a concept we considered identical to faas.execution as per Rename faas.execution to faas.invocation_id and change units for faas.max_memory opentelemetry-specification#3209 above.)

The first two points, I noted above open-telemetry/opentelemetry-specification#3188 (comment) but it was understood then that there is no wider overhaul of OTel semconvs for ECS alignment?

[trask]

@Oberon00 thx for calling out the ECS alignment issue in open-telemetry/opentelemetry-specification#3188, sorry I missed that

@cartersocha @tylerbenson we do intend to factor in ECS alignment across all semantic conventions (open-telemetry/oteps#222)

where there are conflicts, there's no blanket rule about which side (or mix) we will adopt going forward, and so we need to analyze these conflicts on a case-by-case basis, factoring in a few things (among potentially others):

• are there inherent advantages to one modeling or the other?
• is the naming clearer on one side or the other?
• how widespread is adoption of both sides?

this is what we need to gather in this issue so that we can make these case-by-case decisions

[tsloughter]

I think the naming is clearer on our side. These were very recent changes because we felt the naming was clearer.

Also, I don't think faas.trigger.request_id is the same as our invocation_id, but I'm not really sure what it is. It sounds like it'd be like if the trigger is HTTP and that contains a request id header.

[Oberon00]

HTTP is not mentioned. Note that in open-telemetry/opentelemetry-specification#3378 it was noted that our faas.invocation_id might not be suitable for a (job) execution_id, I imagine this might have been different with the old name. So maybe there is indeed space for both an execution(_id) and an invocation_id/request ID

[cartersocha]

@trask the OTel version is slightly more up to date and vendor neutral (although still Lambda skewed). @Oberon00 called out most of the changes already and presciently mentioned ECS previously.

I believe ECS is missing the faas.max_memory field and ECS uses a lambda specific faas.id like we used to. Personally I think there's also a lot of value in having a unique cloud.resource_id that is consistent for all cloud resources. The AWS ARN is already that but it's currently restricted to just being a faas specific field.

[Oberon00]

uses a lambda specific faas.id like we used to

This is not Lambda-specific, it has definitions for Azure and Google Cloud Functions as well

+1 on the other parts

[cartersocha]

Yeah I misspoke (wrote?). main point is it's not a faas specific field but a general cloud resource attribute that functions also have!

[tsloughter]

@Oberon00 HTTP is one of the trigger types.

[Oberon00]

@tsloughter There is a separate http.request.id. https://www.elastic.co/guide/en/ecs/current/ecs-http.html#field-http-request-id

[tsloughter]

@Oberon00 right, seems redundant. Unless it has some other meaning, it doesn't make it clear and the example is just a number.

[Oberon00]

Why do you think that faas.trigger.request_id is specific to HTTP? I could also be used for any other request ID that is used to trigger the Lambda / FaaS function. But I agree that it is very hard to interpret. Someone from ECS needs to tell us what this is meant to be. Or we just drop it and use our own conventions for FaaS

[trask]

@AlexanderWert can you or someone on the ECS side provide clarity on faas.trigger.request_id? (see above discussion for context)

[AlexanderWert]

Also, I don't think faas.trigger.request_id is the same as our invocation_id, but I'm not really sure what it is. It sounds like it'd be like if the trigger is HTTP and that contains a request id header.

The intention of faas.trigger.request_id was to capture an identifier of the trigger event. For HTTP (e.g. through Api Gateway on AWS in case of lambda) it's the API Gateway request ID, in case of S3 events it's the x-amz-request-id (see here: https://docs.aws.amazon.com/lambda/latest/dg/with-s3.html). but I admit that it mostly makes sense with HTTP, since with other events there are often batches of events triggering a function invocation. So it's not the same as faas.invocation_id in OTel SemConv.

In general, the faas.* fields in ECS were introduced based on and in alignment with the existing OTel faas.* attributes at that time (only exception was the faas.trigger.type and faas.trigger.request_id). Also, faas.* fields in ECS are still in Beta, so it's an area where it makes more sense to align with existing OTel attributes.

If we adopt http.request.id from ECS, that could be a good way to capture the above (faas.trigger.request_id).

[AlexanderWert]

General remote service attributes
@AlexanderWert to confirm that ECS doesn't have an equivalent to peer.service.

ECS service.* fields can be self-nested under service.origin.* and service.target.*, which has a similar purpose as the peer.service in OTel. Similar to our previous discussions on client / server and source / destination, here service.origin.* and service.target.* make the relationship more explicit, without the need to know the context of the field.

Messaging
(ECS doesn't have any messaging-specific attributes, @AlexanderWert to confirm)

👍

RPC

Only impacted by network attribute changes above (open-telemetry/opentelemetry-specification#3199 and open-telemetry/opentelemetry-specification#3371).

(ECS doesn't have any RPC-specific attributes, @AlexanderWert to confirm).

👍

Database

Only impacted by network attribute changes above (open-telemetry/opentelemetry-specification#3199 and open-telemetry/opentelemetry-specification#3371).

(ECS doesn't have any database-specific attributes, @AlexanderWert to confirm).

👍

ECS doesn't have any cloud event specific attributes (@AlexanderWert to confirm)

That's correct that there are no cloud events fields in ECS. Though, there is a more generic field set for event.*. So, there is some potential overlap here:

OTel	ECS
cloudevents.event_id	event.id
cloudevents.event_source	event.provider
cloudevents.event_spec_version
cloudevents.event_type	event.type
cloudevents.event_subject

ECS doesn't have any feature flag specific attributes (@AlexanderWert to confirm)

👍

[svrnm]

Is this done?

[trask]

Is this done?

I suspect it can be closed. I'll transfer it to semconv repo and let maintainers there decide.

[AlexanderWert]

Closing as we'll track the individual areas while working on SemConv / continuously contributing ECS fields to SemConv