[receiver/elasticapmintake][processor/elasticapm] Update resource attribute mappings to use semconv to fix aggregated metrics

[isaacaflores2]

Overview

The goal of this PR is to update the intake receiver attribute mappings to use semconv keys to fix missing attributes in aggregate metrics that are generated by the connector/elasticapm.

The connector/elasticapm includes resource attributes defined here. These resource attributes use OTel semconv.

Receiver Updates

The below table summarizes the new mappings.
Note: The language normalization was removed to closely match the apm-data behavior. So if an apm-events has language "Java" we keep it as is now.

Source field	Target field	Notes
service.language.name	telemetry.sdk.language	Now copied directly from intake value (no language normalization like c# -> dotnet).
service.language.version	telemetry.sdk.version	Newly mapped as semconv telemetry SDK version.
service.runtime.name	process.runtime.name	Moved from legacy service.runtime.* to semconv process runtime.
service.runtime.version	process.runtime.version	Moved from legacy service.runtime.* to semconv process runtime.
host.os.platform	os.type	Receiver now emits semconv OS type; connector expects this representation.

Processor Updates

The processor/elasticapm was updated to consider the new semconv fields as supported attributes. The elastic attributes were removed since the receiver/elasticapmintake will no longer emit them.

Exporter

SemConv mappings

These changes rely on the exporter/elasticsearch to map the telemetry.sdk.* attributes back to service.language for the ECS mapping mode: open-telemetry/opentelemetry-collector-contrib#46690. Without the exporter updates the telemetry.sdk.* attributes will be dropped

Host.OS.Type Enrichment in ECS mode

The exporter derives host.os.type here using os.type and os.name. Since this PR converts host.os.platform to os.type the exporter now has a side effect of adding the host.os.type attributes to indexed docs. This field is only needed for otlp events based on the apm-data logic here

Solution:

1. Add host.os.type enrichment to the processor/elasticapm
   • Disable host.os.type enrichment for events ingested by the receiver/elasticapmintake
     PR : [processor/elasticapm] Add host.os.type enrichment for OTLP events with ECS mapping mode only #1087
2. Remove host.os.type enrichment from the exporter/elasticsearch
   I will follow up with separate PRs for this

Testing

1. Updated test cases

I need to be merged #1067 first since I am having version conflict issues on my local collector. This will allow me to finalize my testing

[github-actions]

Generated by PR Review Agent for issue #1069

[coderabbitai]

📝 Walkthrough

Walkthrough

This pull request removes seven elasticattr constants (HostOSPlatform, ProcessRuntimeName, ProcessRuntimeVersion, ServiceLanguageName, ServiceLanguageVersion, ServiceRuntimeName, ServiceRuntimeVersion) and updates attribute mappings across the processor and receiver components. Service language/runtime attributes are remapped to telemetry.sdk.* and process.runtime.* semantic convention keys, or to labels.* keys in processor output. The host.os.platform attribute is removed entirely. Legacy service language translation logic is deleted, and intake mapper code is refactored to use semantic convention keys directly. Test data files are updated to reflect the new attribute structure and mappings.

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch receiver-processor-update-resource-attr-aggr-metrics

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@receiver/elasticapmintakereceiver/internal/mappers/intakeV2ToSemConv.go`:
- Around line 35-37: This change replaces the original service language fields
with telemetry.sdk.* keys which will be dropped by the current elasticsearch
exporter; revert or guard the change so we do not remove service
language/version until the exporter mapping lands. Specifically, in
intakeV2ToSemConv.go where event.Service.Language is handled (the block that
calls putNonEmptyStr(attributes, string(semconv.TelemetrySDKLanguageKey), ...)
and putNonEmptyStr(...TelemetrySDKVersionKey,...)), either (a) also emit the
original service language keys (preserving service.language/name and
service.language/version) in addition to telemetry.sdk.* or (b) wrap emission of
telemetry.sdk.language and telemetry.sdk.version behind a feature-flag or
conditional that checks the exporter mapping readiness, leaving the existing
service language fields intact; ensure you reference event.Service.Language,
putNonEmptyStr, attributes, semconv.TelemetrySDKLanguageKey and
semconv.TelemetrySDKVersionKey when making the change.

In `@receiver/elasticapmintakereceiver/testdata/logs_expected.yaml`:
- Around line 40-42: Update all occurrences of the os.type field in the test
data from the capitalized "Linux" to the lowercase "linux" to comply with
OpenTelemetry semantic conventions; search for the key "os.type" in
receiver/elasticapmintakereceiver/testdata/logs_expected.yaml and replace any
value stringValue: Linux with stringValue: linux across the file so all OS type
values use the required lowercase set (e.g., linux, darwin, windows, etc.).

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f5166557-2495-4332-97df-fc1e50454a43

📥 Commits

Reviewing files that changed from the base of the PR and between 7524288 and 5e6a9ce.

📒 Files selected for processing (22)

• internal/elasticattr/attributes.go
• processor/elasticapmprocessor/internal/ecs/ecs_translation.go
• processor/elasticapmprocessor/internal/ecs/ecs_translation_test.go
• processor/elasticapmprocessor/testdata/ecs/elastic_log/input.yaml
• processor/elasticapmprocessor/testdata/ecs/elastic_log/output.yaml
• processor/elasticapmprocessor/testdata/ecs/elastic_metric/input.yaml
• processor/elasticapmprocessor/testdata/ecs/elastic_metric/output.yaml
• processor/elasticapmprocessor/testdata/ecs/elastic_span_db/input.yaml
• processor/elasticapmprocessor/testdata/ecs/elastic_span_db/output.yaml
• receiver/elasticapmintakereceiver/internal/mappers/intakeV2ToDerivedFields.go
• receiver/elasticapmintakereceiver/internal/mappers/intakeV2ToElasticSpecificFields.go
• receiver/elasticapmintakereceiver/internal/mappers/intakeV2ToSemConv.go
• receiver/elasticapmintakereceiver/testdata/errors_expected.yaml
• receiver/elasticapmintakereceiver/testdata/hostdata_expected.yaml
• receiver/elasticapmintakereceiver/testdata/invalid_ids_expected.yaml
• receiver/elasticapmintakereceiver/testdata/language_name_mapping_expected.yaml
• receiver/elasticapmintakereceiver/testdata/logs_expected.yaml
• receiver/elasticapmintakereceiver/testdata/metricsets_expected.yaml
• receiver/elasticapmintakereceiver/testdata/spans_expected.yaml
• receiver/elasticapmintakereceiver/testdata/transactions_expected.yaml
• receiver/elasticapmintakereceiver/testdata/transactions_spans_expected.yaml
• receiver/elasticapmintakereceiver/testdata/unknown-span-type_expected.yaml

💤 Files with no reviewable changes (6)

• receiver/elasticapmintakereceiver/internal/mappers/intakeV2ToElasticSpecificFields.go
• processor/elasticapmprocessor/testdata/ecs/elastic_log/input.yaml
• internal/elasticattr/attributes.go
• receiver/elasticapmintakereceiver/internal/mappers/intakeV2ToDerivedFields.go
• processor/elasticapmprocessor/testdata/ecs/elastic_span_db/input.yaml
• processor/elasticapmprocessor/testdata/ecs/elastic_metric/input.yaml