[exporter/elasticsearchexporter] Add dynamic_templates to ECS mapping mode metrics encoder

[isaacaflores2]

Description

The current ECS mapping mode metrics encoder does not correctly map histograms. Legacy APM metrics rely on an ingest pipeline that requires metrics to be wrapped in metricset.sample.

Mapping example

Before

Current behavior when indexing a histogram with name http.request.duration

"http": {
  "properties": {
    "request": {
      "properties": {
        "duration": {
          "properties": {
            "counts": {
              "type": "long",
              "index": false
            },
            "values": {
              "type": "float",
              "index": false
            }
          }
        }
      }
    }
  }
}

After

When the exporter sends the dynamic_templates the histogram is correctly mapped

"http": {
    "properties": {
      "request": {
        "properties": {
          "duration": {
            "type": "histogram"
          }
        }
      }
    }

This PR updates the ECS mapping mode metics encode to send dynamic_templates when indexing metrics using the same approach used today for the OTel mapping mode

Link to tracking issue

Related to #46632

Testing

1. Updated existing ecs mode test cases
2. Add additional test cases
3. Build collector and indexed metrics into a local Elasticsearch instance:

Local collector testing details

Collector configuration

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: localhost:4319
        include_metadata: true
      http:
        endpoint: localhost:4320
        include_metadata: true

exporters:
  elasticsearch:
    endpoint: ${env:ES_URL}
    auth:
      authenticator: basicauth
  debug:
    verbosity: detailed

extensions:
  basicauth:
    client_auth:
      username: <username>
      password: <password>

service:
  extensions: [ basicauth ]
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [elasticsearch]
    metrics:
      receivers: [otlp]
      exporters: [elasticsearch,debug]
    logs:
      receivers: [otlp]
      exporters: [elasticsearch]
  telemetry:
    metrics:
      level: detailed
      readers:
        - pull:
            exporter:
              prometheus:
                host: '127.0.0.1'
                port: 8889

Test data

#!/usr/bin/env bash
# Send OTLP metrics (gauge, counter, histogram, summary) via HTTP/JSON.
# Default: http://localhost:4320/v1/metrics (override with OTLP_METRICS_ENDPOINT).
#
# To test: start the minimal collector, then run this script:
#   ./bin/otelcontribcol-minimal_$(go env GOOS)_$(go env GOARCH) --config exporter_test_config_local.yaml
#   ./.send_otlp_metrics
#
set -euo pipefail

OTLP_METRICS_ENDPOINT="${OTLP_METRICS_ENDPOINT:-http://localhost:4320/v1/metrics}"

exit_with_hint() {
  local code=$1
  if [[ $code -ne 0 ]]; then
    echo "Send failed (exit $code). Is the collector running with OTLP HTTP on the endpoint?" >&2
    echo "  Example: ./bin/otelcontribcol-minimal_\$(go env GOOS)_\$(go env GOARCH) --config exporter_test_config_local.yaml" >&2
  fi
  exit "$code"
}

trap 'exit_with_hint $?' EXIT

# Nanoseconds since epoch (second precision; portable on macOS and Linux)
NOW_NS="$(date +%s)000000000"

curl --fail-with-body -sS \
  -X POST "${OTLP_METRICS_ENDPOINT}" \
  -H "Content-Type: application/json" \
  -H "X-Elastic-Mapping-Mode: ecs" \
  --data-binary @- <<EOF
{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          { "key": "service.name", "value": { "stringValue": "metricset-samples-curl" } },
          { "key": "service.namespace", "value": { "stringValue": "dev" } }
        ]
      },
      "scopeMetrics": [
        {
          "scope": {
            "name": "curl.manual",
            "version": "1.0.0"
          },
          "metrics": [
            {
              "name": "demo.gauge",
              "description": "Gauge in same request",
              "unit": "1",
              "gauge": {
                "dataPoints": [
                  {
                    "startTimeUnixNano": "${NOW_NS}",
                    "timeUnixNano": "${NOW_NS}",
                    "asDouble": 42.5,
                    "attributes": [
                      { "key": "metric.type", "value": { "stringValue": "gauge" } }
                    ]
                  }
                ]
              }
            },
            {
              "name": "demo.counter",
              "description": "Monotonic sum (counter) in same request",
              "unit": "1",
              "sum": {
                "aggregationTemporality": 2,
                "isMonotonic": true,
                "dataPoints": [
                  {
                    "startTimeUnixNano": "${NOW_NS}",
                    "timeUnixNano": "${NOW_NS}",
                    "asInt": 7,
                    "attributes": [
                      { "key": "metric.type", "value": { "stringValue": "counter" } }
                    ]
                  }
                ]
              }
            },
            {
              "name": "demo.histogram",
              "description": "Delta histogram in same request",
              "unit": "ms",
              "histogram": {
                "aggregationTemporality": 1,
                "dataPoints": [
                  {
                    "startTimeUnixNano": "${NOW_NS}",
                    "timeUnixNano": "${NOW_NS}",
                    "count": 10,
                    "sum": 123.4,
                    "bucketCounts": [1, 2, 3, 4],
                    "explicitBounds": [10, 50, 100],
                    "attributes": [
                      { "key": "metric.type", "value": { "stringValue": "histogram" } }
                    ]
                  }
                ]
              }
            },
            {
              "name": "demo.summary",
              "description": "Summary in same request",
              "unit": "ms",
              "summary": {
                "dataPoints": [
                  {
                    "startTimeUnixNano": "${NOW_NS}",
                    "timeUnixNano": "${NOW_NS}",
                    "count": 5,
                    "sum": 250.5,
                    "quantileValues": [
                      { "quantile": 0.5, "value": 45.0 },
                      { "quantile": 0.9, "value": 90.0 }
                    ],
                    "attributes": [
                      { "key": "metric.type", "value": { "stringValue": "summary" } }
                    ]
                  }
                ]
              }
            }
          ]
        }
      ]
    }
  ]
}
EOF

Kibana Validation

GET metrics-generic-default/_mapping

mappings

"demo": {
          "properties": {
            "counter": {
              "type": "double",
              "index": false
            },
            "gauge": {
              "type": "double",
              "index": false
            },
            "histogram": {
              "type": "histogram"
            },
            "summary": {
              "type": "aggregate_metric_double",
              "metrics": [
                "sum",
                "value_count"
              ],
              "default_metric": "value_count"
            }
          }
        }

Documentation

Updated the README.md to document the existing OTel mapping mode dynamic templating and added details for the ECS mapping mode changes

[carsonip]

thank you!

[ChrsMark]

LGTM

Approved by code-owner already. Merging.

[otelbot]

Thank you for your contribution @isaacaflores2! 🎉 We would like to hear from you about your experience contributing to OpenTelemetry by taking a few minutes to fill out this survey. If you are getting started contributing, you can also join the CNCF Slack channel #opentelemetry-new-contributors to ask for guidance and get help.