New transform: trace to metric

[kamalmarhubi]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

There's a very real way in which metric instrumentation is redundant with tracing instrumentation. For example, the http.{client,server}.duration metrics defined in the semantic conventions can easily be generated from spans, aggregating along the right attributes.

With some additional VRL functionality, you could imagine some fancier aggregations. For example, total number of SQL queries made in processing an HTTP request, aggregated along the typical HTTP attributes (method, path, status) and database attributes (DB, table, query type). Or total time spent in database queries.

Attempted Solutions

Traces are too new in Vector to have any attempted solutions. However, the spanmetricsprocessor in the OpenTelemetry collector is a spiritual attempt. With VRL, the Vector implementation can be a lot more capable, especially if applied after collecting all spans that make up a trace.

Proposal

Create a new transform that has trace as input and metric as output.

This is vague and fairly inelegant, but a really rough take on what a configuration might look like:

[transforms.my_transform_id]
type = "trace_to_metric"
inputs = [ "traces-source" ]

  # Simple span-to-metric generating the semantic convention HTTP
  # duration metric from semantic convention HTTP spans [0].
  [transforms.my_transform_id.metrics]
  name = "http.server.duration"
  type = "distribution"
  aggregation = "span"
  # A root HTTP server span has no parent and must have http.server_name set; see [0].
  span_filter = "!exists(.trace.parent_id) && .kind == 'server' && exists(.attributes['http.server_name'])"
  value = ".end_time_unix_nano - .start_time_unix_nano"
  include_resource_attributes_as_tags = true  # Or some other convenience thing.

    [transforms.my_transform_id.metrics.tags]
    "http.server_name" = ".attributes['http.server_name']"
    "http.method = ".attributes['http.method']"
    "http.status_code" = ".attributes['http.status_code']"



  # Generate a sql_query_count metric from semantic convention DB spans [1].
  [transforms.my_transform_id.metrics]
  name = "sql_query_count"
  # A SQL server span
  span_filter = ".kind == 'client' && exists(.attributes['db.system'])"
  type = "distribution"
  value = "count()"  # (I'm making stuff up here.)
  aggregation = "trace"
  include_resource_attributes_as_tags = true  # Or some other convenience thing.

    [transforms.my_transform_id.metrics.tags]
    "http.server_name" = ".root.attributes['http.server_name']"
    "http.method = ".root.attributes['http.method']"
    "http.status_code" = ".root.attributes['http.status_code']"
    "db.system" = ".span.attributes['db.system']"
    "db.name" = ".span.attributes['db.name']"
    "db.operation" = ".span.attributes['db.operation']"
    "db.sql.table" = ".span.attributes['db.sql.table']"

[0] https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/http.md
[1] https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/database.md

References

Related to #12029.

Version

No response

[jszwedko]

Thanks for this detailed proposal @kamalmarhubi ! As you noted, traces are a very new data type to Vector, but we will be looking to add functionality like this to generate metrics from them.