Metrics Semantic Conventions not specified as YAML files

[FWinkler79]

What are you trying to achieve?

I looked at the semantic conventions described in YAML and used as a basis for generating markdown.
YAML files exist for resources and trace spans but they don't seem to exist for metrics.
However, there is a semantic convention described for metrics already here which reference sub pages that list metrics-related attributes for e.g. HTTP, Database, etc.

None of these pages seem to contain attribute tables generated from YAML. Is that just temporary or is there a deeper reason why metrics are not yet specified in YAML?

Update: The same is apparently happening for log attributes

What did you expect to see?

Have metrics-related attributes and their semantics defined in YAML and referenced in markdown (as generated tables) instead of simply describing them in markdown manually.

Additional context.

none

[Oberon00]

Thanks for the reminder, this requires open-telemetry/build-tools#100 open-telemetry/build-tools#79

[jsuereth]

@Oberon00 I think you mean: open-telemetry/build-tools#79

Regarding both metrics + logs semantic conventions, the build tooling to generate from markdown was not built / ingested yet.

[FWinkler79]

Thanks, I interpret this as "There is a more up-to-date toolchain required to make this possible". Am I correct in that assumption?

Btw.: I gave it a try to reverse-engineer the metrics http-attributes markdown into yaml.
The below code generates something very close to the original markdown. Maybe someone could have a look at this, if this goes in the right direction.

groups:
  - id: http.metrics
    extends: http
    prefix: http
    brief: 'This document defines semantic conventions for HTTP client and server metrics attributes.'
    attributes:
      - ref: http.method
        requirement_level: required
      - ref: http.host
      - ref: http.scheme
      - ref: http.status_code
      - ref: http.flavor
      - ref: net.sock.peer.addr
        requirement_level: 
          conditionally_required: For clients only.
      - ref: net.sock.peer.port
        requirement_level: 
            conditionally_required: For clients only.
      - ref: net.sock.peer.name
        requirement_level: 
          conditionally_required: For clients only.
      - ref: net.host.name
        requirement_level: 
          conditionally_required: For servers only.
      - ref: net.host.port
        requirement_level: 
          conditionally_required: For servers only.
      - ref: http.server_name
        requirement_level: 
          conditionally_required: For servers only.
  
  - id: http.client.metrics
    extends: http.metrics
    prefix: http.client
    brief: 'Semantic Convention for HTTP Client metrics attributes'
    attributes:
      - id: duration
        type: int
        brief: >
          measures the duration outbound HTTP requests. Unit: milliseconds, ms
        examples: 300
    constraints:
      - any_of:
          - [http.url]
          - [http.scheme, http.host, http.target]
          - [http.scheme, net.peer.name, net.peer.port, http.target]
          - [http.scheme, net.sock.peer.addr, net.sock.peer.port, http.target]
      
  - id: http.client.request.metrics
    extends: http.client.metrics
    prefix: http.client.request
    brief: 'Semantic Convention for HTTP Client request metrics attributes'
    attributes:
      - id: size
        type: int
        brief: >
          measures the size of HTTP request messages (compressed). Unit: bytes, by
        examples: 1024
      
  - id: http.client.response.metrics
    extends: http.client.metrics
    prefix: http.client.response
    brief: 'Semantic Convention for HTTP Client response metrics attributes'
    attributes:
      - id: size
        type: int
        brief: >
          measures the size of HTTP response messages (compressed). Unit: bytes, by
        examples: 1024
    
  - id: http.server.metrics
    extends: http.metrics
    prefix: http.server
    brief: 'Semantic Convention for HTTP Server metrics attributes'
    attributes:
      - id: duration
        type: int
        brief: >
          measures the duration inbound HTTP requests. Unit: milliseconds, ms
        examples: 300
      - id: active_requests
        type: int
        brief: >
          measures the number of concurrent HTTP requests that are currently in-flight. Unit: requests
        examples: 300
    constraints:
      - any_of:
          - [http.scheme, http.host, http.target]
          - [http.scheme, http.server_name, net.host.port, http.target]
          - [http.scheme, net.host.name, net.host.port, http.target]
          - [http.url]

  - id: http.server.request.metrics
    extends: http.server.metrics
    prefix: http.server.request
    brief: 'Semantic Convention for HTTP Server request metrics attributes'
    attributes:
      - id: size
        type: int
        brief: >
          measures the size of HTTP request messages (compressed). Unit: bytes, by
        examples: 1024
  
  - id: http.server.response.metrics
    extends: http.server.metrics
    prefix: http.server.response
    brief: 'Semantic Convention for HTTP Server response metrics attributes'
    attributes:
      - id: size
        type: int
        brief: >
          measures the size of HTTP response messages (compressed). Unit: bytes, by
        examples: 1024

[Oberon00]

Sorry, I meant to link to open-telemetry/build-tools#79, so I'm not sure what you meant by toolchain, but only the actual tool needs to be updated, nothing further down the stack.

[jsuereth]

I'm going to say this is blocked on: open-telemetry/build-tools#105, for which we have a PR: open-telemetry/build-tools#79

We'll need to open another enhancement request for logs (feel free to do so on the build-tools repository), and if you want ot contribute code, please do so over there.

[FWinkler79]

Sorry, I meant to link to open-telemetry/build-tools#79, so I'm not sure what you meant by toolchain, but only the actual tool needs to be updated, nothing further down the stack.

Thanks, indeed I meant the tool. In my YAML approach listed above, I could not recreate the metrics type (histogram, etc.) that the http-attributes table for metrics contains. That limitation (if it really is one) should be fixed by open-telemetry/build-tools#79.

As a bit of context what we are needing this for: we decided to reference your specifications and company-internally extend it with our own attributes. If there is anything relevant, that is not company-specific we will also be happy to contribute back. With resources and traces described in YAML, we have a clean way of extending and refining attributes in our context. With metrics (and log attributes) not being described in YAML yet, this is of course much more difficult, since all we can really reference is the written markdown.

[jamesmoessis]

Hello folks, I've just done a bunch of work on open-telemetry/build-tools#79 and I believe it's close to being ready for merge. Please feel free to leave a review with any thoughts 😄

[lmolkova]

Closing this, thanks to @jamesmoessis we're now describing metrics in yaml.