Track language implementation status in meta schema

[jack-berg]

Followup to #312.

This expands the meta schema to include tracking for language implementation status. Notes:

We want it to be easy for language implementation maintainers to add / update implementation status. Therefore, each language has their own meta_schema_language_{language}.yaml file, with all types listed and kept in sync automatically through build tooling. I could have added a language implementation status to each of the meta schema type entries, but this would be very cumbersome for language maintainers as they would need to scroll through a huge file with lots of irrelevant information just to update their relevant bits.

We want it to be easy for end users to understand implementation status as a glance. End users are typically interested in a specific language. Therefore, the markdown generation has section for the status of each language implementation like this.

We also want config project maintainers to be able to understand the implementation status of particular types / properties across all languages at a glance, for things like deciding whether to promote experimental types to stable. Therefore, the markdown generation has a cross-language support status collapsible section for each type. For example, see "Language support status" section in SpanExporter type definition.

I think we'll need to consider / iterate what implementation status information we want to capture and how best to organize it maintainability. Here's what I sketched out for this PR:

latestSupportedFileFormat: 1.0.0-rc.1
typeSupportStatuses:
  - type: Aggregation
    status: supported
    unsupportedProperties: []
    notes: ""

Notes:

• .latestSupportFileFormat captures the top level file format the implementation supports
• .typeSupportStatuses has entries for each JSON schema type, automatically synced with build tooling
• .typeSupportStatuses[].status is an enum with values: supported, unknown (default until a maintainer updates it). What other options are there?
• .typeSupportStatuses[].unsupportedProperties captures any properties that are unsupported. This is only applicable when status: supported.
• .typeSupportStatuses[].notes open ended string field capturing any specific callouts about the implementation's support of the type.

WDYT? What information is missing? Is there a representation that would be easier for maintainers?

[codeboten]

Thanks @jack-berg, i think this is a good first step. I would definitely take this to the next step of having separate files per languages to make it more manageable.

[marcalff]

Thanks @jack-berg, i think this is a good first step. I would definitely take this to the next step of having separate files per languages to make it more manageable.

Thanks @jack-berg

I second Alex proposal:

Having a file per implementation allows better git history, and allows to have different code owners to review PRs changing the support status for a given language.

[jack-berg]

I would definitely take this to the next step of having separate files per languages to make it more manageable.

Having a file per implementation allows better git history, and allows to have different code owners to review PRs changing the support status for a given language.

👍 done

[marcalff]

RE:

.typeSupportStatuses[].status is an enum with values: supported, unknown (default until a maintainer updates it). What other options are there?

I would like to propose the following enum values as well:

• not implemented: The property is not even parsed. For example, some experimental nodes in opentelemetry-cpp.
• ignored: The property is parsed but is not used in the underlying SDK. For example, opentelemetry-cpp will ignore limits, because this is not implemented: our spec matrix still has holes in it. Ignored imply this is a known limitation, which may change later to supported.
• not applicable: The property is not parsed or is not used in the underlying SDK, because it does not make sense for this SDK. For example, the java instrumentation is not applicable for opentelemetry-cpp. Not applicable is a final state, to differentiate from ignored. Likewise, I would expect the cpp instrumentation node to be not applicable for java, claiming supported is very misleading.

In short, when we know exactly the status for an entry which is not supported, it should not stay blank or unknown but be given a more descriptive status.

[jack-berg]

Suggestion makes sense @marcalff.

With multiple statuses emerging, we'll want to expand the schema for unsupportedProperties to be more specific since we'll want to be able to express things like:

- type: Foo
  status: supported
  propertyOverrides:
    - property: bar
      status: not_implemented
    - property: baz
      status: ignored

The interpretation of this would be: The type Foo and all its properties are supported except bar which is not implemented yet, and baz which is ignored.

In this proposal, propertyOverrides is only meaningful if the type's top level status: supported. I.e. you can't have a type which is not_implemented which has properties which are supported.

[jack-berg]

Here's a concrete example showing usage of propertyOverrides.

In the java implementation, the ExperimentalPrometheusMetricExporter is generally supported, but with some exceptions:

• translation_strategy is not even available in the programmatic config API, and so is not available in declarative config.
• without_scope_info is supported via programmatic config API, but we haven't added declarative support for this property yet.

[jack-berg]

Adding lightweight docs for the schema of the meta_schema*.yaml documents.

[marcalff]

@jack-berg

Thanks for the tooling, and making each language a separate yaml file.

I think we should merge this, and start documenting each language status, it will help to assess readiness for 1.0.0, in term of coverage for each property (if a property is not supported by any language, there is inherent stability risk in the model)

Looking forward to provide the cpp coverage.

[marcalff]

LGTM.

Reviewed the yaml, the doc and generated doc, but not the generation scripts.