Introduce proper meta schema concept

[jack-berg]

Meta schema is the evolution of type_descriptions.yaml, and holds all types of details that don't fit neatly into the JSON schema.

Initial use cases include:

• Property descriptions. Previously done via type_descriptions.yaml, we should evolve this to more precisely model the semantics. I.e. break out high level description, SDK interpretation, default behavior.
• SDK extension plugin interfaces. Tracking which types can reference SDK extension plugin interfaces via the ComponentProvider mechanism.
• SDK implementation status. Track which types and properties are supported by which SDK implementations. Update I've sketched this out on a separate branch here (see generated markdown here). Will open a followup PR to keep things slightly more reviewable.

I wrote a useful summary of the meta schema and associated tasks here: https://github.com/jack-berg/opentelemetry-configuration/blob/meta-schema/CONTRIBUTING.md#meta-schema

Please read while reviewing!

[marcalff]

LGTM, but I did not review scripts/*, since I don't speak js.

[jack-berg]

@open-telemetry/configuration-approvers anybody interested in discussed / reviewing this? Its a decent amount of code, but the stakes are low because its "build" code, as opposed to part of the schema, examples, stability guarantees. My view is we should land this and iterate.

[marcalff]

@open-telemetry/configuration-approvers anybody interested in discussed / reviewing this? Its a decent amount of code, but the stakes are low because its "build" code, as opposed to part of the schema, examples, stability guarantees.

On this:

My view is we should land this and iterate.

Yes, I agree we should merge it.

I reviewed the schema/instrumentation.json part because it affects the schema, and it looks ok.

As for the js code, as noted earlier, I can't review it. Now:

• this is low risk and does not affect the yaml schema itself, so no impact on SIGs implementing file configuration.
• as long at the scripts can run and generate the doc, everything is fine.
• a quick read of the produced document looks ok, validating indirectly the generation scripts.

Pushing the approve button to unblock this, there is no point in letting this work rot, it will serve all of us better in the main branch.

Please rebase, re generate the doc, and ok to merge.

[marcalff]

LGTM.

[marcalff]

• SDK implementation status. Track which types and properties are supported by which SDK implementations. Update I've sketched this out on a separate branch here (see generated markdown here). Will open a followup PR to keep things slightly more reviewable.

This is for later so not part of this review.

Out of curiosity, I looked at the SDK implementation status, and found out that java supports the ExperimentalLanguageSpecificInstrumentation for every SIG, including cpp, not just for java.

@jack-berg

What is the meaning of supported exactly:

• that the file configuration parser in each SIG accepts the property, even if it does nothing about it ?
• that the property is parsed, given to the SDK, and supported by the SDK ?

To clarify once #312 is merged, with the next PR to represent per language support: we may need more statuses, like ignored.

[jack-berg]

To clarify once #312 is merged, with the next PR to represent per language support: we may need more statuses, like ignored.

Yup we got to work out what kind of information we want to capture in support status, and how to represent it so its efficient for maintainers to record / update. I sketched out something quick but imagined we'd probably need an enumeration of possible support statuses, along with the ability to elaborate with an optional description. Let's chat in the followup PR!