Need to be able to publish schema files from specification repo

[tigrannajaryan]

OTEP 0152 introduces the notion of schema files that need to be published at some canonical location for OpenTelemetry schemas. The OTEP proposes to use https://opentelemetry.io/schemas/ as the location for OpenTelemetry schema files.

The schema files are plain YAML file that should be published as is. The source of truth, the original content of the schema files will be in the specification repo, so that it is precisely tied with the version of the specification.

We will need to publish one schema file per each version of the specification.

I am looking for a solution to do that publishing here, in this repository, so that the schema files become available at https://opentelemetry.io/schemas/ and I need maintainer's advice on how they think we can achieve it.

Ideally it should be fully automated, so that whenever a new specification release is made the schema file is automatically published at https://opentelemetry.io/schemas/, however I think we can also settle for a semi-automated approach, since the publishing needs to happen monthly and provided that it is easy to do, it is not going to require much efforts or maintenance.

Once the schema files are published they never change, they are immutable.

[tigrannajaryan]

@flands @mtwo @austinlparker I need your advice on this. Does website SIG do regular meetings? I can join to discuss if you prefer that.

[austinlparker]

We have meetings every two weeks, so if you want to show up next week then we can chat about it there.

From a technical side it's pretty straightforward though, we can create a static directory that serves to whatever path then use the same automation that's available to sync changes between the language repos into the website to sync that on a manual trigger or on a release of the spec.

[austinlparker]

The meeting is on the public calendar, it's 'Communications SIG' on every other thursday

[tigrannajaryan]

@austinlparker Thanks for confirming this is possible. I will join the SIG meeting next week and in the meantime I will try to put together a draft PR that does what you described.

[tigrannajaryan]

@austinlparker can you point me to the current automation that syncs the changes?

[austinlparker]

Sure thing, here's an example in the Java repo: https://github.com/open-telemetry/opentelemetry-java/blob/main/.github/workflows/docs-update.yml

Essentially you just need a source directory (on the spec side) and a target directory (on the website side), then the script will rsync one way from the source to the target on command/trigger.

It would be nice if we could get a bot user for these actions that had permissions across repos so we wouldn't have to use my PAT, but we're using my PAT for now.

[tigrannajaryan]

@austinlparker so, just to clarify:

• In the https://github.com/open-telemetry/opentelemetry-specification repo we will put the source files in some directory, e.g. /schemas, so /schemas/1.4.0 will be a YAML file containing the schema for specification version 1.4.0.
• In this repository we will have all schema files in /static/schemas/.
• When a new version of the specification is planned to be released the corresponding new YAML file should be already created https://github.com/open-telemetry/opentelemetry-specification repo, under the /schemas directory. This is a pre-requisite for spec release (and we can automate verification of this pre-requisite in the future).
• The https://github.com/open-telemetry/opentelemetry-specification repo will have a manually triggered github action that creates a new commit from /schemas and pushes it to this repo as a new PR (similar to what we already have for Java).
• Immediately after a new specification release the github action will be triggered manually by the person who does the spec release, as a result a PR will be created in this repo and once the PR is merged the new schema file will become available at https://opentelemetry.io/schemas/<version> (deployed via Netlify).
• In the future we can automate the process a bit more so that the PR is created automatically when the spec release is made without requiring manual triggering, and possibly also merge the PR automatically if it passes the build.

Does this sound right?

[austinlparker]

Yeah, that all sounds right. There might be an extra piece of config on this side to expose /schemas but I think it should just work as long as /static/schemas/ exists.

[austinlparker]

Just as an FYI Tigran I'm moving house this week so my access will be limited from tomorrow until next Monday, in case we're trying to time this with something else. I'll try to keep an eye on things though.

[tigrannajaryan]

@austinlparker thanks, no rush. We need to publish the first file for 1.4.0 spec release which is still 3 weeks away, so we have plenty of time. And for the very first release we can even do it completely manually, so it shouldn't be a problem.