Deprecation Policy

[jpmckinney]

It's anticipated that properties will someday be deprecated and then renamed/removed. To serialize deprecation, we should use a new deprecated keyword in the schema files, which is a simple boolean. If not present, it's assumed to be false.

As there will be multiple versions of OCDS, its necessary for implementations to declare which version they are implementing.

A new, required conformsTo property should be added to the Release, ReleasePackage and RecordPackage schema with the definition “the version of the Open Contracting Data Standard to which this resource conforms.” Its value should be the URL of a version of the schema, e.g. “http://ocds.open-contracting.org/standard/r/1__0__RC/”.

Given the possibility of extending the schema, it's necessary to allow implementations to refer to a non-canonical, extended schema:

A new, required describedBy property should be added to the Release, ReleasePackage and RecordPackage schema with the definition “the schema against which this resource should be validated.” Its value should be an Open Contracting Data Standard schema, e.g. “http://ocds.open-contracting.org/standard/r/1__0__RC/release-schema.json”. If the canonical schema has been extended, implementations may reference a schema that includes those extensions.

A section of the documentation should list canonical values for conformsTo and describedBy for easy reference.

Note the above is based on https://project-open-data.cio.gov/v1.1/schema/

[practicalparticipation]

+1 - This is definitely something that needs to be addressed.

On the extensions, as these should be implemented as Patches against the master schema, I wondered if the better option would be to provide an 'extensions' array in which users can provide the URLs of each of the extensions they are using, and validation tools can then be responsible for fetching those extensions, and creating their own patched schema to validate against.

Whilst this adds some work to the validation tools, I see the advantages as:

• Limiting a proliferation of people creating and hosting their own schemas;
• Helping with tracking of which are the widely used extensions, and so supporting the evolution of the standard;

[jpmckinney]

I'm not entirely convinced by the "extensions as patches" though. Location should be core. My sector work should be core. Having lots of patches instead of one core schema makes it very difficult to know whether a new extension conflicts with an old one.

It's perhaps useful for implementation-specific changes to the schema to be documented with patches, but I think everything governed by OCP should make changes to the core schema. There should be no official OCP extensions.

[practicalparticipation]

I would agree with the idea that OCP should merge everything it governs into core. I.e. Location should move into core in 1.1 if there is a settled approach by then.

But we would still anticipate different publishers to have their own extensions, and the extension mechanism encourages them to document them well, rather than just add properties to their files without documentation of those properties.

Am I right then in thinking that 'describedBy' can be inferred always from 'conformsTo' (i.e. to confirm to 1.0RC you must use the 1.0RC schema), and that having the two is in order to explicitly point to the master validation schema? In other words, extensions would then be in addition to these (though as above... we would aim to keep its use limited in most cases).

[jpmckinney]

Re-reading the documentation, it seems that extensions are only intended to be published by implementers, which makes this folder somewhat confusing. If the unit weight and delivery location extensions are just meant to be example extensions, they should be moved here to avoid confusion.

If you declare that you are using 1.0RC in conformsTo, you can either have a describedBy that points to the canonical schema, or to your custom 1.0RC-compliant schema, i.e. it cannot always be inferred.

Re: extensions array, it depends to what extent we expect re-use of extensions. If they are implementation-specific, then I wouldn't expect much re-use. Our interest then is just to see what the custom schema changes, in which case a simple diff program would suffice.

If extensions are of general interest, then their use (before consensus and inclusion in core) is actually dangerous to the evolution of the standard. It becomes like HTML before HTML5 where vendors each came up with their idea of how to implement a general-interest feature, e.g. Ajax, and you ended up with a bunch of different, incompatible APIs ("extensions" in our case). Rather than support a proliferation of extensions independent of OCP, OCP should encourage engagement and participation in the core spec.

[practicalparticipation]

I would anticipate two scenarios for extensions:

(1) Standard track extensions: prototyping ideas for inclusion in the full standard in future (which is the case with the current folder of extensions). These should be things of general value to the whole community of users. They should be prototyped within an OCP space, making work on these part of engagement and participation in the core spec, but also allow that they can be tested and move towards reference implementation prior to governance decisions to adopt them into the full standard.

(2) Local user community extensions: e.g. where a few organisations come together with a use-case that does not warrant inclusion in the full standard, but where they want to agree together on ways to represent particular locally relevant properties. For example, the UK have a range of 'flags' for SME Participation, particular statuses of a contract, etc. they would want to include in their data, and which other UK based organisations may also wish to use to be compatible within the country - but which would not make sense to place into a global standard. Allowing them to establish independent governance of that extension reduces the pressure on what will very likely be a constrained capacity governance process for OCP globally.

I would agree that the ideal is to channel as much discussion through OCP processes as possible (even for case (2)... where at least encouraging comment periods to support alignment, and to work out what can be elevated into the core standard is good), but we need pragmatic approaches that will work with the likely capacity of OCP in the coming year.

[jpmckinney]

My issue with (1) is that it sets up two work streams. The main work stream to date is to have GitHub issues about the spec, and to make changes to the spec as issues are resolved. People can test the new draft spec and create new issues with new feedback based on what they've learned from testing. (1) seems to set up a parallel way of accomplishing those same tasks. I'm not clear on the advantage of having two ways of doing the same thing. It seems confusing.

To be clear, after each release, the spec goes back into "draft mode" where anything goes until a new, stable release is ready. This is how many open-source software projects are developed. We might propose the adoption of a "feature freeze" within the release schedule, so that OCDS can have regular points of stability (releases) and not be in a constant state of evolution. Moving the instability into an "extensions" workflow seems like an unnecessary indirection.

Re: (2) the UK folks can get together and agree on additional properties even without the extensions architecture; nothing prevents it. For the specific example, Mexico also has SME flags, so it's not a local issue; even if only the UK had SME flags, it's not clear that a generic "flag" modeling couldn't be developed to serve that (and other) use cases.

I understand OCP doesn't have a lot of capacity, but I don't think the standard will remain a standard if the lack of capacity leads to groups developing the spec themselves in conflicting directions. I think a better way of matching capacity is to have the standard develop at a pace that matches capacity; a slow standard development is not a bad thing!

[timgdavies]

We're currently working on updated write-up of extensions process. Assigning to @kindly to consider in initial draft - and will then share things here for further discussion.

[timgdavies]

A proposal on the deprecating policy can be found in #367

Our updated approach to handling extensions, which aims to build upon the points raised in this thread, can be found at #335