MXNet extensions semantic versioning

[samskalicky]

Description

Currently the lib_api.h has a version number in it that is required to match the one in MXNet when loading the library.

Every change increments the version number:
https://github.com/apache/incubator-mxnet/blob/f1acda73e0b2ca5f1b6ff89f25c17a838819be82/include/mxnet/lib_api.h#L56
And the version number is required to match:
https://github.com/apache/incubator-mxnet/blob/f1acda73e0b2ca5f1b6ff89f25c17a838819be82/src/c_api/c_api.cc#L1500-L1503
Each release of MXNet will require libraries to have the same version of lib_api.h. The versioning of the Extensions APIs is different than MXNet's C API.

This simplifies testing and ensures that libraries will work with a specific version of MXNet. It also allows the APIs to evolve without being restrained.

Starting in the 2.0 release onwards, should we change this to follow semantic versioning? And if so, how do we test and verify the compatibility hasnt been broken? How do we ensure that for future major releases we can break compatibility and check and report incompatibility errors?

[szha]

In general, features in MXNet must follow semantic versioning. This means that if a feature is offered as an official, non-experimental API in a release, then backward-incompatible change to it is no longer allowed in future revision. This applies to both 1.x and 2.x.

Specific to the recent changes in extensions in #19117, it looks they fall in the category of adding functionality. One way to make it work is to only add new API without changing existing API. @samskalicky would this approach work?

[samskalicky]

In previous 1.x releases we're been terribly inconsistent:

• 1.6.0 didnt have a MX_LIBRARY_VERSION number: https://github.com/apache/incubator-mxnet/blob/1.6.0/include/mxnet/lib_api.h#L25-L33
• 1.7.0 had version number 7: https://github.com/apache/incubator-mxnet/blob/1.7.0/include/mxnet/lib_api.h#L52
• 1.8.0 will have version number 10: https://github.com/apache/incubator-mxnet/blob/35feed663427ceafd3369a55c47d7140476851ea/include/mxnet/lib_api.h#L56
  From 1.7 we're consistently checking the version number at least.

Lets focus the discussion on how to do things better for 2.0.

We'll probably need to keep the version number (to avoid backward incompatible libraries from earlier versions). But I guess we can stop incrementing it for minor releases and do as you suggest on API changes to enable backwards compatibility for 2.x minor versions.

And then in major version changes we can increment the version number to demarcate backward incompatibility.

What do you think?