[legacy versions] how do we support old versions

[chenqin]

As xgboost is getting into production with critical use cases, I would like to open a discussion on how we support old versions. This is important because

1. changes from different versions may change model performance
2. change from different versions may require code change or dependency change

Both of which will put extra work and more importantly a risk factor to platform teams (ML as a service)

Here is similar discussion from another Apache project.
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Time-based-releases-in-Flink-td15386.html#a15394

@hcho3 @CodingCat

[CodingCat]

we have achieved the agreement on how to support old version and how to achieve backward compatibility. check https://xgboost.readthedocs.io/en/latest/contrib/release.html

In most of cases (not releasing a major version), the following two cases are not to be there unless in a form of bug

1. changes from different versions may change model performance
2. change from different versions may require code change or dependency change

[hcho3]

@CodingCat

the following two cases are not to be there unless in a form of bug

Are we sure that we won't have situations like 0.82 -> 0.90 upgrade?

@chenqin Can you be more specific? Do you mean that we should support old releases, e.g. backport bug fixes to previous releases to some time? See #4389, where we previously decided that we didn't have enough manpower and resources to maintain legacy releases.

[CodingCat]

Are we sure that we won't have situations like 0.82 -> 0.90 upgrade?

1. changes from different versions may change model performance

isn't this a bug? I don't think any new change will shift or shifted the output of algorithm significantly given the current status of algorithms (cpu at least, gpu maybe)

2. change from different versions may require code change or dependency change

this breaks into two parts

(1) code change: no, we will try our best to avoid the case like #4349 in future (the major reason for me to put a simplified solution like that is we were in 0.x where a lot of flexibilities exist), if anything similar appeared in future, we should consider behavior consistency

(2) dependency change: I cannot find it, but I think in some thread, we decided to drop the binary compatibility guarantee in the end. Not sure if it is worth bringing back the discussion here. On the other side, as a project maintainer who is also responsible to advertise the usage of xgb, we should be strategic for major dependency upgrading (like Spark). For instance, what's the right time to move forward from 2.4 to 3.0. (if Spark 3 was going to release right before next Spark Summit, shall we wait...if Spark 3 is releasing next month, shall we upgrade exclusively or compatibly, given the risk that the first major release of spark is always full of surprise)

[chenqin]

where we previously decided that we didn't have enough manpower and resources to maintain legacy releases.

Got it. sounds good with 0.x as long as we have a plan.

dependency change:
sounds good.