Exercise versioning

[iHiD]

As Concept Exercises change, the feedback given on representations may need to change. Similarly as a language adds new features, the feedback given in an exercise might need to change.

To solve this, I am considering adding a version key to an exercises config.json:

• It will use semvar.
• A change to the "major" number will invalidate the feedback on existing representations (although the feedback will be suggested to a new representation-feedback author).
• I suggest all exercises gets locked to 1.0.0 until a track launches in v3 mode (ie, you can just ignore any changes to the exercises between now and launch, leaving them at 1.0.0)
• We will script-patch all existing exercises

Does anyone see any issues with this before we document and patch the existing ones?

(cc @exercism/track-maintainers)

[SaschaMann]

I suggest all exercises gets locked to 1.0.0 until a track launches in v3 mode (ie, you can just ignore any changes to the exercises between now and launch, leaving them at 1.0.0)

What do you think about setting them to 1.0.0-DEV instead so that one can mark exercises as "done" by setting them to 1.0.0 and to better represent their state in general?

---

Eventually configlet should have a check whether or not the version was bumped if this is introduced

[cmcaine]

Do the minor and patch numbers have any meaning yet?

[iHiD]

What do you think about setting them to 1.0.0-DEV instead so that one can mark exercises as "done" by setting them to 1.0.0 and to better represent their state in general?

I imagine every exercise will have some updates before launch, even if that exercise feels good now. So I'd rather use a set of specific flags on an exercise or a status column to represent a checklist of things that need to be done (the resolution of #1455 basically). My aim in locking to 1.0.0 for now is to remove the overhead of anyone having to think about this for the next ~3 months. That said, whatever is easiest for you is the aim here so if the consensus is that 1.0.0-dev flag makes sense, I'm fine with that.

Do the minor and patch numbers have any meaning yet?

Not for me, but I believe they have very well defined meanings in problem-specifications and that the existing maintainers have found them valuable. Hopefully someone more knowledgeable than me can clarify. (cc @petertseng @ErikSchierboom @yawpitch @rpottsoh @coriolinus as you're all often active in problem-specs)

[coriolinus]

Adapting the test data versioning schema from the problem specifications makes sense to me.

[ErikSchierboom]

☝️ I agree with adopting the test data versioning schema from the problem specifications, as they provide relatively clear rules on how to interpret changes to exercises.

To solve this, I am considering adding a version key to an exercises config.json

I like having a version be associated with an exercise! While the config.json is a very natural place, I have one concern: the information in the config.json is currently not available to tooling like analyzers or test runners. Analyzers in particular could really benefit from having a version number being available, as they can then run different logic for different versions. Two obvious solutions to this problem are:

• Store the version information in another file that would be downloaded with the exercise (possibly the meta/config.json file, which already can include language version information)
• Pass the information from the config.json to the tooling

[coriolinus]

At the same time, global config.json is more convenient to work with than exercise-specific .meta/config.json.

Proposal: tooling always injects an exercise's entry from the global config.json into .meta/config.json before delivery to the analyzer/test runner/representer/etc.

[ErikSchierboom]

At the same time, global config.json is more convenient to work with than exercise-specific .meta/config.json.

Yep, that is absolutely true. Let's see if we can figure out something that is convenient for everyone :)

[iHiD]

What keys from the global config.json would be needed by test runners etc?

[ErikSchierboom]

Hmmm, another way to look at the config.json vs .meta/config.json files is that maybe the config.json should contain the bare minimum of information needed to link a track's exercises. Anything exercise-specific (like the version) could then go into the .meta/config.json.