Backwards compatibility validator

[sldblog]

Validates that during change to reference data CSVs,

• entries are not deleted (instead, add a start_date/end_date column pair)
• data structure changes are backwards-compatible in the same file:
  • cannot remove existing columns
  • cannot rename existing columns
  • can add new columns

To bypass these validations, a new file has to be created

---

Currently, there's no check that

each entry still has a unique ID, which has to be in the first column

because it's failing, as it looks like we may need composite PK columns:

OFFENCES.csv:
  ❗ error: duplicate key: ID `LM82008` appears more than once
  ❗ error: duplicate key: ID `LM82009` appears more than once
<snip>

How does it work?

1. Retrieves the *.csv files from a git reference, by default origin/main (VersionedRegisters)
2. Then compares all of them against the version in the local filesystem (CheckCompatibility)

(The thing is orchestrated by App)

Why?

To make it extremely easy to update reference data for anyone (github text editor, basically). It also gives us history 🎉

And to provide guarantees for applications loading this data up that there won't be suddenly missing entries or incompatible payloads.

Notes

As the app already ignores test files, there's scope to introduce a convention to ignore certain files that are currently experimental, i.e. ones with -draft in them, or similar.

[sldblog]

because the app logic relies on git history; fetch-depth: 0 means fetch the entire history, not just a single commit

[sldblog]

I'll consider the silence as 👍 and sneak this in -- we can always revert