Deprecate data_version and introduce checksum for DataPackages.

[ThePhar]

As discussed a bit more in Proposed Changes to DataPackage and Network Protocol in discord, I'm proposing we make the following changes to the DataPackage and Network Protocol to deprecate and eventually stop using data_version to indicate if a given world definition has an updated items/locations table. Instead, clients can reference the checksum of a given game and request the server for an updated DataPackage if it does not match their local cache's checksum. Then, a client can choose to override (or store additionally) that new DataPackage if the cache_safe property is true as well. This can be helpful for games that are still in development or testing and there may be rapid change to the items/locations table and we don't want to cache every single change. In worlds, they can set a development_mode member to True to indicate that it is not cache_safe.

Also made the updates to the documentation to show the new changes and which properties are deprecated to allow time for library and game maintainers to update before it can be removed entirely.

Updates to RoomInfo:

# Mark as deprecated.
datapackage_versions: Dict[str, int]
# A dictionary of Games to hex encoded SHA1 hash of the DataPackage for a particular game.
datapackage_checksums: Dict[str, str]

Updates to GameData (DataPackage Data):

# Mark as deprecated.
version: int
# Base64 encoded SHA1 hash of the DataPackage for a particular game.
checksum: str

The reason for this change is to hopefully avoid situations where integrators forget to increment the data_version and allow clients to be 100% sure that if their checksum matches the server, their DataPackage is exactly the same as the servers. This should incentivize library maintainers to cache their data as they can be sure the DataPackage has not updated.

@Berserker66 @Ijwu @JarnoWesthof @black-sliver @KonoTyran @espeon65536 @FelicitusNeko @N00byKing as you all have libraries (or help maintain them) that I am aware of. I have the thread in Discord as well.

[ThePhar]

Crossed out the stuff talking about cache_safe as I removed that in the previous commits.

[Jarno458]

This PR needs to be updated as the data_versions added to some games are outdated, and the version(s) in the docs are nolonger depricated but already removed

[ThePhar]

Now that I'm back in action PC-wise, plan is to finish wrapping this up over the next week so we can have it ready to deploy when we're ready to do all the planned protocol changes.

[Jarno458]

^^ And that was almost a month ago

[ThePhar]

Yeah, my sense of time is bad.

[Berserker66]

0 currently still implies to embed the relevant game's datapackage into the resulting .archipelago file.
This PR should probably explain this or offer an alternative, as with this default and explanation I'd expect people to use version 0 now, as in not change it, which is undesired.

[ThePhar]

Hopefully the changes to the comment should be more clear.

[beauxq]

This change should probably be in a different pull request as a bug fix.
The only reason it's working without this fix is that it's getting the 1 value from the base class.

[ThePhar]

Originally this set data_version to 0 as default to ensure new worlds were automatically in testing mode and that would have affected this world. It could be reverted now that I changed the default back to 1, but honestly it's such a small tweak that it's probably just fine in my opinion, but if you insist, I can revert it.

[beauxq]

Saying that it should be in a different PR is not saying that it shouldn't be in this PR.
It's not a problem if it's both here and in another PR.

[black-sliver]

If i were to continue this - what is still missing?
Caching in the client,
caching in the server,
always including datapackage in generated games,
stripping datapackage when storing generated games on the webhost,
anything else?

Do we want to truncate the hash for filename (MAX_PATH=256 may be a problem for some games?) If so, wouldn't a web-safe b64 representation be less likely to collide even if stored CI?

>>> len(hashlib.sha1(dump.encode()).hexdigest())
40
>>> len(urlsafe_b64encode(hashlib.sha1(dump.encode()).digest()).rstrip(b"=").decode())
27
>>> len(f"C:\\Documents and Settings\\A Longer Username\\Local Settings\\Application Data\\My Great Client\\datapackage\\A Link To The Past\\{'0' * 40}.json")
168

hex would still kinda work in the worst case if we don't add more to it than I did above.

[Berserker66]

approving in so far that I read through it. Unfortunately did not find the time for decent testing and doubt I will this week.

[kindasneaki]

While I don't fully understand the inner workings. I ran ror2 explore mode with this with the world removed and everything worked just fine.

[shananas]

tested with KH2 same as kindasneaki with the same results everything works fine