Core: Remove Universally Unique ID Requirements (Per-Game Data Packages)

[ThePhar]

What is this fixing or adding?

Note: This is a breaking change that may require some clients (and libraries) to handle potential ID conflicts.

The goal of this PR is to remove the requirement that item and location IDs must be unique across all worlds. IDs must still remain unique to their own world, but other worlds may now utilize that ID space for their implementations. This means it's expected for a client to check which game an ID is relevant for, prior to lookup.

This will allow any world developer to use more sensible IDs without need of a base_id_offset`-type addition when building item and location tables, without worry of an ID conflict with another hypothetical world, if so desired.

Data Package Changes / Expectations

Clients that rely on looking up names for location and item IDs will most likely need to change how they perform these lookups to support this paradigm, otherwise, at best, names could be incorrect, and at worst, games will not function. The only changes to the Data Package itself will be the removal of version as this update also drops data_version support. If a client or library relies on version for data package caching, it will also stop working.

For library and client developers, the recommended way to perform name lookup for an ID is to check the relevant player's NetworkSlot data for their world, then lookup from within the corresponding game's GamePackage.

An example lookup scenario:

1. You connect and authenticate to the Archipelago server and the server responds with a Connected packet.

2. You locally cache the slot_info data from Connected packet for later lookup.

3. Eventually, you need to scout information in a location, so you send out a LocationScouts packet for location: 100.

4. Server responds with a LocationInfo packet with the following payload:

   [{
       "cmd": "LocationInfo",
       "locations": [{ 
           "item": 1, 
           "location": 100, 
           "player": 4, 
           "flags": 1
       }]
   }] 
   

5. You check player 4 in the slot_info data you locally cached on connection and find out they're playing Blique.

6. You can now do your lookup inside the Blique game package and get the name for item 1.

CommonClient/CommonContext changes

CommonContext.item_names and CommonContext.location_names is now a specialized dict (called NameLookupDict) for looking up names, separated by game. To make it easier for developers who derive from CommonContext, some additional helper methods were created to make getting these names without explicitly finding the game name yourself.

def lookup_in_slot(self, code: int, slot: Optional[int] = None) -> str:
"""Returns the name for an item/location id in the context of a specific slot or own slot if `slot` is omitted."""

def lookup_in_game(self, code: int, game_name: Optional[str] = None) -> str:
"""Returns the name for an item/location id in the context of a specific game or own game if `game` is omitted."""

# Example usage:
# received_item: NetworkItem
# sender_name: str

item_name = ctx.item_names.lookup_in_slot(received_item.item)
location_name = ctx.location_names.lookup_in_slot(received_item.location, received_item.player)
logger.log(f"Received {item_name} from {sender_name} ({location_name})")

It is still possible to lookup names via {type}_names[id], but this should be considered deprecated and may fail to load the correct name if multiple items or locations utilize the same id. Instead, developers should switch to using the above helper methods or defining the game if accessing directly from the dict itself (e.g., item_names[game][id]) Eventually, this implicit fallback will be removed.

Core Changes

This change also increments the version of Archipelago to 0.5.0 considering the level of breaking change this contains.

How was this tested?

Used a modified version of Clique that has ids that match another game and confirmed MultiServer and TextClient worked fine.

Couple unit tests also added for changes to CommonContext.

I have provided a website for "Clique1933" which shares the same IDs as "Clique" to test the client and server changes, as well as to test your own clients/lib to ensure it doesn't crash when this game interacts. See link for apworld.

Clique1933 Client: http://darkshare.site.nfoservers.com/clique_1933/
Clique1933 APWorld: http://darkshare.site.nfoservers.com/clique_1933/clique_1933.apworld

Please be sure to mark libraries and clients that have updated for this change in #3394.

If this makes graphical changes, please attach screenshots.

N/A

[black-sliver]

Other things that should probably be part of this PR

• update docs and protocol
• define and require min client version
• reject upload of non-checksum seeds in webhost
• either upgrade the DB or disable spinning up of non-checksum seeds - upgrade could be done on demand or as a click. If we decide to disable old seeds instead, we should probably clean out the database at some point.
• remove code path from custom server and/or MultiServer that loads the Datapackage from worlds/

[ThePhar]

I know Berserker and I were talking about potentially deleting old seeds. @Berserker66 would probably have a better idea on the feasibility/practicality of upgrading the DB.

[ScipioWright]

The Noita mod/client now supports this change. It has a cache for the room to link up the slot number for each player with the game they are playing, and uses that as a reference for item/location names.

Edit: okay rereading this, this comment isn't super clear. I'm just following the "ideal way" as mentioned above with no real variation.

[el-u]

I don't disagree with the import and string changes in general, but including so many unrelated changes in this already large PR unnecessarily makes it less clear to read.

[ThePhar]

I don't disagree with the import and string changes in general, but including so many unrelated changes in this already large PR unnecessarily makes it less clear to read.

This PR is going to be renamed later. Part 1 of tracker clean up goes hand in hand with this PR. Not going to hackily make tracker.py work and then redo all my work again afterwards.

[ThePhar]

Yep it was hints, just had to pass along player info in the node. Seems to work for me now.

[Berserker66]

Why is this getting deleted in this PR?

[ThePhar]

Because it snuck back into main somehow. The real tracker was renamed tracker__OcarinaOfTime.html so this isn't evne used anymore.

[ThePhar]

I suppose it could have been in another PR.

[Berserker66]

Looked it over, remarked what I noticed and my clients seem to still work.