[MP] Implement "watched" properties (reliable/on change).

[Faless]

Tentative implementation for godotengine/godot-proposals#4429.

How it works

"Watched" properties are deep duplicated and then compared (Variant::hash_compare) according to the delta_interval property of the MultiplayerSynchronizer (default = every frame). Re-duplication only happens when a change is detected, and the time of change is also stored for each property.

Clients receives progressive (delta) changes which only includes the changed properties. The transmission happens reliably, so all changes "across frames" (delta_interval) are guaranteed to be received by the clients.

There is a hard limit of 64 watched properties per synchronizer.
There is a (quite arbitrary) limit of 64k bytes per delta packet which probably needs adjustment.

WIP / RFC:

• More testing (e.g. comparison method). Seems to work fine in my tests, we can always fix that in a follow up PR.
• More feedback (e.g. custom comparison function, comparison frequency, reliable vs unreliable + ack). Well, it's been a month, I think I can mark the PR as ready.
• Discuss replication config editor redesign (e.g. make "Initial/Sync/Delta" exclusive, or use the current one in the proposal). Let's do that in a separate PR/proposal.
• Some further cleanup / internal command ids change. Kinda unrelated to this PR.
• Nice to have: Implement editor profiler (needs redesign proposal due to space constraints 😅 ). Needs design proposal.

[Calinou]

There is a hard limit of 64 watched properties per synchronizer.

If we go with this design, remember to make the editor UI show an error dialog when trying to make a property watchable when there are already 64 watched properties.

There is a (quite arbitrary) limit of 64k bytes per delta packet which probably needs adjustment.

I assume this is done to avoid UDP fragmentation, therefore reducing potential issues with latency?

[Faless]

If we go with this design, remember to make the editor UI show an error dialog when trying to make a property watchable when there are already 64 watched properties.

Will do.

I assume this is done to avoid UDP fragmentation, therefore reducing potential issues with latency?

Well, yes and no, UDP fragmentation starts at around 1400 bytes, so a 64k packet will already be fragmented in ~64 packets.
The goal of the limit is to still keep the fragmentation to what I believe is a reasonable number and avoid clogging too much the underlying peer with huge packets.
But honestly it's pretty arbitrary as I mentioned, so it should probably be configurable. I'll look into that too.

[Faless]

Updated:
Editor error when adding > 64 properties.
Expose sync_mtu/delta_mtu for configuration.
Use usec instead of msec in synchronizer/replicator.

[dsnopek]

This looks great! Regarding the overall approach: it's hard to strike a balance between saving network bandwidth and the processing necessary to do so, and I think this does a pretty good job :-)

[mhilbrunner]

@Faless I found nothing besides what is already mentioned above, docs look good too, and the implementation is clean and minimal, nice work :)

I'll see if I can find some time to do some tests and play around with it, but once the above is addressed this IMO is good to merge. Should not impact anything else anyway.

[Faless]

Thanks everyone for the reviews ❤️ , I believe I've addressed all the comments and suggestions.

[akien-mga]

Thanks!

[FeralBytes]

This is awesome but it is also causing the server side to crash often. I will add more details to this issue here.