Payload limit exceeded when there's too much Surveyor data

[Cynthia7979]

Internal Exception: io.netty.handler.codec.DecoderException: java.lang.IllegalArgumentException: Payload may not be larger than 32767 bytes

When Surveyor's chunk data for a server reaches a certain threshold, one or more of the packets it sends to the server exceeds the payload size limit of 32 KiB, causing the player to disconnect immediately after joining a server.

A workaround we found is to delete several arbitrary chunk files in data/surveyor/<serverID>/minecraft/<dimension>. Deleting the whole folder works as well.

Haven't looked at the code yet but what I suspect is happening is that, upon entering the server or a new dimension, Surveyor sends a single big packet containing a summary of which chunks it has on the client side. This would explain why the invalid payload appears only after having explored a large portion of the map.

Context

Me and a few of my friends are playing on a modded server that hosts a very large RPG map called Drehmal: Apotheosis. As we progressed through the map, an increasing number of players reported getting the error "Payload may not be larger than 32767 bytes" and being kicked immediately upon joining the server, similar to the behavior of a chunkban or bookban.

Apart from the error message shown in the screenshot above, nothing in the logs were remotely relevant to the issue. I was unable to reproduce the issue using the same player .dat file on a local save. To see if anything in the map accidentally caused a bookban situation, I checked the following files using an NBT viewer, but none of them turned out to have an abnormal size or corrupted content:

• Region .mca file of the region the player's supposed to be in
• Entity data .mca file of the region the player's supposed to be in
• Player .dat file

We then found that if the player reinstalls the modpack, dumping all stored mod data in the process, then they will get to play again with no issue. I soon ran into the same problem and did some testing myself, and learned that:

1. The issue disappears when Surveyor is disabled (using the disable mod feature in Modrinth App)
2. Re-downloading Minecraft and mods, without resetting mod data, does not do anything
3. If I delete a few files in data/surveyor/../minecraft/<dimension I'm in>, then the issue fixes itself without needing a full reinstallation.

There were some 250+ files in said folder, the sizes of which ranging from 4 KiB to 434 KiB. I prioritized deleting the larger ones, but it seems like deleting any arbitrary few would have done the same.

Specs

• Minecraft: 1.20.1
• Mod loader: Fabric 0.16.7
• Launcher: Modrinth App
• Surveyor: 0.6.23+
• Surveystones: 1.3.1
• Antique Atlas 4: 2.9.17+
• Wraith Waystones (Fabric Waystones/fwaystones): 3.3.2

Can provide list of other mods on client/server if requested.

Related Issues

#17 - We didn't use /surveyor share but the workaround seems similar.
#23 - Also encountered the up-to-a-minute loading time when connecting to a server. The loading time goes away completely if Surveyor (and Atlas) is disabled. Loading time also seems to be related to portion of map explored.
#25 - Similar "crash/disconnect as soon as entering world" behavior. We don't have any structure mod installed, and the map was originally generated and built in Vanilla, so the cause is probably not the same.

[sisby-folk]

So the big question is "what packet is it" - most packets we expect to be large are built to split when they're too large, and provide a clear error when an atomic packet exceeds the size limit. If you provide a crash or latest.log via https://mclo.gs/ the stack trace might reveal which, otherwise I'll need your big surveyor data to test with.

[Cynthia7979]

This does not cause a crash (although the game does freeze for a short time when it sends the giant packet that will eventually be rejected by the server), so there's no crash log unfortunately.

I found a log one of our players shared when the bug first occurred, but it seems like the reason for server disconnect isn't logged: https://mclo.gs/xHAdw10

I'll go change my log4j configuration so it logs DEBUGs, and try reproducing the issue the next time I play on the map. If that still doesn't produce a verbose enough log at least I'll have the Surveyor data available 👍

[sisby-folk]

thanks!