Chunk restoration on compression data check failure

[BinarySplit]

I just had some awesome success restoring a corrupted chunk by simply decompressing and recompressing it, while muffling the data check exception. This is a feature request to add a restoration command in the REPL.

The chunk was from a world running the Minecraft 1.7.10 Regrowth modpack. It was corrupted due to a random game crash. After using Minecraft-Region-Fixer to find the block/chunk ID, this is how I fixed it via the Python repl:

from nbt.region import *
import zlib
r = RegionFile("r.-3.4.mca")
r.get_blockdata(0, 23)
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
#   File "nbt\region.py", line 541, in get_blockdata
#     raise ChunkDataError(err)
# nbt.region.ChunkDataError: Error -3 while decompressing data: incorrect data check
m = r.metadata[0, 23]
r.file.seek(m.blockstart * SECTOR_LENGTH + 5)
chunk = r.file.read(m.length - 1)
zlib.decompress(chunk)
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
# zlib.error: Error -3 while decompressing data: incorrect data check
dc = zlib.decompressobj()
out = ""
for c in chunk:
    out += dc.decompress(c)
# Traceback (most recent call last):
#   File "<stdin>", line 2, in <module>
# zlib.error: Error -3 while decompressing: incorrect data check
#### (NOTE: out now contains all retrievable bytes up until the byte that threw the exception)
len(out)
#128738
r.write_blockdata(0, 23, out)
#### Checking that roughly the correct number of bytes were salvaged:
len(chunk)
#8974
len(zlib.compress(out))
#8974

Booting up Minecraft, the chunk appears completely fine. There's probably a speck of dust in the wrong place somewhere, but it's still much better than losing 1/4th of my base.

[macfreek]

Thanks, great report! Given the success of your method, I can only second your request to make this functionality available in Minecraft-Region-Fixer.

I'm maintainer of the NBT library. After reading your report, I briefly considered if this type of fall-back should be able in NBT as well. I don't think so, as there is no guaranteed way to check if all data is recovered: the metadata of a MCA file does not list the size of a decompressed NBT data structure. I'm open for other suggestions though (I think the functionality should be there, I'm just wondering were it should go: NBT or Minecraft-Region-Fixer).

@Fenixin, @BinarySplit: is there any particular functionality you miss from NBT? For example, a method to read the raw, uncompressed data? We did split get_blockdata from get_nbt (previously, it was one -even longer- method). Would it be useful to split this even further? Is the functionality to parse all NBT data sufficient? E.g. what should be the desired functionality if a NBT data structure is truncated? And would this ever happen? E.g. with some of the uncompressed files.

[Fenixin]

Hello! And sorry for the time it took me to answer.

Thanks for the suggestion and the code, this feature sounds neat! Is something that region fixer really needs.

@macfreek: I'm not really good deciding where functions should be, that said, a function to read raw data from region files in NBT sounds like a good way to implement this in region-fixer. Another idea is to add an argument to get_blockdata to ignore compression errors, don't know if its a good one though, I haven't read this code in months. I don't really know what is the desired functionality if a NBT data structure is truncated, I think I have never found one. This is probably because almost all NBT files are always compressed, and if something goes wrong is (well, I thought it was) completely lost. If we start to restore corrupted chunks I can see this happening but I have no idea which will be the best way to handle it.

[Fenixin]

Hello!

Better late than never. It only took me 5 years to do this.

The branch bugfix has a implementation of this. It seems to work, the problem is that I only have one region file to test it, and it can't decomrpess a sigle byte from it, so I don't know if it works.

Thanks a lot for your report and the idea. I'm closing this, feel free open/comment o whatever you want.