Sparse files can be incompatible with GS/OS #49
Comments
|
I've seen Copy ][+ fail on such files (see issue #15), but I was not aware that the ProDOS FST had issues. What version of GS/OS are you using? Can you attach a disk image with a file that fails? |
|
A ran some tests. A file with 1024 zeroes stored in a ShrinkIt archive occupies 3 blocks when extracted by P8 ShrinkIt, but only 1 block when extracted by GS/ShrinkIt, and 1 block when created by CiderPress. Looking at the directory entry, GS/OS set the storage type to 5 (forked) for all files, while P8 set it to 2. P8 ShrinkIt is able to "type" files extracted by either (it just can't open resource forks). So both GS/OS and P8 are able to handle files with an initial sparse block. I'm not sure why you're seeing failures / corruption. |
|
I've been able to reproduce an issue. It appears that a file with nothing but zeroes can be represented as fully sparse, but a file that starts with zeroes and then has some nonzero values must have an explicitly-allocated block 0, or you can get failures. So far I've only seen them happen when the file is > 128KB (the transition from "sapling" to "tree" structure). I suspect the problem is that there must be at least one block allocated in the first 128KB, so that the first block in the tree's master block list is allocated. |
|
Glad you have been able to reproduce it. I noticed it on a tree, but another person on the Slack has seen it on a 34 KB file (i.e. sapling). |
It appears that some programs, notably Copy ][+ and the GS/OS FST, expect that the first block of a sparse file will always be allocated, even if it's just 512 zeroes. I've only seen this fail in cases where the first entry in the master index block of a "tree" file is zero, because the first 128KB are sparse, but rumor has it that the issue can affect "sapling" files as well. This changes CiderPress to work the way ProDOS does: always allocate a block to hold the first 512 bytes of the file. The only exception to this is a file that is nothing but zeroes (perhaps populated by creating a file and setting its EOF to something large), which can be stored as a seedling. (for issues #15 and #49)
|
This should fix the issue: https://github.com/fadden/ciderpress/releases/tag/v4.1.0-d3 I'm always nervous messing with the filesystem stuff, but I threw a bunch of tests at it and nothing seemed to break. Let me know if it does the trick. |
|
That's fixed it! Thanks a lot! |
|
@fadden Indeed, good unit tests are your friend. |
If you have a file which has 512 bytes or more of zeros at the start of the file, Ciderpress will write the first block as a sparse block.
If you attempt to read this using GS/OS / the ProDOS FST it will not be read in as sparse! As a result the file will be corrupt.
If you write 512 bytes or more of zeros from within a GS/OS program it will not write the first block as a sparse block! Other blocks will be written as sparse, but not the first. As a result, this issue was never encountered prior to the invention of external tools.
There even appears to be code within GS/OS when reading the file to bypass sparse handling for the first block. So it appears to be deliberate behaviour, even though the reasons are unknown.
The text was updated successfully, but these errors were encountered: