[Feature Request] Add the ability to check and fix the errors in gocryptfs file system

[yanjiew1]

Sometimes, the file system may be corrupted due to the bad cloud synchronizing apps or provider as well as memory error or silent bit rot on the disk.

Checking and fixing errors in the gocryptfs file system may be a good feature.

[rfjakob]

Yes, good idea. I have also thought about it.

…

On Jan 7, 2018 02:06, "jaywang0tw" ***@***.***> wrote: Sometimes, the file system may be corrupted due to the bad cloud synchronizing apps or provider as well as memory error or silent bit rot on the disk. Checking and fixing errors in the gocryptfs file system may be a good feature. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#191>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARgf2CQNPGTFe3tJrGP0gkaYMyU5oZGks5tIBiTgaJpZM4RVfz0> .

[rfjakob]

For now, @yahesh posted this a quickl way to check the files in a mounted directory: #221 (comment)

find . -type f -exec shasum {} \; | grep "shasum:"

[rfjakob]

The -fsck feature is almost done - read-only, though! The users will have to fix errors (and/or delete corrupt files) themselves.

If you want to test-drive it, it's available in the fsck branch,

git remote update
git checkout fsck
./build.bash

[rfjakob]

Merged to master.

[pos42]

Will the fsck be distributed with the binary package? Or do I have to build it myself?

Asking as I think it should be included in the gocryptfs binary itself or at least be distributed along with the gocryptfs binary.

[pos42]

I see... You have added it to the binary, but not added it to "gocryptfs --help"

[rfjakob]

Hmm yes, fsck should probably be in the short help

[pos42]

Is it still read only in 1.6?

[rfjakob]

Yes

[pos42]

Can we expect it to be non read only in the future? Timeframe? Asking as I am starting to put a lot of stuff in my rocket fast gocryptfs folders (love it…). The best overlay encryption out there... I would like to see something like Cryptomator:s ”sanitizer".

Two days ago I had a problem where a few folders within my gocryptfs could not be deleted. Probably due to cloud problems. This was a head ace.

So I am waiting for a writeable fsck version to be able to trust it even more… OR instruction of how to handle the read only version with manual repair. But it you have many errors a manual repair is not optimal.

Love your work with gocryptfs !

[rfjakob]

About "fixing" corruption... The thing is, in general there is not much that can be done but deleting the corrupt file. Would this still be useful for you?

[rfjakob]

Or better, move the corrupt file to a lost+found folder. That should work. Maybe for 1.7!

[pos42]

Yes

Great idea!

If you consider a file dead and it is locked and can´t be deleted, what could you do? Create a new gocryptfs folder or just leave it there for ever? If it cannot be removed, sync programs like rsync could give errors every time as well as the file is not deletable.

So an fsck that asks about deletion of corrupted files would be great (and move them to lost+found). Perhaps a "-y" flag to it if it is many like std fsck.

[pos42]

B t w

You should reconsider regarding std paypal donations to the project. Maybe call it a "wife flower" fund to be used because all hours you spent on this.

[rfjakob]

I have created a new ticket for this at #263

[dumblob]

What'll happen if the bitrot affected a directory node. Will the whole subtree become unreachable forever (i.e. potentially the whole mountpoint)?

Or is there any way to recover those files in quick/performant way (I don't consider recovery from backup to fall into this category)?