Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gocryptfs on mergerfs: backing file inode number crosses reserved space #457

Closed
Suika opened this issue Mar 2, 2020 · 3 comments
Closed
Labels

Comments

@Suika
Copy link

Suika commented Mar 2, 2020

I'm currently using mergerfs to aggregate a few HDDs into a single directory and apply gocryptfs in reverse mode ontop of it to create a backup.

@trapexit had made a change to his inode algorithm in trapexit/mergerfs@a646fe0 which triggered the error in gocryptfs.
GetAttr "": backing file inode number 14920808378658256559 crosses reserved space, max=1000000000000000000. Returning EOVERFLOW.

I can circumvent the problem of the inodes, by removing the use_ino from mergerfs and using the fuse inodes, but the question is why it's done that way and why there is a limit.
It's a bit problematic for me, since I don't know to whom to report the problem.

@trapexit
Copy link

trapexit commented Mar 2, 2020

mergerfs is using the full 64bits of the inode (which AFAIK is entirely legit and not unique). I could add as special inode calculation mode that MODs the value by 10^19 but I imagine it's a matter of time till some other filesystem runs into this issue.

I understand the desire to reserve values for your own use but perhaps you could use the fact that filesystems don't need to have unique values. Or do you use public inode values internally for something?

@rfjakob
Copy link
Owner

rfjakob commented Mar 3, 2020

I'd say the problem is with gocryptfs. I don't think mergerfs does anything wrong, just unusual. I'll see if there is way to fix it.

@rfjakob rfjakob added the bug label Mar 3, 2020
@rfjakob rfjakob changed the title backing file inode number crosses reserved space gocryptfs on mergerfs: backing file inode number crosses reserved space Mar 3, 2020
@trumblejoe
Copy link

@trapexit Hard linking would be the reason. Which programs like radarr and sonarr use.

rfjakob added a commit that referenced this issue Apr 19, 2020
Adding flags allows to use inomap in reverse mode,
replacing the clunky inoBaseDirIV/inoBaseNameFile
logic that causes problems with high underlying
inode numbers ( #457 )

Microbenchmarks (values below) show that the "SingleDev"
case is now much slower due to an extra map lookup,
but this has no visible effects in ./test.bash results,
so there was no time spent optimizing the case further.

$ go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/rfjakob/gocryptfs/internal/inomap
BenchmarkTranslateSingleDev-4   	18757510	        61.5 ns/op
BenchmarkTranslateManyDevs-4    	18061515	        64.5 ns/op
PASS
ok  	github.com/rfjakob/gocryptfs/internal/inomap	2.467s
@rfjakob rfjakob closed this as completed in 518771e May 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants