PermissionError in post-processing

[azzuron]

Running this in a docker container with NFS mounts will seemingly cause the app to crash/get stuck after a download. it will copy the file into the destination and the old file in the download temp path will stay. You get a few errors in the log and the system stops until the docker container is closed out and restarted.

The first error is:

kapowarr-kapowarr-1 | os.rename(src, real_dst) kapowarr-kapowarr-1 | OSError: [Errno 18] Invalid cross-device link: '/downloads/file.cbr' -> '/comics-1/Path/path/file.cbr'

The second error is:

kapowarr-kapowarr-1 | os.rename(src, real_dst) kapowarr-kapowarr-1 | OSError: [Errno 18] Invalid cross-device link: '/downloads/The Walking Dead 193 (2019) (Digital) (Zone-Empire).cbr' -> '/comics-1/Path/path/file.cbr'

To Reproduce

1. Run Kapowarr in docker with NFS mounts for the download temp path and the destination path.
2. Add a series
3. Search for series or issue so it will download
4. It will download
5. Log will produce the errors

Expected behaviour
The file is moved into the destination and the download temp copy is removed. The app then proceeds to the next file.

Version info

Kapowarr version	v1.0.0-beta-3
3.8.17.final.0
10
/app/db/Kapowarr.db
/app

Browser: Firefox 120.0.1
Windows (Kapowarr runs in docker container)

[Casvt]

Yeah someone else also encountered this a few days ago. I already know how to fix it, just have to do it.

[Casvt]

It turns out to be more complicated than it originally seemed. Could you share the complete traceback?

[azzuron]

kapowarr-kapowarr-1 | Exception in thread Download Handler:
kapowarr-kapowarr-1 | Traceback (most recent call last):
kapowarr-kapowarr-1 | File "/usr/local/lib/python3.8/shutil.py", line 791, in move
kapowarr-kapowarr-1 | os.rename(src, real_dst)
kapowarr-kapowarr-1 | OSError: [Errno 18] Invalid cross-device link: '/downloads/file.cbr' -> '/comics-1/path/path/file.cbr'

[azzuron]

kapowarr-kapowarr-1 | During handling of the above exception, another exception occurred:
kapowarr-kapowarr-1 |
kapowarr-kapowarr-1 | Traceback (most recent call last):
kapowarr-kapowarr-1 | File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
kapowarr-kapowarr-1 | self.run()
kapowarr-kapowarr-1 | File "/usr/local/lib/python3.8/threading.py", line 870, in run
kapowarr-kapowarr-1 | self._target(*self._args, **self._kwargs)
kapowarr-kapowarr-1 | File "/app/backend/download.py", line 65, in __run_download
kapowarr-kapowarr-1 | PostProcessing(download, self.queue).full()
kapowarr-kapowarr-1 | File "/app/backend/post_processing.py", line 152, in full
kapowarr-kapowarr-1 | self.__run_actions(self.actions_full)
kapowarr-kapowarr-1 | File "/app/backend/post_processing.py", line 138, in __run_actions
kapowarr-kapowarr-1 | action()
kapowarr-kapowarr-1 | File "/app/backend/post_processing.py", line 107, in _move_file
kapowarr-kapowarr-1 | move(self.download.file, file_dest)
kapowarr-kapowarr-1 | File "/usr/local/lib/python3.8/shutil.py", line 811, in move
kapowarr-kapowarr-1 | copy_function(src, real_dst)
kapowarr-kapowarr-1 | File "/usr/local/lib/python3.8/shutil.py", line 436, in copy2
kapowarr-kapowarr-1 | copystat(src, dst, follow_symlinks=follow_symlinks)
kapowarr-kapowarr-1 | File "/usr/local/lib/python3.8/shutil.py", line 381, in copystat
kapowarr-kapowarr-1 | lookup("chmod")(dst, mode, follow_symlinks=follow)
kapowarr-kapowarr-1 | PermissionError: [Errno 1] Operation not permitted: '/comics-1/path/path/file.cbr'

[Casvt]

Note to reader:

On the discord server, the following was discovered:

1. shutil.move is used everywhere and supports moving between file systems.
2. The OSError is raised because it first tries the normal os.rename, fails (because of mismatch in FS), falls back to a cross-FS safe method, raises PermissionError in fall-back, which also raises the original OSError.
3. The problem is the PermissionError, not the OSError.

---

@azzuron could you explain the weird filepath /comics-1/path/path/file.cbr?

[azzuron]

When i mount the path of my file server into the docker container using the volumes i have to provide it a path in the docker container that it will mount to. that path is /comics-1
the /path/path/file.cbr is just generic for the actual file path after the mount point. I left the default folder structure in the config. So i think its like /Series/volume/filename.

So in the container where your software runs it tries to write the cbr files to /comics-1/series/volume/file.cbr.

I note in the log that chmod might be getting invoked for some reason. I believe that will always fail when the target file is on the NFS mount, at least with Truenas. I am not sure of the details on that but i know i have seen something like that before.

kapowarr-kapowarr-1 | lookup("chmod")(dst, mode, follow_symlinks=follow)
kapowarr-kapowarr-1 | PermissionError: [Errno 1] Operation not permitted: '/comics-1/path/path/file.cbr'

I would also like to note that the file is placed where it should go so the app does have permission to write to the NFS mountpoint. This error seems to cancel all other operations however so the file in the download folder is not cleaned up and also it seems to block further activity until a restart.

[Casvt]

Yeah so for some reason TrueNAS, SMB (I assume you're mounting using SMB), or Python does not allow changing the permissions of the file (which chmod does). Anything in your TrueNAS config that would restrict changing the permissions? Maybe the docker container runner (prob. root) is not the owner of the file or doesn't have sufficient permissions, so it can't change the permissions?

[azzuron]

It is NFS. The share permissions are 777 so full access. As i mentioned the file can be written you can write and delete. I tested that from CLI in the docker container. There is no need to CHMOD files within docker i guess. You should be knowing the user id of the process accessing the files on the share and have pre-configured appropriate permissions on it prior to mounting. I believe i only have one other container that tries to do this and it also fails but since the permissions are somewhat relaxed and it is able to just fail and continue the app works because it can read/write to the path without issues in the end.