Queue gets stuck with DB locked

[dervish666]

Description of the bug
Have performed a search and it's found a load of files to import, on the activity screen it imports the first file to 100% and then stops.

To Reproduce
When restarting the docker container it automatically starts to try to import the files, when it does it reaches 100% on the first one and fails.

Expected behaviour
It should import the files.

Screenshots
Included the relevant log entries....

07/09/2023 10:35:18
[2023-09-07 10:35:18][Download Importer][INFO] Adding https://getcomics.org/links/vlPuyCNEBjZvNuF4VdB/a8jmA6p6Actu0adhieEhsE5FWuKASTHgQ9z8bMVswuAKF6fxkw2bZA30rlrHcen8/fwJz73G3veABAPuKmXVBNFeapqF02Oezf/dWQebo0CIocm7h1B0dJsW5FbU3+PEL02fjPodF4+4IL3gT8/zE6Y=:Tykix0LhmcNiJPzmRisFfA== to blocklist with reason "Link broken"
07/09/2023 10:35:18
[2023-09-07 10:35:18][Download Importer][INFO] Adding download for volume 11: https://getcomics.org/other-comics/firefly-28-2021/
07/09/2023 10:35:18
[2023-09-07 10:35:18][Task Handler][ERROR] An error occured while trying to run a task:
07/09/2023 10:35:18
Traceback (most recent call last):
07/09/2023 10:35:18
File "/app/backend/tasks.py", line 292, in __run_task
07/09/2023 10:35:18
result = task.run()
07/09/2023 10:35:18
File "/app/backend/tasks.py", line 226, in run
07/09/2023 10:35:18
refresh_and_scan()
07/09/2023 10:35:18
File "/app/backend/volumes.py", line 412, in refresh_and_scan
07/09/2023 10:35:18
cursor.executemany(
07/09/2023 10:35:18
sqlite3.OperationalError: database is locked
07/09/2023 10:35:19
[2023-09-07 10:35:19][Task Handler][INFO] Starting auto search for volume 1
07/09/2023 10:35:19
[2023-09-07 10:35:19][Task Handler][INFO] Starting auto search for volume 2

Version info

Kapowarr version	v1.0.0-beta-3
3.8.17.final.0
10
/app/db/Kapowarr.db
/app

Using Unraid, and chrome.

Additional context
I have tried deleting the offending entry but it then doesn't continue to the next one, when I then restart the original entry comes back and behaviour is the same.

[Casvt]

I already noticed this myself. It happens when loads of downloads are added to the queue and are completing and importing all at the same time. The database locks up and "crashes" the download queue, making it stall. I'm already looking for ways to fix this, but it's a bit complicated.

[dervish666]

Thanks for the quick reply, let me know if you would like me to test anything.

[dervish666]

Is there any way to add a "import now" button so when it fails you can start the process when the rest of the app has calmed down? Or a way to trigger an import of an individual file?

[Casvt]

That's avoiding the problem instead of fixing it. I don't really like such solutions.

[Casvt]

Fixed this. Will be included in the next release.

The download importer (which fills up the queue after a restart), would in your case add a link to the blocklist. That requires a write to the database and from then on the database is locked until the download importer is done. In the mean time, something else would use the database and throw the error that the database is locked. I fixed it by immediately commiting the database after something is added to the blocklist to not lock up the database. It should be fixed now.