[Bug] Book analysis is slow for webp images #279
Comments
|
Komga doesn't count pages during the analysis, it does much more than that. The analysis for pdf is faster, because the pages are always rendered in JPEG at the same dimensions, so the analysis doesn't need to analyze each page to detect the media type and size of each page. |
|
The message The webp library used in Komga is quite beta and not very fast, which would explain why your book is long to analyze. Analysis time will also depend on your hardware (cpu) and disk or network access speed to access the books. |
|
@gotson It only took Komga a fraction of the time to scan the identical files in previous versions, though. Just for reference, this is a colocated server with NVMe SSDs, two 12c/24t CPUs and 128GB of RAM, so I don't think a lack of processing power is causing Komga to take 20 minutes to scan a single file. |
|
It's doing more in recent versions than it did before. Where are your files located and how are they accessed? |
|
@gotson The files are stored on the same server; they're not being accessed via NFS or another network file system. I run Komga as a regular systemd service, so it's accessing the files like any other application would. Are you able to reproduce the issue with my test file? Does it also take several minutes for Komga to finish scanning it on your machine? I have about 2,200 files in my collection, most of which are tankobons. Based on my tests, it takes Komga around three seconds to process one page. Assuming an average page count of 100 pages, that would work out to an initial scan time of 7 days, 15 hours and 20 minutes, not including thumbnail generation. Even taking the fact that the "AnalyzeBook" task does more than it did in previous versions into account, that figure seems extremely high to me. Again, just for reference, the initial library scan in previous versions only took a few hours to finish and that's including the thumbnail generation process (which accounted for the vast majority of that time). |
|
I'm on my phone so cannot test now. A cbz with 100 pages is analyzed in 5s on my system. |
ghost
changed the title
|
I managed ti reproduce, it only happens with webp images. The latest versions of Komga perform an image analysis on each image to get the dimensions. This takes around 3s per image on my system for a webp. There are no other webp libraries for Java. There's an open issue at twelve monkeys (a java image library) to have a proper one, but I discussed with the author and the performance would be most likely similar. Another solution would be to use native libraries, and to that end I did submit a PR some time ago on a project to properly package native libraries for multiple architectures, but it's still pending (sejda-pdf/webp-imageio#6). To come back to your issue, it's only gonna impact webp files. I don't know how many you have, I found them to be pretty rare in the western comics and European bds, and only saw a few for mangas. |
|
@gotson Thanks for the explanation. I sadly converted my entire library to WEBP months ago due to the significant space savings and improved page loading times compared to JPEG. Oh well, at least I have confirmation that this is expected behavior now. 😅 |
|
It is really not wise to convert from a lossy format like JPEG to another format. You should only ever convert from lossless to lossy, never from lossy to anything else! |
gotson
changed the title
|
@gotson I would agree if we were talking about music, but lossless sources for almost all other types of media simply don't exist. I personally think the almost imperceptible quality loss is well worth the significant space and bandwidth savings. |
|
I think you are wrong, but you do what you want. |
|
@gotson I didn't say that lossless image formats don't exist, I said that lossless sources don't exist for almost any type of media other than music. A lot of people rip CDs directly to FLAC files and release those on the Internet, but not even official digital manga publishers like Viz release chapters or volumes in a lossless image format. |
|
A scanner gives you a RAW. But that's not the debate here. |
|
I just encountered this recently as well where Webp pages takes much longer to process. If the library is the issue, would it be possible to brute force the problem with really crude multi-threading? I have no java/kotlin experience but maybe you can giving each page to a different core or even different books for each core? My unraid server has 16 cores but there's only one core struggling with the webp. This may be more trouble than it's worth but it could help for people with a lot of webp books if it's doable. |
|
The current library is the issue, but i plan to switch to a native library once sejda-pdf/webp-imageio#6 is merged. The current version doesn't support all the architectures that Komga supports. I just did some tests with a 236 pages book with webp images, the native library performs the analysis in 4s. |
|
@WhaleHub @blkjack410 - until the upstream webp-imageio issue is resolved, here are some summary instructions on how to construct a Komga JAR with the fast decoder library included. Three massive caveats, though:
To modify the komga JAR:
|
|
@mihailim I could kiss you right now. Komga is analyzing my library at breakneck speeds with this build. 😁 @blkjack410 Here's a little Bash script I wrote to automate the build process for new Komga releases. It is to be executed like this: #!/bin/bash
VERSION="$1"
BUILD_PATH="/tmp/komga-build"
mkdir "${BUILD_PATH:?}" &&
cd "${BUILD_PATH:?}" &&
curl -sSL -o komga-tmp.jar https://github.com/gotson/komga/releases/download/v"${VERSION:?}"/komga-"${VERSION:?}".jar &&
unzip -q komga-tmp.jar &&
curl -sSL -o BOOT-INF/lib/webp-imageio-0.1.6.jar https://search.maven.org/remotecontent?filepath=org/sejda/imageio/webp-imageio/0.1.6/webp-imageio-0.1.6.jar &&
rm BOOT-INF/lib/webp-imageio-decoder-plugin-0.2.jar &&
sed -i 's|webp-imageio-decoder-plugin-0.2.jar|webp-imageio-0.1.6.jar|g' BOOT-INF/classpath.idx &&
zip -q -0 -r komga.jar META-INF org BOOT-INF &&
mv komga.jar /usr/local/bin/komga.jar &&
rm -rf "${BUILD_PATH:?}" |
|
@mihailim Thanks soooo much!! It's so fast now and I finally can upgrade komga to the last version. |
|
Any idea on how to do this for the docker container? I tried tossing |
|
@CWhits Since containers are ephemeral by nature, all changes that are made inside of them while they are running are discarded when they are restarted. To make use of the faster WEBP library, you have to either build a custom Docker image or map the modified JAR file from the host to the container using Docker's volume feature. |
that's only true for recreating a container, not restarting a container. regardless of how many times I restart the container, any changes I made will remain until I recreate the container. it also doesn't appear that the jar exists inside the container, or, if it does it's the unzipped version of the jar. I don't know anything about Java so I'm at a loss there. I guess I could map BOOT-INF, META-INF, and org as volumes but that seems a lil crazy. |
|
well it's gross but it worked. unzipped the jar (added/removed the webp libraries, edited the classpath) and mounted it to /app. 🤷♀️ |
|
For docker you can leverage on bind mounts to only change a single file. In your - type: bind
source: /path/to/this/webp-imageio-0.1.6.jar
target: /app/BOOT-INF/lib/webp-imageio-decoder-plugin-0.2.jar
read_only: trueRecreate the container. That's it! |
|
@blkjack410 @mihailim @Kiwi-13-plo @CWhits Just wanted to share a discovery I made earlier today with my fellow WEBP adopters. I always took it for granted that WEBP would be universally better at compressing manga/comics compared to the alternatives, but it turns out that it depends largely on the encoder you use. I've been using WEBP with ImageMagick @ 80% for my conversions up until now and tried out the JPG encoder MozJPEG @ 80% today, only to discover that the output images produced by the latter are not only smaller, but also either identical or higher in quality compared to their WEBP counterparts. YMMV, but check out the results of a test I did using a very detailed and high quality page from Berserk. JPG (Original): 7.59MB Original vs WEBP I'm probably gonna redownload all my stuff and encode it with MozJPEG this time around because once JPEG XL becomes standardized and widely supported, it will be possible to compress these images a further 20% or so "while allowing the original JPEG to be recovered byte-by-byte" (see brunsli). |
|
That heavily depends on the input file, @WhaleHub -- I've run tests over hundreds of heterogenous images using combinations between MozJPEG, jpegoptim, jpeg-recompress from jpeg-archive (my preferred JPEG tool, which internally uses MozJPEG and manages to keep closer to the original while usually producing smaller images when using MS-SSIM or SmallFry), libwebp etc. Sometimes one wins over the other, but it's hard to compare apples to apples, since the quality metric does not necessarily mean the same thing for all of them. In my experience, webp@q80 still beats jpeg-recompress@q80 by 18-25% overall in file size while retaining better color fidelity. That being said, it still makes the best sense to keep around the original files as long as enough space is available. I use webp for remote disaster recovery backups, where using webp versus jpg means I can cut my storage costs in half; I wouldn't even consider going a lossy-to-lossy re-encoding route if I didn't archive the original files. |
|
Since the author of https://github.com/sejda-pdf/webp-imageio did not reply to my PR, I have forked the repo and published my own version onto JCenter. Komga will use the native library if possible, and fallback on the java implementation (slower) if the native library is not available or cannot be loaded. Currently the following OS/Arch are supported:
|
## [0.64.2](v0.64.1...v0.64.2) (2020-10-25) ### Bug Fixes * native webp library ([15caea3](15caea3)), closes [#279](#279)
|
🎉 This issue has been resolved in version 0.64.2 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
# [0.30.0](v0.29.1...v0.30.0) (2021-01-06) ### Bug Fixes * fix sqlite library detection for aarch64 ([10df531](10df531)) * **opds:** duplicate series if in multiple collections ([c7e6209](c7e6209)), closes [gotson#363](https://github.com/tahngarth/komga/issues/363) * don't append volume to series title if equals to 1 ([1cb9ae6](1cb9ae6)), closes [gotson#347](https://github.com/tahngarth/komga/issues/347) * enhanced logging for sqlite native lib loading ([f1a894d](f1a894d)) * scanner crashes if folders are not readable ([f43a386](f43a386)), closes [gotson#342](https://github.com/tahngarth/komga/issues/342) * use bionic base image ([85fe674](85fe674)), closes [gotson#349](https://github.com/tahngarth/komga/issues/349) * **api:** support accept header in getBookPage ([gotson#352](https://github.com/tahngarth/komga/issues/352)) ([e123d38](e123d38)), closes [gotson#350](https://github.com/tahngarth/komga/issues/350) * native webp library ([15caea3](15caea3)), closes [gotson#279](https://github.com/tahngarth/komga/issues/279) * use own distrib of sqlite with freebsd support ([3d2793a](3d2793a)), closes [gotson#319](https://github.com/tahngarth/komga/issues/319) * **api:** incorrect parameter type ([342f04c](342f04c)) * **api:** large file download ([d100db2](d100db2)) * **api:** openAPI spec fields marked as required ([8f228db](8f228db)), closes [gotson#308](https://github.com/tahngarth/komga/issues/308) * **opds:** cannot download file with semicolon in filename ([e1e251e](e1e251e)), closes [gotson#309](https://github.com/tahngarth/komga/issues/309) * **opds:** latest series sort inverted ([bddc008](bddc008)), closes [gotson#301](https://github.com/tahngarth/komga/issues/301) * **opds:** url encode publisher query param ([8347fdc](8347fdc)), closes [gotson#335](https://github.com/tahngarth/komga/issues/335) * **webreader:** images smaller than viewport would not fill width ([81d9a00](81d9a00)), closes [gotson#311](https://github.com/tahngarth/komga/issues/311) * trim and lowercase tags and genres in metadata ([656f23a](656f23a)), closes [gotson#302](https://github.com/tahngarth/komga/issues/302) * **api:** do not return empty language ([2ab3c9c](2ab3c9c)) * **webreader:** incorrect display on safari and firefox ([22b0b7e](22b0b7e)), closes [gotson#262](https://github.com/tahngarth/komga/issues/262) * **webreader:** webtoon reader not loading images ([1a030b4](1a030b4)), closes [gotson#259](https://github.com/tahngarth/komga/issues/259) * **webui:** change grid size for sm screens ([18280be](18280be)) * **webui:** clear selection after action performed ([e0cff70](e0cff70)), closes [gotson#303](https://github.com/tahngarth/komga/issues/303) * **webui:** compute scrollability on mount, fix right chevron when using display scaling ([8928959](8928959)) * **webui:** contextual filters ([f515819](f515819)), closes [gotson#290](https://github.com/tahngarth/komga/issues/290) * **webui:** correct label in library edit dialog ([c532e50](c532e50)) * **webui:** disable filter panels with no values ([57cc6c4](57cc6c4)) * **webui:** edit series dialog could incorrectly display MIXED placeholder ([5d3f061](5d3f061)) * **webui:** field should not display if empty ([968e297](968e297)) * **webui:** filter values could be duplicated ([e91954f](e91954f)) * **webui:** restore library filter for language and age rating ([dc064f2](dc064f2)) * **webui:** theme is not restored properly ([432dc91](432dc91)), closes [gotson#304](https://github.com/tahngarth/komga/issues/304) * database migration could fail ([1c9f3d0](1c9f3d0)) * database migration failure ([2ca6077](2ca6077)) * **analysis:** page number was not persisted ([99f800c](99f800c)), closes [gotson#189](https://github.com/tahngarth/komga/issues/189) * **api:** add count of in progress books in SeriesDto ([0d2713a](0d2713a)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **api:** add missing sort fields for Books ([797535d](797535d)) * **api:** books could disappear for users if read by others ([3d1f0e0](3d1f0e0)) * **api:** collection without element would return incorrect dto ([0891981](0891981)) * **api:** incorrect number of books per series ([bf2d0ba](bf2d0ba)) * **api:** incorrect number of books returned ([544c873](544c873)), closes [gotson#177](https://github.com/tahngarth/komga/issues/177) * **api:** pagination for collection's series ([6dd0704](6dd0704)) * **api:** sort libraries case insensitive ([1650aec](1650aec)) * **docker:** docker image doesn't start ([8bce80b](8bce80b)) * **metadata:** recognize ComicInfo Day tag and "MA15+" rating ([3ad438d](3ad438d)) * **opds:** prepend position for series in ordered collection ([7e5a141](7e5a141)) * **webreader:** cycle image fit would not persist setting ([e5272d2](e5272d2)) * **webui:** missing settings icon in webreader ([9693829](9693829)) * fix database migration errors ([f84ba17](f84ba17)), closes [gotson#239](https://github.com/tahngarth/komga/issues/239) [gotson#238](https://github.com/tahngarth/komga/issues/238) [gotson#237](https://github.com/tahngarth/komga/issues/237) [gotson#240](https://github.com/tahngarth/komga/issues/240) * ignore blank metadata fields ([c8c5df2](c8c5df2)), closes [gotson#288](https://github.com/tahngarth/komga/issues/288) * incorrect book number after adding book in series ([3b1eb36](3b1eb36)), closes [gotson#269](https://github.com/tahngarth/komga/issues/269) * incorrect read progress for books ([107d7db](107d7db)), closes [gotson#286](https://github.com/tahngarth/komga/issues/286) * potential incorrect read progress for series ([7b90244](7b90244)) * series genre is not imported from metadata ([58fac36](58fac36)) * series genre is not imported from metadata ([d0815dd](d0815dd)) * **api:** exception on unpaged empty content ([2cc27f2](2cc27f2)) * **scanner:** add TRACE logs for file update times ([5433567](5433567)), closes [gotson#159](https://github.com/tahngarth/komga/issues/159) * **webui:** action menu not showing for non-admin ([e60666c](e60666c)) * **webui:** adjust padding for grids ([02e9168](02e9168)) * **webui:** adjust series unread count when marking books ([31e21fe](31e21fe)) * **webui:** browse collection should show all series ([8629ea9](8629ea9)), closes [gotson#226](https://github.com/tahngarth/komga/issues/226) * **webui:** clear selection when reloading search results ([7d808c5](7d808c5)) * **webui:** display collection after adding in series browsing screen ([8ce5a39](8ce5a39)) * **webui:** drag series by handle when editing collection ([e72f4ab](e72f4ab)), closes [gotson#214](https://github.com/tahngarth/komga/issues/214) * **webui:** filter status not reset properly when changing library ([fea5431](fea5431)) * **webui:** incorrect count of collections in browse collections screen ([2254929](2254929)) * **webui:** lazy http call for CollectionEditDialog ([4716a1b](4716a1b)) * **webui:** lazy load collections on browse series ([d89533d](d89533d)) * **webui:** library navigation not highlighted correctly ([e28c070](e28c070)) * **webui:** make card title as link ([d6e4b80](d6e4b80)), closes [gotson#224](https://github.com/tahngarth/komga/issues/224) * **webui:** missing settings icon in sidebar ([b9e4bea](b9e4bea)) * **webui:** prevent cropping on book thumbnails ([e038857](e038857)), closes [gotson#191](https://github.com/tahngarth/komga/issues/191) * **webui:** read button link underline on hover ([863258d](863258d)) * **webui:** reset edit book dialog to first tab on reopen ([5760a06](5760a06)) * **webui:** thumbnails explorer navigation misaligned ([2ffed28](2ffed28)) * **webui:** update option title in library dialog ([29d9726](29d9726)) * order of pages is not loaded correctly from database ([d2288dd](d2288dd)), closes [gotson#189](https://github.com/tahngarth/komga/issues/189) * properly expand home dir in config ([5d86d3e](5d86d3e)), closes [gotson#195](https://github.com/tahngarth/komga/issues/195) [gotson#203](https://github.com/tahngarth/komga/issues/203) * some PDF pages are blurry ([9ad1cfe](9ad1cfe)), closes [gotson#260](https://github.com/tahngarth/komga/issues/260) * **webui:** replace mark as read/unread buttons with icons ([709ee1e](709ee1e)) * **webui:** series grid not expanding to full width ([2f7d2a4](2f7d2a4)) * **webui:** simplify unread filter ([bb60f10](bb60f10)) * **webui:** unread tick not showing properly on books ([0d0b998](0d0b998)) * **webui:** use pagination for browsing screens ([5867db7](5867db7)), closes [gotson#91](https://github.com/tahngarth/komga/issues/91) * **webui:** wider display for collection expansion panel ([96c95ea](96c95ea)) * strip accented characters for title sort when creating series ([6f1e36e](6f1e36e)), closes [gotson#188](https://github.com/tahngarth/komga/issues/188) ### Features * **api:** filter /series endpoint by library_id ([1603a96](1603a96)), closes [gotson#360](https://github.com/tahngarth/komga/issues/360) * append volume to series name ([2567796](2567796)), closes [gotson#343](https://github.com/tahngarth/komga/issues/343) [gotson#132](https://github.com/tahngarth/komga/issues/132) * **api:** filter collection's series ([cfa06a9](cfa06a9)) * **api:** filter referential data by library or series ([982983e](982983e)), closes [gotson#290](https://github.com/tahngarth/komga/issues/290) * **api:** get all age ratings ([be80d86](be80d86)) * **api:** get all languages ([115ad42](115ad42)) * **api:** get all publishers ([1929061](1929061)) * **api:** get referential data by collection ([d7fd296](d7fd296)) * **api:** http eTag caching for all API calls ([fe22cb5](fe22cb5)) * **api:** search books by tag ([f46f1a0](f46f1a0)) * **api:** search series by age rating ([f51d575](f51d575)) * **api:** search series by genre ([da4a0aa](da4a0aa)) * **api:** search series by language ([723d7c1](723d7c1)) * **api:** search series by publisher ([e4b0b2d](e4b0b2d)) * **api:** search series by tag ([7bd1de6](7bd1de6)) * **api:** search series' books by tag ([940d5d3](940d5d3)) * **api:** unpaged queries for books ([8f8054e](8f8054e)) * **opds:** browse series by publishers ([f3fcab4](f3fcab4)), closes [gotson#332](https://github.com/tahngarth/komga/issues/332) * **opds:** recently added books ([dd7bc9c](dd7bc9c)), closes [gotson#327](https://github.com/tahngarth/komga/issues/327) * order books by release date ([e3bb8bc](e3bb8bc)), closes [gotson#318](https://github.com/tahngarth/komga/issues/318) * **webreader:** add double page no cover layout ([5fe015e](5fe015e)), closes [gotson#103](https://github.com/tahngarth/komga/issues/103) * **webreader:** add shortcut information menu ([1885f32](1885f32)) * **webui:** add system theme option ([8f22f01](8f22f01)) * **webui:** change internals for shortcuts. add new shortcuts for settings ([b402817](b402817)), closes [gotson#172](https://github.com/tahngarth/komga/issues/172) * **webui:** dark theme ([efe2021](efe2021)), closes [gotson#179](https://github.com/tahngarth/komga/issues/179) * **webui:** filter collection content ([fc905ef](fc905ef)), closes [gotson#270](https://github.com/tahngarth/komga/issues/270) * **webui:** filter series by age rating ([01eef83](01eef83)) * handle multiple story arcs in ComicInfoProvider ([f4451bf](f4451bf)), closes [gotson#282](https://github.com/tahngarth/komga/issues/282) * **webreader:** better display of landscape images ([09984a4](09984a4)), closes [gotson#123](https://github.com/tahngarth/komga/issues/123) * **webreader:** side padding, original scaling for Webtoon ([6499788](6499788)), closes [gotson#266](https://github.com/tahngarth/komga/issues/266) [gotson#264](https://github.com/tahngarth/komga/issues/264) * **webreader:** webtoon reader, fit to screen ([44c814a](44c814a)), closes [gotson#81](https://github.com/tahngarth/komga/issues/81) [gotson#145](https://github.com/tahngarth/komga/issues/145) * **webui:** color coded chips for series status ([fb8a8c4](fb8a8c4)) * **webui:** handle new metadata fields ([5567adc](5567adc)), closes [gotson#276](https://github.com/tahngarth/komga/issues/276) * **webui:** more filter criteria ([4d22d9c](4d22d9c)), closes [gotson#283](https://github.com/tahngarth/komga/issues/283) [gotson#34](https://github.com/tahngarth/komga/issues/34) * **webui:** navigation drawer for sort/filter ([28598cb](28598cb)), closes [gotson#283](https://github.com/tahngarth/komga/issues/283) * add/rearrange metadata fields ([9e406e3](9e406e3)), closes [gotson#276](https://github.com/tahngarth/komga/issues/276) * **webui:** read lists ([27edf17](27edf17)), closes [gotson#106](https://github.com/tahngarth/komga/issues/106) * read lists ([f0c864f](f0c864f)), closes [gotson#106](https://github.com/tahngarth/komga/issues/106) * **webui:** library option to import local artwork ([cba0a2e](cba0a2e)) * **webui:** navigate to series books from book details screen ([48e92c7](48e92c7)), closes [gotson#272](https://github.com/tahngarth/komga/issues/272) * library option to import local artwork ([eefb802](eefb802)) * **webui:** scanner option per library ([fca0180](fca0180)) * change database from H2 to SQLite ([20b2b39](20b2b39)), closes [gotson#218](https://github.com/tahngarth/komga/issues/218) * custom thumbnails for series ([f5f423f](f5f423f)), closes [gotson#63](https://github.com/tahngarth/komga/issues/63) * get images dimension during media analysis ([f9d55ec](f9d55ec)), closes [gotson#123](https://github.com/tahngarth/komga/issues/123) * scanner option per library ([4da6ff9](4da6ff9)) * sidecar thumbnails for books ([d01b29f](d01b29f)) * **analysis:** handle read progress during book analysis ([1fc893e](1fc893e)) * **api:** add read_status parameter to series books ([fc5c502](fc5c502)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **api:** add read/unread books count in SeriesDto ([3ca50d7](3ca50d7)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **api:** claim status ([47dd2f6](47dd2f6)), closes [gotson#207](https://github.com/tahngarth/komga/issues/207) * **api:** collections are pageable ([449a27e](449a27e)), closes [gotson#216](https://github.com/tahngarth/komga/issues/216) * **api:** collections management ([c2f9403](c2f9403)), closes [gotson#30](https://github.com/tahngarth/komga/issues/30) * **api:** manage book read progress per user ([17c80cd](17c80cd)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **api:** mark all books in series as read or unread ([75b7216](75b7216)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **api:** metadata import settings per library ([6824212](6824212)), closes [gotson#199](https://github.com/tahngarth/komga/issues/199) * **api:** on deck books ([1b6a030](1b6a030)), closes [gotson#131](https://github.com/tahngarth/komga/issues/131) * **api:** read progress as search criteria for Series ([885c891](885c891)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **api:** restrict page streaming and file download with roles ([6291dab](6291dab)), closes [gotson#146](https://github.com/tahngarth/komga/issues/146) * **api:** search books by read status ([7f3c492](7f3c492)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **api:** search series by collection ids ([ca91af7](ca91af7)) * **opds:** browse by collection ([15f9c82](15f9c82)) * **webreader:** display reading direction on book opening ([ebf2aac](ebf2aac)) * **webreader:** mark progress while reading ([10895a3](10895a3)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **webui:** action menu on item cards ([37d790d](37d790d)) * **webui:** add On Deck section on dashboard ([37c935e](37c935e)), closes [gotson#131](https://github.com/tahngarth/komga/issues/131) * **webui:** add read status filter when browsing Series ([cc39ce8](cc39ce8)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **webui:** add simple next/previous book buttons to reader toolbar ([e428115](e428115)), closes [gotson#233](https://github.com/tahngarth/komga/issues/233) * **webui:** claim server from login screen ([d4810bd](d4810bd)), closes [gotson#207](https://github.com/tahngarth/komga/issues/207) * generate collections from ComicInfo SeriesGroup ([277cdcd](277cdcd)), closes [gotson#210](https://github.com/tahngarth/komga/issues/210) * **webui:** allow direct input of library path in add dialog ([6ece7b1](6ece7b1)), closes [gotson#88](https://github.com/tahngarth/komga/issues/88) * **webui:** better handling of library deletion ([0297210](0297210)) * **webui:** collection edition on card in browse collections ([e45389d](e45389d)) * **webui:** collection edition on card in search results ([68fe4fd](68fe4fd)) * **webui:** collections management ([2f8255a](2f8255a)), closes [gotson#30](https://github.com/tahngarth/komga/issues/30) * **webui:** display collections in search results and search box ([82aec45](82aec45)), closes [gotson#212](https://github.com/tahngarth/komga/issues/212) * **webui:** display unread count on series card ([4962f17](4962f17)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **webui:** enable edit on books and series cards in search screen ([32dad62](32dad62)) * **webui:** hide pagination when there is only 1 page ([675b4a1](675b4a1)) * **webui:** keep reading section in dashboard ([7984cef](7984cef)), closes [gotson#131](https://github.com/tahngarth/komga/issues/131) * **webui:** mark books as read or unread ([24c994f](24c994f)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **webui:** mark series as read/unread ([4d7e243](4d7e243)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **webui:** metadata import settings per library ([521cc42](521cc42)), closes [gotson#199](https://github.com/tahngarth/komga/issues/199) * **webui:** multi-select in dashboard page ([8e33be7](8e33be7)) * **webui:** multi-select in search results page ([c5417ac](c5417ac)), closes [gotson#213](https://github.com/tahngarth/komga/issues/213) * **webui:** pagination for collections ([50b516d](50b516d)), closes [gotson#216](https://github.com/tahngarth/komga/issues/216) * **webui:** read button on item card ([a59f263](a59f263)), closes [gotson#133](https://github.com/tahngarth/komga/issues/133) * **webui:** read progress filter for Series when browsing Libraries ([0c046a7](0c046a7)), closes [gotson#25](https://github.com/tahngarth/komga/issues/25) * **webui:** refresh browse collection screen on action menu actions ([8f2c453](8f2c453)) * **webui:** refresh dashboard on action menu actions ([b6bd735](b6bd735)) * **webui:** restrict page streaming and file download per user ([381b196](381b196)), closes [gotson#146](https://github.com/tahngarth/komga/issues/146) * **webui:** search results page ([89039a4](89039a4)), closes [gotson#29](https://github.com/tahngarth/komga/issues/29) * **webui:** sort/filter settings are persisted per library ([bf737de](bf737de)), closes [gotson#190](https://github.com/tahngarth/komga/issues/190) * automatic database backup ([bbb9f7c](bbb9f7c)), closes [gotson#138](https://github.com/tahngarth/komga/issues/138) * migrate DAO from Hibernate to jOOQ ([75e1079](75e1079)) * switch to JRE for docker image ([3206495](3206495)) ### Performance Improvements * add sql indexes ([41f3e8d](41f3e8d)) * lazy sql joins ([6eb7669](6eb7669))
Komga environment
jarDescribe the bug
After noticing that a library scan on a fresh installation of Komga had been stuck on counting the number of pages inside my CBZ files for several hours earlier today, I aborted the scan and set up a new installation of Komga for this bug report. I continued by creating a new library containing only a single 10MB/35 pages CBZ file, started a library scan and observed that it took Komga almost two minutes to complete the "AnalyzeBook" task. The thumbnail generation task, meanwhile, finished in just a couple of seconds.
Steps to reproduce
.zipfile, rename it to.cbzand create a new folder for it: Locke & Key - Welcome to Lovecraft v01.zipExpected behavior
Komga only takes a fraction of a second to finish counting the number of pages inside the CBZ file.
Actual behavior
Komga takes almost two minutes to finish counting the number of pages inside the CBZ file. While testing different CBZ files in my collection, I observed that this time increases linearly with the number of pages. For example, it took Komga almost 20 minutes to finish scanning a 70MB/221 pages CBZ file.
Additional context
I think this issue is at least tangentially related to #278. I've been using Komga with the same media collection on the same hardware since about December 2019 and never had any issues with these files. In older versions, it only took Komga a fraction of a second each to finish the "AnalyzeBook" task for the CBZ files in my collection.
Edit: It looks like this issue does not affect PDF files. Komga only takes 270ms to finish scanning a 50+ pages PDF file for me.
Log file
I've noticed that Komga prints the error message
ERROR 30813 --- [DefaultMessageListenerContainer-1] unknown.jul.logger: TODOexactly once for each corresponding page in the CBZ file, for a total of 35 times.Spoiler
The text was updated successfully, but these errors were encountered: