Cannot read JPEG2000 image: Java Advanced Imaging (JAI) Image I/O Tools are not installed

[sardaukar]

I have a few JPEG2000 PDFs I guess, and Komga just generates blank covers and pages (with no error on the UI, but the title of the issue shows up in the log file).

Example file

[gotson]

I looked up the thing with jpeg2000, and it's complicated :/

There was a project stopped in 2010, which is still available but has a dodgy license. There seems to be another project that could provide the feature, but i would need to try what that means in terms of added dependencies and portability.

Just to have an idea, how many of those jpeg2000 PDFs do you have?

[gotson]

I managed to add some dependencies that helps with JPEG2000 as well as the JBIG2 compression format (which is apparently used in your file). The code can read the files.

Now i'm faced with another problem, as the files are considered jpeg, they are sent to the browser as is, but the browser cannot read them and display a blank image instead.

I need to figure out how to detect that the images are jpeg2000 in the PDF and convert them to classic jpeg.

[gotson]

I had the browser cache enabled, which was loading the previous blank page. After disabling the browser cache it works fine.

[gotson]

@sardaukar assuming you are using docker, would you be able to try the latest gotson/komga:beta and confirm it it solves your issues?

[sardaukar]

Trying it now.

[sardaukar]

Well, it seems to have fewer errors in the logs, but the web UI still stops working after a while. The scanner still seems to be working, but after

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "http-nio-8080-ClientPoller"

shows up, the web UI is unresponsive and reloading it just gives me 504s.

[sardaukar]

A lot of

2020-01-04 15:03:59.631  INFO 1 --- [kTaskExecutor-2] org.hibernate.orm.cache                  : HHH90001005: Cache[cache.media.collection.pages] Key[org.gotson.komga.domain.model.Media._pages#27123]
A soft-locked cache entry was expired by the underlying cache. If this happens regularly you should consider increasing the cache timeouts and/or capacity limits
2020-01-04 15:03:59.634  INFO 1 --- [kTaskExecutor-2] o.h.c.s.support.AbstractReadWriteAccess  : Cached entry expired : org.gotson.komga.domain.model.Media._pages#27123

still.

[sardaukar]

I'll let the scanner finish and then restart the container to see if the web UI comes back.

[sardaukar]

I also see a few

java.util.NoSuchElementException: List is empty.
        at kotlin.collections.CollectionsKt___CollectionsKt.first(_Collections.kt:196) ~[kotlin-stdlib-1.3.61.jar:1.3.61-release-180 (1.3.61)]
        at org.gotson.komga.domain.service.BookAnalyzer.analyze(BookAnalyzer.kt:53) ~[app/:na]
        at org.gotson.komga.domain.service.BookLifecycle.analyzeAndPersist(BookLifecycle.kt:35) ~[app/:na]
        at org.gotson.komga.domain.service.BookLifecycle$$FastClassBySpringCGLIB$$6608c007.invoke(<generated>) ~[app/:na]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) ~[spring-tx-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99) ~[spring-tx-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115) ~[spring-aop-5.2.2.RELEASE.jar:5.2.2.RELEASE]
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
        at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]

and here's the unzip test output for the file mentioned before this in the log:

Archive:  IDW Ultimate Collection (2002-2017)/24 (001-005)(2014).zip                                                                          
    testing: 24 (001-005)(2014)/      OK
    testing: 24 (001-005)(2014)/24 002 (2014) (digital) (Minutemen-Slayer).cbr   OK                                                                             
    testing: 24 (001-005)(2014)/24 004 (2014) (digital) (Minutemen-Slayer).cbr   OK                                                                             
    testing: 24 (001-005)(2014)/24 001 (2014) (digital) (Minutemen-Slayer).cbr   OK                                                                             
    testing: 24 (001-005)(2014)/24 005 (2014) (digital) (Minutemen-Slayer).cbr   OK                                                                             
    testing: 24 (001-005)(2014)/24 003 (2014) (digital) (Minutemen-Slayer).cbr   OK                                                                             
No errors detected in compressed data of IDW Ultimate Collection (2002-2017)/24 (001-005)(2014).zip.                                          

Looks like this just contains other comics files, have to unzip it.

[gotson]

I hoped you could try a smaller library just with the jpeg2000 files to confirm it works 😅

[gotson]

🎉 This issue has been resolved in version 0.11.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀