[Kavita] Add ability to read a comic by panel

[Gunnm92]

Idea Description

A like read comic by panel it's more confortable a better example for that is comic trim : https://play.google.com/store/apps/details?id=com.comic.trim&hl=en_US (the application is no longer available on the google store)

On apk pure :
https://apkpure.net/comic-trim/com.comic.trim/download

Idea Category

User Experience

Duration of Using Kavita

No response

Before submitting

• I've already searched for existing ideas before posting.

[majora2007]

I looked at the linked app but not understanding the difference between our reader and what you're expecting.

[Gunnm92]

In this application, you can read a comic book panel by panel, which makes the experience much more enjoyable. There's no need to constantly zoom in, and it's more dynamic. It utilizes the panel's frame to identify it.

[DieselTech]

How do you tell what panel should come in which order? There is no metadata surrounding the panel.

[Gunnm92]

Maybe we can use it to give metadata of panel:
https://framagit.org/nicooo/kumiko
https://github.com/CodeMinion/C.A.P.E
https://github.com/pedrovgs/DeepPanel

[Gunnm92]

I don't know how he coded the panel recognition, but it works very well in the application. Typically, we read from left to right or from right to left.

[mushroommunk]

I came to suggest the same thing as it only recently occurred to me. The link is dead for me, but it seems like the request is what is normally known as Guided View and for those of us who read mostly on our phones this would be fantastic. It would replace the current constant pinching to zoom and wiggle the image to see each box; not having to add yet another layer on top of Kavita would be even better in my book (I have too many apps around already).

It seems to me the best way to implement this would be to expand the Anansi Comic Info Standard that Kavita already supports (I don't know if @majora2007 has seen this come up already there or not, should I maybe create a parallel discussion there?). Expanding this metadata is something that I think should be done in general as many readers are haphazardly implementing things of that nature, might as well try to standardize it right? Either that or implement the Advanced Comic Book Format which seems to already have this information alongside most the same things as ComicInfo, but that would obviously splinter things and seems to just duplicate work. It also doesn't make sense to me to create a second PanelInfo.xml or something just to hold this data.

Something like this would probably work (It's been a while since I've touched an xsd, probably needs tweaking):
<xs:complexType name="ComicPageInfo"> <xs:attribute name="Image" type="xs:int" use="required"/> <xs:attribute default="Story" name="Type" type="ComicPageType"/> <xs:attribute default="false" name="DoublePage" type="xs:boolean"/> <xs:attribute default="0" name="ImageSize" type="xs:long"/> <xs:attribute default="" name="Key" type="xs:string"/> <xs:attribute default="" name="Bookmark" type="xs:string"/> <xs:attribute default="-1" name="ImageWidth" type="xs:int"/> <xs:attribute default="-1" name="ImageHeight" type="xs:int"/> <xs:element minOccurs="0" maxOccurs="1" name="Panels" type="ArrayOfComicPagePanels"/> </xs:complexType> <xs:complexType name="ArrayOfComicPagePanels"> <xs:sequence> <xs:element minOccurs="0" maxOccurs="unbounded" name="Panel" nillable="true" type="ComicPagePanelInfo"/> </xs:sequence> </xs:complexType> <xs:simpleType name="ComicPagePanelInfo"> <xs:attribute name="X" type="xs:int"/> <xs:attribute name="Y" type="xs:int"/> <xs:attribute name="W" type="xs:int"/> <xs:attribute name="H" type="xs:int"/> </xs:simpleType>

Under ComicPagePanelInfo it might be prudent to support a series of points to define a more custom shape, but that does add complexity and adding later shouldn't break compatibility if done right.

Looking over manga-reader.component.ts it seems like it wouldn't be too painful (I know, famous last words) to then add a new display mode alongside Single and Double that scales and slides the image and adds black bars outside the specified box while looping through the panels the same way pages are looped through. It wouldn't matter then how the metadata got in the ComicInfo.xml file since any number of apps as Gunnm92 pointed out already can generate it. It looks like at least some of the metadata is already fetched and accessible in manga-reader component so don't think a server change would even be needed (though I haven't looked too close).

I'd even be happy to do the dev work if a consensus can be reached. Web apps in C# and Vue front end is my day job and I use Kavita a decent amount.

[majora2007]

Hi mushroommunk, I'd be open to a proof of concept for this guided view in a dedicated reader (like you see the webtoon reader is in a separate component). Showcasing the idea and how it might work would be interesting.

I took a look at acbf, but I've never seen it in practice. I think on the fly processing might be an appropriate first step for a poc then identify if storing in the DB or in a dedicated xml/json would be beneficial during an analyze task.

As for Anansi, I am a member already, but moving things through into the official standard has proven to be difficult without many members approvals, so I don't think that is an avenue. That is mainly around maintaining the ComicInfo.xml spec, which doesn't seem to be a good fit for something like this.

I do think something like this would likely lead to a good experience on mobile, but there are quite many edge cases to consider and test against to see if it can be included into Kavita.

[mushroommunk]

Sure, I'll try and get something together over the next few days.

[majora2007]

No rush but excited to see what can be put together. Reach out if you need any help.

[mizaki]

I'll add this https://maxhalford.github.io/blog/comic-book-panel-segmentation/ for the splitting. And an implementation: https://sourceforge.net/projects/comic-panel-detection/ (that I've not tried yet).

As for external file formats. The problem I see with ACBF frames is having to be in order. (Kumiko's JSON is also in order but there is a direction field.)

To my mind having the order be split into columns and rows would enable easy reversing. That way it doesn't matter for the processing (oops I didn't pick the right direction) and left-to-right or right-to-left can be decided by the reader.

There is also Digital Visual Narratives Profile which, if I'm reading it right, another field could be added to the media fragment i.e. image1.jpg#xywh=160,120,320,240&rowcol=1,3 or image1.jpg#xywh=160,120,320,240&row=1&col=3 etc.

[mizaki]

After some more looking around I found https://github.com/Seeneva/seeneva-reader-android which uses ML to find the speech bubble and expands them one after another. So, it occurred that bubble pan/zoom might be more desirable as panels can be full page width etc.

A certain level of panel detection is still required (text below other text without panel knowledge could be the next text bubble in sequence (same panel) or 10 texts (and x panels) away) but I would imagine the "difficult" panels wouldn't be such a problem using text bubbles for guidance. The user being able to set a percentage zoom say for text bubbles.

I've found a few papers and githubs with examples but https://github.com/dmMaze/comic-text-detector which is used by https://github.com/kha-white/mokuro and https://github.com/dmMaze/BallonsTranslator seems to be used at least.

Making up a new format and trying to get similar people in form Anansi seems like a good idea (if the format moves fast enough)? Otherwise ACBF or Divina etc. Like information format, it makes sense (to me) that people can use whatever they want to generate the guiding boxes (panels or text).

I may try and wrap comic-text-detector to output an ACBF/Divina file.