BookReader

[davidar]

https://archive.org/details/BookReader

IPFS Demo

• add more books!
• add to ipfs/examples?

CC: @jbenet

[jbenet]

@davidar this is so, so, so cool! I just read a book on IPFS! \o/

Yeah please do.

@davidar knows this, but note to everyone else: please be careful to make sure books we rehost are ok to be rehosted by us (i.e. look at the license/copyright)

We should talk to archive folks about using IPFS to replicate all the archive's books sooner than the rest of the stuff.

cc @brewsterkahle in case he checks github.

[brewsterkahle]

so great!

yes, we can help with selecting materials.

-brewster

On 9/13/15 4:03 PM, Juan Benet wrote:

@davidar https://github.com/davidar this is so, so, so cool! I just
read a book on IPFS! \o/

Yeah please do.

@davidar https://github.com/davidar knows this, but note to everyone
else: please be careful to make sure books we rehost are ok to be
rehosted by us (i.e. look at the license/copyright)

We should talk to archive folks about using IPFS to replicate all the
archive's books sooner than the rest of the stuff.

cc @brewsterkahle https://github.com/brewsterkahle in case he checks
github.

—
Reply to this email directly or view it on GitHub
#14 (comment).

[davidar]

Wow, @brewsterkahle commenting on one of my issues :D

The problem I ran into was browsers still not supporting JPEG2000 for some inexplicable reason, so I guess the best strategy would be to batch convert everything to regular JPEG first? I also tried some javascript JPEG2000 decoders, but they seemed way too slow (especially on mobile).

+1 for getting all the scanned books into IPFS, that would be amazing.

[brewsterkahle]

Yes, the archive does server-side reformatting of pages (turns out jpg
is not very good for scanned book pages).

This is one of the things we have to solve in a distributed web-- having
sophisticated website specific client side javascript to be things like
mini-db's and search engines.

Marcel van der Peijl [email protected] did a search engine for my
blog:
http://gateway.ipfs.io/ipfs/QmavE42xtK1VovJFVTVkCR5Jdf761QWtxmvak9Zx718TVr/

in the case of the book reader, I think we would have to download the
jp2's and convert a few of them ahead of time. the book reader
currently does something similar by predownloading a set of pages.
without that lookahead, the performance is klunky.

-brewster

On 9/14/15 1:42 AM, David A Roberts wrote:

Wow, @brewsterkahle https://github.com/brewsterkahle commenting on one
of my issues :D

The problem I ran into was browsers still not supporting JPEG2000 for
some inexplicable reason, so I guess the best strategy would be to batch
convert everything to regular JPEG first? I also tried some javascript
JPEG2000 decoders, but they seemed way too slow (especially on mobile).

+1 for getting all the scanned books into IPFS, that would be amazing.

—
Reply to this email directly or view it on GitHub
#14 (comment).

[rht]

There is https://archive.org/details/gutenberg, which can be readily mirrored with https://www.gutenberg.org/wiki/Gutenberg:Mirroring_How-To (s/rsync/ipfs/).

[davidar]

@rht I already mirrored a small subset of Gutenberg a little while ago (check the irc logs), so mirroring the rest is definitely on my radar. Part of the reason I've been holding back on that so far is that I think it would be a really cool demonstration of the search engine #8 when it materialises.

[davidar]

@brewsterkahle I guess a combination of:

1. low quality jpeg versions (thumbnails)
2. aggressive preloading and rendering of high quality jpeg2k

where the user is shown 1 while waiting for 2 to load, might work quite well in practice.

@jbenet I'm still concerned about mobile performance, but there may be other things that can be done to help with that. For example, perhaps we could release an "IPFS browser" that catches ipfs URLs (or appropriate URIs ipfs/kubo#1678 ) and provides support for some of the computationally expensive things traditionally done server side (like native image processing libraries). JavaScript is OK for this on a powerful computer, but not so much on mobile.

[davidar]

This is one of the things we have to solve in a distributed web-- having
sophisticated website specific client side javascript to be things like
mini-db's and search engines.

I'm also quite interested in this, but on a larger scale, see #8 and ipld/go-ipld-deprecated#8

[jbenet]

For example, perhaps we could release an "IPFS browser" that catches ipfs URLs (or appropriate URIs ipfs/kubo#1678 ) and provides support for some of the computationally expensive things traditionally done server side (like native image processing libraries)

@davidar i think we can do everything in javascript. https://github.com/ipfs/node-ipfs is not that far from landing. (the one stopgap is chrome/ff extensions with go-ipfs bundled to resolve ipfs links)

JavaScript is OK for this on a powerful computer, but not so much on mobile.

Yeah, mobile is an interesting question. not sure, i need to talk to people at Mozilla + Chrome about this-- stunted mobile browsers are such a pain :(

[davidar]

Yeah, mobile is an interesting question. not sure, i need to talk to people at Mozilla + Chrome about this-- stunted mobile browsers are such a pain :(

Thanks, that would be great.

[davidar]

BPG could be another option: the JS decoder for BPG seems noticeably faster than the JP2 ones, and it also seems to have slightly better image quality for similar files sizes (demo).

[RichardLitt]

If there's anything I can do to help, I am 💯% in.

[davidar]

@RichardLitt I'm trying to experiment with BPG (here) but I keep getting Cannot enlarge memory arrays errors. I've tried recompiling bpgdec.js with emscripten's -s ALLOW_MEMORY_GROWTH=1 flag, but that's also erroring (I suspect my LLVM package might be too old). If you were able to get this working, I'd really appreciate it :)

Alternatively, if you wanted to get the bookreader working with a JS JPEG2000 decoder, that would also be great.

[davidar]

I've created a new issue (ipfs/apps#2) to discuss getting BookReader working on IPFS.

I'll leave this issue open to discuss actually getting scanned books mirrored to IPFS (in a format compatible with the BookReader).

[mekarpeles]

re: #14 (comment)
You can use the Archive's IIIF API to fetch Archive.org book pages on the fly in any format, e.g.:

https://iiif.archivelab.org/iiif/adventuresoftoms00twaiiala$1/full/full/0/default.jpg
https://iiif.archivelab.org/iiif/adventuresoftoms00twaiiala$1/full/full/0/default.png etc.

The syntax is iiif/:id$page#