Skip to content
This repository has been archived by the owner on Apr 16, 2020. It is now read-only.

BookReader #14

Open
2 tasks
davidar opened this issue Sep 13, 2015 · 15 comments
Open
2 tasks

BookReader #14

davidar opened this issue Sep 13, 2015 · 15 comments

Comments

@davidar
Copy link
Collaborator

davidar commented Sep 13, 2015

https://archive.org/details/BookReader

IPFS Demo

  • add more books!
  • add to ipfs/examples?

CC: @jbenet

@jbenet
Copy link
Contributor

jbenet commented Sep 13, 2015

@davidar this is so, so, so cool! I just read a book on IPFS! \o/

Yeah please do.

@davidar knows this, but note to everyone else: please be careful to make sure books we rehost are ok to be rehosted by us (i.e. look at the license/copyright)

We should talk to archive folks about using IPFS to replicate all the archive's books sooner than the rest of the stuff.

cc @brewsterkahle in case he checks github.

@brewsterkahle
Copy link

so great!

yes, we can help with selecting materials.

-brewster

On 9/13/15 4:03 PM, Juan Benet wrote:

@davidar https://github.com/davidar this is so, so, so cool! I just
read a book on IPFS! \o/

Yeah please do.

@davidar https://github.com/davidar knows this, but note to everyone
else: please be careful to make sure books we rehost are ok to be
rehosted by us (i.e. look at the license/copyright)

We should talk to archive folks about using IPFS to replicate all the
archive's books sooner than the rest of the stuff.

cc @brewsterkahle https://github.com/brewsterkahle in case he checks
github.


Reply to this email directly or view it on GitHub
#14 (comment).

@davidar
Copy link
Collaborator Author

davidar commented Sep 14, 2015

Wow, @brewsterkahle commenting on one of my issues :D

The problem I ran into was browsers still not supporting JPEG2000 for some inexplicable reason, so I guess the best strategy would be to batch convert everything to regular JPEG first? I also tried some javascript JPEG2000 decoders, but they seemed way too slow (especially on mobile).

+1 for getting all the scanned books into IPFS, that would be amazing.

@brewsterkahle
Copy link

Yes, the archive does server-side reformatting of pages (turns out jpg
is not very good for scanned book pages).

This is one of the things we have to solve in a distributed web-- having
sophisticated website specific client side javascript to be things like
mini-db's and search engines.

Marcel van der Peijl [email protected] did a search engine for my
blog:
http://gateway.ipfs.io/ipfs/QmavE42xtK1VovJFVTVkCR5Jdf761QWtxmvak9Zx718TVr/

in the case of the book reader, I think we would have to download the
jp2's and convert a few of them ahead of time. the book reader
currently does something similar by predownloading a set of pages.
without that lookahead, the performance is klunky.

-brewster

On 9/14/15 1:42 AM, David A Roberts wrote:

Wow, @brewsterkahle https://github.com/brewsterkahle commenting on one
of my issues :D

The problem I ran into was browsers still not supporting JPEG2000 for
some inexplicable reason, so I guess the best strategy would be to batch
convert everything to regular JPEG first? I also tried some javascript
JPEG2000 decoders, but they seemed way too slow (especially on mobile).

+1 for getting all the scanned books into IPFS, that would be amazing.


Reply to this email directly or view it on GitHub
#14 (comment).

@rht
Copy link

rht commented Sep 15, 2015

There is https://archive.org/details/gutenberg, which can be readily mirrored with https://www.gutenberg.org/wiki/Gutenberg:Mirroring_How-To (s/rsync/ipfs/).

@davidar
Copy link
Collaborator Author

davidar commented Sep 15, 2015

@rht I already mirrored a small subset of Gutenberg a little while ago (check the irc logs), so mirroring the rest is definitely on my radar. Part of the reason I've been holding back on that so far is that I think it would be a really cool demonstration of the search engine #8 when it materialises.

@davidar
Copy link
Collaborator Author

davidar commented Sep 15, 2015

@brewsterkahle I guess a combination of:

  1. low quality jpeg versions (thumbnails)
  2. aggressive preloading and rendering of high quality jpeg2k

where the user is shown 1 while waiting for 2 to load, might work quite well in practice.

@jbenet I'm still concerned about mobile performance, but there may be other things that can be done to help with that. For example, perhaps we could release an "IPFS browser" that catches ipfs URLs (or appropriate URIs ipfs/kubo#1678 ) and provides support for some of the computationally expensive things traditionally done server side (like native image processing libraries). JavaScript is OK for this on a powerful computer, but not so much on mobile.

@davidar
Copy link
Collaborator Author

davidar commented Sep 15, 2015

This is one of the things we have to solve in a distributed web-- having
sophisticated website specific client side javascript to be things like
mini-db's and search engines.

I'm also quite interested in this, but on a larger scale, see #8 and ipld/go-ipld-deprecated#8

@jbenet
Copy link
Contributor

jbenet commented Sep 15, 2015

For example, perhaps we could release an "IPFS browser" that catches ipfs URLs (or appropriate URIs ipfs/kubo#1678 ) and provides support for some of the computationally expensive things traditionally done server side (like native image processing libraries)

@davidar i think we can do everything in javascript. https://github.com/ipfs/node-ipfs is not that far from landing. (the one stopgap is chrome/ff extensions with go-ipfs bundled to resolve ipfs links)

JavaScript is OK for this on a powerful computer, but not so much on mobile.

Yeah, mobile is an interesting question. not sure, i need to talk to people at Mozilla + Chrome about this-- stunted mobile browsers are such a pain :(

@davidar
Copy link
Collaborator Author

davidar commented Sep 18, 2015

Yeah, mobile is an interesting question. not sure, i need to talk to people at Mozilla + Chrome about this-- stunted mobile browsers are such a pain :(

Thanks, that would be great.

@davidar davidar self-assigned this Sep 19, 2015
@davidar
Copy link
Collaborator Author

davidar commented Sep 23, 2015

BPG could be another option: the JS decoder for BPG seems noticeably faster than the JP2 ones, and it also seems to have slightly better image quality for similar files sizes (demo).

@RichardLitt
Copy link
Contributor

If there's anything I can do to help, I am 💯% in.

@davidar
Copy link
Collaborator Author

davidar commented Sep 25, 2015

@RichardLitt I'm trying to experiment with BPG (here) but I keep getting Cannot enlarge memory arrays errors. I've tried recompiling bpgdec.js with emscripten's -s ALLOW_MEMORY_GROWTH=1 flag, but that's also erroring (I suspect my LLVM package might be too old). If you were able to get this working, I'd really appreciate it :)

Alternatively, if you wanted to get the bookreader working with a JS JPEG2000 decoder, that would also be great.

This was referenced Sep 29, 2015
@davidar
Copy link
Collaborator Author

davidar commented Sep 30, 2015

I've created a new issue (ipfs/apps#2) to discuss getting BookReader working on IPFS.

I'll leave this issue open to discuss actually getting scanned books mirrored to IPFS (in a format compatible with the BookReader).

@ghost ghost mentioned this issue Sep 30, 2015
88 tasks
@mekarpeles
Copy link

re: #14 (comment)
You can use the Archive's IIIF API to fetch Archive.org book pages on the fly in any format, e.g.:

https://iiif.archivelab.org/iiif/adventuresoftoms00twaiiala$1/full/full/0/default.jpg
https://iiif.archivelab.org/iiif/adventuresoftoms00twaiiala$1/full/full/0/default.png etc.

The syntax is iiif/:id$page#

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants