Use a higher quality image resizing algorithm

[keyeh]

Most browsers use linear interpolation when resizing images. This leads to extreme aliasing and Moire patterns which is a deal breaker for anyone resizing images for art/photo galleries, albums, etc. These kinds of artifacts are impossible to remove after the fact.

Edit: For anyone looking for a solution this is much better than limby-resize.

A solution I've found is using limby-resize, which resizes images using a custom algorithm to a level of quality similar to imagemagick or photoshop. Here is a demo. Fine Uploader uses the default browser image resizing algorithm AKA the "Crappy" method.

My solution was to replace drawImage in the else statement at line#168 in megapix-image.js.

I replaced:

else {
      ctx.drawImage(img, 0, 0, width, height);
}

canvas.qqImageRendered && canvas.qqImageRendered();

With:

else {
    var tmpCanvas = document.createElement("canvas");
    tmpCanvas.width = iw;
    tmpCanvas.height = ih;
    var tmpCtx = tmpCanvas.getContext("2d");
    tmpCtx.drawImage(img, 0, 0);
    canvasResize(tmpCanvas, canvas, function () {
        alert("Image resized by limby-resize");
        canvas.qqImageRendered && canvas.qqImageRendered();
    });
}

And included the limby-resize js file above fineuploader js in my HTML.

It works well and I think this, or another proper image resizing algorithm should be included in Fine-Uploader. it should be an option the user can set in the resizing options as it is significantly more CPU intensive.

[rnicholus]

Duplicate of #1524. I'll close that one if favor of this, since your description is more detailed.

[keyeh]

I was the person on Stack Overflow, wasn't sure if you saw it so I posted here.

[rnicholus]

Cool. Thanks much for your answer on SO and for opening this issue. This approach looks great and seems like it will solve a long-standing issue with image quality of the thumbnail feature and the image scaling feature.

[keyeh]

If this is implemented it's also a good idea to show a "Resizing..." status text to the user as creating many different sizes of very large images can take 30+ seconds. The status text should be visible on both the scaled and the original to account for the hideScaled and sendOriginal options.

Perhaps adding onResize and onResizeComplete callbacks would be more appropriate

[rnicholus]

if this type of message appears to be needed once I've investigated further, I'll look into an appropriate solution. Thanks for bringing this up

[rnicholus]

Usually, a message is not appropriate as Fine Uploader users don't always use the built in UI

[classofoutcasts]

Pica seems to be a considerably faster implementation than limby for high quality down scaling.
Any progress on hq scaling?

http://nodeca.github.io/pica/demo/

[rnicholus]

You can follow this case for progress updates and commits.

[keyeh]

@classofoutcasts Yeah, Pica is way faster. Do you have some good unsharp settings? I'm using this:

else {
            var tmpCanvas = document.createElement("canvas");
            tmpCanvas.width = iw;
            tmpCanvas.height = ih;
            var tmpCtx = tmpCanvas.getContext("2d");
            tmpCtx.drawImage(img, 0, 0);
            window.pica.resizeCanvas(tmpCanvas, canvas, {
                quality: 3,
                alpha: false,
                unsharpAmount: 30,
                unsharpRadius: 0.6,
                unsharpThreshold: 0,
                transferable: true
                },
                function () {
                    tmpCanvas = tmpCtx = null;
                    canvas.qqImageRendered && canvas.qqImageRendered();
            });
        }

[rnicholus]

I'm not sure about integrating this library into the core Fine Uploader code, but perhaps Fine Uploader could be modified to allow this or any other scaling library to be integrated, replacing the internal module. This sort of extreme flexibility is an area where Fine Uploader is lacking a bit. While Fine Uploader has been continually evolved to solve a great number of problems and workflows, extensibility was not a concern. I have been writing up plan for a new uploader library that would follow a vastly different architecture, focusing on extensibility (among other things) and make this sort of integration much easier (ideally). Let me know what you think. Note that this is still very early on in the design phase.

[keyeh]

Modules are definitely a better solution. In addition to the scaling I have a lot of very ugly hacks to customize Fine uploader for my needs (particularly S3 uploads and scaling).

However if modern-uploader is still far away it may be reasonable to put a note in the Fine uploader documentation about scaling quality & alternative scaling libraries.

[rnicholus]

I would say that Modern Uploader is not around the corner just yet (since I'm still writing up the design docs). I'll make it a point to look into either documenting or making adjustments to Fine Uploader to allow the image generation code to be swapped out with something else.

If you can comment more on the "hacks" you've employed to fit the scaling and S3 uploader to your needs, that would be helpful, particularly for future development, such as Modern Uploader.

[keyeh]

In addition to the scaling library:

• Send image dimensions in header to S3 metadata/create endpoint
• Events/Callbacks for scaling images for UI
• Send Base64 encoded scaled image
• Edge case where scaled images sometimes hit create endpoint after the parent if the parent filesize is very small. (not really an issue, accounting for it on server side)

Wish list

• Scale images when added to upload queue, not when the upload starts.
• Different jpg quality per scaled image (smaller images handle more compression before degrading)

My platform is for content creators so they all have powerful computers. We get to save some costs by scaling on their machines.

[rnicholus]

Thank you for sharing this information.

Scale images when added to upload queue, not when the upload starts.

The current behavior is intentional. It would be very easy to crash the browser (due to lack of memory) or freeze the page (due to the scaling effort) if all files are scaled as soon as they are added to the queue. The current approach waits until just before the file is to be uploaded, at which point the scaled Blobs are created, and then thrown away after the upload completes. This means that, by default, only three files have scaled image Blobs consuming memory at any time. Changing this would have potentially devastating consequences for most users. It's not clear to me what the issue is with the current behavior. Can you explain?

Send Base64 encoded scaled image

What use do you have for this?

[keyeh]

Scaling all the images at the same was just a thought to compensate for limby-resize being slow. Sometimes a file would finish uploading before the next is done scaling. When you're scaling and uploading hundreds of 20 megapixel images the delay adds up. But it's not an issue with unmodified fine uploader.

We embed base64 images in HTML to use as a placeholder while the actual, larger image loads.

• Embed a small 20px image in HTML
• Stretch the image to the actual image's dimensions
• Apply Gaussian blur
• Wait for the actual image to finish loading (standard or 2x retina)
• Fade out blurred image to actual image

It's similar to Facebook's cover page loads. It gives a nice effect while images load. (If you haven't guessed already I'm working on an extremely photo-heavy platform)

[rnicholus]

We embed base64 images in HTML to use as a placeholder while the actual, larger image loads.

Are you talking about the scaling feature or the thumbnail preview generation feature? First I thought you were talking about the latter, but now it seems like you are in fact talking about thumbnails. If you are, the thumbnails used to be generated all at once after they were submitted, and in a sense they still are, but there is code in place to prevent too many from being generated at once due to issues with garbage collection that effected image generation in Chrome. See #1279 for details.

I'd like to see your Fine Uploader integration code, out of curiosity. Can you send me a link?

[keyeh]

Sorry if I was unclear, the base64 images are not used for fine uploader; I'm not using the thumbnails feature at all. They are for public pages like photographer's portfolios, photo albums, etc.

As for the code for that I'm basically (trying) to call scaleImage() in onComplete(), converting the returned blob to b64 and POSTing it to another endpoint.

I'll get back to you with the full code in a bit

[classofoutcasts]

Appreciate your efforts, no other file uploader has this yet, afaik!
Hopefully it will become part of the future html5 canvas features...

[rnicholus]

I'm starting to look into this now as one of the primary features of v5.10.0. Any help or offers to review code or test solutions will be much appreciated.

[rnicholus]

To allow for freedom in choosing a resizing algorithm/library, I expect to create a new option or callback that accepts a function value. This function, if provided, will be called by Fine Uploader, passing an HTMLCanvasElement containing the image, the expected new width and height, and the associated file ID. This is expected to be a promissory function. The promise returned by this function should be fulfilled w/ the resized HTMLCanvasElement once the resize operation is complete (or rejected on error).

If a resize callback function is not provided, Fine Uploader will continue to use the internal scaling code. I'm not sure, at the moment, how well limby-resize and pica account for image subsampling in iOS. If they do not properly account for this, that may be an issue for Fine Uploader users, and I may want to also provide a way to easily switch to the internal scaling code (which does handle iOS image subsampling) on-the-fly.

[rnicholus]

Follow progress in #1576.

[rnicholus]

Looking for testers before I release. Please comment in #1576 if interested.

[rnicholus]

@keyeh The primary focus of this issue, and what will be part of 5.10.0, is the ability to easily substitute Fine Uploader's internal image scaling algorithm (for thumbnails and scaled uploads) with any other library of your choice. I now you mentioned some other items on your wish list. Perhaps the most important ones can be opened up as separate feature requests. I noticed a number of +1s under that comment, so if anyone else sees benefit in specific items in your wishlist, please feel free to comment and/or open up a case.

[davidtgq]

Would it be possible to use this mechanism (of delegating image processing to an external library) to not scale the photo, but to (for example) optimize it? A usage of this may be using a jpeg encoding library to optimize the rescaled and thumbnailed images outputted by pica or limby. In other words, run multiple external processing libraries in series, where the first library's output is the input for the second library, and so on.

(Admittedly this is something that can be done manually with the getFile and addFiles method, which may be a more suitable solution for such rare usage cases)

[rnicholus]

@davidtgq Currently, the solution that is a candidate for 5.10.0 (please see the code in #1576) passes a <canvas> with the original image drawn onto it, and a <canvas> to receive the resized image. As long as the resulting image can be placed onto this target <canvas>, you can do whatever you want as far as image processing. While I would have preferred a more pure functional approach (pass in a <canvas> and expect either a <canvas> or Blob as a return value) that would have required more widespread changes to the internal code of Fine Uploader, and the two JS scaling libraries I am aware of offer this same exact API (pass in a source and target <canvas>).

[rnicholus]

This feature is set to be released tomorrow as Fine Uploader 5.10.0.