Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace CanvasPixelArray with Uint8ClampedArray; optimizations; bugfixes #604

Merged
merged 7 commits into from
Aug 29, 2015
Merged

Conversation

zbjornson
Copy link
Collaborator

This updates quite a bit of stuff:

Tested primarily on node 2.4.0 (iojs) but uses v8 APIs available to node 0.12 and later. Node 3.0 (when v8 upgrades to 4.4) will need a quick API update but I'm poised to make those changes. I can't get node-gyp to work with iojs 3.0 nightlies right now, else I'd put in compiler directives.

@zbjornson
Copy link
Collaborator Author

CI failed for node 0.8 and 0.10 as expected. Package.json specifies the engine must be 0.12 or later.

@LinusU
Copy link
Collaborator

LinusU commented Aug 1, 2015

This is really cool! Good job 👍

  • What is warning 4512?
  • Which is the api call only available on >0.12?

Tests is passing on my local machine, code looks good. I would like it if we could support Node.js 0.10 but maybe that just isn't viable.

Could you update the .travis.yml file to remove Nodejs 0.8 and 0.10? That way the tests will pass.

@LinusU
Copy link
Collaborator

LinusU commented Aug 1, 2015

Realized from the failed tests that it's ArrayBuffer and UInt8ClampedArray that isn't available on Node.js 0.10.x so I guess we can't support that...

@zbjornson
Copy link
Collaborator Author

Thanks :)

  • Warning 4512 is "assignment operator could not be generated" -- google briefly but didn't see a quick fix. Messages were not introduced by my changes, and if I recall only appear on more recent versions of node.
  • Ya, the TypedArray stuff (which is what changes again in v8 4.4, ugh).
  • Updated .travis.yml

@LinusU
Copy link
Collaborator

LinusU commented Aug 1, 2015

Could you amend the last commit to also remove line 6 of the travis config, that was specific only to node.js version 0.8.

@zbjornson
Copy link
Collaborator Author

Done

@zbjornson
Copy link
Collaborator Author

As far as node 0.10 support, could make this version 2 and update the readme to be clear that v2 is for node 0.12+ only.

@LinusU LinusU self-assigned this Aug 1, 2015
@LinusU
Copy link
Collaborator

LinusU commented Aug 1, 2015

Perfect 👍

Looks good to me, I'm happy to merge but I'll give the other maintainers some time to give their comments.

I also think that it might be a good idea to release this as version 2.0.0. Need to think a bit about this.

@kangax
Copy link
Collaborator

kangax commented Aug 2, 2015

Yep, backwards-incompatible changes need new major version since we follow semver

@LinusU
Copy link
Collaborator

LinusU commented Aug 2, 2015

@zbjornson I think I detected a small "bug" with your performance fix thought. When a is 0 all the other values should be zero as well since it uses pre-multiplied values. It might not affect anything since it will probably skip drawing that anyhow.

Maybe it's better to check if a is 0 and then just set all the other values to 0, we don't even need to read them from the source.

@zbjornson
Copy link
Collaborator Author

@LinusU I was actually a bit confused about that and should have brought it up. I saw someone else added the equivalent-ish to that to getImageData (https://github.com/Automattic/node-canvas/pull/549/files), but I didn't understand why the same behavior should happen for a == 0 as for a == 255. Because going to/from premultiplied RGB values is lossy, it seems like there's no way to recover the value at all if a == 0, so you just have to return the premultiplied value in that case I guess (which is 0)?

(The biggest gain actually comes from how the source is accessed and destination is set -- working entirely with uint8s instead of bitshifting in/out of a uint32.)

This look okay to you?

// Context2d::PutImageData
if (a == 0) {
  *dstRow++ = 0;
  // ...
} else if (a == 255) {
  *dstRow++ = b;
  // ...
} else {
  *dstRow++ = b * alpha;
  // ...
}

@LinusU
Copy link
Collaborator

LinusU commented Aug 2, 2015

That looks very good, I also think that the intent is clearer. When I look at that I immediately understand it.

Also, in GetImageData, couldn't we do something like this to get rid of the bitshifting there?

-      uint32_t *pixel = row + x + sx;
-      uint8_t a = *pixel >> 24;
-      uint8_t r = *pixel >> 16;
-      uint8_t g = *pixel >> 8;
-      uint8_t b = *pixel;
+      uint8_t *pixel = row + x + sx;

And then access as pixel[0] instead of a, pixel[1] instead of r, etc... My C++ skills are bit rusty so there might be a reason why we can't do this.

@zbjornson
Copy link
Collaborator Author

Yeah, we can do the same thing in GetImageData -- I tried it and it was only slightly faster (not worth the regression risk), but I'll benchmark it again now that I'm more comfortable with what these functions do and have the test suit working. Commit coming in a few hours. Thanks!

@zbjornson
Copy link
Collaborator Author

OK, bug fixed in putImageData.

Benchmarked two variations of GetImageData, both of which were slower: uint8_t *pixel = row + x + sx; uint8_t a = pixel[0]; and uint8_t *pixel = row + s + sx; uint8_t a = *pixel++;. Left as-is.

(Aside, I tried using the benchmarking suite in the repo, which gave some really surprising timings. For example, fillRect took 3.5 seconds for one iteration and consumed a ton of memory. That looks like essentially a pass-through function to cairo, so wonder if it's a bug in cairo [on Windows].)

@LinusU
Copy link
Collaborator

LinusU commented Aug 6, 2015

This would be very good to merge before any more work goes into moving towards nan 2.0.

@zbjornson Is there anything more you would like to work on or do you feel comfortable with merging this?

@zbjornson
Copy link
Collaborator Author

@LinusU I'm good with merging. Thanks!

Hopefully sometime soon I can work on some other bugs :)

@kkoopa
Copy link
Contributor

kkoopa commented Aug 14, 2015

This still uses the external array data API, which has been entirely removed.

@zbjornson
Copy link
Collaborator Author

Right -- it's removed from the version of v8 used by node 3.0. (That's the API change I was referring to in my original PR message.) As soon as node-gyp works with iojs 3, I will add compiler directives that change which API is used. The API we need to switch to is not present in node 0.12-2.x (the non-externalizing API).

@kkoopa
Copy link
Contributor

kkoopa commented Aug 14, 2015

node-gyp works with io.js 3, so does pangyp

@kkoopa
Copy link
Contributor

kkoopa commented Aug 14, 2015

Would it not be possible to retain support for 0.10 (and / or 0.8) via https://github.com/joyent/node/blob/v0.10.40-release/src/v8_typed_array.cc ?

@zbjornson
Copy link
Collaborator Author

Possibly -- let me look at that. Will also try to get node-gyp working with iojs 3 again.

@kkoopa kkoopa mentioned this pull request Aug 18, 2015
@inssein inssein mentioned this pull request Aug 22, 2015
@LinusU
Copy link
Collaborator

LinusU commented Aug 22, 2015

@zbjornson Did you get a chance to look at the part that kkoopa mentioned?

@zbjornson
Copy link
Collaborator Author

Sorry -- swamped last week. Working on it today.

@LinusU
Copy link
Collaborator

LinusU commented Aug 22, 2015

No problem, tell me if you need help with anything!

@zbjornson
Copy link
Collaborator Author

Erm, do I need to fork (or cherry-pick) from #580 to get a version of nan that works with iojs 3?

@rvagg

@LinusU
Copy link
Collaborator

LinusU commented Aug 22, 2015

I don't think that that branch is done yet. My plan was to get this merged and then I would take a stab at completing the Nan switch.

Do you need anything from Nan 2.x?

@zbjornson
Copy link
Collaborator Author

It looks like Nan 2.0 is required for iojs 3.0, so I can only work on supporting node 0.8/0.10 without it.

I'm working on 0.8/0.10 now. Perhaps after I finish that, we merge this branch with support only through 2.x, update/rebase #580 and then I can make the IndexedProperties changes for 3.0 there? Let me know what you want to do.

@LinusU
Copy link
Collaborator

LinusU commented Aug 22, 2015

That sounds perfect 👍

@zbjornson
Copy link
Collaborator Author

Done. No regressions in the browser/visual tests. Set travis to iojs 2.4.0, but 2.5.0 works as well.

clampedArray->SetIndexedPropertiesToExternalArrayData(dst, kExternalUint8ClampedArray, size);
#else
// TODO
#endif
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing nothing if NODE_MAJOR_VERSION < 3 I would rather that we gave a compiler error. Basically, I would prefer it if line 754, 756, 757 and 758 where gone.

Makes sense?

@LinusU
Copy link
Collaborator

LinusU commented Aug 23, 2015

@zbjornson Could you remove your changes to .travis.yml and rebase on master? I've added in specific iojs versions there.

Added some argument testing/manipulation to match WebKit/Moz behaviors.

Additionally benchmarked:
* branching on `if (a == 0 || a == 255)`, as it is
* not branching (always doing the alpha calculation)
* Mozilla's implementation found here: https://dxr.mozilla.org/mozilla-central/source/dom/canvas/CanvasRenderingContext2D.cpp#5083
Mozilla's is insignificantly faster (p=0.17) :) so left it as-is.
Benchmark:
var canvas = new Canvas(300, 600);
var ctx = canvas.getContext("2d");
// any manipulation of canvas/ctx here.
var data = ctx.getImageData(0,0,300,600);
// time 1000x:
ctx.putImageData(data, 0, 0);
Negative arguments were causing source image data to be painted wrapped-around the other side of the canvas.
@zbjornson
Copy link
Collaborator Author

All done!

@rvagg
Copy link
Contributor

rvagg commented Aug 24, 2015

@zbjornson if you want to jump to NAN v2, you can either battle with merging in my PR or you can do it yourself. Use this script https://gist.github.com/thlorenz/7e9d8ad15566c99fd116 as a first pass but it'll need some love afterwards. Mostly, the Nan::New<String>().ToLocalChecked() replacements will get messed up because it doesn't count parens, so you'll have .ToLocalChecked() dangling in all the wrong places. Also, try and replace all references to Handle to Local while you're at, the former is being deprecated in the next V8 so now's the time to move away from it.

@zbjornson
Copy link
Collaborator Author

@LinusU are you waiting for anything from me? Probably best to do the nan 2 upgrade in a separate PR.

@LinusU
Copy link
Collaborator

LinusU commented Aug 29, 2015

Sorry for the delay, this is awesome! Merging now :)

I'll release this as a patch bump since it's backwards compatible, and since it doesn't introduce any new functionality, sounds good?

LinusU added a commit that referenced this pull request Aug 29, 2015
Replace CanvasPixelArray with Uint8ClampedArray; optimizations; bugfixes
@LinusU LinusU merged commit 6633300 into Automattic:master Aug 29, 2015
@zbjornson
Copy link
Collaborator Author

Sounds perfect. Will start working on iojs 3/nan 2 this week. Thanks!

@LinusU
Copy link
Collaborator

LinusU commented Sep 14, 2015

So I somehow managed to miss the actual cutting of a release. I'll get this out asap.

@LinusU
Copy link
Collaborator

LinusU commented Sep 14, 2015

Okay, here is what happened. I tagged the release, pushed it to npm, updated History.md and then pushed it to my own fork.

I have now pushed it to the main repo 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants