[Bridge] Array Buffers / Typed Arrays for WebSockets, Ajax

[phoboslab]

It's currently not possible to send/received ArrayBuffers with React Native.

Am I right to assume that this is largely due to the missing TypedArray implementation in JSC? I.e. we're not able to transfer raw memory in a sane and performant way between JavaScript and native.

I've opened an issue over at webkit.org (https://bugs.webkit.org/show_bug.cgi?id=120112), asking for a TypedArray API a few years ago and have written a working implementation for this API (https://github.com/phoboslab/JavaScriptCore-iOS/blob/master/JavaScriptCore/JSTypedArray.h) in my fork of JSC. Forking JSC has worked okay-ish for my own library, but now I'm stumbling over this issue again with React Native.

It would be really useful to be able to load and handle binary data for a number of use cases. So I'm wondering if you guys would be interested to weigh in on a TypedArray API or if you have any ideas how this would be workaroundable without the huge performance penalty that we currently face.

[ide]

There are two roadblocks here. One is the API limitation you described and one solution would be to work with the WebKit team to patch the API for the next-next release of JSC.

The second issue is that the RN bridge uses JSON to communicate between JS and native. There are no shared references so you currently can't access a TypedArray from Obj-C that JS can also access. This could change at the cost of losing the Chrome debugger but if shared memory enables another class of apps that do photo processing, etc, that might be the right trade off.

[vkurchatkin]

We can have a separate ArrayBuffer implementation that is just a handle for raw memory. They can be produced and consumed by some APIs and behave just as usual array buffers, except no direct access to bytes via typed arrays (maybe DataView is also possible).

[sebmarkbage]

Even without shared memory, the RN bridge could benefit from binary serialization and transferring ownership of memory. However, given the direction of the language, it might as well just be a shared reference.

[rclai]

This is also an issue for any kind of Bluetooth related functionality. Perhaps we should document current work-arounds for this?

[phoboslab]

FYI: I recently put a lot of time into a workaround for reading/writing Typed Arrays in native code and it seems to perform reasonably well. About 7ms/MB for reading, 20ms/MB for writing. The overhead of the read/write function call itself (regardless of array size) is ~0.2ms. The whole journey is documented here:
http://phoboslab.org/log/2015/11/the-absolute-worst-way-to-read-typed-array-data-with-javascriptcore

Final implementation: EJConvertTypedArray.h, EJConvertTypedArray.m

[vkurchatkin]

it looks like this API is finally being added to JSC

[rclai]

Wow really? You have a link for this?

[vkurchatkin]

@rclai https://bugs.webkit.org/show_bug.cgi?id=120112

[fungilation]

Looks like this is resolved and merged? #4483

[rclai]

Not quite, it is just a fix for WebSocket. There's more being requested in this ticket.

[springuper]

I'm writing a react-native-thrift lib, but find out that binary protocol can't be sent over xhr, Typed Arrays will make it work as MDN says, can you continue to work for this feature?

[springuper]

maybe a relative pr: #6327

[edgesite]

@springuper I tried to modify RCTNetworking and XMLHttpRequest.js to add basic binary support, it might help. See edgesite@a40a0a1

UPDATE:
I've refined base64 decode procedure, now responseText support utf8 string and arraybuffer/blob handle by native Uint8Array.
See edgesite@a4486d6.

[cvermilion]

Looks like Typed Array support will indeed be in iOS 10: https://developer.apple.com/library/prerelease/content/releasenotes/General/iOS10APIDiffs/Objective-C/JavaScriptCore.html

[saulshanabrook]

@cvermilion Will this let it work on Android as well?

[cvermilion]

@saulshanabrook I'm not sure, but I think the Android version uses JavaScriptCore directly, so I would think so. But looking at the ReactAndroid build files it looks like it's using a pretty old commit of JSC (r174650, from 2014), so having TypedArray access might require upgrading JSC on that side?

[saulshanabrook]

I think it does, I was checking it out. Will verify that.

[sulliwane]

fetching .gz file in react-native android 0.30 fails with response.arrayBuffer():

fetch('http://127.0.0.1:8125/bonjour.txt.gz', {
  mode: 'no-cors'
})
.then(function(response) {
  console.log('response %o: ', response);
  return response.arrayBuffer();
})
.then(function(buffer) {
  console.log('buffer %o: ', buffer);
})

Here is the result from react-native running on my android device (chrome debugger):

You can see _blobBlob and _blobInit in the response, as well as arrayBuffer size of 54. It should be 42 as you can see from the correct result from firefox, running on the same android device:

firefox android and react-native android run the same javascript code. Why Firefox can properly use arrayBuffer on a fetch() response object, but not react-native?

note: I'm trying to download a .gz file as an arrayBuffer to then decompress it using pako.

note2: A workaround to my problem is to use react-native-fetch-blob. I would prefer use a simple fetch however :)

[robclouth]

So TypedArray support is now in iOS 10. For earlier versions we could either use the code @phoboslab wrote, or show an unsupported error. On Android we'd need to rebuild android-jsc with a newer version of WebKit, and add it to maven. That would be TypedArray support on the JSC side done.
How would we start adapting the bridge?

[lacker]

We are trying to keep issues focused on bugs and this is more of a feature request, albeit a very cool and valuable feature request. I am going to close this issue but if people are interested in working to improve this then that would be pretty great!

[rclai]

Is there at least a Product Pain for this?

[lacker]

I don't see one, you could create a new one! Maybe we should flush out Product Pains every once in a while to keep it fresh....

[idibidiart]

@sebmarkbage how would a shared reference get implemented? seems too low level for JS.

@ALL can we take care of the bridge--is-the-bottleneck issue? seems that the JS<->Native interop is the "weakest link" in the RN performance story.

[gre]

We are discussing and looking for a way to implement a UInt8Array pixels => Promise<encoded image uri> at rnc-archive/react-native-webgl#2 (comment) and unfortunately hitting on this missing support. However, in Expo EXGLView (and react-native-webgl fork) there are support for typedarray, via using C++ and accessing JSC environment with a fallback when not supported (iOS < 10, android). (see https://github.com/expo/expo/blob/master/cpp/UEXGL.cpp#L339-L403 & https://github.com/expo/expo/blob/master/cpp/EXJSConvertTypedArray.c )

I wonder if it could be generalized in RN? and if so, what would it mean when in ObjC iOS/Java Android (and not just in C++ support) – would it also mean RN to provide an abstract new type on top of this?