Feature-test moz-chunked-arraybuffer before use #5531

Rob--W · 2014-12-08T23:28:01Z

Resubmission of #5457, rebased to master. The previous one was closed with the following comment:

Closing for now. #5448 has landed, so the issue itself should be fixed with less code than in this PR. Optimizions for the landed fix can be proposed in a new (rebased) PR.

This PR indeed has a bit more code, but that is caused by the fact that this PR uses feature detection instead of try-catch.
The other diff is just some re-indention after moving the Mozilla-specific APIs between preprocessor conditionals.

Rob--W · 2015-06-05T21:42:12Z

@yurydelendik Could you merge this? Since Chrome 44.0.2360.0, the console prints the following warning:

The provided value 'moz-chunked-arraybuffer' is not a valid enum value of type XMLHttpRequestResponseType.

Snuffleupagus · 2015-06-05T22:23:07Z

src/core/network.js

@@ -75,6 +75,21 @@ var NetworkManager = (function NetworkManagerClosure() {
    return array.buffer;
  }

+//#if (GENERIC || FIREFOX || B2G || MOZCENTRAL)


I'm just wondering: Since we know that moz-chunked-arraybuffer is supported for MOZCENTRAL (and FIREFOX too, for all versions that we realistically care about), couldn't we avoid doing any additional checks for those cases? E.g. something like this:

//#if (FIREFOX || MOZCENTRAL) // var supportsMozChunked = true; //#else var supportsMozChunked = (function supportsMozChunkedClosure() { var x = new XMLHttpRequest(); . . . //#endif

for all versions that we realistically care about

If that's the case, then yes, we can. Do others feel the same?

Some additional information:
It seems that moz-chunked-arraybuffer was added in https://bugzilla.mozilla.org/show_bug.cgi?id=687087, which landed in Firefox 9 (released on 2011-12-20). I would be very surprised if the addon even works in such an old version of Firefox anyway, so it should be fine.

I have added another commit to the PR that completely eliminates this extra variable in Firefox. Could you take a look?

Snuffleupagus · 2015-06-06T09:56:16Z

src/core/network.js

+//#if (FIREFOX || MOZCENTRAL)
+//var supportsMozChunked = true;
+//#endif
+//#if GENERIC


Nit: Since there is a SINGLE_FILE build target as well, would it perhaps make more sense to just use //#else instead of

//#endif //#if GENERIC

I wanted to exclude the feature detection logic from the CHROMIUM builds. And AFAIK there is no #elseif in our preprocessor, hence I've used #endif + #if GENERIC.

That's a good idea. Let's use that logic for FIREFOX as well, e.g. use supportMozChunked only when !(FIREFOX || MOZCENTRAL || CHROMIUM)

#elseif shall be expressed in terms of #if CASE1, #if CASE2, #if !(CASE1 || CASE2)

Snuffleupagus · 2015-06-06T12:03:37Z

I'd still like @yurydelendik to sign off on the patch, but let's check that the tests pass.

/botio test

pdfjsbot · 2015-06-06T12:03:38Z

From: Bot.io (Linux)

Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.21.233.14:8877/795d9c3547006e2/output.txt

pdfjsbot · 2015-06-06T12:03:38Z

From: Bot.io (Windows)

Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.22.172.223:8877/bd277a69974edbe/output.txt

pdfjsbot · 2015-06-06T12:21:22Z

From: Bot.io (Windows)

Success

Full output at http://107.22.172.223:8877/bd277a69974edbe/output.txt

Total script time: 17.74 mins

Font tests: Passed
Unit tests: Passed
Regression tests: Passed

pdfjsbot · 2015-06-06T12:21:56Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/795d9c3547006e2/output.txt

Total script time: 18.30 mins

Font tests: Passed
Unit tests: Passed
Regression tests: Passed

Rob--W · 2015-06-28T22:47:05Z

@yurydelendik Gentle ping to get this PR merged. I want to get rid of console spam (#5531 (comment)) (the log spam already appears in the current version of Chrome, 43.0.2357.81).

yurydelendik · 2015-06-30T15:14:40Z

src/core/network.js

      } else {
        xhr.responseType = 'arraybuffer';
      }
+//#else


make it #if CHROMIUM and move #else branch up

Why?
Do you want something like this:

//#if CHROMIUM xhr.responseType = 'arraybuffer'; //#endif //#if (GENERIC || FIREFOX || MOZCENTRAL) ... //#endif

or

//#if CHROMIUM xhr.responseType = 'arraybuffer'; //#else ... //#endif

We don't want to miss any other future targets (in the past it was B2G), so I'm looking into something like:

//#if CHROMIUM // xhr.responseType = 'arraybuffer'; //#endif //#if (FIREFOX || MOZCENTRAL) // .... //#endif //#if !(CHROMIUM || FIREFOX || MOZCENTRAL) ... //#endif

Since supportsMozChunked is always set for non-Chromium builds, we can do either of the following (the FIREFOX|MOZCENTRAL build does not really need the boolean, but in my previous PR a reviewer suggested to not abuse the preprocessor to eliminate the boolean, so I've introduced the boolean). WDYT?

//#if CHROMIUM // xhr.responseType = 'arraybuffer'; //#else if (args.onProgressiveData && supportsMozChunked) { xhr.responseType = 'moz-chunked-arraybuffer'; pendingRequest.onProgressiveData = args.onProgressiveData; pendingRequest.mozChunked = true; } else { xhr.responseType = 'arraybuffer'; } //#endif

or

xhr.responseType = 'arraybuffer'; //#if !CHROMIUM if (args.onProgressiveData && supportsMozChunked) { xhr.responseType = 'moz-chunked-arraybuffer'; pendingRequest.onProgressiveData = args.onProgressiveData; pendingRequest.mozChunked = true; } //#endif

or something like #5531 (diff)

but in my previous PR a reviewer suggested to not abuse the preprocessor to eliminate the boolean

Missed that bit -- I'm sorry for confusion. (Now I have to convince @Snuffleupagus to agree with me :) But since we are doing special case for CHROMIUM, I would like to remove a compatibility code for MOZCENTRAL as well, and also to be consistent with stuff we did for fonts in the past.

I'm thinking creating XMLHttpRequest object for feature test during scripts loading is not cheap process, so my goal to not include this code into production FF code base.

(Side note about //#if CASE1 if (check) { //#endif -- I don't think we shall use incomplete if-/while-/try-statements in the preprecessor)

The supportsMozChunked is always true for MOZCENTRAL and FIREFOX, but it can also be true or false for non-FF/CHROMIUM. Because of this, and because we don't want preprocessor magic to eliminate the boolean, the supportsMozChunked check always happens in non-Chromium builds.
The appearance of the preprocessor special case for Chromium can be removed by adding

//#if CHROMIUM //var supportsMozChunked = false; //#endif

but that doesn't gain us much, does it?

Do you have a preference for any of the presented alternatives (this comment and my previous comment)?

and because we don't want preprocessor magic to eliminate the boolean,

Au contraire, I want to eliminate the boolean for FF/CHROMIUM, means that in non-FF/CHROMIUM code (which will be uncommented by default) the supportsMozChunked feature test and check will be present. For FF/CHROMIUM cases, we use preprocessor to provide optimized versions of the same code (which will be commented) with assumption of supportsMozChunked=true/false. (Don't worry about code duplication)

Does it make more sense?

In the future, I hope we can re-write this code and define better classes/interfaces than XHR that will be more suitable for our operations.

That certainly makes sense, and I've also understood it in that way.

By "preprocessor magic", I meant #5531 (diff). Assuming that setting .responseType multiple times is inexpensive (which probably is), the complete code (with minimal use of the boolean) could be:

xhr.responseType = 'arraybuffer'; //#if (MOZCENTRAL || FIREFOX) // if (args.onProgressiveData) { //#endif //#if !(MOZCENTRAL || FIREFOX || CHROMIUM) if (args.onProgressiveData && supportsMozChunked) { //#endif //#if !CHROMIUM xhr.responseType = 'moz-chunked-arraybuffer'; pendingRequest.onProgressiveData = args.onProgressiveData; pendingRequest.mozChunked = true; } //#endif

But in a previous comment, you've expressed that you don't want this if-in-ifdef construct.

(The alternative (without boolean AND without if-in-ifdef) is duplication of code (copy & paste of the full if block, and include supportsMozChunked in non-FF/CR builds, and omit supportsMozChunked in FF builds). This is even uglier and undesirable.)

In the future, I hope we can re-write this code and define better classes/interfaces than XHR that will be more suitable for our operations.

Definitely. The current code is too tightly coupled with XHR, and trying to replace it with the fetch API will be a challenge (in Chrome 43+, fetch supports readable streams, which enables PDF.js to display PDFs even if the server does not support range requests - #5319 & #6126).

duplication of code might be fine in this case. okay, may I suggest a compromise that will remove the test for MOZCENTRAL/CHROMIUM, however 'moz-chunked-arraybuffer' branch will still be present (it will be mostly likely removed by JIT)?

var useMozChunkedLoading = false; //#if (FIREFOX || MOZCENTRAL) // useMozChunkedLoading = !!args.onProgressiveData; //#endif //#if !(CHROMIUM || FIREFOX || MOZCENTRAL) useMozChunkedLoading = supportsMozChunked && !!args.onProgressiveData; //#endif if (useMozChunkedLoading) {

I've submitted a variant of this, please take a look. To emphasize that the options are mutually exclusive, I've moved var useMozChunkedLoading = false; inside a #ifdef CHROMIUM and added var in front of the other options.

yurydelendik · 2015-06-30T18:44:49Z

Looks good. Good to land after testing.

/botio test

pdfjsbot · 2015-06-30T18:44:51Z

From: Bot.io (Windows)

Received

Command cmd_test from @yurydelendik received. Current queue size: 1

Live output at: http://107.22.172.223:8877/500677f2e55c729/output.txt

pdfjsbot · 2015-06-30T18:44:51Z

From: Bot.io (Linux)

Received

Command cmd_test from @yurydelendik received. Current queue size: 1

Live output at: http://107.21.233.14:8877/5c8ca61cc6ee9ad/output.txt

yurydelendik · 2015-06-30T18:44:56Z

/botio-linux preview

pdfjsbot · 2015-06-30T18:44:56Z

From: Bot.io (Linux)

Received

Command cmd_preview from @yurydelendik received. Current queue size: 2

Live output at: http://107.21.233.14:8877/7ee6b6f9ef78a3b/output.txt

pdfjsbot · 2015-06-30T19:14:12Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/5c8ca61cc6ee9ad/output.txt

Total script time: 18.07 mins

Font tests: Passed
Unit tests: Passed
Regression tests: Passed

pdfjsbot · 2015-06-30T19:14:29Z

From: Bot.io (Windows)

Success

Full output at http://107.22.172.223:8877/500677f2e55c729/output.txt

Total script time: 18.24 mins

Font tests: Passed
Unit tests: Passed
Regression tests: Passed

pdfjsbot · 2015-06-30T19:14:35Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/7ee6b6f9ef78a3b/output.txt

Total script time: 0.39 mins

Published

yurydelendik · 2015-06-30T19:48:09Z

node mode generic fails with:

### Creating generic viewer
Could not evalute line '!(CHROMIUM || FIREFOX || MOZCENTRAL)' at /mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/build/pdf.worker.js:1674

/mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/external/builder/builder.js:72
            throw e;
                  ^
ReferenceError: CHROMIUM is not defined
    at evalmachine.<anonymous>:1:3
    at preprocess (/mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/external/builder/builder.js:68:24)
    at /mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/external/builder/builder.js:221:7
    at Array.forEach (native)
    at /mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/external/builder/builder.js:215:13
    at Array.forEach (native)
    at Object.build (/mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/external/builder/builder.js:210:20)
    at Function.target.generic (/mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/make.js:152:11)
    at Object.global.target.(anonymous function) [as generic] (/mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/node_modules/shelljs/make.js:28:26)
    at Function.target.web (/mnt/botio-files-pdfjs/private/7ee6b6f9ef78a3b/make.js:224:10)
``

timvandermeij · 2015-06-30T19:50:44Z

CHROMIUM should be CHROME everywhere in this PR according to the targets at https://github.com/mozilla/pdf.js/blob/master/make.js#L74

yurydelendik · 2015-06-30T19:58:22Z

/botio-windows test
/botio-linux preview

pdfjsbot · 2015-06-30T19:58:22Z

From: Bot.io (Windows)

Received

Command cmd_test from @yurydelendik received. Current queue size: 0

Live output at: http://107.22.172.223:8877/75ab13080d8abb8/output.txt

pdfjsbot · 2015-06-30T19:58:22Z

From: Bot.io (Linux)

Received

Command cmd_preview from @yurydelendik received. Current queue size: 0

Live output at: http://107.21.233.14:8877/91fc9230a6f1995/output.txt

pdfjsbot · 2015-06-30T19:59:06Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/91fc9230a6f1995/output.txt

Total script time: 0.72 mins

Published

pdfjsbot · 2015-06-30T20:16:39Z

From: Bot.io (Windows)

Success

Full output at http://107.22.172.223:8877/75ab13080d8abb8/output.txt

Total script time: 18.27 mins

Font tests: Passed
Unit tests: Passed
Regression tests: Passed

Snuffleupagus · 2015-07-01T10:17:17Z

src/core/network.js

+    var x = new XMLHttpRequest();
+    // Firefox requires that .open() is called before setting responseType.
+    // https://bugzilla.mozilla.org/show_bug.cgi?id=707484
+    x.open('GET', 'http://example.com');


Is it possible to move this line into the try block instead.

The reason that I'm asking is that when testing the preview (i.e. the GENERIC build), Firefox threw an error on this line and the viewer thus failed to load the PDF file. I tracked this down to the fact that I'm using the NoScript addon, which blocked this request (since I hadn't white-listed that particular domain).

Considering that NoScript is a fairly common addon, it's conceivable that more people could run into this. In this case, it thus seems better to me that streaming gets disabled, than the entire viewer breaking.

Done.

diff --git a/src/core/network.js b/src/core/network.js index 0c714f8..e3a9b63 100644 --- a/src/core/network.js +++ b/src/core/network.js @@ -78,9 +78,16 @@ var NetworkManager = (function NetworkManagerClosure() { //#if !(CHROME || FIREFOX || MOZCENTRAL) var supportsMozChunked = (function supportsMozChunkedClosure() { var x = new XMLHttpRequest(); - // Firefox requires that .open() is called before setting responseType. - // https://bugzilla.mozilla.org/show_bug.cgi?id=707484 - x.open('GET', 'http://example.com'); + try { + // Firefox 37- required .open() to be called before setting responseType. + // https://bugzilla.mozilla.org/show_bug.cgi?id=707484 + x.open('GET', 'https://example.com'); + } catch (e) { + // Even though the URL is not visited, .open() could fail if the URL is + // blocked, e.g. via the connect-src CSP directive or the NoScript addon. + // When this error occurs, this feature detection method will mistakenly + // report that moz-chunked-arraybuffer is not supported in Firefox 37-. + } try { x.responseType = 'moz-chunked-arraybuffer'; return x.responseType === 'moz-chunked-arraybuffer';

Snuffleupagus · 2015-07-02T10:20:58Z

/botio test

pdfjsbot · 2015-07-02T10:21:00Z

From: Bot.io (Linux)

Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.21.233.14:8877/559be50aa5ebc0d/output.txt

pdfjsbot · 2015-07-02T10:21:00Z

From: Bot.io (Windows)

Received

Command cmd_test from @Snuffleupagus received. Current queue size: 0

Live output at: http://107.22.172.223:8877/ed880e31d361ac3/output.txt

pdfjsbot · 2015-07-02T10:39:31Z

From: Bot.io (Linux)

Success

Full output at http://107.21.233.14:8877/559be50aa5ebc0d/output.txt

Total script time: 18.52 mins

Font tests: Passed
Unit tests: Passed
Regression tests: Passed

pdfjsbot · 2015-07-02T10:39:31Z

From: Bot.io (Windows)

Success

Full output at http://107.22.172.223:8877/ed880e31d361ac3/output.txt

Total script time: 18.52 mins

Font tests: Passed
Unit tests: Passed
Regression tests: Passed

Feature-test moz-chunked-arraybuffer before use

yurydelendik · 2015-07-02T14:08:16Z

Thank you for the patch

timvandermeij added the viewer label Dec 8, 2014

Snuffleupagus added core and removed viewer labels Dec 8, 2014

yurydelendik self-assigned this May 18, 2015

Snuffleupagus reviewed Jun 5, 2015
View reviewed changes

Rob--W force-pushed the moz-chunked-only-moz branch from 07dbb74 to d9410f3 Compare June 6, 2015 09:47

Snuffleupagus reviewed Jun 6, 2015
View reviewed changes

yurydelendik reviewed Jun 30, 2015
View reviewed changes

Rob--W force-pushed the moz-chunked-only-moz branch from d9410f3 to 8a87bff Compare June 30, 2015 18:37

Rob--W force-pushed the moz-chunked-only-moz branch from 8a87bff to 5578d6a Compare June 30, 2015 19:56

Snuffleupagus reviewed Jul 1, 2015
View reviewed changes

Feature-test moz-chunked-arraybuffer before use

2e63dcd

Rob--W force-pushed the moz-chunked-only-moz branch from 5578d6a to 2e63dcd Compare July 1, 2015 13:32

yurydelendik added a commit that referenced this pull request Jul 2, 2015

Merge pull request #5531 from Rob--W/moz-chunked-only-moz

9ad6af4

Feature-test moz-chunked-arraybuffer before use

yurydelendik merged commit 9ad6af4 into mozilla:master Jul 2, 2015

timvandermeij mentioned this pull request Jul 4, 2015

scrambled garbage #6174

Closed

CodingFabian mentioned this pull request Aug 4, 2015

"The provided value 'moz-chunked-arraybuffer' is not a valid enum value... " #6306

Closed

yurydelendik mentioned this pull request Aug 12, 2015

Fixes supportsMozChunked for node.js #6349

Merged

yurydelendik mentioned this pull request Aug 19, 2016

Warning on load #7553

Closed

Feature-test moz-chunked-arraybuffer before use #5531

Feature-test moz-chunked-arraybuffer before use #5531

Conversation

Rob--W commented Dec 8, 2014

Rob--W commented Jun 5, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Snuffleupagus commented Jun 6, 2015

pdfjsbot commented Jun 6, 2015

From: Bot.io (Linux)

Received

pdfjsbot commented Jun 6, 2015

From: Bot.io (Windows)

Received

pdfjsbot commented Jun 6, 2015

From: Bot.io (Windows)

Success

pdfjsbot commented Jun 6, 2015

From: Bot.io (Linux)

Success

Rob--W commented Jun 28, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yurydelendik commented Jun 30, 2015

pdfjsbot commented Jun 30, 2015

From: Bot.io (Windows)

Received

pdfjsbot commented Jun 30, 2015

From: Bot.io (Linux)

Received

yurydelendik commented Jun 30, 2015

pdfjsbot commented Jun 30, 2015

From: Bot.io (Linux)

Received

pdfjsbot commented Jun 30, 2015

From: Bot.io (Linux)

Success

pdfjsbot commented Jun 30, 2015

From: Bot.io (Windows)

Success

pdfjsbot commented Jun 30, 2015

From: Bot.io (Linux)

Success

Published

yurydelendik commented Jun 30, 2015

timvandermeij commented Jun 30, 2015

yurydelendik commented Jun 30, 2015

pdfjsbot commented Jun 30, 2015

From: Bot.io (Windows)

Received

pdfjsbot commented Jun 30, 2015

From: Bot.io (Linux)

Received

pdfjsbot commented Jun 30, 2015

From: Bot.io (Linux)

Success

Published

pdfjsbot commented Jun 30, 2015

From: Bot.io (Windows)

Success

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Snuffleupagus commented Jul 2, 2015

pdfjsbot commented Jul 2, 2015

From: Bot.io (Linux)

Received

pdfjsbot commented Jul 2, 2015