[web] ES6 modules #10913

josephrocca · 2022-03-17T13:29:14Z

Is your feature request related to a problem? Please describe.
I'm trying to get ONNX Runtime Web working in Deno. Deno maintains very close compatibility with the browser, but there's no concept of HTML/DOM since it's a server-side language, so the usual script tag import method is not possible.

System information

ONNX Runtime version (you are using): 1.10.0

Describe the solution you'd like
It would be great if the package had an ES6 modules build available so that it could be imported like this:

import ort from "https://cdn.jsdelivr.net/npm/[email protected]/dist/ort.es6.js"

josephrocca · 2022-03-20T06:58:33Z

I've just realised that you can simply write this: (Edit: wrong - read next comment)

import "https://cdn.jsdelivr.net/npm/[email protected]/dist/ort.js"

And it will create the window.ort global. Not ideal since it pollutes the global namespace, but at least this means it's possible to import it into Deno.

I'll leave this issue open since it still would be nice to have proper ES6 module support.

josephrocca · 2022-03-20T08:01:12Z

Oh, but now I see that a new problem arises. If we do a bare module import like this:

import "https://cdn.jsdelivr.net/npm/[email protected]/dist/ort.js";

then ort.js isn't able to find the wasm file because document.currentScript.src doesn't exist in modules (and ort.js is treated like a module in this case). So, to fix this, all references to document.currentScript.src should try to fall back to import.meta.url.

Here's an example of some code in ort.js trying to detect where it's running, but I think this code is actually generated by webpack? (I haven't used webpack before)

var _scriptDir = typeof document !== 'undefined' && document.currentScript ? document.currentScript.src : undefined;

You can see other examples by searching [email protected]/dist/ort.js for "currentScript" using Ctrl+F.

I was only able to find one reference to document.currentScript.src within the ONNX Runtime Web codebase itself.

@hanbitmyths, @fs-eire Wondering if you could provide guidance on if/how I can submit a pull request to fall back to import.meta.url when ort.js is trying to determine its own location? Specifically in regard to how this _scriptDir stuff is generated by the bundler. This would allow the library to work in Deno and with a "bare" ES6 import statement in the browser.

Note that currently it falls back to __filename which has a default hard-coded value of /index.js, which causes ort.js to try to load the wasm file from /ort-wasm-simd-threaded.wasm. So, if it's possible (?) to change the default __filename to a runtime-generated value, then a quick solution might be to set it to import.meta.url by default.

fs-eire · 2022-03-28T22:30:19Z

I will take a look at this issue.

nestarz · 2022-12-13T00:02:26Z

Any workarounds to make onnx runtime works on Deno ?

josephrocca · 2022-12-13T15:20:42Z

@fs-eire Wondering if you could just add this to the top of the published ort.js file?

if(typeof document === "undefined") {
  var document = {currentScript:{src:import.meta.url}};
}

I.e. just add a "dummy" document to fix the issue in Deno and other JS runtimes that don't have document/DOM stuff. It's not a "proper" fix, but is simple enough that it might make sense until the webpack stuff mentioned in my previous comment can be sorted out?

fs-eire · 2022-12-14T21:13:46Z

I found that Emscripten actually supports this feature. I am trying to make a change to support it for ORTWEB.

nestarz · 2023-02-07T17:41:20Z

I published a repo with a working version of onnx-runtime-web on Deno (note: it discard multithreading !).

https://github.com/nestarz/onnx_runtime

I built the wasm files with:

set_property(TARGET onnxruntime_webassembly APPEND_STRING PROPERTY LINK_FLAGS " -s EXPORT_ES6=1 -s MODULARIZE=1 -s USE_ES6_IMPORT_META=1")

fs-eire · 2023-02-08T01:24:57Z

@nestarz thank you so much for this example! it's a good learning for me.

Currently, we have 3 different scenarios for consuming the onnxruntime-web library:

consume from script tag ( <script src="https://some-cdn-server/ort.min.js">):
consume from module import ( import * as ort from 'onnxruntime-web' )
consume from module URL (import * as ort from "https://deno.land/x/onnx_runtime/mod.ts";)

There are a few hacks we currently doing in our build process :

the Emscripten generated file ort-wasm.js and ort-wasm-simd.js are identical in content. To reduce the final file size (ort.min.js), we force it to always use ort-wasm.js. This also apply to ort-wasm-threaded.js vs ort-wasm-simd-threaded.js.

the standard way:

import ortWasmFactory from './binding/ort-wasm.js';
import ortWasmSimdFactory from './binding/ort-wasm-simd.js';

...
if (simd) {
    ortWasmSimdFactory()...
} else {
    ortWasmFactory()...
}

onnxruntime-web currently doing:

import ortWasmFactory from './binding/ort-wasm.js';

...
const config = {
   locateFile: (fileName: string, scriptDirectory: string) => {
      ...
      if (fileName == 'ort-wasm.wasm' && simd) {
         return 'ort-wasm-simd.wasm';
      }
   }
};
ortWasmFactory(config)...

This indeed helped to reduce the final javascript size, but it caused the problem that webpack failed to analyze the file dependency of ort-wasm-simd.wasm, so it will not copy the web assembly file and modify the file path as it does by default.

inline worker for proxy mode. The motivation of doing this is for simple distribution (scenario.1). By using the worker-loader webpack plugin, we are able to put whole content of the worker source code into an inline string so that we can distribute single file (ort.min.js).

standard way:
```
proxyWorker = new Worker('./proxy-worker/main');
```
onnxruntime-web currently doing:
```
proxyWorker = require('worker-loader?inline=no-fallback!./proxy-worker/main').default() as Worker;
```
the require(worker-loader?...) can only be understood by webpack with configured to use worker-loader plugin. Webpack recommended to use its builtin worker loader however it does not support inline mode.

override processing of /ort-wasm.*\.worker\.js$/ to use asset/source. This also helps to inline the .worker.js file, but instead of using in proxy mode, this file is generated by Emscripten in multi-thread build. The file is processed as "asset" instead of source code, to work with special code so that we can still distribute a single file.

standard way:
```ts
// no extra code
```

onnxruntime-web currently doing:
```ts
const config = {
   locateFile: (fileName: string, scriptDirectory: string) => {
      if (useThreads && fileName.endsWith('.worker.js') && typeof Blob !== 'undefined') {
          return URL.createObjectURL(new Blob(
              [ require('./binding/ort-wasm-threaded.worker.js') ],
              {type: 'text/javascript'}));
      }
    }
  }
  ...
```

use function.toString() on the exported variable from file ort-wasm-threaded.js to reduce file size. This hack specifically rely on the UMD output type for the generated files and webpack config to preserve symbol name of _scriptDir and ortWasmThreaded to work.

standard way:
```ts
// no extra code
```

onnxruntime-web currently doing:
```ts
const config = {
    ...
};

if (useThreads) {
  if (typeof Blob === 'undefined') {
    config.mainScriptUrlOrBlob = path.join(__dirname, 'ort-wasm-threaded.js');
  } else {
    const scriptSourceCode = `var ortWasmThreaded=(function(){var _scriptDir;return ${factory.toString()}})();`;
    config.mainScriptUrlOrBlob = new Blob([scriptSourceCode], {type: 'text/javascript'});
  }
}
```

combining of the hacks, it makes ORT web very difficult to support ES6 module for now. We need to re-think the balance between the benefit of (single-file distribution and file size) VS. (easy-to-maintain code and webpack friendly)

fs-eire · 2023-05-02T06:47:58Z

#15772: This change turns onnxruntime-common into a ESModule/commonJS "dual" package, which means it is able to comsumed as either a ESM or CJS module.

When this is merged, following-up work will start for onnxruntime-web.

josephrocca · 2023-06-16T11:18:10Z

Just realised that OR Web currently works with Deno by simply setting ort.env.wasm.wasmPaths to avoid the issue I mentioned in my earlier comment. For example, this works:

import "https://cdn.jsdelivr.net/npm/[email protected]/dist/ort.js";
ort.env.wasm.wasmPaths = "https://cdn.jsdelivr.net/npm/[email protected]/dist/";
let onnxImageSession = await ort.InferenceSession.create("https://huggingface.co/rocca/openai-clip-js/resolve/main/clip-image-vit-32-float32.onnx", { executionProviders: ["wasm"] });

fs-eire · 2023-10-11T23:27:41Z

added ESM exports for onnxruntime-web in latest main branch. ( will later paste nightly build version )

However ort.env.wasm.wasmPaths may still need to manually set.

Please try it out. If it's still not working, let me know the repro steps and I will fix

josephrocca · 2023-10-12T15:09:29Z

Awesome! Awaiting nightly script CDN URL 🫡

fs-eire · 2023-10-16T16:08:22Z

Please try 1.17.0-dev.20231014-ae211999dd

josephrocca · 2023-10-16T17:47:07Z

Hmm, for some reason on JSDelivr I get a 403 HTTP error with this message: "Package size exceeded the configured limit of 150 MB.":

let ort = await import("https://cdn.jsdelivr.net/npm/[email protected]/dist/esm/ort.js");

I tried npm install [email protected] and then loaded from node_modules like this:

let ort = await import("./node_modules/onnxruntime-web/dist/esm/ort.js");

Then it loads but then throws:

error: Uncaught (in promise) Error: no available backend found. ERR: [wasm] ReferenceError: __dirname is not defined
    at resolveBackend (file:///pathtofolder/node_modules/onnxruntime-web/dist/esm/ort.js:105:13)
    at async Function.create (file:///pathtofolder/node_modules/onnxruntime-web/dist/esm/ort.js:1025:26)
    at async file:///pathtofolder/main.mjs:35:24

And if I try:

import ort from "./node_modules/onnxruntime-web/dist/esm/ort.js";

Then it throws:

error: Uncaught SyntaxError: The requested module './node_modules/onnxruntime-web/dist/esm/ort.js' does not provide an export named 'default'
import ort from "./node_modules/onnxruntime-web/dist/esm/ort.js";

And using esm.sh like this:

import ort from "https://esm.sh/[email protected]/dist/esm/ort.js";

also throws:

error: Uncaught SyntaxError: The requested module 'https://esm.sh/[email protected]/dist/esm/ort.js' does not provide an export named 'default'
import ort from "https://esm.sh/[email protected]/dist/esm/ort.js";

### Description This PR tries to fix a part of the NPM package consuming problems for onnxruntime-web (ES module) as described in #10913: - reduce the package size to fit the 150MB restriction in jsdelivr, by removing dev build targets for uncommon exports - add default export to support `import ort from 'onnxruntime-web';` (currently only support `import * as ort from 'onnxruntime-web';`

### Description This PR tries to fix a part of the NPM package consuming problems for onnxruntime-web (ES module) as described in microsoft#10913: - reduce the package size to fit the 150MB restriction in jsdelivr, by removing dev build targets for uncommon exports - add default export to support `import ort from 'onnxruntime-web';` (currently only support `import * as ort from 'onnxruntime-web';`

fs-eire self-assigned this Mar 28, 2022

fs-eire added the component:ort-web label Mar 28, 2022

sophies927 added platform:web issues related to ONNX Runtime web; typically submitted using template and removed component:ort-web labels Aug 12, 2022

fs-eire mentioned this issue Feb 1, 2023

[js][feature tracking] JavaScript library consume #14529

Open

xenova mentioned this issue Apr 8, 2023

[Feature request] Deno Support huggingface/transformers.js#78

Closed

kungfooman mentioned this issue Apr 25, 2023

Es6 rewrite huggingface/transformers.js#91

Closed

josephrocca mentioned this issue Oct 12, 2023

Can't import onnxruntime in pyodide pyodide/pyodide#4220

Open

fs-eire mentioned this issue Oct 26, 2023

[js/web] fix a few package consuming problems #18109

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[web] ES6 modules #10913

[web] ES6 modules #10913

josephrocca commented Mar 17, 2022 •

edited

Loading

josephrocca commented Mar 20, 2022 •

edited

Loading

josephrocca commented Mar 20, 2022 •

edited

Loading

fs-eire commented Mar 28, 2022

nestarz commented Dec 13, 2022

josephrocca commented Dec 13, 2022

fs-eire commented Dec 14, 2022 •

edited

Loading

nestarz commented Feb 7, 2023 •

edited

Loading

fs-eire commented Feb 8, 2023

fs-eire commented May 2, 2023

josephrocca commented Jun 16, 2023

fs-eire commented Oct 11, 2023

josephrocca commented Oct 12, 2023 •

edited

Loading

fs-eire commented Oct 16, 2023

josephrocca commented Oct 16, 2023 •

edited

Loading

[web] ES6 modules #10913

[web] ES6 modules #10913

Comments

josephrocca commented Mar 17, 2022 • edited Loading

josephrocca commented Mar 20, 2022 • edited Loading

josephrocca commented Mar 20, 2022 • edited Loading

fs-eire commented Mar 28, 2022

nestarz commented Dec 13, 2022

josephrocca commented Dec 13, 2022

fs-eire commented Dec 14, 2022 • edited Loading

nestarz commented Feb 7, 2023 • edited Loading

fs-eire commented Feb 8, 2023

fs-eire commented May 2, 2023

josephrocca commented Jun 16, 2023

fs-eire commented Oct 11, 2023

josephrocca commented Oct 12, 2023 • edited Loading

fs-eire commented Oct 16, 2023

josephrocca commented Oct 16, 2023 • edited Loading

josephrocca commented Mar 17, 2022 •

edited

Loading

josephrocca commented Mar 20, 2022 •

edited

Loading

josephrocca commented Mar 20, 2022 •

edited

Loading

fs-eire commented Dec 14, 2022 •

edited

Loading

nestarz commented Feb 7, 2023 •

edited

Loading

josephrocca commented Oct 12, 2023 •

edited

Loading

josephrocca commented Oct 16, 2023 •

edited

Loading