Skip to content

Commit bfd3787

Browse files
authored
[Next][Fix] Remove require() in index.js for scriptDir (mlc-ai#397)
### Overview This PR post-processes web-llm's `index.js` by replacing all `new (require('u' + 'rl').URL)('file:' + __filename).href` with `"MLC_DUMMY_PATH"`: ```javascript var _scriptDir = (typeof document === 'undefined' && typeof location === 'undefined' ? "MLC_DUMMY_PATH": typeof document === 'undefined' ? location.href : (document.currentScript && document.currentScript.src || new URL('index.js', document.baseURI).href) ); ``` which previously would raise error as shown in mlc-ai#383 and other issues. Other occurrences of `"MLC_DUMMY_PATH"` are only in branches of `if (ENVIRONMENT_IS_NODE)` in runtime, which we do not consider / support as of now. We use `"MLC_DUMMY_PATH"` instead of `null` because we do not expect the value to be used at all. If that is not the case, it would be easier to debug with `"MLC_DUMMY_PATH"`. ### Details When building projects that use web-llm with `next` (e.g. `examples/next-simple-chat`), the **compile time** would complain about the call for `require()`; runtime does not run into it because `document` is not `undefined` when evaluating `_scriptDir`. Other examples, like `examples/chrome-extension`, do not have this issue because they build with `parcel`, which would fix it for us with `@parcel/resolver-default`: ![image](https://github.com/mlc-ai/web-llm/assets/53290280/0b9df99a-f80e-4fed-8c19-88deb8aabfbd) This PR's fix does not affect correctness because, by inspecting `index.js`, `_scriptDir` is used to populate `scriptDirectory`, which is used in the function `locateFile()`, which currently is only used for `wasmBinaryFile` (but `isDataURI(wasmBinaryFile)` would never evaluate to `false`): ```javascript function locateFile(path) { if (Module["locateFile"]) { return Module["locateFile"](path, scriptDirectory) } return scriptDirectory + path } if (!isDataURI(wasmBinaryFile)) { wasmBinaryFile = locateFile(wasmBinaryFile); } ``` We also do not remove other `require()` in `index.js` as of now, as from the current understanding, they would not cause issues -- but we can come back later when they do. One observation that is not yet explainable is that, if we set `"@mlc-ai/web-llm": "^0.2.35",` in `examples/next-simple-chat/package.json`, mlc-ai#383 would be observed. However, if we use `"@mlc-ai/web-llm": "../..",`, no issue is observed -- we are able to use `require()` in compile time.
1 parent 1943eb9 commit bfd3787

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

cleanup-index-js.sh

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@
44
sed -e s/"const{createRequire:createRequire}=await import('module');"//g -i .backup lib/index.js
55
sed -e s/"const{createRequire:createRequire}=await import('module');"//g -i .backup lib/index.js.map
66

7+
# Replace string "new (require('u' + 'rl').URL)('file:' + __filename).href" with "MLC_DUMMY_PATH"
8+
# This is required for building nextJS projects -- its compile time would complain about `require()`
9+
# See https://github.com/mlc-ai/web-llm/issues/383 and the fixing PR's description for more.
10+
sed -e s/"new (require('u' + 'rl').URL)('file:' + __filename).href"/"\"MLC_DUMMY_PATH\""/g -i .backup lib/index.js
11+
sed -e s/"new (require('u' + 'rl').URL)('file:' + __filename).href"/"\"MLC_DUMMY_PATH\""/g -i .backup lib/index.js.map
12+
713
# Cleanup backup files
814
rm lib/index.js.backup
915
rm lib/index.js.map.backup

0 commit comments

Comments
 (0)