Skip to content

Commit 0a77990

Browse files
penalosapenalosa
authored
Support modules with --no-bundle (#2769)
* Support modules with `--no-bundle`
1 parent 191b23f commit 0a77990

31 files changed

+1295
-85
lines changed

‎.changeset/beige-eagles-wave.md

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
---
2+
"wrangler": minor
3+
"no-bundle-import": patch
4+
---
5+
6+
feature: Support modules with `--no-bundle`
7+
8+
When the `--no-bundle` flag is set, Wrangler now has support for uploading additional modules alongside the entrypoint. This will allow modules to be imported at runtime on Cloudflare's Edge. This respects Wrangler's [module rules](https://developers.cloudflare.com/workers/wrangler/configuration/#bundling) configuration, which means that only imports of non-JS modules will trigger an upload by default. For instance, the following code will now work with `--no-bundle` (assuming the `example.wasm` file exists at the correct path):
9+
10+
```js
11+
// index.js
12+
import wasm from './example.wasm'
13+
14+
export default {
15+
async fetch() {
16+
await WebAssembly.instantiate(wasm, ...)
17+
...
18+
}
19+
}
20+
```
21+
22+
For JS modules, it's necessary to specify an additional [module rule](https://developers.cloudflare.com/workers/wrangler/configuration/#bundling) (or rules) in your `wrangler.toml` to configure your modules as ES modules or Common JS modules. For instance, to upload additional JavaScript files as ES modules, add the following module rule to your `wrangler.toml`, which tells Wrangler that all `**/*.js` files are ES modules.
23+
24+
```toml
25+
rules = [
26+
{ type = "ESModule", globs = ["**/*.js"]},
27+
]
28+
```
29+
30+
If you have Common JS modules, you'd configure Wrangler with a CommonJS rule (the following rule tells Wrangler that all `.cjs` files are Common JS modules):
31+
32+
```toml
33+
rules = [
34+
{ type = "CommonJS", globs = ["**/*.cjs"]},
35+
]
36+
```
37+
38+
In most projects, adding a single rule will be sufficient. However, for advanced usecases where you're mixing ES modules and Common JS modules, you'll need to use multiple rule definitions. For instance, the following set of rules will match all `.mjs` files as ES modules, all `.cjs` files as Common JS modules, and the `nested/say-hello.js` file as Common JS.
39+
40+
```toml
41+
rules = [
42+
{ type = "CommonJS", globs = ["nested/say-hello.js", "**/*.cjs"]},
43+
{ type = "ESModule", globs = ["**/*.mjs"]}
44+
]
45+
```
46+
47+
If multiple rules overlap, Wrangler will log a warning about the duplicate rules, and will discard additional rules that matches a module. For example, the following rule configuration classifies `dep.js` as both a Common JS module and an ES module:
48+
49+
```toml
50+
rules = [
51+
{ type = "CommonJS", globs = ["dep.js"]},
52+
{ type = "ESModule", globs = ["dep.js"]}
53+
]
54+
```
55+
56+
Wrangler will treat `dep.js` as a Common JS module, since that was the first rule that matched, and will log the following warning:
57+
58+
```
59+
▲ [WARNING] Ignoring duplicate module: dep.js (esm)
60+
```
61+
62+
This also adds a new configuration option to `wrangler.toml`: `base_dir`. Defaulting to the directory of your Worker's main entrypoint, this tells Wrangler where your additional modules are located, and determines the module paths against which your module rule globs are matched.
63+
64+
For instance, given the following directory structure:
65+
66+
```
67+
- wrangler.toml
68+
- src/
69+
- index.html
70+
- vendor/
71+
- dependency.js
72+
- js/
73+
- index.js
74+
```
75+
76+
If your `wrangler.toml` had `main = "src/js/index.js"`, you would need to set `base_dir = "src"` in order to be able to import `src/vendor/dependency.js` and `src/index.html` from `src/js/index.js`.

‎fixtures/no-bundle-import/README.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# `no-bundle-import`
2+
3+
This worker exercises the module collection system when `--no-bundle` is specified. It demonstrates dynamic import and static import, as well as module rules whcih treat different JS files as CommonJS or ESModule

‎fixtures/no-bundle-import/package.json

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "no-bundle-import",
3+
"version": "0.0.0",
4+
"private": true,
5+
"scripts": {
6+
"deploy": "wrangler publish",
7+
"start": "wrangler dev",
8+
"test": "vitest"
9+
},
10+
"devDependencies": {
11+
"get-port": "^6.1.2",
12+
"wrangler": "^2.10.0"
13+
}
14+
}

‎fixtures/no-bundle-import/src/data.bin

4 Bytes
Binary file not shown.

‎fixtures/no-bundle-import/src/data.txt

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
TEST DATA

‎fixtures/no-bundle-import/src/dynamic.cjs

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
module.exports = "cjs-string";

‎fixtures/no-bundle-import/src/dynamic.js

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export default "dynamic";

‎fixtures/no-bundle-import/src/index.js

+75
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
import { sayHello } from "./say-hello.js";
2+
import cjs from "./say-hello.cjs";
3+
4+
import { johnSmith } from "./nested/index.js";
5+
import WASM from "./simple.wasm";
6+
import nestedWasm from "./nested/simple.wasm";
7+
8+
import text from "./data.txt";
9+
import binData from "./data.bin";
10+
export default {
11+
async fetch(request, env, ctx) {
12+
const url = new URL(request.url);
13+
if (url.pathname === "/dynamic") {
14+
return new Response(`${(await import("./dynamic.js")).default}`);
15+
}
16+
if (url.pathname === "/wasm") {
17+
return new Response(
18+
await new Promise(async (resolve) => {
19+
const moduleImport = {
20+
imports: {
21+
imported_func(arg) {
22+
resolve(arg);
23+
},
24+
},
25+
};
26+
const module1 = await WebAssembly.instantiate(WASM, moduleImport);
27+
module1.exports.exported_func();
28+
})
29+
);
30+
}
31+
if (url.pathname === "/wasm-nested") {
32+
return new Response(
33+
await new Promise(async (resolve) => {
34+
const moduleImport = {
35+
imports: {
36+
imported_func(arg) {
37+
resolve("nested" + arg);
38+
},
39+
},
40+
};
41+
const m = await WebAssembly.instantiate(nestedWasm, moduleImport);
42+
m.exports.exported_func();
43+
})
44+
);
45+
}
46+
if (url.pathname === "/wasm-dynamic") {
47+
return new Response(
48+
`${await (await import("./nested/index.js")).loadWasm()}`
49+
);
50+
}
51+
52+
if (url.pathname.startsWith("/lang")) {
53+
const language = url.pathname.split("/lang/")[1];
54+
return new Response(
55+
`${JSON.parse((await import(`./lang/${language}`)).default).hello}`
56+
);
57+
}
58+
59+
if (url.pathname === "/txt") {
60+
return new Response(text);
61+
}
62+
if (url.pathname === "/bin") {
63+
return new Response(binData);
64+
}
65+
if (url.pathname === "/cjs") {
66+
return new Response(
67+
`CJS: ${cjs.sayHello("Jane Smith")} and ${johnSmith}`
68+
);
69+
}
70+
if (url.pathname === "/cjs-loop") {
71+
return new Response(`CJS: ${cjs.loop}`);
72+
}
73+
return new Response(`${sayHello("Jane Smith")} and ${johnSmith}`);
74+
},
75+
};

‎fixtures/no-bundle-import/src/index.test.ts

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
import path from "path";
2+
import { describe, expect, test, beforeAll, afterAll } from "vitest";
3+
import { unstable_dev } from "../../../packages/wrangler/wrangler-dist/cli.js";
4+
import type { UnstableDevWorker } from "../../../packages/wrangler/wrangler-dist/cli.js";
5+
6+
describe("Worker", () => {
7+
let worker: UnstableDevWorker;
8+
9+
// TODO: Remove this when `workerd` has Windows support
10+
if (process.env.RUNNER_OS === "Windows") {
11+
test("dummy windows test", () => {
12+
expect(process.env.RUNNER_OS).toStrictEqual("Windows");
13+
});
14+
return;
15+
}
16+
17+
beforeAll(async () => {
18+
worker = await unstable_dev(path.resolve(__dirname, "index.js"), {
19+
bundle: false,
20+
experimental: { experimentalLocal: true },
21+
});
22+
}, 30_000);
23+
24+
afterAll(() => worker.stop());
25+
26+
test("module traversal results in correct response", async () => {
27+
const resp = await worker.fetch();
28+
const text = await resp.text();
29+
expect(text).toMatchInlineSnapshot(
30+
`"Hello Jane Smith and Hello John Smith"`
31+
);
32+
});
33+
34+
test("module traversal results in correct response for CommonJS", async () => {
35+
const resp = await worker.fetch("/cjs");
36+
const text = await resp.text();
37+
expect(text).toMatchInlineSnapshot(
38+
`"CJS: Hello Jane Smith and Hello John Smith"`
39+
);
40+
});
41+
42+
test("correct response for CommonJS which imports ESM", async () => {
43+
const resp = await worker.fetch("/cjs-loop");
44+
const text = await resp.text();
45+
expect(text).toMatchInlineSnapshot('"CJS: cjs-string"');
46+
});
47+
48+
test("support for dynamic imports", async () => {
49+
const resp = await worker.fetch("/dynamic");
50+
const text = await resp.text();
51+
expect(text).toMatchInlineSnapshot(`"dynamic"`);
52+
});
53+
54+
test("basic wasm support", async () => {
55+
const resp = await worker.fetch("/wasm");
56+
const text = await resp.text();
57+
expect(text).toMatchInlineSnapshot('"42"');
58+
});
59+
60+
test("resolves wasm import paths relative to root", async () => {
61+
const resp = await worker.fetch("/wasm-nested");
62+
const text = await resp.text();
63+
expect(text).toMatchInlineSnapshot('"nested42"');
64+
});
65+
66+
test("wasm can be imported from a dynamic import", async () => {
67+
const resp = await worker.fetch("/wasm-dynamic");
68+
const text = await resp.text();
69+
expect(text).toMatchInlineSnapshot('"sibling42subdirectory42"');
70+
});
71+
72+
test("text data can be imported", async () => {
73+
const resp = await worker.fetch("/txt");
74+
const text = await resp.text();
75+
expect(text).toMatchInlineSnapshot('"TEST DATA"');
76+
});
77+
78+
test("binary data can be imported", async () => {
79+
const resp = await worker.fetch("/bin");
80+
const bin = await resp.arrayBuffer();
81+
const expected = new Uint8Array(new ArrayBuffer(4));
82+
expected.set([0, 1, 2, 10]);
83+
expect(new Uint8Array(bin)).toEqual(expected);
84+
});
85+
});

‎fixtures/no-bundle-import/src/lang/en.json

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"hello": "Hello"
3+
}

‎fixtures/no-bundle-import/src/lang/fr.json

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"hello": "Bonjour"
3+
}

‎fixtures/no-bundle-import/src/nested/index.js

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import { sayHello } from "../say-hello.js";
2+
import cjs from "./say-hello.js";
3+
import subWasm from "../simple.wasm";
4+
import sibWasm from "./simple.wasm";
5+
export const johnSmith =
6+
sayHello("John Smith") === cjs.sayHello("John Smith")
7+
? sayHello("John Smith")
8+
: false;
9+
10+
export async function loadWasm() {
11+
const sibling = await new Promise(async (resolve) => {
12+
const moduleImport = {
13+
imports: {
14+
imported_func(arg) {
15+
resolve("sibling" + arg);
16+
},
17+
},
18+
};
19+
const m = await WebAssembly.instantiate(sibWasm, moduleImport);
20+
m.exports.exported_func();
21+
});
22+
23+
const subdirectory = await new Promise(async (resolve) => {
24+
const moduleImport = {
25+
imports: {
26+
imported_func(arg) {
27+
resolve("subdirectory" + arg);
28+
},
29+
},
30+
};
31+
const m = await WebAssembly.instantiate(subWasm, moduleImport);
32+
m.exports.exported_func();
33+
});
34+
return sibling + subdirectory;
35+
}

‎fixtures/no-bundle-import/src/nested/say-hello.js

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
module.exports.sayHello = (name) => `Hello ${name}`;

‎fixtures/no-bundle-import/src/nested/simple.wasm

78 Bytes
Binary file not shown.

‎fixtures/no-bundle-import/src/say-hello.cjs

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
module.exports.sayHello = (name) => `Hello ${name}`;
2+
3+
module.exports.loop = require("./dynamic.cjs");

‎fixtures/no-bundle-import/src/say-hello.js

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export const sayHello = (name) => `Hello ${name}`;

‎fixtures/no-bundle-import/src/simple.wasm

78 Bytes
Binary file not shown.

‎fixtures/no-bundle-import/wrangler.toml

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
name = "no-bundle-import"
2+
main = "src/index.js"
3+
compatibility_date = "2023-02-20"
4+
5+
rules = [
6+
{ type = "CommonJS", globs = ["nested/say-hello.js", "**/*.cjs"]},
7+
{ type = "ESModule", globs = ["**/*.js"]},
8+
{ type = "Text", globs = ["**/*.json"], fallthrough = true}
9+
]

0 commit comments

Comments
 (0)