Repository navigation
bun test --isolate: reclaim the previous file's module graph on global swap #31772
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
6607b9b
bun test --isolate: reclaim the previous file's module graph on globa…
robobun 07903ee
[autofix.ci] apply automated fixes
autofix-ci[bot] 3d2491f
review: extract module-registry teardown into GlobalObject::clearModu…
robobun d894fb9
ci: retrigger
robobun 7825f61
isolate: reclaim detached module graph via async GC, not a sync full GC
robobun c542deb
parallel workers: run mimalloc_cleanup before each isolation swap
robobun 5c8a351
isolation.test.ts: drop per-test timeout that undercut the ASAN file …
robobun 270a691
worker teardown: route module-registry clear through clearModuleRegistry
robobun 1749f67
isolate swap: satisfy exception-check validation across the FFI boundary
robobun 5a1544e
isolate swap: use release-safe exception assert; trim comments
robobun 4cbd6b0
isolate: dedup forbidExecution through clearModuleRegistry; one-line …
robobun b078401
test(31771): shrink isolate-RSS workload, tighten timeout
robobun d89f273
test(31771): drop dead post-exit VmHWM read
robobun 480f4fe
[autofix.ci] apply automated fixes
autofix-ci[bot] 8ae0d70
VM: narrow collect_full_async to pub(crate), matching collect_async
robobun ea01f7c
parallel worker: hoist mimalloc_cleanup above the isolation branch
robobun File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| import { expect, test } from "bun:test"; | ||
| import { bunEnv, bunExe, isLinux, tempDir } from "harness"; | ||
| import fs from "node:fs"; | ||
| import path from "node:path"; | ||
|
|
||
| // https://github.com/oven-sh/bun/issues/31771 | ||
| // | ||
| // `bun test --isolate` runs each file in a fresh global within ONE process. | ||
| // The module records / namespace objects instantiated for one file must be | ||
| // reclaimed when the global is swapped for the next file. Before the fix the | ||
| // outgoing global's module registry and require cache were never cleared, so | ||
| // each finished file's module graph was retained and peak RSS grew linearly | ||
| // with the number of files (proportional to each file's imported graph size), | ||
| // OOMing large suites. After the fix peak RSS stays roughly flat, as it does | ||
| // in default (shared-global) mode. | ||
| // | ||
| // Measured by polling VmHWM (peak RSS high-water mark, monotonic) of the child | ||
| // from /proc. Serial --isolate is a single process, so this is exact. The | ||
| // signal is the SLOPE (RSS grows with file count), not an absolute number, so | ||
| // the threshold is robust across platforms and build types. | ||
|
|
||
| // A 500-module import graph shared by every test file. Before the fix this | ||
| // retained several MB/file; the difference between a small and large file | ||
| // count is tens of MB — far above allocator noise. Kept modest so the run | ||
| // completes well within the timeout under ASAN (the debug/gate build). | ||
| const MODULE_COUNT = 500; | ||
| const SMALL_N = 4; | ||
| const LARGE_N = 10; | ||
|
|
||
| function makeFixtures() { | ||
| const files: Record<string, string> = {}; | ||
| let graph = ""; | ||
| for (let i = 0; i < MODULE_COUNT; i++) { | ||
| files[`mods/m${i}.ts`] = `export function f${i}() { return ${i}; }\n`; | ||
| graph += `import "./mods/m${i}";\n`; | ||
| } | ||
| files["graph.ts"] = graph; | ||
| for (let i = 0; i < LARGE_N; i++) { | ||
| files[`t${i}.test.ts`] = `import "./graph";\nimport { test } from "bun:test";\ntest("noop", () => {});\n`; | ||
| } | ||
| return files; | ||
| } | ||
|
|
||
| async function peakRssMb(dir: string, n: number): Promise<number> { | ||
| const files = Array.from({ length: n }, (_, i) => `./t${i}.test.ts`); | ||
| await using proc = Bun.spawn({ | ||
| cmd: [bunExe(), "test", "--isolate", ...files], | ||
| env: bunEnv, | ||
| cwd: dir, | ||
| stdout: "ignore", | ||
| stderr: "pipe", | ||
| }); | ||
|
|
||
| let peakKb = 0; | ||
| const statusPath = `/proc/${proc.pid}/status`; | ||
| let running = true; | ||
| const poll = (async () => { | ||
| while (running) { | ||
| try { | ||
| const status = fs.readFileSync(statusPath, "utf8"); | ||
| const m = status.match(/VmHWM:\s*(\d+)\s*kB/); | ||
| if (m) peakKb = Math.max(peakKb, parseInt(m[1], 10)); | ||
| } catch { | ||
| // process gone | ||
| } | ||
| await Bun.sleep(2); | ||
| } | ||
| })(); | ||
|
|
||
| const [stderr, exitCode] = await Promise.all([proc.stderr.text(), proc.exited]); | ||
| running = false; | ||
| await poll; | ||
|
|
||
| // Combined assertion so a failed/killed child surfaces its stderr tail, | ||
| // exit code, and signal together in one diff. | ||
| const summaryOk = stderr.includes(`${n} pass`) && stderr.includes("0 fail"); | ||
| expect({ | ||
| summary: summaryOk ? "ok" : stderr.slice(-2000), | ||
| exitCode, | ||
| signalCode: proc.signalCode, | ||
| sawPeakRss: peakKb > 0, | ||
| }).toEqual({ summary: "ok", exitCode: 0, signalCode: null, sawPeakRss: true }); | ||
| return peakKb / 1024; | ||
| } | ||
|
|
||
| test.skipIf(!isLinux)( | ||
| "bun test --isolate does not retain finished files' module graph (peak RSS stays flat)", | ||
| async () => { | ||
| using dir = tempDir("isolate-rss-31771", makeFixtures()); | ||
| const root = String(dir); | ||
|
|
||
| // Sanity: fixtures materialized. | ||
| expect(fs.existsSync(path.join(root, "graph.ts"))).toBe(true); | ||
|
|
||
| const small = await peakRssMb(root, SMALL_N); | ||
| const large = await peakRssMb(root, LARGE_N); | ||
|
|
||
| const perFile = (large - small) / (LARGE_N - SMALL_N); | ||
|
|
||
| // Before the fix the 500-module graph is retained per file, so the slope is | ||
| // several MB/file (tens of MB across the file-count delta). After the fix | ||
| // the graph is reclaimed on each swap and the slope is near zero. 0.5 MB/file | ||
| // sits comfortably between the two (post-fix is <0.1 MB/file even under | ||
| // ASAN's noisier heap accounting). | ||
| expect(perFile).toBeLessThan(0.5); | ||
| }, | ||
| // Outlier per root CLAUDE.md: spawns two child `bun test --isolate` runs that | ||
| // each transpile a 500-module graph under debug+ASAN (~14s total), which no | ||
| // module count small enough to fit the default budget would still discriminate. | ||
| 30_000, | ||
| ); | ||
|
robobun marked this conversation as resolved.
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.