process.memoryUsage: report heapUsed from the most recent collection - #39593
Conversation
Process_functionMemoryUsage read heap.sizeAfterLastEdenCollection(). JSC updates that counter after eden collections only. After a full collection heapUsed kept the figure of the last eden collection, and it stayed 0 for the life of a process that runs without the JIT, where every collection is a full one. JSVMClientData now owns a HeapObserver that is attached for the life of the VM. When a collection finishes, it copies the counter of the scope that just ran. process.memoryUsage() reports that value. The diagnostics_channel leak test compared heapUsed across a full collection. The two numbers used to be equal because heapUsed did not move. It now checks that unsubscribed channels are collectable.
|
Updated 4:24 PM PT - Aug 18th, 2026
⏳ @robobun, your commit 7334569 is still building in |
|
Status: reproduced on the released build (1.4.0). After |
|
Warning Review limit reached
Next review available in: 2 minutes Limit details: You’ve used all 5 included reviews currently available. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
Comment |
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. Because it adds a per-VM HeapObserver with lifecycle tied to JSVMClientData teardown and a collector-thread write read by the mutator, plus rewrites the diagnostics_channel leak test's assertion, a human look would still be worthwhile — particularly on the observer detach ordering and the new WeakRef-survival threshold across CI configs.
What was reviewed
- Observer lifecycle:
~JSVMClientDataruns whilevm.heapis alive (confirmed by the existingJSHeapDataDeletercomment), soremoveObserveris safe; init-list order matches declaration order. - Cross-thread
m_sizeAfterLastCollectionwrite/read: same contract as JSC's ownsizeAfterLast*Collection()counters and the existingGCProfilerObserverinNodeV8.h. - Checked other
sizeAfterLastEdenCollection()call sites — each already reads the counter for the scope it just ran, so no sibling sites need the same fix. BUN_GC_TIMER_DISABLEused in the new test is a real env knob (src/jsc/GarbageCollectionController.rs).
Extended reasoning...
Overview
The PR fixes process.memoryUsage().heapUsed to report the size measured by the most recent GC (eden or full) instead of only the last eden collection. It does this by adding Bun::HeapSizeAfterLastCollection, a JSC::HeapObserver owned by JSVMClientData, that copies sizeAfterLast{Full,Eden}Collection() in didGarbageCollect(scope). Process_functionMemoryUsage reads the copy via clientData(vm)->heapSizeAfterLastCollection(). It also rewrites the diagnostics_channel "references are not leaked" test (previously vacuous because heapUsed did not move across a full collection) to assert on WeakRef survival, and adds two subprocess tests in process.test.js that pin heapUsed to the return of Bun.gc(true) / edenGC().
Security risks
None. This is a diagnostic/metrics read path with no user-controlled input reaching native code.
Level of scrutiny
Medium-high. The logic is small and well-argued, but it lives in per-VM native state: it registers a heap observer at VM creation and detaches it in the client-data destructor, and the observer callback runs in the GC end phase (potentially on the collector thread) writing a plain size_t that the mutator later reads. The PR correctly notes this is the same happens-before contract JSC's own m_sizeAfterLast*Collect counters rely on, and GCProfilerObserver in NodeV8.h already follows the identical pattern. Still, GC-observer lifecycle and cross-thread reads in BunClientData are the sort of thing a maintainer should sign off on rather than land on bot approval alone.
Other factors
- The diagnostics_channel test rewrite changes what is asserted. The argument that the old comparison was always trivially equal is convincing, and the new WeakRef check with a
< length/10bound accounts for conservative stack scanning, but it would be good for a maintainer to confirm that threshold holds under debug/ASAN across platforms. - The PR flags overlap with #39541 (same line in
BunProcess.cpp) and #33368 (changes whatheapUsed/heapTotalmeasure); coordination is a human call. - The other in-tree readers of
sizeAfterLastEdenCollection()(BunJSCModule.h'sedenGC,NodeV8.h's profiler) are scope-correct already, so no sibling fixes are missing. - CI is still building (#100859) at the time of review.
|
On the two points left for a human look: Detach ordering. The survival bound in the diagnostics_channel test. The bound is 100 of 1000. Measured values: 1 survivor with the debug ASAN build, 2 with the release build, in repeated runs. With a strong reference kept on purpose, all 1000 survive. The survivors come from conservative stack scanning, which pins the channels whose pointers the last loop iterations left on the stack, so the count does not scale with the platform or the build type. The test is synchronous on purpose: |
…ollection heapUsed comes from the figure JSC measured at the end of the most recent collection (#39593). With no collection requested at startup there is no figure until the first one, and nothing has been freed yet either, so report heapTotal until then. external is only measured by a collection and stays 0 until then, which is the one line of test-memory-usage.js that cannot pass without a collection, so that node test is removed.
### What does this PR do? Bun requested a garbage collection right before it waited on the entry module's promise. `VirtualMachine::load_entry_point`, its worker and test runner variants, and `load_preloads()` (once per preload module) each called `perform_gc()`, which is `JSC::VM::collectAsync()`. At that point the heap holds little more than the new global object, so the collection frees nothing. JSC serves the request at the next allocation slow path, inside `loadAndEvaluateModule`. That is a synchronous eden collection on the main thread, and it also starts the marker threads, before the first line of the program runs. `BUN_JSC_logGC=1 bun empty.js` logs it on every start: ``` [GC<0x...>: START M 400kb => EdenCollection, ... ] ``` This PR removes those requests. JSC's own activity callbacks and Bun's idle GC timer still collect once there is something to collect. The full collection that `web_worker.rs` runs after a worker's entry point is a different mechanism and is not changed. Windows x64 release build, hyperfine `-N` (150 to 200 runs) plus per-launch counters: | | before | after | |---|---|---| | `bun empty.js` | 28.7 ms | 26.8 ms | | `bun hello.js` | 29.8 ms | 27.4 ms | | CPU cycles per launch | 55.7 M | 47.0 M | | threads created | 15 | 8 | | peak commit | 85.9 MB | 68.7 MB | Two tests depended on the startup collection. Both read numbers that JSC only fills in while it collects. - `process.memoryUsage()` (`BunProcess.cpp`) reports `heapUsed` from the figure JSC measured at the end of the most recent collection (#39593), and `external` from `heap.extraMemorySize()`. With no collection, both are 0. Before the first collection `heapUsed` now reports `heapTotal`: nothing has been freed yet, so the whole heap is in use. This is one branch, no collection. `external` stays 0 until the first collection. JSC only measures it while it collects, and there is no public counter for it. `test/js/node/test/parallel/test-memory-usage.js` asserts `r.external > 0` at startup, which only a collection can satisfy, so that node test is removed. A `Heap` accessor for the bytes allocated since the last collection in the WebKit fork would make both numbers exact without a collection and bring the test back. - `test/js/web/abort/abort.test.ts` counts `AbortSignal` wrappers with `heapStats()` before and after `Bun.gc(true)`. `heapStats()` runs a full collection itself when nothing has collected yet, so the first count was taken after the wrappers were gone and the difference was 0. The test now collects once before it creates the signals. ### How did you verify your code works? - `test/js/bun/gc/gc-controller-cadence.test.ts`: new tests start a script, a script with a preload, a test file, and a worker with `BUN_JSC_logGC` and check that no eden collection is logged. On main they see one per entry point and per preload. The full collections Bun runs on purpose (VM teardown on the ASAN lanes, a worker after its entry point and at teardown) are not counted. - `test/js/node/process/process.test.js`: a new case in the #39593 describe block reads `process.memoryUsage()` in a fresh process with `BUN_GC_TIMER_DISABLE=1` and expects `heapUsed` to equal `heapTotal`. On main it reports `heapUsed: 0`. - With the debug build, rebased on main at 91cdf15: the #39593 cases in `process.test.js`, `test/js/web/abort/abort.test.ts`, `test/js/bun/globals.test.js` ("cleans up memory"), `test/js/node/diagnostics_channel/diagnostics_channel.test.ts` ("references are not leaked"), `test/js/bun/jsc/bun-jsc.test.ts`, `test/js/node/v8/v8-module.test.ts`, and the node tests `test-v8-stats.js`, `test-worker-heap-statistics.js`, `test-v8-collect-gc-profile.js`, `test-gc-tls-external-memory.js`, `test-vm-measure-memory.js` pass. `test-memory-usage.js` fails on its `r.external > 0` line only. - `BUN_JSC_logGC=1 bun hello.js` no longer logs a collection before the entry module evaluates. ETW traces of 25 launches no longer contain `Heap::collectInMutatorThread` or marker thread creation during startup. ### Background JSC's collector is generational. An eden collection only examines the objects allocated since the previous collection. A full collection examines everything. At the end of a collection, `Heap` stores the live size in the counter for that kind of collection only: `sizeAfterLastEdenCollection()` or `sizeAfterLastFullCollection()`. `extraMemorySize()` is the memory outside the heap (string contents, buffers) that the cells visited by the collections so far reported. `heap.size()` is computed from the mark bits, so it is also 0 before the first collection, and it walks every block of the heap on each call. That is why `process.memoryUsage()` does not use it. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/node/process/process.test.js test/js/web/abort/abort.test.ts <!-- robobun:evidence:end --> --------- Co-authored-by: robobun <117481402+robobun@users.noreply.github.com>
Problem
process.memoryUsage().heapUseddoes not change after a full collection. It keeps the figure of the last eden collection. AfterBun.gc(true)frees 10 MB,heapUsedstill reports the 10 MB, and it is larger thanheapTotal.BUN_JSC_useJIT=0),heapUsedis 0 for the life of the process. JSC turns off generational collection in that mode (VM::isInMiniMode()), so every collection is a full one.heapUsedis 0 right afterBun.gc(true). The full collection serves the pending startup request, so no eden collection has run.Process_functionMemoryUsageinsrc/jsc/bindings/BunProcess.cppreadsheap.sizeAfterLastEdenCollection().Heap::updateAllocationLimits()writes that counter after an eden collection only. A full collection writesm_sizeAfterLastFullCollect. The counter that is current after both,m_sizeAfterLastCollect, has no accessor.Fix
JSVMClientDataowns aJSC::HeapObserver(Bun::HeapSizeAfterLastCollection,src/jsc/bindings/BunClientData.h). It attaches to the heap when the VM is created and detaches when the VM is destroyed.didGarbageCollect(scope)copies the counter of the scope that just ran.process.memoryUsage()reports that copy.Heap::runEndPhase()callsupdateAllocationLimits(), which stores the size of the collection in the counter for its scope and inm_sizeAfterLastCollect, and thendidFinishCollection(), which notifies the observers with the same scope. So the copy always equalsm_sizeAfterLastCollect.process.memoryUsage()stays usable in a monitoring loop.heap.size()would be current too, but it walks every block of the heap.sizeAfterLast*Collection()counters today.Zig__GlobalObject__create), before any global object exists, so the observer sees every collection of the heap. Each worker has its own VM and its own copy.~VMdeletes the client data while the heap is still alive, so the detach is safe.test/js/node/diagnostics_channel/diagnostics_channel.test.ts, "references are not leaked", comparedheapUsedfrom before a loop withheapUsedafter a full collection. The two numbers were always equal, becauseheapUseddid not move across the full collection. With this change the first number dates from an earlier collection in the file and the comparison fails. The test now checks what the node test is after: once unsubscribed, nothing holds the channels. It holds aWeakRefper channel and counts the ones that survivegc(true). Locally 1 or 2 of 1000 survive (conservative stack scanning). A retained reference keeps all 1000 alive.test/js/node/process/process.test.js, describe "process.memoryUsage().heapUsed reports the most recent collection". One child runs a full, an eden, and a full collection and checks thatheapUsedequals the figure each one returned. One child runs withBUN_JSC_useJIT=false. Both fail on the released build (heapUsedis[0, eden, eden]and0) and pass with this change.test/js/node/diagnostics_channel/diagnostics_channel.test.ts,test/js/bun/globals.test.js("cleans up memory" now measures a real drop),test/js/bun/jsc/bun-jsc.test.ts,test/js/node/v8/v8-module.test.ts, and the node teststest-memory-usage.js,test-sqlite-template-tag.js(its leak check reads 2.71 MB before and 2.76 MB after, against a 1.5x bound),test-v8-collect-gc-profile*.js,test-v8-stats.js,test-worker-heap-statistics.js,test-gc-tls-external-memory.js.Related PRs. #39541 touches the same line of
BunProcess.cpp. It adds a fallback to the full counter when the eden counter is 0, for the fresh process case above, and it collects once when both are 0. With this observer in place, that fallback reduces to one check ofheapSizeAfterLastCollection(). Whichever lands second resolves a small conflict there. #33368 changes whatheapUsedandheapTotalmeasure (it takes both from the marked space, for #20793). This PR keeps the current measure and only fixes which collection it comes from.Background
JSC's collector is generational. An eden collection marks only the objects allocated since the previous collection. A full collection marks the whole heap. At the end of either one,
Heap::updateAllocationLimits()computes the live size (bytes visited plus the extra memory that live cells reported). It stores it inm_sizeAfterLastEdenCollectorm_sizeAfterLastFullCollect, depending on the scope of the collection, and always inm_sizeAfterLastCollect. Only the first two have accessors.A
JSC::HeapObserveris an interface withwillGarbageCollect()anddidGarbageCollect(CollectionScope).Heap::addObserver()registers one. The heap callsdidGarbageCollectfromHeap::didFinishCollection(), in the end phase of every collection. The end phase runs with the world stopped (worldShouldBeSuspended()inCollectorPhase.cpp), which can be on the collector thread.GCProfilerObserverinsrc/jsc/bindings/NodeV8.his the existing observer in this codebase. It reads the same counters by scope.JSVMClientData(src/jsc/bindings/BunClientData.h) is Bun's per VM data. It is created inJSVMClientData::create()right after the VM, and~VMdeletes it afterheap.lastChanceToFinalize(), while the heap, a member of the VM, is still alive.Repro on the released build and with this change
Released build (1.4.0):
With this change:
Without the JIT, released build, after allocating 300000 objects:
heapUsedis0. With this change:11833741.