Repository navigation
Sync upstream dev3 (2026-09-12, 636510a3, v3.5.2) - #37
Merged
Merged
Conversation
fix numa node count on linux: return the count, not the highest node …
Add theap stats retrieval
v3.5.0 release
…tions/actions/checkout-7.0.1 Bump actions/checkout from 6.1.0 to 7.0.1
…tions/actions/stale-11.0.0 Bump actions/stale from 10.4.0 to 11.0.0
…tions/minor-and-patch-c937a527f7 Bump the minor-and-patch group with 2 updates
Improve behavior on multi-config generators
…rd at exit on Windows) Brings in #38 and #39 from the fork branch (8565de2, 1b894af, 707d90b) on top of the merged upstream arena code. src/arena.c, src/scavenger.c, src/subproc.c and include/mimalloc/internal.h merge without overlap: upstream did not touch `_mi_arenas_try_purge`, `_mi_arenas_collect` or `mi_arena_schedule_purge` between the two bases. The one upstream change next to them is in `mi_arena_try_purge`, which now leaves an arena whose expire is 0 alone for a forced pass as well. That is compatible: a free arms the arena before it sets its purge bits, and the pass reads the arena expire with the 0 -> 0 CAS after the walk, as before. Conflicts: - CMakeLists.txt: the test list keeps `profile` in place of `prof`/`prof-adversarial` and the `MI_TRACK STREQUAL "ASAN"` condition from the upstream side, and gains `forced-purge` and `purge-behind-pass`. - src/init.c (`mi_process_done_once`): the Windows release of the purge guard goes before the heap snapshot; the call to `_mi_prof_on_exit` stays removed with the fork's profiler. test-forced-purge and test-purge-behind-pass set `minimal_purge_size` to one slice. They compare the `purged` counter with the bytes they freed, and with upstream's MI_ALLOW_THP=FULL (the Linux default of a standalone build) a pass purges aligned 2 MiB units only: each freed run keeps its edges (the runs break at the 256 MiB aligned page meta), so a release build read 252 of 256 MiB and 316 of 320 MiB.
Takes upstream v3.5.2 (636510a, 2026-09-12): the process-init theap is associated with the thread-done key (`mi_process_setup_auto_thread_done` moves into `mi_process_init_once`, pr microsoft#1393), `mi_usable_size` always unaligns the pointer, `mi_page_ptr_unalign_ex` returns the offset to `mi_page_ptr_block_check`, `_mi_page_profile_free` becomes `_mi_page_profile_on_free`, the generic `_mi_memzero_block` (the `rep stosb/movsb` paths and `_mi_cpu_movsb_max`/ `_mi_cpu_stosb_max` are gone), small pages keep their page info in front of the slice up to `MI_SMALL_MAX_OBJ_SIZE` and not for OS-aligned blocks (636510a), the multi-config cmake generator support, the `execinfo.h` check, 32-bit warning fixes, and the action version bumps (all SHA-pinned). Conflicts: - .github/workflows/test.yaml: upstream edited the Alpine jobs (arm32 asm upload); they stay removed here (setup-alpine's nested action is not SHA-pinned). The other workflow changes are taken as they are. Everything under src/ and include/ merged without overlap with the fork's changes.
…ported as a double free Two slips in upstream code, both still in upstream dev3 at 636510a: - `_mi_page_usable_size` (d68195c) compares the unaligned block with `page` where it means `p`. A block is never at the address of its page, so every call went through the out-of-line `mi_page_usable_aligned_size_of`. The result was right (the offset is 0 there), only slower. - `mi_check_padding_on_free` read `is_double_free` without a value when the canary was intact and only the padding bytes behind the block were overwritten: `mi_page_decode_padding` sets it on a bad canary only. The message for a buffer overflow then depended on what was on the stack (gcc: -Wmaybe-uninitialized with MI_SECURE=FULL).
A third of the blocks of each heap stayed allocated for good after `mi_heap_delete` had moved them to the main heap. Each pins its page, so a release build grew by 3 GiB a second (38 GiB RSS at the end of the 6 s run on a 64 core machine), and with MI_SECURE=FULL, where a guard page follows every page, the process ran into `vm.max_map_count` (65530) within two seconds: `pthread_create` failed and the test crashed in `pthread_join`. The owner now frees those blocks after the delete, which also covers frees into the pages that the delete handed to the main heap. After a delete by another thread it waits for that delete to return first: frees by the thread that owns the theap are not safe against a concurrent `mi_heap_delete`, that is what the park is for. Release: 0.8 GiB RSS, and 3.4 times as many heaps released in the same time.
64 KiB is two slices on a 32-bit target (32 KiB slices), where a pass would again leave the odd slice at the edge of a run. The option value is rounded up to an OS page, and that is never more than a slice. No-Verification-Needed: test-only change (test/test-forced-purge.c, test/test-purge-behind-pass.c)
Jarred-Sumner
changed the title
…letes its old heap The previous commit made the owner wait, after its park, for a delete by a freer thread to return before it freed the blocks that stayed live in that heap. That took away a case the test had: the owner allocating and freeing in its next heap while its old one is still being deleted elsewhere. It now sets those blocks aside, goes on, and frees them when `deleted[self]` says the delete is over (at the latest before it hands the next heap to a freer, and when it stops). The block count per heap and the size of the two arrays come from one pair of constants, and there is one place that frees what was kept. No-Verification-Needed: test-only change (test/test-heap-release-mt.c)
… tests It was run with MIMALLOC_GUARDED_SAMPLE_RATE=0 because it kept a third of its blocks for good and every guarded block is a mapping of its own. It frees them now: with MI_GUARDED a release build passes at sample rates 1024, 64 and 1, with and without the scavenger. So it takes the default rate of 1024, and the -no-scavenger variant whatever the job sets. No-Verification-Needed: test wiring only (CMakeLists.txt add_test environment)
The prepare handler takes the purge guard (last, under its locks: a pass takes none of them and waits for no one, so the one in progress ends), the parent and the child release it. test-park-handoff "fork while parked" kept failing now and then on macOS Debug builds: the forked child died with SIGBUS. The diagnostics the test got earlier now say where (CI run 34751251554, basic/macos-latest/cxx, Debug): forked child: signal 10 at address 0x20001210000 (in a mimalloc heap: 1) mi_block_set_next < mi_page_free_list_extend < mi_page_extend_free < _mi_page_init < mi_arenas_page_regular_alloc < _mi_arenas_page_alloc < mi_page_fresh_alloc < mi_page_fresh < mi_page_queue_find_free_ex < _mi_malloc_generic < mi_malloc Not in a free list that the interrupted hole sweep was rewriting, but on the first write into a page that the child had just allocated from the arena, at a slice aligned address: a range that the arena bitmaps call free and committed, and that is not writable. Whole slices become PROT_NONE in one place only, the decommit of an arena purge, and only with MI_DEBUG or MI_SECURE (`_mi_prim_decommit`), which fits "Debug only". A park ends in `_mi_arenas_purge_now`, so the scavenger is in an arena purge pass when the test forks a few hundred microseconds later. The prepare handler ended the park and took every lock, but a pass takes no lock, so it ran across the fork. A pass claims a range, decommits it, clears its commit bits and releases it; a consistent copy of that is harmless to the child (it sees the range claimed), a copy with the bitmaps from before and the protection from after is not. That the kernel produced such a copy is an inference, the missing quiescence is not. With the guard held by the fork no pass is cut off, so the child has nothing to repair: the release of a dead purger's guard and the "make a pass due" of 1b894af are replaced by a plain release. `_mi_arenas_purge_guard_reset` becomes `_mi_arenas_purge_guard_acquire`/`_release`; the process exit on Windows uses the latter. test-purge-behind-pass "a fork in a pass" passes as it is: the fork waits for the pass, and the child purges the arena that was not yet due when its time comes.
`page->used` became the low 16 bits of the packed word `mi_used_t xused` (used, alloc count, last used, last alloc) with the upstream merge. Both fast paths then load the word, change it, store it, and `mi_free` tests the low half, where they used to do `incl`/`decl` on a 32-bit field: three more instructions in `mi_free`, two more in `mi_malloc`. - `mi_free` decrements the 16-bit count in place, through the `used_count` member the union already had: `decw 0x18(%rdi); je`. Little-endian only; the code for the whole word stays for everything else. (On a used count of 0, a double free that nothing caught, the count wraps to 0xFFFF where it used to borrow from the alloc count.) - `mi_malloc` adds 0x10001 to the word in memory once it knows the free list is not empty: `addq $0x10001,0x18(%rcx)`. The load before the test and the compiler barrier that forced it are gone. Instructions for 25 M malloc + 25 M free of 16..1039 bytes (perf stat, user mode), clean bun-dev3-v2 / before / after: cmake Release, gcc: 2.4142 G / 2.5394 G / 2.4127 G Bun's flags (clang -O3 as C++): 2.2463 G / 2.3411 G / 2.2153 G (with MI_PROFILE=0: 2.3097 G / 2.1838 G) `mi_free` is 22 instructions (23 on bun-dev3-v2, 25 before), `mi_malloc` 18 (19, 20). Cycles do not differ between the three on this benchmark on a loaded machine (1.24 to 1.28 G each).
Jarred-Sumner
marked this pull request as ready for review
September 13, 2026 12:14
Jarred-Sumner
added a commit
to oven-sh/bun
that referenced
this pull request
Sep 13, 2026
Pins ab13501334a8, the merge of oven-sh/mimalloc#37 on bun-dev3-v2. On top of the two purge fixes this branch already carried (#38, #39) it brings: - microsoft/mimalloc dev3 through v3.5.2 (301 commits): always-on statistics (MI_STATS=1), sampled profiler hooks (mi_profiler_t, MI_PROFILE=1; the fork's own prof.c and mi_prof_* are gone, bun used none of it), codegen work in free/zalloc/memzero, double-free detection through the padding canary - fork(): no arena purge pass runs across it any more (the macOS Debug SIGBUS of a forked child, and a whole class of torn purge state in the child) - mi_heap_destroy of a heap with a block still on a page's thread-free list no longer reports corrupted meta-data; two slips in upstream's free.c fixed (mi_usable_size always took its slow path; an overflow could be reported as a double free) - malloc/free fast paths update the packed used count with one instruction: 25 M malloc + 25 M free with bun's flags execute 2.2403 G instructions in this configuration, 2.2463 G with the old pin MI_FREE_USE_PAGEMAP=1 keeps the page-map lookup in mi_free. Upstream's new default puts page meta data at 256 MiB boundaries and needs every arena to start on one; mi_manage_os_memory_ex then cannot take JSC's structure heap when its reservation is 256 MiB or less, and bun aborted on startup under `ulimit -v 3000000` or BUN_JSC_structureHeapSizeInKB<=262144. With the define that works down to 64 MiB as before. The aligned layout is worth another 1.1% of instructions on that benchmark (2.2153 G) once WebKit hands mimalloc an aligned region. Every mi_* function and mi_option number bun declares is unchanged (options 0..46; 47 is collect_merges_stats now, snapshot_on_exit 48). Release build, old pin against this one, 5 interleaved runs, user-mode instructions / peak RSS: bun -e 1 9.17 M / 25.9 MB 9.16 M / 25.7 MB Bun.serve hello 51748 / request 51787 / request (+0.08%), 48.2 MB / 48.5 MB JSON round trips 5102.3 M / 42.6 MB 5102.9 M / 42.6 MB string concat 1939.8 M / 58.9 MB 1904.3 M / 57.8 MB (run to run spread 2%)
Jarred-Sumner
added a commit
to oven-sh/bun
that referenced
this pull request
Sep 13, 2026
…freed during a pass is purged by the next (#42571) Both fork PRs are merged: oven-sh/mimalloc#38 and oven-sh/mimalloc#39. The pin `707d90be` is on `bun-dev3-v2` (they were merged with merge commits, so the sha did not change). This PR now carries both bumps and both tests; it replaces #42543. ### Two fixes in the fork **1. A forced purge waits for the pass in progress (oven-sh/mimalloc#38).** - `Bun.gc(true)` ends with `mi_collect(true)` since #42428. mimalloc dropped that purge when another thread held its purge guard, and the purge thread holds it for 36 to 47 ms while it purges what was already freed. `Bun.gc(true)` then returned with all of it still resident. - `test/js/bun/spawn/spawn-pipe-leak.test.ts` fails on main because of it: `Peak RSS: warmup 46 MB -> main 312 MB (573%)`, 7 of 10 runs pass. With this pin: 25 of 25. - Only a forced collect waits. bun's one caller is `garbage_collect_from_js`. It takes 29 to 47 ms in place of 1 to 2 ms when it collides with a pass over 384 MB, and the memory is back with the OS when it returns. - Test: `test/js/bun/gc/gc-controller-cadence.test.ts`, "Bun.gc(true) returns what is free to the OS while the purge thread is at work". Fails 3 of 3 on main's release build (38 to 42 MB released, expected more than 300). **2. What is freed during a purge pass is purged by the next pass (oven-sh/mimalloc#39).** The rest of this description. ### Problem - Memory freed while mimalloc's purge thread (the scavenger) is in a pass can stay resident until the event loop goes idle or something forces a collection. With one arena (`Malloc=1`) a busy script keeps 110 of 384 MB: `{"released":262.6,"purged":274.4}`. - In the fork, `_mi_arenas_try_purge` (`src/arena.c`) keeps its old `subproc->purge_expire` during a pass, so a free in that time arms the arena only. The scavenger then clears the value, and no later free sets it again. - It reaches bun's default configuration. JSC's structure heap is a second arena that rarely purges, which hides the case of one busy thread. With several busy threads and no ticking event loop it shows each time: 8 workers that churn `ArrayBuffer`s and then spin hold 578 MB RSS on main's release build and fall to 91 MB with this pin. ### Fix - Bump `MIMALLOC_COMMIT` to `707d90be` (oven-sh/mimalloc#39, which contains oven-sh/mimalloc#38). The bump also picks up `0fecc729`, a 32-bit shift fix in `src/prof.c` that was already on `bun-dev3-v2`. - In the fork a pass now resets `subproc->purge_expire` first and puts back what is pending at its end, so a free behind the pass schedules the next one. - Verified: the new test in `test/js/bun/jsc/heapStats-mimalloc.test.ts` fails 3 of 3 on main's release build, passes 20 of 20 with this pin. `bun bd test` skips it (ASAN). ### Background - A purge returns freed mimalloc arena memory to the OS (`madvise`), 100 ms after the free. - The scavenger is the fork's thread that runs those purges. It sleeps until `subproc->purge_expire`. - The first free into an arena after a pass arms it (`arena->purge_expire`). Only that free looks at `subproc->purge_expire`. <details><summary>Notes</summary> **The two commits of oven-sh/mimalloc#39.** - `1b894afe` arena: what is freed during a purge pass is purged by the next pass (the fix) - `707d90be` init: release the purge guard at process exit on Windows **The free path (`mi_arena_schedule_purge`) is unchanged.** Only the pass and the scavenger loop change. **The defect in full.** A free arms its arena (`arena->purge_expire`) and, only if the arena was not armed yet, arms the subprocess (`subproc->purge_expire`, 0 -> set), which is what the scavenger sleeps on. A pass resets the arena expire when it goes into the arena, but it kept its old subprocess expire until its end. The first free into the arena behind the pass armed the arena again, found the subprocess expire still set, and left it alone. Then `_mi_arenas_try_purge` skipped its update (the break for `max_purge_count` also fires on the last arena, so a pass in which each arena purged ended with `all_visited = false`), and `mi_scavenger_run` set `subproc->purge_expire` to 0. The arena was armed and the subprocess was not. Each later free into that arena saw an armed arena and stopped there. The 30 s timeout of the scavenger did not help: with nothing scheduled it only waits again. **Reach in bun.** JSC registers its structure heap as an exclusive mimalloc arena (`mi_manage_os_memory_ex` in `StructureAlignedMemoryAllocator.cpp`). A pass over two arenas in which only the main one purges ends with `all_visited = true`, reads the re-armed expire of the main arena after its walk, and installs it. So the same script that loses 128 MB with one arena gets its second pass on time with two (traced on main's release build: first pass at 107 to 123 ms, second at 207 ms). It shows when both arenas purge in the same pass, when a process has more arenas (more than 1 GiB of mimalloc memory) and each purges, and with `Malloc=1`. The event loop's `mi_on_thread_idle_start` on each `epoll_wait` heals it when the sweep runs to its end. Under load the owner wakes first and the sweep stops before `_mi_arenas_purge_now`. The state is absorbing: once an arena is armed and the subprocess is not, nothing but a park or a forced collect ends it. So a low chance per pass is enough for a process whose threads stay busy. Measured with stock two arena bun (release, Linux x64): 8 workers allocate and free 1 to 8 MB `ArrayBuffer`s for 3 s (`transfer(0)` frees at once), free everything, and spin, while the main thread sits in `Atomics.wait`. After a quiet 2.5 s RSS is 578 MB on main and 91 MB with this pin (`arena_count` 2 in both). The same happens with only the one line fix for `all_visited` in the fork, which is why the pass resets the subprocess expire first. **The test.** A child frees 256 MB with `ArrayBuffer.prototype.transfer(0)` (no collection involved), spins until RSS fell by 64 MB (the purge thread is in its pass), frees 128 MB more, and spins without an `await` until RSS fell by 336 MB, for at most 2 s. It reports that drop and the growth of `heapStats().mimalloc.purged`. Both have to be at least 336 MB. The drop in RSS is taken before the last `heapStats()` call, because that call polls the allocator (`mi_collect(false)`), and a poll runs a pass that is due by itself. `Malloc=1` makes JSC allocate through `malloc`, which is mimalloc on Linux, so there is one arena. Linux only (the wait reads RSS) and not ASAN (malloc is not mimalloc there). | build | result | | --- | --- | | release build of main (`6a92015fc`, pin `6a64e1ba`) | fails 3 of 3: `{"released":262.6,"purged":274.4}`, expected >= 336 | | `bun run build:release` with this pin | passes 20 of 20, about 450 ms | | `bun bd test` with this pin (debug, ASAN) | new test skipped, the other 4 tests in the file pass | The change is under `scripts/`, so a checkout of `src/` from main does not undo it. The fail-before check needs a release build of main. **About `707d90be`.** `1b894afe` alone failed the 5 Windows jobs of the fork's CI. Its new test returns from `main` while the scavenger is in its last pass. On Windows `mi_process_done` runs in the process detach callback, after the system terminated every other thread, so the purge guard stays held, and the forced collect at exit waits for it without end (that wait is what oven-sh/mimalloc#38 adds). `707d90be` releases the guard there. bun builds mimalloc with `MI_NO_PROCESS_DETACH` and `MI_SKIP_COLLECT_ON_EXIT`, so bun does not reach that code on any platform. Fork CI at `707d90be`: 15 of 15 jobs pass (Windows x64 and arm64, mingw, macOS, Linux x64 and arm64 with ASAN, UBSAN, TSAN and guarded builds, FreeBSD). **In the fork.** New `test/test-purge-behind-pass.c`: 5 of 5 runs fail before (`266 of 320 MiB`), 30 of 30 pass after. A stress harness with 8 to 16 busy threads: 15 of 15 rounds end stuck before, with about 530 MiB of free committed ranges unpurged. After: 750 rounds, 0 stuck. Details are in oven-sh/mimalloc#39. **Effect on purge activity.** Ranges that stayed resident by accident are now purged 100 ms after the free, which is what `purge_delay` promises. **Suites run with a release build of this branch (Linux x64).** `test/js/bun/jsc/heapStats-mimalloc.test.ts`, `test/js/bun/gc/gc-controller-cadence.test.ts` (11 pass), `test/js/bun/spawn/spawn-pipe-leak.test.ts` (3 pass), `test/js/bun/spawn/spawn-noread-leak.test.ts`, `test/js/bun/jsc/bun-jsc.test.ts` (38 pass). Nothing here was run on macOS or Windows. </details> ### Now pins the merged upstream sync `MIMALLOC_COMMIT` is `ab13501334a8da2b64ab2e5f4f552c3a39b96350`: the merge of oven-sh/mimalloc#37 on `bun-dev3-v2`. It contains `707d90be` (both fixes above), so everything in this description still holds. **What it adds beyond oven-sh/mimalloc#38 and oven-sh/mimalloc#39** - microsoft/mimalloc `dev3` through v3.5.2 (301 commits): always-on statistics (`MI_STATS=1`), sampled profiler hooks (`mi_profiler_t`, `MI_PROFILE=1`), codegen work in free/zalloc/memzero, double-free detection through the padding canary. - The fork's own heap profiler is gone (`src/prof.c`, `mi_prof_*`, `MIMALLOC_PROF_*`); upstream's hooks replace it. bun referenced none of it. - `fork()`: no arena purge pass runs across it any more. That was the macOS Debug SIGBUS of a forked child in the fork's CI, and a whole class of torn purge state in the child. - `mi_heap_destroy` of a heap that still has a block on a page's thread-free list no longer reports corrupted meta-data. - Two slips in upstream's `free.c` are fixed: `mi_usable_size` always took its slow path, and an overflow could be reported as a double free. - The malloc/free fast paths update upstream's packed used count with one instruction (`decw` / `addq` on memory). **One new define: `MI_FREE_USE_PAGEMAP=1`** - Upstream's new default keeps page meta data at 256 MiB boundaries so that `mi_free` needs no page map. Every arena then has to start on such a boundary. - JSC gives mimalloc its structure heap as an arena (`mi_manage_os_memory_ex(start + 16 KiB, ...)`, under a `RELEASE_ASSERT`), and halves that reservation when address space is short. With the new default `mi_manage_os_memory_ex` refuses it once the reservation is 256 MiB or less. - Measured with a release build of the new pin without the define: `ulimit -v 3000000; bun -e 1` aborts on startup, and so does `BUN_JSC_structureHeapSizeInKB=262144`. The old pin runs both. With the default 4 GiB reservation it works but loses the first 256 MiB of structure address space. - With the define, `mi_free` looks the page up through the page map as it does today, and the structure heap works down to 64 MiB as before. The fork's CI runs this configuration (debug and release) on Linux x64 and arm64. - The aligned layout is worth another 1.1% of instructions on the malloc/free benchmark below. Getting it needs WebKit to hand mimalloc an aligned region (`mi_arena_min_alignment()`), which is a separate change. **Not changed: `MI_PROFILE` stays at its default of 1.** Nothing in bun attaches a profiler yet, and `MI_PROFILE=0` would save about 1.4% of instructions on the same benchmark. The hooks are what a `Bun.pprof.heap` API would build on, so they stay compiled in. **Bindings.** Every `mi_*` function that `src/mimalloc_sys/mimalloc.rs` and the C++ side declare is exported with an unchanged declaration. `mi_heap_area_t` has the same layout. `mi_option_t` 0..46 did not move (47 is `collect_merges_stats` now, `snapshot_on_exit` 48); bun sets option 0 only. The `heapStats().mimalloc` keys the tests read (`purged`, `page_bins`, `pages`, `committed`) are emitted as before. **Allocator micro benchmark** (25 M `mi_malloc` + 25 M `mi_free` of 16..1039 bytes, bun's compile flags, user-mode instructions): | | instructions | | --- | --- | | old pin (`707d90be`) | 2.2463 G | | this pin, `MI_FREE_USE_PAGEMAP=1` (what this PR builds) | **2.2403 G** (-0.3%) | | this pin, upstream's aligned layout | 2.2153 G (-1.4%) | **bun, release build, old pin against this commit** (same tree, only the pin and the define differ; 5 interleaved runs pinned to 8 cores; user-mode instructions and peak RSS, medians): | | old pin | this commit | | --- | --- | --- | | `bun -e 1` | 9.17 M, 25.9 MB | 9.16 M (-0.06%), 25.7 MB | | `Bun.serve` hello, per request (20000 sequential `fetch`) | 51748, 48.2 MB | 51787 (+0.08%), 48.5 MB | | `JSON.stringify` + `JSON.parse` of a 200-user object, 3000 times | 5102.3 M, 42.6 MB | 5102.9 M (+0.01%), 42.6 MB | | string concat + `Buffer.from` + `split`, 300 x 5000 | 1939.8 M, 58.9 MB | 1904.3 M (-1.8%, the spread between runs is 2%), 57.8 MB | | idle after start (`Bun.sleep`), RSS from `smaps_rollup` | 32.4 MB (anon 3.85) | 32.5 MB (anon 3.96) | Nothing is slower by more than 0.1%. **Tests with a release build of this commit (Linux x64)** - `test/js/bun/spawn/spawn-pipe-leak.test.ts`: 25 of 25 runs pass. `test/js/bun/jsc/heapStats-mimalloc.test.ts` and `test/js/bun/gc/gc-controller-cadence.test.ts`: 10 of 10 each. - 137 test files, 4280 tests pass: the two files of this PR, `spawn-noread-leak`, `require-cache`, `heap-snapshot`, `bun-jsc`, `socket-retention`, `serve.test.ts`, all of `test/js/node/child_process`, `node-net`, `bunshell`, `fetch.test.ts`, `bun-install.test.ts`, `bundler_edgecase`, all of `test/js/web/workers` and `test/js/node/worker_threads`, `test/js/bun/resolve`, `test/js/bun/jsc`, the `--compile` and bundler files, `test/bake/dev`. - 8 tests fail, and the same 8 fail with a release build of the old pin: they need a non-root user (a port below 1024 that must be refused, `EPERM` after dropping privileges, unreadable files and directories). - Small structure heaps: `BUN_JSC_structureHeapSizeInKB` = 262144, 131072, 65536 and `ulimit -v` = 3000000, 2000000, 1500000 all start. - Debug (ASAN) build: `heapStats-mimalloc`, `gc-controller-cadence`, `test/js/web/workers/worker.test.ts` pass; `child_process.test.ts` has the one root-only failure. - `src/static.c` compiles with the new define and bun's flags for macOS arm64/x64 and Windows x64/arm64 (release and debug). Nothing here was run on macOS or Windows. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/jsc/heapStats-mimalloc.test.ts, test/js/bun/gc/gc-controller-cadence.test.ts <!-- robobun:evidence:end --> --------- Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
usrbinkat
pushed a commit
to usrbinkat/bun
that referenced
this pull request
Sep 15, 2026
…freed during a pass is purged by the next (oven-sh#42571) Both fork PRs are merged: oven-sh/mimalloc#38 and oven-sh/mimalloc#39. The pin `707d90be` is on `bun-dev3-v2` (they were merged with merge commits, so the sha did not change). This PR now carries both bumps and both tests; it replaces oven-sh#42543. ### Two fixes in the fork **1. A forced purge waits for the pass in progress (oven-sh/mimalloc#38).** - `Bun.gc(true)` ends with `mi_collect(true)` since oven-sh#42428. mimalloc dropped that purge when another thread held its purge guard, and the purge thread holds it for 36 to 47 ms while it purges what was already freed. `Bun.gc(true)` then returned with all of it still resident. - `test/js/bun/spawn/spawn-pipe-leak.test.ts` fails on main because of it: `Peak RSS: warmup 46 MB -> main 312 MB (573%)`, 7 of 10 runs pass. With this pin: 25 of 25. - Only a forced collect waits. bun's one caller is `garbage_collect_from_js`. It takes 29 to 47 ms in place of 1 to 2 ms when it collides with a pass over 384 MB, and the memory is back with the OS when it returns. - Test: `test/js/bun/gc/gc-controller-cadence.test.ts`, "Bun.gc(true) returns what is free to the OS while the purge thread is at work". Fails 3 of 3 on main's release build (38 to 42 MB released, expected more than 300). **2. What is freed during a purge pass is purged by the next pass (oven-sh/mimalloc#39).** The rest of this description. ### Problem - Memory freed while mimalloc's purge thread (the scavenger) is in a pass can stay resident until the event loop goes idle or something forces a collection. With one arena (`Malloc=1`) a busy script keeps 110 of 384 MB: `{"released":262.6,"purged":274.4}`. - In the fork, `_mi_arenas_try_purge` (`src/arena.c`) keeps its old `subproc->purge_expire` during a pass, so a free in that time arms the arena only. The scavenger then clears the value, and no later free sets it again. - It reaches bun's default configuration. JSC's structure heap is a second arena that rarely purges, which hides the case of one busy thread. With several busy threads and no ticking event loop it shows each time: 8 workers that churn `ArrayBuffer`s and then spin hold 578 MB RSS on main's release build and fall to 91 MB with this pin. ### Fix - Bump `MIMALLOC_COMMIT` to `707d90be` (oven-sh/mimalloc#39, which contains oven-sh/mimalloc#38). The bump also picks up `0fecc729`, a 32-bit shift fix in `src/prof.c` that was already on `bun-dev3-v2`. - In the fork a pass now resets `subproc->purge_expire` first and puts back what is pending at its end, so a free behind the pass schedules the next one. - Verified: the new test in `test/js/bun/jsc/heapStats-mimalloc.test.ts` fails 3 of 3 on main's release build, passes 20 of 20 with this pin. `bun bd test` skips it (ASAN). ### Background - A purge returns freed mimalloc arena memory to the OS (`madvise`), 100 ms after the free. - The scavenger is the fork's thread that runs those purges. It sleeps until `subproc->purge_expire`. - The first free into an arena after a pass arms it (`arena->purge_expire`). Only that free looks at `subproc->purge_expire`. <details><summary>Notes</summary> **The two commits of oven-sh/mimalloc#39.** - `1b894afe` arena: what is freed during a purge pass is purged by the next pass (the fix) - `707d90be` init: release the purge guard at process exit on Windows **The free path (`mi_arena_schedule_purge`) is unchanged.** Only the pass and the scavenger loop change. **The defect in full.** A free arms its arena (`arena->purge_expire`) and, only if the arena was not armed yet, arms the subprocess (`subproc->purge_expire`, 0 -> set), which is what the scavenger sleeps on. A pass resets the arena expire when it goes into the arena, but it kept its old subprocess expire until its end. The first free into the arena behind the pass armed the arena again, found the subprocess expire still set, and left it alone. Then `_mi_arenas_try_purge` skipped its update (the break for `max_purge_count` also fires on the last arena, so a pass in which each arena purged ended with `all_visited = false`), and `mi_scavenger_run` set `subproc->purge_expire` to 0. The arena was armed and the subprocess was not. Each later free into that arena saw an armed arena and stopped there. The 30 s timeout of the scavenger did not help: with nothing scheduled it only waits again. **Reach in bun.** JSC registers its structure heap as an exclusive mimalloc arena (`mi_manage_os_memory_ex` in `StructureAlignedMemoryAllocator.cpp`). A pass over two arenas in which only the main one purges ends with `all_visited = true`, reads the re-armed expire of the main arena after its walk, and installs it. So the same script that loses 128 MB with one arena gets its second pass on time with two (traced on main's release build: first pass at 107 to 123 ms, second at 207 ms). It shows when both arenas purge in the same pass, when a process has more arenas (more than 1 GiB of mimalloc memory) and each purges, and with `Malloc=1`. The event loop's `mi_on_thread_idle_start` on each `epoll_wait` heals it when the sweep runs to its end. Under load the owner wakes first and the sweep stops before `_mi_arenas_purge_now`. The state is absorbing: once an arena is armed and the subprocess is not, nothing but a park or a forced collect ends it. So a low chance per pass is enough for a process whose threads stay busy. Measured with stock two arena bun (release, Linux x64): 8 workers allocate and free 1 to 8 MB `ArrayBuffer`s for 3 s (`transfer(0)` frees at once), free everything, and spin, while the main thread sits in `Atomics.wait`. After a quiet 2.5 s RSS is 578 MB on main and 91 MB with this pin (`arena_count` 2 in both). The same happens with only the one line fix for `all_visited` in the fork, which is why the pass resets the subprocess expire first. **The test.** A child frees 256 MB with `ArrayBuffer.prototype.transfer(0)` (no collection involved), spins until RSS fell by 64 MB (the purge thread is in its pass), frees 128 MB more, and spins without an `await` until RSS fell by 336 MB, for at most 2 s. It reports that drop and the growth of `heapStats().mimalloc.purged`. Both have to be at least 336 MB. The drop in RSS is taken before the last `heapStats()` call, because that call polls the allocator (`mi_collect(false)`), and a poll runs a pass that is due by itself. `Malloc=1` makes JSC allocate through `malloc`, which is mimalloc on Linux, so there is one arena. Linux only (the wait reads RSS) and not ASAN (malloc is not mimalloc there). | build | result | | --- | --- | | release build of main (`6a92015fc`, pin `6a64e1ba`) | fails 3 of 3: `{"released":262.6,"purged":274.4}`, expected >= 336 | | `bun run build:release` with this pin | passes 20 of 20, about 450 ms | | `bun bd test` with this pin (debug, ASAN) | new test skipped, the other 4 tests in the file pass | The change is under `scripts/`, so a checkout of `src/` from main does not undo it. The fail-before check needs a release build of main. **About `707d90be`.** `1b894afe` alone failed the 5 Windows jobs of the fork's CI. Its new test returns from `main` while the scavenger is in its last pass. On Windows `mi_process_done` runs in the process detach callback, after the system terminated every other thread, so the purge guard stays held, and the forced collect at exit waits for it without end (that wait is what oven-sh/mimalloc#38 adds). `707d90be` releases the guard there. bun builds mimalloc with `MI_NO_PROCESS_DETACH` and `MI_SKIP_COLLECT_ON_EXIT`, so bun does not reach that code on any platform. Fork CI at `707d90be`: 15 of 15 jobs pass (Windows x64 and arm64, mingw, macOS, Linux x64 and arm64 with ASAN, UBSAN, TSAN and guarded builds, FreeBSD). **In the fork.** New `test/test-purge-behind-pass.c`: 5 of 5 runs fail before (`266 of 320 MiB`), 30 of 30 pass after. A stress harness with 8 to 16 busy threads: 15 of 15 rounds end stuck before, with about 530 MiB of free committed ranges unpurged. After: 750 rounds, 0 stuck. Details are in oven-sh/mimalloc#39. **Effect on purge activity.** Ranges that stayed resident by accident are now purged 100 ms after the free, which is what `purge_delay` promises. **Suites run with a release build of this branch (Linux x64).** `test/js/bun/jsc/heapStats-mimalloc.test.ts`, `test/js/bun/gc/gc-controller-cadence.test.ts` (11 pass), `test/js/bun/spawn/spawn-pipe-leak.test.ts` (3 pass), `test/js/bun/spawn/spawn-noread-leak.test.ts`, `test/js/bun/jsc/bun-jsc.test.ts` (38 pass). Nothing here was run on macOS or Windows. </details> ### Now pins the merged upstream sync `MIMALLOC_COMMIT` is `ab13501334a8da2b64ab2e5f4f552c3a39b96350`: the merge of oven-sh/mimalloc#37 on `bun-dev3-v2`. It contains `707d90be` (both fixes above), so everything in this description still holds. **What it adds beyond oven-sh/mimalloc#38 and oven-sh/mimalloc#39** - microsoft/mimalloc `dev3` through v3.5.2 (301 commits): always-on statistics (`MI_STATS=1`), sampled profiler hooks (`mi_profiler_t`, `MI_PROFILE=1`), codegen work in free/zalloc/memzero, double-free detection through the padding canary. - The fork's own heap profiler is gone (`src/prof.c`, `mi_prof_*`, `MIMALLOC_PROF_*`); upstream's hooks replace it. bun referenced none of it. - `fork()`: no arena purge pass runs across it any more. That was the macOS Debug SIGBUS of a forked child in the fork's CI, and a whole class of torn purge state in the child. - `mi_heap_destroy` of a heap that still has a block on a page's thread-free list no longer reports corrupted meta-data. - Two slips in upstream's `free.c` are fixed: `mi_usable_size` always took its slow path, and an overflow could be reported as a double free. - The malloc/free fast paths update upstream's packed used count with one instruction (`decw` / `addq` on memory). **One new define: `MI_FREE_USE_PAGEMAP=1`** - Upstream's new default keeps page meta data at 256 MiB boundaries so that `mi_free` needs no page map. Every arena then has to start on such a boundary. - JSC gives mimalloc its structure heap as an arena (`mi_manage_os_memory_ex(start + 16 KiB, ...)`, under a `RELEASE_ASSERT`), and halves that reservation when address space is short. With the new default `mi_manage_os_memory_ex` refuses it once the reservation is 256 MiB or less. - Measured with a release build of the new pin without the define: `ulimit -v 3000000; bun -e 1` aborts on startup, and so does `BUN_JSC_structureHeapSizeInKB=262144`. The old pin runs both. With the default 4 GiB reservation it works but loses the first 256 MiB of structure address space. - With the define, `mi_free` looks the page up through the page map as it does today, and the structure heap works down to 64 MiB as before. The fork's CI runs this configuration (debug and release) on Linux x64 and arm64. - The aligned layout is worth another 1.1% of instructions on the malloc/free benchmark below. Getting it needs WebKit to hand mimalloc an aligned region (`mi_arena_min_alignment()`), which is a separate change. **Not changed: `MI_PROFILE` stays at its default of 1.** Nothing in bun attaches a profiler yet, and `MI_PROFILE=0` would save about 1.4% of instructions on the same benchmark. The hooks are what a `Bun.pprof.heap` API would build on, so they stay compiled in. **Bindings.** Every `mi_*` function that `src/mimalloc_sys/mimalloc.rs` and the C++ side declare is exported with an unchanged declaration. `mi_heap_area_t` has the same layout. `mi_option_t` 0..46 did not move (47 is `collect_merges_stats` now, `snapshot_on_exit` 48); bun sets option 0 only. The `heapStats().mimalloc` keys the tests read (`purged`, `page_bins`, `pages`, `committed`) are emitted as before. **Allocator micro benchmark** (25 M `mi_malloc` + 25 M `mi_free` of 16..1039 bytes, bun's compile flags, user-mode instructions): | | instructions | | --- | --- | | old pin (`707d90be`) | 2.2463 G | | this pin, `MI_FREE_USE_PAGEMAP=1` (what this PR builds) | **2.2403 G** (-0.3%) | | this pin, upstream's aligned layout | 2.2153 G (-1.4%) | **bun, release build, old pin against this commit** (same tree, only the pin and the define differ; 5 interleaved runs pinned to 8 cores; user-mode instructions and peak RSS, medians): | | old pin | this commit | | --- | --- | --- | | `bun -e 1` | 9.17 M, 25.9 MB | 9.16 M (-0.06%), 25.7 MB | | `Bun.serve` hello, per request (20000 sequential `fetch`) | 51748, 48.2 MB | 51787 (+0.08%), 48.5 MB | | `JSON.stringify` + `JSON.parse` of a 200-user object, 3000 times | 5102.3 M, 42.6 MB | 5102.9 M (+0.01%), 42.6 MB | | string concat + `Buffer.from` + `split`, 300 x 5000 | 1939.8 M, 58.9 MB | 1904.3 M (-1.8%, the spread between runs is 2%), 57.8 MB | | idle after start (`Bun.sleep`), RSS from `smaps_rollup` | 32.4 MB (anon 3.85) | 32.5 MB (anon 3.96) | Nothing is slower by more than 0.1%. **Tests with a release build of this commit (Linux x64)** - `test/js/bun/spawn/spawn-pipe-leak.test.ts`: 25 of 25 runs pass. `test/js/bun/jsc/heapStats-mimalloc.test.ts` and `test/js/bun/gc/gc-controller-cadence.test.ts`: 10 of 10 each. - 137 test files, 4280 tests pass: the two files of this PR, `spawn-noread-leak`, `require-cache`, `heap-snapshot`, `bun-jsc`, `socket-retention`, `serve.test.ts`, all of `test/js/node/child_process`, `node-net`, `bunshell`, `fetch.test.ts`, `bun-install.test.ts`, `bundler_edgecase`, all of `test/js/web/workers` and `test/js/node/worker_threads`, `test/js/bun/resolve`, `test/js/bun/jsc`, the `--compile` and bundler files, `test/bake/dev`. - 8 tests fail, and the same 8 fail with a release build of the old pin: they need a non-root user (a port below 1024 that must be refused, `EPERM` after dropping privileges, unreadable files and directories). - Small structure heaps: `BUN_JSC_structureHeapSizeInKB` = 262144, 131072, 65536 and `ulimit -v` = 3000000, 2000000, 1500000 all start. - Debug (ASAN) build: `heapStats-mimalloc`, `gc-controller-cadence`, `test/js/web/workers/worker.test.ts` pass; `child_process.test.ts` has the one root-only failure. - `src/static.c` compiles with the new define and bun's flags for macOS arm64/x64 and Windows x64/arm64 (release and debug). Nothing here was run on macOS or Windows. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/jsc/heapStats-mimalloc.test.ts, test/js/bun/gc/gc-controller-cadence.test.ts <!-- robobun:evidence:end --> --------- Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges microsoft/mimalloc
dev3at 636510a (2026-09-12, v3.5.2; 301 commits since the last sync at 6def7be) intobun-dev3-v2. These are real merge commits: upstream at 117d439 (272 commits), thenbun-dev3-v2with #38 and #39, then upstream again at 636510a (29 commits). A few small commits follow each merge: fixes for fork bugs that the wider test matrix turned up, the Windows C-mode guard, two slips in upstream code, and test fixes.Refresh of 2026-09-13 (head 5af105d)
Merged from
bun-dev3-v2(876ea41): #38 and #39, that is 8565de2 "arena: a forced purge waits for the pass in progress instead of being dropped", 1b894af "arena: what is freed during a purge pass is purged by the next pass", 707d90b "init: release the purge guard at process exit on Windows".src/arena.c,src/scavenger.c,src/subproc.c,include/mimalloc/internal.hmerged without overlap. Upstream did not touch_mi_arenas_try_purge,_mi_arenas_collectormi_arena_schedule_purgein the 272 commits. The one upstream change next to them is inmi_arena_try_purge: an arena whose expire is 0 is now left alone by a forced pass too. That fits the new protocol: a free arms the arena before it sets its purge bits, and the pass still reads the arena expire with the 0 -> 0 CAS after its walk. So the fork semantics stand as merged in arena: a forced purge waits for the pass in progress instead of being dropped #38/arena: what is freed during a purge pass is purged by the next pass #39: a forced purge waits for its turn, what is freed behind a pass is purged by the next one, the guard is released at process exit on Windows. (What arena: what is freed during a purge pass is purged by the next pass #39 did in the fork child is replaced in 93eb9c7: the fork itself now holds the guard, see open item 2.)CMakeLists.txt(test list): keepsprofilein place ofprof/prof-adversarialand theMI_TRACK STREQUAL "ASAN"condition from the upstream side, addsforced-purgeandpurge-behind-pass.src/init.c(mi_process_done_once): the Windows release of the purge guard goes in front of the heap snapshot; the call to_mi_prof_on_exitstays removed with the fork's profiler.purgedcounter with the bytes they freed. With upstream'sMI_ALLOW_THP=FULL(the Linux default of a standalone build, not of Bun's) a pass purges aligned 2 MiB units only, and each freed run keeps its edges (the runs break at the 256 MiB aligned page meta). A release build read252 of 256 MiBand316 of 320 MiB. Both tests now setminimal_purge_sizeto one OS page (7aae82a: not 64 KiB, which is two slices on 32-bit). Bun builds withMI_DEFAULT_ALLOW_THP=0, where the unit is an OS page.Merged from upstream (47dd31f), 117d439..636510a: the process-init theap is associated with the thread-done key (
mi_process_setup_auto_thread_donemoves intomi_process_init_once, upstream pr 1393),mi_usable_sizealways unaligns,mi_page_ptr_unalign_ex,_mi_page_profile_free->_mi_page_profile_on_free, a generic_mi_memzero_block(therep stosb/movsbpaths and_mi_cpu_movsb_max/_mi_cpu_stosb_maxare gone), small pages keep their page info in front of the slice up toMI_SMALL_MAX_OBJ_SIZEand not for OS-aligned blocks (636510a "fix wrong small free size alignment"), multi-config cmake generators, theexecinfo.hcheck, 32-bit warning fixes, action version bumps (all SHA-pinned).src/andinclude/merged without overlap with fork code..github/workflows/test.yaml: upstream edited the Alpine jobs (arm32 asm upload). They stay removed here (setup-alpine's nested action is not SHA-pinned); the other workflow changes are taken as they are.Follow-ups:
alloc, free: the fast paths update the packed used count with one instruction on memory. The merge now executes fewer instructions thanbun-dev3-v2on the malloc/free benchmark. See open item 1 below.fork: no arena purge pass runs across afork(). See open item 2 below.free: two slips in upstream code, both still in upstream at 636510a and worth reporting there._mi_page_usable_size(upstream d68195c) compares the unaligned block withpagewhere it meansp, so everymi_usable_sizecall took the out-of-line path (right result, slower).mi_check_padding_on_freereadis_double_freeuninitialized when the canary was intact and only the padding bytes were overwritten, so a buffer overflow could be reported as a double free.test-heap-release-mtfreed nothing of the third of its blocks that it kept live acrossmi_heap_delete. A release build grew to 38 GiB RSS in its 6 s on a 64 core machine, and withMI_SECURE=FULL(a guard page behind every page) it ran intovm.max_map_countwithin 2 s and crashed inpthread_join(this failed at the previous head e98ec96 as well). The owner now frees those blocks after the delete. After a delete by another thread it sets them aside, goes on with its next heap while that delete may still run (as before), and frees them once the delete has returned (fe2a5e5). 0.7 GiB RSS, 3 times as many heaps released in the same time. With that the test no longer needsMIMALLOC_GUARDED_SAMPLE_RATE=0in guarded builds (it passes at rates 1024, 64 and 1), so cmake runs it with the default rate like the other tests.What comes in from upstream
include/mimalloc-profile.h,src/sample-profile.c,src/sample-guarded.c,MI_PROFILE(default 1),mi_profiler_twithon_alloc/on_free,mi_profile,mi_heap_profile,mi_subproc_profileMI_STATis nowMI_STATS(default 1 in release),page->usedbecame the packedmi_used_t xused, statistics are folded in lazily by_mi_page_update_statsMI_PAGE_META_IS_ALIGNED):mi_freefinds the page without the page map; arenas and OS pages are 256 MiB aligned,arena->start,page->selfMI_ALLOW_THP=[OFF,ON,FULL](FULL by default on Linux), multi-choice cmake options (MI_DEBUG=,MI_TRACK=,MI_GUARDED=,MI_SECURE=; the old spellings still work),mi_theap_stats_get,mi_theap_stats_merge_to_heap,mi_option_collect_merges_stats,mi_cfreereturnsboolbun*branches and pull requestsRemoved: the fork's heap profiler
src/prof.c,mi_prof_enable/reset/dump/dump_buf/dump_to_file,mi_option_prof_sample_rate/MIMALLOC_PROF_SAMPLE_RATE/MIMALLOC_PROF_PATH,MI_PAGE_HAS_PROF_SAMPLES(page flags are 2 bits again), theprof_*theap/tld fields andtest-prof*. Upstream'smi_profiler_treplaces it; the pprof writer lives in the embedder from now on.mi_option_t:collect_merges_stats= 47 (upstream's slot),snapshot_on_exitmoves 47 -> 48,scavenger= 49 ...purge_holes_full_every= 53 unchanged,_mi_option_last= 54 unchanged. Nothing up toarena_is_numa_local= 46 moved.Exported symbols removed compared to
bun-dev3-v2(fromnmof the static libraries):mi_prof_dump,mi_prof_dump_buf,mi_prof_dump_to_file,mi_prof_enable,mi_prof_reset(+ internal_mi_prof_*,_mi_free_in_page,_mi_free_subproc_safe_in_page,_mi_theap_malloc_zero_ex,__mi_stat_counter_increase[_mt], and now_mi_cpu_movsb_max/_mi_cpu_stosb_max). No other public function changed signature exceptmi_cfree(void->bool).For Bun's bindings (checked again at this head against bun main): each function that
src/mimalloc_sys/mimalloc.rsdeclares, and the ones the C++ side declares (mi_stats_get_json,mi_heap_dump_json,mi_thread_set_in_threadpool,mi_is_in_heap_region,mi_process_info, ...), is still exported with a declaration identical to the one at Bun's current pin (6a64e1b).mi_heap_area_thas the same layout. Option numbers 0..46 are unchanged (Bun passesshow_errors= 0). Bun uses none ofmi_prof_*.src/static.ccompiles as C++20 with Bun's flags for Linux (release, debug, ASAN, musl flags), macOS arm64/x64 and Windows x64/arm64.Fork features kept
Heap delete/destroy protocol, lazy abandoned bitmaps, lazily zeroed
pages_meta(non-aligned configuration only now), hole purging, scavenger and park handoff, the purge protocol of #38/#39,mi_on_thread_idle*, fork handlers and lock order, per-mapping THP opt-out (MI_DEFAULT_ALLOW_THP=0), ProcessPrng seed, heap snapshot +mi-heapview, the macOS zone enumerator, and every fork test.arena_purge_multstays 1 (upstream went to 4 with its 1000 mspurge_delay; here the scavenger purges with 100 ms).Fixes that the combination needed
mi_heap_destroyof a heap that has an abandoned page with a block still on its thread-free list (another thread freed into a full page) reported "corrupted meta-data in thread-free list" and aborted in debug/secure builds: destroy resets the used count, and upstream's new collect inside the page free then sees more freed blocks than used ones. The delete/destroy walk folds the list in first. Upstream dev3 has the same bug (worth reporting there); new caseforeign-freesintest-heap-teardown.mi_heap_deletemay be merging that theap; TSAN intest-heap-mt)test-purge-holes)allow_reclaim && optioninmi_free_try_collect_mtmapped the option value -1 to 1Follow-up commits (fork bugs, present before the merge)
_mi_thread_donefreed the dynamic thread locals before_mi_park_leave(an artifact of the previous upstream merge)theap_meta_lockand then allocate the abandoned bitmap from itself: self-deadlock (test-stress-subprocshangs in Debug +MI_SECURE=FULL)atomic.h, which only models word-sized atomics. The park/scavenger state here uses 32-bit atomics, so cmake switches such builds to C++ (with a warning),atomic.hrefuses the C wrapper, and the CI drops the Windows C variants of the basic jobsNot changed, but worth knowing
_mi_page_profile_on_freereadspage->heapon a cross-thread free before it owns the page; harmless unless a profiler is attached andmi_heap_deleteruns concurrentlysrc/sample-guarded.c:143: gcc warns about the unused parameterpin a releaseMI_GUARDEDbuild (upstream commented out its only use outside an assertion)CPU: the malloc/free fast path (open item 1)
Method: 50 M iterations over 256 slots, a slot is freed if it is full and else filled with
mi_malloc(16 + r % 1024); 25 M mallocs, 25 M frees, one thread; user-mode instructions fromperf stat -e instructions:u, exact to 4 digits between runs; per-function counts from callgrind.-O3 -march=nehalemas C++bun-dev3-v2(961ac76)MI_STATS=1,MI_PROFILE=1)MI_STATS=0 MI_PROFILE=0MI_PROFILE=0MI_STATS=0 MI_PROFILE=0The 29 new upstream commits and #38/#39 change none of these numbers (the previous head read 2.5394 G / 2.3411 G too).
Where the increase came from (disassembly of
mi_freeandmi_malloc, both compilers agree; clang numbers, gcc in brackets):mi_free, local fast path: 23 -> 25 instructions (21 -> 24). Finding the page costs one instruction less and one load in place of three dependent ones (page0 = (p & ~256MiB) | index*176; page = page0->self, no page map). The used count cost three more:decl 0x10(%rdi); jebecamemov 0x18(%rdi),%rax; dec %rax; mov %rax,0x18(%rdi); test %ax,%ax; je, becauseusedis now the low 16 bits of the packed 64-bitmi_used_t xused(used, alloc count, last used, last alloc in one word).mi_malloc, small fast path: 19 -> 20 (16 -> 18).incl 0x10(%rcx)became load,add $0x10001, store onxused(+2; upstream puts a compiler barrier there so that the load is issued before the free list test), and themovq $0,(%rax)that clearedblock->nextis gone (-1)._mi_malloc_genericand below got cheaper: 113.6 M -> 102.6 M instructions with both switches off. With the defaultsmi_page_update_sample_countdown(24.6 M,MI_PROFILE) and the lazy statistics (MI_STATS) add to it.MI_STATS/MI_PROFILE/MI_GUARDEDare compiled in.The last commit keeps upstream's packed word and changes how the two fast paths update it:
mi_freedecrements the 16-bit used count in place (decw 0x18(%rdi); je, through theused_countmember that the union already had; little-endian only, the old code stays for the rest), andmi_mallocadds0x10001to the word in memory after the free list test (addq $0x10001,0x18(%rcx)) without the early load and the barrier.mi_freeis 22 instructions now andmi_malloc18, one belowbun-dev3-v2each.User-mode cycles on the same runs (shared, loaded machine, 7 interleaved runs pinned to 8 cores): 1.236 to 1.279 G for
bun-dev3-v2, 1.239 to 1.269 G for the merge before the last commit, 1.243 to 1.287 G with it. The three do not differ in cycles on this benchmark; the instruction counts do.MI_PROFILE=0is worth another 1.4% of instructions if Bun does not attach a profiler,MI_STATS=0nothing measurable.Testing
Linux x64 at 5af105d,
ctest100% in each of: Release (30 tests), DebugMI_DEBUG=FULL(31), ASAN (28), TSAN (25), UBSAN (28),MI_SECURE=ON(30),MI_SECURE=FULLrelease (30) and debug (31),MI_GUARDED=FULLrelease (30), C++ (clang++) debug (32) and release (31),MI_FREE_USE_PAGEMAPdebug (31),MI_PROFILE=0release (29),MI_STATS=0 MI_PROFILE=0release (29). That includestest-park-handoff(+-no-scavenger,-eager),test-heap-teardownwithforeign-frees,test-forced-purgeandtest-purge-behind-pass.GitHub CI: all 23 jobs passed at b5bc53f (the merges, the follow-ups and the test changes; run 34751790726) at 93eb9c7 (with the fork change; run 34753001747) and at 5af105d (the head; run 34754401015). The run of fe2a5e5 before them was cut short by the next push with 12 jobs passed and 1 failed: the macOS Debug SIGBUS of open item 2. Extra runs of the macOS jobs on 93eb9c7 and later: see open item 2.
Open item 2:
test-park-handoff"fork while parked" on macOS DebugIt failed intermittently on
macos-latestDebug builds (the forked child died with SIGBUS) in two CI runs of the earlier head (4 of 6 executions) and in none of the ~30 executions around them. It is not reproducible on Linux (30/30 with the same settings).It fired again in the first CI run of this refresh (run 34751251554,
basic, macos-latest, cxx, Debug, round 7), and this time the diagnostics that the test got in e98ec96 printed where:So the child does not fault in a free list that the interrupted hole sweep was rewriting, which is what the test was written for. It faults on the first write into a page that it just allocated from the arena, at a slice aligned address: the arena handed out a range that its bitmaps call free and committed, and the memory is not writable. On macOS a write to a
PROT_NONEpage is a SIGBUS. The only thing that makes whole arena slicesPROT_NONEis the decommit of an arena purge, and only withMI_DEBUGorMI_SECURE(_mi_prim_decommit:mprotect(PROT_NONE), a release build only callsmadvise). That fits "Debug only". It also fits the timing: a park ends in_mi_arenas_purge_now, which pulls every scheduled arena purge forward, so the scavenger is in an arena purge pass when this test forks 150 to 550 us after the park. The prepare handler took the forking thread out of its park and took every lock, but a purge pass takes no lock, so it ran across thefork().A pass claims a free range, decommits it, clears its commit bits and releases it. A child that gets a consistent copy of that sees the range claimed (and leaks it), which is harmless. To see it free and committed and
PROT_NONEthe child must get the bitmaps (at the start of the arena) from before the pass reached the range and the protection from after. My reading is that the kernel copies the address space entry by entry and can let anmprotectfrom another thread in between; I have not proven that, and none of it can be run here.Change in this PR (the last commit): the fork prepare handler takes the purge guard, last, under its locks (a pass takes none of them and waits for no one, so the one in progress ends); parent and child release it. No purge pass runs across a
fork()any more, which removes this whole class whatever the exact interleaving was. The guard became a file-scope variable in #38, which is what makes this possible. The child side of #39 (release the guard of the dead purger, make a pass due) goes away with it: the fork holds the guard, so no pass is cut off, and eachpurge_expireis as a finished pass left it.test-purge-behind-pass"a fork in a pass" still passes: the fork now waits for the pass, and the child purges the arena that was not yet due when its time comes.What is confirmed and what is not: the fault location and the missing quiescence are facts. That this was the cause is an inference from elimination. Evidence since the change: before it 3 of about 7 complete CI runs of this branch failed on it (two at the earlier head, one at fe2a5e5). With it, the macOS arm64 jobs (
macos-latestc/cxx/extra,macos-14c/cxx; each runs the Debug test in three variants) passed in both pull request runs (93eb9c7, 5af105d) and in 7 extraworkflow_dispatchruns of the same heads: 45 job runs, no failure; the threemacos-15-inteljobs passed in each of the 3 runs where they were waited for.What else this refresh changes on that path: none of the 29 upstream commits touches
src/prim/,subproc.c,scavenger.c,theap.cor the fork handlers. 636510a changes the layout of small pages only whereMI_PAGE_META_SMALL_IS_ALIGNEDis set, which Debug builds do not (they are guarded). Upstream's pr 1393 only matters when a thread ends.Seen on the way, not changed
FreeLibrary:mi_process_doneruns under the loader lock,_mi_scavenger_stopwaits for the scavenger thread withWaitForSingleObject(INFINITE), and that thread needs the loader lock to exit (DLL_THREAD_DETACH).ExitProcess(the other threads are gone by then) andMI_NO_PROCESS_DETACHbuilds such as Bun's are not affected. Same code onbun-dev3-v2. Found by reading, not run.