Update mimalloc to the upstream dev3 (v3.4.3) sync - #36431
Conversation
Pins mimalloc at oven-sh/mimalloc#11, which merges upstream microsoft/mimalloc dev3 (v3.4.2 + v3.4.3 + follow-ups) into our fork: the thread-init/TLS rework, the audit fixes (usable size on aligned blocks, realloc_aligned alignment check, heap delete for OS-allocated pages), and the arm64/apple codegen improvements. The fork keeps its macOS fixed TLS slots (96/97), the scavenger/idle-sweep, hole purging and the Darwin MADV_FREE_REUSE skip. Also updates the process.versions expectation to the new commit.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 45 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (3)
WalkthroughChangesmimalloc revision update
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Found 6 issues this PR may fix:
🤖 Generated with Claude Code |
Windows compiles mimalloc as C++ (clang-cl -x c++); upstream's new file-scope atomic used a `= NULL` copy-initializer, which does not compile there. Points at the fork commit that drops the redundant initializer.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| import type { Dependency, DirectBuild } from "../source.ts"; | ||
|
|
||
| const MIMALLOC_COMMIT = "acd9924a0af3ba7c341910b48815106f2944ffa0"; | ||
| const MIMALLOC_COMMIT = "0e6e51895669632b0610effafa96a85a283c6933"; |
There was a problem hiding this comment.
🟡 The PR description lists -mno-outline on apple as an upstream perf change picked up, but that flag lives in upstream's CMakeLists.txt and Bun's mimalloc dep uses kind: "direct", which bypasses upstream cmake entirely — the cflags array here doesn't add it, and there's no repo-wide -mno-outline in flags.ts either. Either add if (cfg.darwin && cfg.arch === "aarch64") cflags.push("-mno-outline") to match upstream's gate, or drop the claim from the description. Worth diffing upstream's CMakeLists.txt between the old and new pins for any other new cmake-gated defines/flags introduced in the dev3 sync.
Extended reasoning...
What the finding is
The PR description's "upstream change picked up | why we care" table lists:
arm64
mi_freecodegen, faster ptr→page check,-mno-outlineon apple | perf
Of those three items, the first two are source-level changes in mimalloc's .c/.h files and are picked up automatically by bumping MIMALLOC_COMMIT. But -mno-outline is a compiler flag that upstream added to its CMakeLists.txt for the apple/arm64 target. Bun's mimalloc build never consults that file.
Why the flag is not applied
scripts/build/deps/mimalloc.ts returns a spec with kind: "direct". Per scripts/build/CLAUDE.md, a DirectBuild "skips a sub-process configure entirely" — upstream's CMakeLists.txt is never invoked. The compiler command line is exactly computeDepFlags(cfg) (the global flag table in scripts/build/flags.ts) plus the cflags array in this file.
The cflags array in this file contains only:
-fvisibility=hidden-Wno-deprecated,-Wno-static-in-inline-DMI_CMAKE_BUILD_TYPE=...-ftls-model=...(non-Windows)-fno-builtin-malloc(when overriding)-w(Windows)
No -mno-outline. A repo-wide grep for mno-outline / no-outline returns zero hits — it's not in flags.ts's global tables either. So on apple/arm64, mimalloc's src/static.c compiles without the flag.
Step-by-step trace
mimalloc.build(cfg)is called withcfg.darwin === true, cfg.arch === "aarch64".- It builds the
cflagsarray shown above; the darwin/arm64 branch adds nothing extra (the only platform-gated cflags pushes are!cfg.windowsfor-ftls-model,overridefor-fno-builtin-malloc, andcfg.windowsfor-w). source.tscompilessrc/static.cwith[...computeDepFlags(cfg), ...spec.cflags].computeDepFlagsinflags.tshas no outline-related entries.- Upstream's
CMakeLists.txt— where the-mno-outlineaddition lives — is never read by the DirectBuild path. - Result: the flag is not on the clang command line for the apple/arm64 mimalloc build, so the "perf" item advertised in the PR description is not actually applied.
Why existing code doesn't cover it
scripts/build/deps/README.md explicitly documents this responsibility for direct deps: "For direct deps: the source list is hardcoded, so a bump that adds or removes a .c/.cpp upstream needs a matching list edit here … the cue to diff the upstream CMakeLists.txt." That guidance applies equally to new cmake-gated compiler flags/defines — the DirectBuild spec must hand-replicate them, and this bump did not.
Impact
Nothing breaks — mimalloc compiles and runs correctly without -mno-outline; the function-outlining pass just isn't disabled, so the arm64 mi_free codegen micro-optimization upstream intended is partially left on the table on macOS/arm64. The concrete problem is that the PR description advertises a perf win that isn't in the build, which .claude/docs/landing-prs.md §Docs flags: "Don't publish claims you haven't verified — verify, scope down, or drop them."
Fix
Either replicate the flag in the DirectBuild spec (matching upstream's gate), e.g.:
// Match upstream CMakeLists.txt: disable clang's machine outliner on apple/arm64
// so the mi_free hot path doesn't get a function-call detour.
if (cfg.darwin && cfg.arch === "aarch64") cflags.push("-mno-outline");…or drop the -mno-outline on apple mention from the PR description. It's also worth diffing upstream's CMakeLists.txt between the old and new pins for any other new cmake-gated defines introduced in the dev3 sync (e.g. anything the prim-tls.h rework gates on) that would similarly need hand-replicating here.
mimalloc's default VM tag (100) is VM_MEMORY_IOACCELERATOR, so vmmap and profilers built on it attributed all of Bun's heap to the GPU driver. The fork now defaults to 240 (VM_MEMORY_APPLICATION_SPECIFIC_1), which is in the range mach reserves for applications; the memory shows up as "Memory Tag 240". Adds a vmmap-based test so a future mimalloc bump cannot regress it.
| import type { Dependency, DirectBuild } from "../source.ts"; | ||
|
|
||
| const MIMALLOC_COMMIT = "acd9924a0af3ba7c341910b48815106f2944ffa0"; | ||
| const MIMALLOC_COMMIT = "aba864aa275982ec4171711dd566c12ada8cca99"; |
There was a problem hiding this comment.
🟣 Pre-existing (dead at the old pin too, not introduced by this bump): defines.MI_NO_SET_VMA_NAME = 1 at line 71 and its 8-line comment claiming it "Skip[s] prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, ...) after each mmap" do nothing — the mimalloc source at both acd9924a and aba864aa never reads MI_NO_SET_VMA_NAME; src/prim/unix/prim.c:310 gates the prctl solely on #if defined(__linux__) && defined(PR_SET_VMA), and glibc force-defines PR_SET_VMA via the <linux/prctl.h> fallback at lines 36-38, so the prctl still fires on every mmap. Since a dep bump is the documented re-audit point for build defines: either add a fork patch gating the prctl on !defined(MI_NO_SET_VMA_NAME), or delete the define + comment (and the matching flag in scripts/bench-json-rust.sh:37).
Extended reasoning...
What the finding is
scripts/build/deps/mimalloc.ts:71 sets defines.MI_NO_SET_VMA_NAME = 1 on Linux, and the 8-line comment above it (lines 63-70) claims this will "Skip prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, ...) after each mmap" because the syscall is "pure syscall overhead on the startup path" on kernels without CONFIG_ANON_VMA_NAME. But the macro is never read by mimalloc at either the old pin (acd9924a) or the new pin (aba864aa), so the prctl call is still compiled in and still fires on every mmap on Linux/glibc. The define is dead code and the comment describes behavior that does not exist.
The specific code path
At the new pin aba864aa (and identically at the old pin), src/prim/unix/prim.c gates the VMA-naming prctl like this:
- Lines 34-38: on glibc, if
PR_SET_VMAis not already defined by<sys/prctl.h>, mimalloc#include <linux/prctl.h>as a fallback. This meansPR_SET_VMAis always defined on Linux/glibc regardless of toolchain. - Line 310:
#if defined(__linux__) && defined(PR_SET_VMA)— the only guard aroundprctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, start, size, name). There is no&& !defined(MI_NO_SET_VMA_NAME)clause.
A grep across every mimalloc source file, header, and CMakeLists.txt at both pins (and at bun-dev3 tip ffa38ab8) returns zero hits for MI_NO_SET_VMA_NAME. Repo-wide, the only two occurrences are the setter in mimalloc.ts:71 and a mirror in scripts/bench-json-rust.sh:37 — both write the macro, nothing reads it.
Why existing code doesn't prevent it
The mimalloc dep spec has no patches: entry, so nothing injects an #if !defined(MI_NO_SET_VMA_NAME) guard into prim.c at build time. The build is kind: "direct" compiling src/static.c verbatim from the fork tarball — whatever preprocessor guards exist in the fetched source are exactly what compile. Since the fork source never references the macro, passing -DMI_NO_SET_VMA_NAME=1 on the command line is a no-op.
Step-by-step proof
mimalloc.build(cfg)is called withcfg.linux === true.- Line 71 sets
defines.MI_NO_SET_VMA_NAME = 1, which becomes-DMI_NO_SET_VMA_NAME=1on the clang command line. src/static.cis compiled, which#includessrc/prim/prim.c→src/prim/unix/prim.c.- At
prim.c:34-38,__linux__is defined and__GLIBC__is defined; if<sys/prctl.h>didn't definePR_SET_VMA,<linux/prctl.h>is included and does. Either way,PR_SET_VMAis now defined. - At
prim.c:310, the guard#if defined(__linux__) && defined(PR_SET_VMA)evaluates true.MI_NO_SET_VMA_NAMEis not consulted. - The
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, ...)call is compiled into the object file. - At runtime, every
unix_mmap_primcall issues the prctl — exactly the syscall the comment claims is skipped.
You can confirm empirically: strace -e trace=prctl bun-debug -e 1 2>&1 | grep PR_SET_VMA on a glibc Linux box will show the calls.
Impact
The runtime cost is minor (one extra syscall per mmap, typically returning EINVAL on kernels < 5.17). The real problem is what the repo's own review guidance flags: per landing-prs.md §Docs, "a comment contradicting the code is a correctness bug, not a nit" — a future maintainer reading this comment will believe the prctl is disabled and won't investigate startup-path syscall counts, or will "fix" something else based on a false premise. And per §Dependencies & vendoring, a dep bump is the point at which build defines should be re-audited against the vendored source ("rebase every local patch and verify"), which is why this is worth surfacing on this PR even though it predates it.
How to fix
Two options:
- Make the define real: add a fork patch (or commit to
oven-sh/mimalloc) that changesprim.c:310to#if defined(__linux__) && defined(PR_SET_VMA) && !defined(MI_NO_SET_VMA_NAME). This makes the comment true and actually saves the syscall. - Delete the dead code: remove
mimalloc.ts:63-71(the comment block + the define) and drop-DMI_NO_SET_VMA_NAME=1fromscripts/bench-json-rust.sh:37.
Severity
Pre-existing. The define was already dead at the old pin acd9924a; this PR's commit bump does not change its status in either direction. Not blocking for this PR — but worth cleaning up while the mimalloc build spec is being touched.
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ... # Conflicts: # src/jsc/bindings/BunDebugger.cpp
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ...
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ... # Conflicts: # src/js/internal/debugger.ts
* upstream/main: (422 commits) install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681) Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431) compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430) Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849) test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424) test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429) Deflake a few tests no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414) GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356) exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383) FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250) test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919) test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166) test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413) fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187) event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703) dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199) fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145) bundler: don't panic on unterminated naming template placeholders (oven-sh#36325) Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420) ...
…ailure (#37915) ### Problem - Build lanes die in a dep fetch: `error: Failed to download after 5 attempts: https://github.com/oven-sh/lol-html/archive/725ce499....tar.gz`, `cause: fetch failed`. Build 93382 lost linux x64-asan on lolhtml. Build 93392 (a one-file PR) lost four lanes: x64-musl and x64-android on cares, mimalloc and the WebKit tarball, freebsd aarch64 on lolhtml, windows aarch64 on cares, libuv, mimalloc and WebKit. - Those are the downloads that miss the image's prefetch cache (everything else in the same logs says `using prefetch cache`): the deps whose pins moved after the images were baked in #34782 (lolhtml #36733, mimalloc #36431, c-ares #34007, libuv #36839, WebKit several times a week), plus WebKit on every lane other than linux arm64, because `prefetch-deps.ts` only enumerates the bake host's own target (handed off separately). Each build makes on the order of a hundred live github.com downloads. - `downloadWithRetry` (`scripts/build/download.ts:156`) made 5 attempts with 2+4+8+16s of backoff, about 30s in total. The logs show agent-wide outages longer than that: on the x64-musl lane three downloads that started together failed on all five attempts over roughly two minutes, after lolhtml had succeeded on the same agent seconds earlier; on the freebsd lane lolhtml failed five times in a row while cares and mimalloc went through and WebKit succeeded on its second try. - `BuildError.format()` (`scripts/build/error.ts:33`) prints one level of cause. node's fetch throws `TypeError: fetch failed` and keeps the real error (DNS, connect timeout, reset) in `.cause`, so every one of these logs says only `cause: fetch failed`, and the retry lines say nothing about what failed. ### Fix - `downloadRetry`: 10 attempts, backoff doubling from 2s and capped at 30s, 180s of backoff in total instead of 30s. Exported as a `RetryPolicy` (optional last parameter of `downloadWithRetry`) so the test can run the production attempt count with the backoff zeroed. - 408 and 429 are retried along with 5xx and network errors. Other 4xx still fail on the first attempt and are still thrown unwrapped, which `prefetch-deps.ts` relies on to tell a 404 (variant not published) from a transient failure. - Each retry line names the failure it is retrying, e.g. `retry 2/10 in 2000ms (fetch failed: other side closed)`, and `format()` prints the whole cause chain through the new `describeError()`, so the next one of these says what the network did. - Why here: the prefetch cache goes stale by design as soon as a pin moves, and WebKit moves faster than images get rebaked (the images were rebaked on July 21 and this came back within two weeks; #30095 was an earlier rebake for the same symptom), so the live path is permanently on every build's critical path and has to outlast the outages CI actually sees. The wider window only costs time while github.com is actually down, when the lane would otherwise have failed; build-bun's step timeout is 60 minutes. - Verified with `test/internal/build-download-retry.test.ts` (`bun bd test`, 7 pass). It drives the real `downloadWithRetry` against a local server that drops connections or returns scripted statuses, checks the retry lines and `format()` output, and pins the schedule's total backoff at two minutes or more. Against the previous `download.ts`/`error.ts` the file fails at import (`downloadRetry` and `describeError` did not exist); each behavioral case is something the old loop did not do (10 attempts, 429 retried, reason on the retry line, cause chain in `format()`). - Also ran the loop under node, the runtime CI builds with, against a dropping server: retry lines read `(fetch failed: other side closed)` and the final report prints `cause: fetch failed: other side closed`. `bunx tsc -p scripts/build/tsconfig.json` reports nothing for these files. ### Background - Dep fetching: configure emits one ninja `dep_fetch` edge per vendored dep, which runs `scripts/build/fetch-cli.ts`; that calls `downloadWithRetry` on `https://github.com/<repo>/archive/<commit>.tar.gz`, and `fetchPrebuilt` uses the same function for release tarballs such as WebKit. Both URL kinds start with a 302 from github.com (to codeload.github.com and objects.githubusercontent.com respectively), which is why one github.com problem takes out both kinds at once. - Prefetch cache: CI images run `scripts/prefetch-deps.ts` at bake time, storing each tarball under `/opt/bun-prefetch/by-url/<sha256(url)>`. `downloadWithRetry` looks there before touching the network, so a dep is served from the image only while its pinned URL is the one that was current at bake time; anything bumped later downloads live until the next `[publish images]` rebake. - `BuildError` is the build system's error type; `fetch-cli.ts` and `build.ts` print failures through its `format()`, which produces the `error:` / `hint:` / `cause:` lines seen in the build log.
…ap labels (#40044) ### Problem - `heapStats-mimalloc.test.ts` > `arena memory is tagged as application memory, not IOAccelerator` fails on macOS 27 beta (26A5416b): `expect(appTag).toBeGreaterThan(64)`, `Received: 0`. It passes on macOS 14, 15 and 26. Build 103104, lane `darwin 27 aarch64`. - The test sums `vmmap --summary` lines that start with `Memory Tag 24` / `Memory Tag 25`. vmmap on macOS 27 prints tag 240 as `App-Specific Tag 1` (255 as `App-Specific Tag 16`), so nothing matches. The kernel still stores tag 240 on every mimalloc mapping. Bun has no regression. ### Fix - The child reads the tags from the kernel with `mach_vm_region(VM_REGION_EXTENDED_INFO)` through `bun:ffi` and sums the region sizes per `user_tag`. `ioaccelerator` is the total for tag 100, `appTag` the total for tags 240 to 255. The assertions do not change. - This is the same quantity as before (vmmap's VIRTUAL column is the sum of region sizes per tag), without the label table that Apple renames between releases. - Verified on the macOS 27 box with bun 1.4.0: the same walk reports tag 240 at 1042.4 MB, tag 100 at 0 MB. On Linux the test is skipped. The other four tests in the file pass with `bun bd test`. ### Background - A VM tag is an 8-bit label on each kernel map entry. On Darwin, `mmap` takes it in the `fd` argument as `VM_MAKE_TAG(tag)` when `MAP_ANON` is set. mimalloc passes 240 (`VM_MEMORY_APPLICATION_SPECIFIC_1`) since #36431. The old value, 100, is `VM_MEMORY_IOACCELERATOR`, so Instruments showed Bun's heap as GPU memory. - `vmmap` turns tags into display names with a private table. `mach_vm_region` returns the raw tag in `vm_region_extended_info.user_tag`, the second 32-bit field of the struct. Fixes #40037 <details><summary>Notes</summary> Data from the macOS 27 box (`darwin-arm64-bingus`, xnu-13432.1.9~3), collected with a plain C probe and with the same probe through `bun:ffi` in bun 1.4.0. Kernel view, one mapping per tag through `mmap(fd=VM_MAKE_TAG(tag))` and `mach_vm_allocate(VM_FLAGS_ANYWHERE | VM_MAKE_TAG(tag))`, read back with `mach_vm_region`: ``` mmap requested=100 stored=100 mmap requested=240 stored=240 mach_vm_allocate requested=240 stored=240 ... mmap requested=255 stored=255 ``` vmmap labels on macOS 27 for the same mappings: ``` App-Specific Tag 1 .. 16 tags 240 .. 255 (macOS 26: "Memory Tag 240" ..) Rosetta Tag 10 tag 239 VM_MEMORY_150 tag 150 (macOS 26: "Memory Tag 150") Sanitizer tag 99 IOAccelerator tag 100 (unchanged) Untagged tag 0 (macOS 26: "VM_ALLOCATE") ``` bun 1.4.0 on macOS 27, `vmmap --summary` of the test's child (96 x 1 MiB latin1 strings kept alive): ``` App-Specific Tag 1 260.4M 146.9M 121.2M ... 9 App-Specific Tag 1 (reserved) 768.0M 0K 0K ... 1 reserved VM address space (unallocated) ``` The same process, kernel walk via `bun:ffi` (what the test now does): ``` tag 100: (absent) tag 240: virtual= 1042.4M resident= 146.9M regions=17 ``` bun 1.3.13 (before the tag change) on macOS 27 shows the arena as `IOAccelerator 132.3M` plus `IOAccelerator (reserved) 896.0M`, so the `ioaccelerator` assertion still catches the old default on 27. macOS 27's vmmap also retitles the other rows (`Malloc Small`, `Stack Guard`, `Guard`, `OS Alloc Once`) and splits unallocated address space into `<label> (reserved)` rows. None of that matters once the test stops parsing the summary. The gate cannot exercise this test: it is `skipIf(!isMacOS)` and there is no `src/` change. The darwin CI lanes run it on macOS 14, 15 and 26. The macOS 27 lane from #40019 runs it once that PR lands. </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/jsc/heapStats-mimalloc.test.ts <!-- robobun:evidence:end -->
1 participant
Pins mimalloc at oven-sh/mimalloc#11 — a merge of upstream
microsoft/mimallocdev3 (v3.4.3+) into our fork. Fork intents (macOS fixed TLS slots 96/97, scavenger + idle-sweep handoff, hole purging, DarwinMADV_FREE_REUSEskip) are preserved; see #11 for the conflict-resolution table.mi_usable_sizeon over-aligned blocksrealloc_alignedalignment check / power-of-two guardmi_realloc_alignedprim-tls.h)mi_freecodegen, faster ptr→page check,-mno-outlineon appleAlso updates
process.versions.mimallocinprocess.test.jsto the new commit.MI_DEBUG_FULL) / releasetheap-sentinel,park-handoff×3,purge-holes,heap-churn)purge_delay)bun bdon the new pinserve-body-leak.test.ts8/8,heap-snapshot.test.ts9/9, serve/worker/spawn/install/bundler smoke clean