Skip to content

windows: commit one page of stack per thread and check the stack sizes of the linked exe - #45094

Open
robobun wants to merge 5 commits into
mainfrom
robobun/ca5656d7/windows-stack-commit
Open

robobun wants to merge 5 commits into
mainfrom
robobun/ca5656d7/windows-stack-commit

Conversation

@robobun

@robobun robobun commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • On Windows, a bun process near the commit limit ends when JSC starts a helper thread: WTFCrashWithInfo under WTF::Thread::create, exit code 0xC0000409 (561 Sentry events in 90 days).
  • The link flag /STACK:0x1200000,0x200000 (scripts/build/flags.ts:1020) sets SizeOfStackCommit. Windows charges it when any thread starts, and refuses the thread when it cannot.

Fix

  • Link with /STACK:0x1200000,0x1000, as node.exe and WebKit do. A thread start commits 28 to 57 KB, not 2.1 MB.
  • verify-binary.ts checks both header fields of bun.exe and the .bin shim on every CI link.
  • The MemoryPressure watcher and the IO pool reserve the 2 MiB the old commit gave them.
  • Verified: test/internal/source-lints/windows-cross-config.test.ts, and on Windows test/js/bun/binary/tls-segment-size.test.ts. Both fail with the old flag.

Background

  • A Windows stack has a reserve (address space) and a commit (pages charged against RAM plus page file). The commit grows as the stack is touched.
  • WTF, Rust std, mimalloc and the OS take the commit from the header. No call site can lower it.
  • Considered a commit size in WTF's thread creation: it covers 1 of 7 thread creators.

Downsides

Notes

How the Windows numbers were taken. On the CI test machines (Windows Server 2019 x64 and Windows 11 arm64, 4 cores each, build 124918). "4k" is the bun.exe of this PR. "2m" is a copy of it with the 8 bytes of SizeOfStackCommit set back to 0x200000: the same code, linked the way main links it. On a stub, a link with ,0x200000 and this edit give the same file.

Commit per thread start (32 threads parked in Sleep, PrivateUsage delta / 32):

2m 4k
x64, CreateThread size 0 2,151,680 28,800
x64, 18 MB reserve with STACK_SIZE_PARAM_IS_A_RESERVATION 2,149,760 28,800
arm64, size 0 2,127,488 55,936
arm64, 18 MB reserve 2,125,952 56,960

The test in this PR printed 50,176 / 44,416 (x64) and 45,952 / 49,920 (arm64).

What a thread gets (reserve KB / initial commit KB, from the TEB of a thread created suspended, same on x64 and arm64):

request 2m 4k
size 0 18432 / 2048 18432 / 4
reserve 64 KB 2048 / 2048 64 / 4
reserve 512 KB 2048 / 2048 512 / 4
reserve 2 MB 2048 / 2048 2048 / 4
reserve 18 MB 18432 / 2048 18432 / 4

Under the 2 MB commit, a thread that asked for 2 MB or less got a 2 MB stack that was committed in full, with no guard page. That covers the MemoryPressure watcher, the IO pool and every Rust thread with the default size. They have a guard page again.

The reported failure, under a job object commit limit. The process limits itself to its current commit plus a headroom.

  • Start 8 threads with size 0. 2m: 0 start at 256 KB, 1 MB and 2 MB, 1 at 4 MB, 3 at 8 MB, 7 at 16 MB, 8 at 32 MB. The refusal is error 1455, the code in the HTTP thread panics of Report a refused HTTP, bundle, IO or process waiter thread as an error instead of a crash #39864. 4k: 8 start at every headroom.
  • Collect, stay idle 25 s (the GC helper threads exit), set the limit, collect again. Smallest headroom that survives: x64 8 MB (2m), 2 MB (4k). arm64 4 MB (2m), 1 MB (4k). Below it the process ends with exit byte 9 or 3. Bun.spawn reports the low byte of a Windows exit code: 9 is 0xC0000409, the code of the report, and 3 is bun's crash handler.

Idle Bun.serve commit, MB (PrivateUsage; at start / 5 s / 35 s / after one fetch / after a GC):

  • x64: 106.0 / 98.2 / 91.3 / 102.1 / 104.6 (2m), 80.7 / 76.2 / 75.5 / 82.3 / 82.7 (4k)
  • arm64: 101.4 / 95.5 / 86.8 / 97.5 / 99.9 (2m), 85.5 / 80.3 / 79.3 / 86.2 / 86.7 (4k)

Startup (CreateProcessW to exit, median of interleaved runs. "noise" is a second copy of 2m against 2m.)

2m ms 4k ms delta noise page faults peak commit KB
x64 --version (20) 15.73 15.67 -0.06 +0.26 774 to 770 10,760 to 6,672
x64 -e 0 (30) 25.79 26.25 +0.46 +0.05 3,549 to 3,568 57,042 to 52,948
x64 empty.js (40) 26.05 26.85 +0.80 -0.06 3,488 to 3,509 56,448 to 52,352
x64 16 Workers (8) 85.21 98.53 +13.32 -3.54 30,650 to 30,653 472,552 to 459,430
arm64 --version (20) 21.21 20.87 -0.34 -0.05 1,448 to 1,444 10,942 to 6,852
arm64 -e 0 (30) 30.36 30.59 +0.23 -0.01 4,247 to 4,265 57,248 to 53,152
arm64 empty.js (40) 30.11 30.26 +0.15 +0.13 4,194 to 4,212 56,636 to 52,540
arm64 16 Workers (8) 72.19 79.17 +6.98 +0.81 31,677 to 31,631 379,948 to 348,064

The cost is per JS thread and comes from JSC's pre-commit loop (preCommitStackMemory). Each of the first 512 pages was a plain page fault and is now a guard-page move. The number of faults does not change. A process that starts no VM (--version) does not pay.

Regexp frames on JIT threads. DFGStrengthReductionPhase runs RegExp::matchConcurrently on a JIT worklist thread when the pattern and the subject are constants. The Yarr prologue moves the stack pointer by the whole frame without a probe, and nothing commits a JIT thread's stack ahead of it. Script: /^(a?){N}b$/ written out, .test("aaab") in a loop of 30,000, run as a file.

frame 2m 4k
5.9 KB, 10 KB, 15.6 KB pass pass
31.3 KB, 46.9 KB, 125 KB pass exit byte 5 (0xC0000005)
125 KB, subject not constant pass pass

Same on x64 and arm64. Upstream JSC has the same gap with its own one-page header. A commit of 64 KB in the header was the fallback if a frame of real size had failed. It did not. The fix in JSC (a limit from the committed stack, or a probe in the Yarr prologue) is separate work.

Merge order. The regexp crash goes away with a fix in JSC (a probe in the Yarr prologue, or a stack limit taken from the committed stack). After the WebKit bump that carries it, this PR adds no new failure. Before it, the window is a folded match with a Yarr frame above about 16 KB.

Other checks.

  • Linked header in CI: bun-profile 18874368 bytes of stack reserved, 4096 committed per thread, bun-shim-impl 1048576 bytes of stack reserved, 4096 committed per thread, on the x64 and the aarch64 build.
  • Binary size: 0 bytes. A stub linked with the old and the new flag differs in 2 bytes, both in SizeOfStackCommit. /STACK:0x1200000 and /STACK:0x1200000,0x1000 give the same file.
  • bun build --compile keeps the template's stack fields (checked with stub templates: 4096 in gives 4096 out, 2097152 in gives 2097152 out). An executable compiled from an older downloaded template keeps 2 MB.
  • RangeError depth of one recursive function: 53,199 frames (x64) and 53,206 (arm64) with both headers. Runs where the JIT tiers up on the way go deeper with both.
  • verify-binary.ts on a stub linked with the old flag: SizeOfStackCommit 2097152, expected 4096, exit 1. It starts 3 child processes before and after.
  • Windows test lanes of build 124918: no failure other than the two measurement files, which failed on purpose and are removed.

Sentry. WTFCrashWithInfo under WTF::Thread::create under WTF::AutomaticThread::start: BUN-2V38 (GC helper pool, from JSC::Heap::runBeginPhase) and BUN-2V2T (JIT worklist). The helper threads exit when idle and start again, so an idle process dies at a GC timer tick.

Other designs. Only surviving a refused start (oven-sh/WebKit#569, #39864) is needed too, but the 2 MB request stays on every thread. A commit size in WTF's Thread::create leaves the main thread, Rust std, mimalloc, zstd, sqlite and the OS's threads at 2 MB.

History. The commit operand has no recorded reason. It was 2 MB in the first Windows build work, 1 MB in #6938, and 2 MB again in #15985 by a commit named "Double the thread stack size on Windows". The operand is the initial commit. The stack size (the reserve) stayed 18 MB.

Related, not in this PR.


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/binary/tls-segment-size.test.ts, test/internal/source-lints/windows-cross-config.test.ts

robobun and others added 2 commits October 12, 2026 21:58
bun.exe linked with /STACK:0x1200000,0x200000. The second operand is
SizeOfStackCommit in the PE header. Windows charges it against the system
commit limit when a thread starts, for every thread that passes no commit
size of its own, and does not start a thread it cannot charge it for. That
is every WTF thread, every Rust std thread, mimalloc, zstd, sqlite and the
threads the OS adds to the process. JSC's GC helper and JIT threads exit
when idle and start again later, and WTF aborts when one does not start.

- Link with /STACK:0x1200000,0x1000. One page is the linker's default, and
  what node.exe and WebKit's own executables ship. The stack is committed
  as it is touched.
- verify-binary.ts checks SizeOfStackReserve and SizeOfStackCommit of
  bun.exe and of the .bin shim against binary-expectations.ts.
- The MemoryPressure watcher and the IO pool reserve 2 MiB on Windows.
  They asked for 64 KB and 512 KB, less than the old commit size.

Co-authored-by: Shelley Vohr <2036040+codebytere@users.noreply.github.com>
Two test files that fail on purpose on Windows and print measurements:
commit per thread start, real stack sizes, startup time and page faults,
an idle server, a process at a commit limit, and unprobed regexp frames
on JIT threads. Each compares the executable under test with a copy whose
SizeOfStackCommit is set back to 2 MB. To be removed before this PR is
ready.
@robobun

robobun commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:24 AM PT - Oct 15th, 2026

✅ @robobun, your commit e083680107c81183d924a98949618e60a6ada95f passed in Build #125870! 🎉


🧪   To try this PR locally:

bunx bun-pr 45094

That installs a local version of the PR into your bun-45094 executable, so you can run:

bun-45094 --bun

@robobun

robobun commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for review.

  • Reproduction: on the Windows test machines (Server 2019 x64, Windows 11 arm64), with the bun.exe of this PR and a copy of it whose SizeOfStackCommit is set back to 2 MB. Each process limits its own commit with a job object.
    • The 2 MB copy starts no thread with 2 MB of headroom (error 1455) and 1 of 8 with 4 MB. This PR's exe starts 8 of 8 with 256 KB.
    • After 25 idle seconds a collection has to start the GC helper threads again. The 2 MB copy ends (exit byte 9, the low byte of 0xC0000409) unless it has 8 MB of headroom on x64 and 4 MB on arm64. This PR's exe needs 2 MB and 1 MB.
  • The measurement files are removed. The numbers are in the description.
  • PR: windows: commit one page of stack per thread and check the stack sizes of the linked exe #45094

…to small stacks

The two stack-commit-measure test files did their work on the Windows
test machines (Server 2019 x64, Windows 11 arm64). The numbers are in the
PR description.

One of them goes into the comment on WINDOWS_STACK_COMMIT: under the 2 MB
commit, a thread that asked for a 64 KB, 512 KB or 2 MB reserve got a
2 MB stack that was committed in full, with no guard page.
…ows-stack-commit

# Conflicts:
#	scripts/build/verify-binary.ts
Comment thread src/threading/io_thread_pool.rs Outdated
@robobun
robobun marked this pull request as ready for review October 15, 2026 11:46
@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: oven-sh/bun/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: b07103c1-2a64-4aa1-858e-56bb48047cc1
📥 Commits

Reviewing files that changed from the base of the PR and between 5f5799e and e083680.

📒 Files selected for processing: 8
  • scripts/build/CLAUDE.md
  • scripts/build/binary-expectations.ts
  • scripts/build/flags.ts
  • scripts/build/verify-binary.ts
  • src/runtime/node/memory_pressure.rs
  • src/threading/io_thread_pool.rs
  • test/internal/source-lints/windows-cross-config.test.ts
  • test/js/bun/binary/tls-segment-size.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.

Walkthrough

The Windows build now configures an 18 MB PE stack reserve and a one-page commit. Binary checks and tests cover these values and per-thread commit charges. The memory-pressure watcher and IO thread pool also use updated stack sizes.

Changes

Windows Stack Sizing

Layer / File(s) Summary
PE stack configuration and expectations
scripts/build/flags.ts, scripts/build/binary-expectations.ts
The Windows linker uses an 18 MB stack reserve and a one-page commit. Binary expectations record these values for bun and the linker defaults for the shim.
Runtime thread stack sizes
src/runtime/node/memory_pressure.rs, src/threading/io_thread_pool.rs
The Windows memory-pressure watcher uses a 2 MiB stack. The IO thread pool uses 2 MiB on Windows and 512 KiB on other platforms.
Binary checks and regression tests
scripts/build/verify-binary.ts, test/internal/source-lints/windows-cross-config.test.ts, test/js/bun/binary/tls-segment-size.test.ts
Binary verification checks PE stack reserve and commit values. Tests cover Windows linker flags, PE header values, and commit charges when starting threads.

Suggested reviewers: jarred-sumner

Priority: ⬆️ High

Merge Risk: ⚪ Minimal · up to e0836

No actionable issue remains before merge. Windows-specific validation is still expected in CI.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly summarizes the main change: reducing Windows per-thread stack commitment to one page while checking linked executable stack sizes.
Description check Passed The description clearly explains the problem, fix, tradeoffs, validation, and known limitations. It does not use the template headings, but it includes the required content in equivalent sections.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline finding, I also checked the two Rust stack-size bumps: the MemoryPressure watcher (src/runtime/node/memory_pressure.rs:457) and the IO pool (src/threading/io_thread_pool.rs:34) now request 2 MiB, which on Windows is a reservation only and matches the stack they effectively had under the old 2 MB commit floor, so neither loses usable stack. The shim's expected 0x100000/0x1000 in binary-expectations.ts matches lld-link's defaults for a link that passes no /STACK, so that check is not a new CI hazard.

Extended reasoning...

The change switches the Windows link to /STACK:0x1200000,0x1000 (one page of initial commit per thread), adds a PE header reserve/commit check to verify-binary.ts with literal expectations for bun.exe and the shim, raises the Windows stack request of two Rust threads to 2 MiB, and adds a cross-config lint test plus a Windows-only test that reads the PE header and measures commit per thread start. It touches no auth, crypto, or input-parsing surface. A human look is warranted because the inline finding is a real behavioral trade-off the author documents (Yarr frames on JIT worklist threads can now fault where the base ran), which is a product decision rather than a code defect.

Comment thread scripts/build/flags.ts
},
{
flag: ["/STACK:0x1200000,0x200000", "/errorlimit:0"],
flag: [`/STACK:${WINDOWS_STACK_RESERVE},${WINDOWS_STACK_COMMIT}`, "/errorlimit:0"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Windows users whose script has a large constant regexp tested against a constant string get an access-violation crash (exit 0xC0000005) that the base build does not have. The /STACK:...,0x1000 operand at scripts/build/flags.ts:1044 leaves a JIT worklist thread with a 4 KB committed stack plus one guard page, and Yarr's JIT prologue moves the stack pointer by the whole frame without a probe when DFGStrengthReductionPhase folds the match on that thread. Fix: before shipping a one-page header, make every thread that runs JIT-generated Yarr code safe for frames above one page: give the JIT worklist threads a commit size of their own (WTF Thread::create) or add a stack probe to the Yarr prologue, so the guard page cannot be skipped.

Why this was flagged

A script with a constant regexp of many capture groups, e.g. /^(a?){N}b$/ written out, tested in a hot loop against a constant subject, so the DFG constant-folds the match on a JIT worklist thread via RegExp::matchConcurrently. With scripts/build/flags.ts:1044 now emitting /STACK:0x1200000,0x1000 (WINDOWS_STACK_COMMIT at flags.ts:53), that thread is created by WTF with size 0 and starts with 4096 bytes committed and a single guard page; nothing pre-commits a JIT thread's stack (preCommitStackMemory only runs on JS threads). The Yarr JIT prologue subtracts the full frame from the stack pointer without a probe, so a frame larger than about 16 KB lands past the guard page and the process dies with 0xC0000005. On the base the header commits 0x200000 bytes, so the same frame lands on committed pages and the script runs. The hardening/stack checks in scripts/build/verify-binary.ts:636-644 only confirm the header values and cannot catch this. The PR description measures 15.6 KB passing and 31.3 KB, 46.9 KB and 125 KB failing on both x64 and arm64.

Verification: On Windows, a script whose hot function tests a large constant regexp against a constant string, so the DFG folds the match on a JIT worklist thread. scripts/build/flags.ts:1044 now emits /STACK:${WINDOWS_STACK_RESERVE},${WINDOWS_STACK_COMMIT} with WINDOWS_STACK_COMMIT = "0x1000" (flags.ts:53), replacing /STACK:0x1200000,0x200000. JIT-generated Yarr code emits no __chkstk-style probe.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct. It is the second bullet under Downsides, and the numbers quoted here are the ones measured for this PR.

The gap is in JSC. For a match on a compiler thread, Yarr takes its stack limit from the reserve of the thread, and nothing commits or probes the frame. Upstream JSC has the same gap with its own one-page header. The 2 MB commit covered it for frames below about 1.9 MB.

Both fixes named here are changes in oven-sh/WebKit:

  • A commit size of their own for the JIT worklist threads puts the large request back on the threads of one of the two crash groups this PR is about (BUN-2V2T is JITWorklist::wakeThreads).
  • A probe in the Yarr prologue, or a stack limit taken from the committed stack so that the DFG does not fold the match, fixes it where it is. That fix is separate work, with a Windows test for the 31.3 KB frame.

The order is a decision for the maintainer:

  • This PR after the WebKit bump that carries the Yarr fix: no new failure.
  • This PR first: until that bump, a match that the DFG folds crashes when its Yarr frame is above about 16 KB. The largest frame in 4,219 regexes from packages is 6 KB, and the match needs a constant subject in hot code.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants