Skip to content

worker: survive pthread_create EAGAIN at the thread/pids limit - #34705

Closed
robobun wants to merge 2 commits into
mainfrom
claude/farm/2865a2d2/wtf-thread-trycreate
Closed

robobun wants to merge 2 commits into
mainfrom
claude/farm/2865a2d2/wtf-thread-trycreate

Conversation

@robobun

@robobun robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator

What

new Worker() at a kernel thread limit (cgroup pids.max, RLIMIT_NPROC) currently aborts the whole process instead of failing that one Worker. Node at the same limit emits a per-Worker error and fully recovers.

The load-bearing fix is in oven-sh/WebKit#311; this PR adds the bun-side regression test and will bump WEBKIT_VERSION once that lands.

Repro

# as any non-root user on Linux
bash -c 'ulimit -u 16 && exec bun worker-churn.mjs'
worker-churn.mjs
import { mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
const w = join(mkdtempSync(join(tmpdir(), "t-")), "w.js");
writeFileSync(w, `self.onmessage = e => postMessage(e.data + 1)`);
for (let r = 0; r < 12; r++) {
  const ws = [];
  for (let i = 0; i < 4; i++) { try { ws.push(new Worker(w)); } catch {} }
  await Promise.all(ws.map(x => new Promise(res => {
    const t = setTimeout(res, 1200);
    x.onmessage = () => { clearTimeout(t); res() };
    x.onerror = () => { clearTimeout(t); res() };
    x.postMessage(r);
  })));
  ws.forEach(x => x.terminate());
  Bun.gc(true);
}
console.log("SURVIVED");

Stock 1.4.0 and current main: SIGABRT (134), silent in release. With debug assertions:

ASSERTION FAILED: success
vendor/WebKit/Source/WTF/wtf/Threading.cpp(331) : static Ref<Thread> WTF::Thread::create(...)

Cause

Bun's own Worker OS thread (std::thread::Builder::spawn in web_worker.rs) already turns spawn failure into a JS TypeError. The abort comes from the JSC/WTF/bmalloc helper threads that the Worker churn triggers:

  • AutomaticThread::start (GC marker ParallelHelperPool, JIT/Wasm worklists, per-VM Heap collector) calls Thread::create which RELEASE_ASSERTs on pthread_create EAGAIN.
  • bmalloc's pas_scavenger does a raw pthread_create + PAS_ASSERT(!result).
  • VMTraps' process-global signal-sender WorkQueue is created lazily on the first fireTrap, which the first worker.terminate() reaches after Workers have consumed every pid slot.

Fix (oven-sh/WebKit#311)

  • New WTF::Thread::tryCreate returning RefPtr<Thread>.
  • AutomaticThread::start uses it and backs off to the no-underlying-thread state so the next notify retries.
  • Heap::finishRelinquishingConn re-takes the conn when the collector thread has no underlying thread, so the mutator drives GC instead of waitForCollector parking forever.
  • pas_scavenger drops back to no_thread on failure instead of asserting.
  • VMTraps::initializeSignals creates the signal-sender queue eagerly.

Verification

Patched the five objects into the prebuilt debug-asan libWTF.a/libbmalloc.a/libJavaScriptCore.a and relinked; swept ulimit -u 13 through 28 on the repro. Every value now exits 0 where the unpatched build aborts. All existing test/js/web/workers and test/js/node/worker_threads tests pass against the patched build.

The new test (test/js/web/workers/worker-thread-limit.test.ts) runs the repro under ulimit -u and asserts exit 0. Linux-only, skipped under uid 0. It will stay red until WEBKIT_VERSION is bumped to include oven-sh/WebKit#311.

Status

Draft until oven-sh/WebKit#311 merges and autobuilds; then this PR bumps scripts/build/deps/webkit.ts and goes green.


no test proof · iteration 0 · Platform-specific test-only change; deferring to CI.

Regression test for Worker churn at RLIMIT_NPROC (the same EAGAIN path
a cgroup pids.max budget hits). Before oven-sh/WebKit#311 the child
process dies with SIGABRT from RELEASE_ASSERT in WTF::Thread::create
or PAS_ASSERT in bmalloc's pas_scavenger; with it the child exits 0
after churning Workers that individually fail.

Linux-only and skipped under uid 0 (the kernel ignores RLIMIT_NPROC
for root). Requires the WebKit bump once #311 lands.
@robobun

robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 7:17 AM PT - Jul 19th, 2026

❌ @autofix-ci[bot], your commit 74647af has 2 failures in Build #75759 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 34705

That installs a local version of the PR into your bun-34705 executable, so you can run:

bun-34705 --bun

@robobun

robobun commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: superseded by #41587 and oven-sh/WebKit#569. That WebKit change covers the same aborts and also the synchronous GC that spun in waitForCollector with no collector thread, and it keeps the VMTraps queue lazy instead of creating a thread at startup. The new PR carries a test for both the GC and the Worker case and is verified against the preview tarballs.

@robobun robobun closed this Sep 6, 2026
Bewinxed added a commit to Bewinxed/cawco that referenced this pull request Oct 8, 2026
…er runs out of room itself

A Bun process that cannot create a thread aborts: WTF::Thread::create
asserts on a refused pthread_create (oven-sh/WebKit#311 and oven-sh/bun#34705
closed unmerged; 1.4.2 is the latest release). Stage 2's joiner keeper died
that way at its container's 307-task limit, in JITWorklist::enqueue, and
took every child it held. Reproduced on a scratch keeper under
`prlimit --nproc=120`: 150 spawns all acked applied (81 alive), then the
first connection after a 40 s idle aborted it.

- Before each spawn the keeper reads its room under the tightest limit: the
  user's process limit (/proc/self/limits against the user's tasks in
  /proc/*/status) and cgroup v2 pids.max along its cgroup chain. A child is
  started only with 64 tasks to come up (a Claude Code CLI reached 17
  threads starting) above the 64 the keeper keeps for its own threads and
  its `ps`; otherwise the spawn is refused with `process-limit: <limit>`.
- Spawns run one at a time, so a burst is weighed child by child.
- A started child gets RLIMIT_NPROC 64 below the keeper's (prlimit(2) via
  bun:ffi): it and all it starts can never take the keeper's room.
- The ack waits for the child's `spawn` or `error`: a fork the kernel
  refuses (EAGAIN) is a process-limit refusal, not an applied spawn that
  dies unexplained.
- The agent says it as "<machine> can't start another process right now (its
  process limit is reached), so <session> didn't start."

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant