Repository navigation
Conversation
The conversion of a 16-bit string to UTF-8 allocates the second buffer, for a string with an unpaired surrogate, at its exact size. Before, it grew the first buffer by 1.5x, and the process aborted when that was more than a Vector can hold (a first buffer of 1,431,655,766 bytes or more). Tests in test/js/bun/util/utf8-conversion-limit.test.ts: the long string through bun:sqlite and node:sqlite (release builds), the exact size and the error for a refused second buffer (debug builds), and the bytes of strings with an unpaired surrogate around the buffer sizes.
|
Updated 4:45 PM PT - Oct 10th, 2026
❌ @robobun, your commit b08d964 has 1 failures in
🧪 To try this PR locally: bunx bun-pr 44949That installs a local version of the PR into your bun-44949 --bun |
|
Status: reproduced on main The fix is oven-sh/WebKit#791. This pull request stays a draft until that one merges. CI on |
Draft until oven-sh/WebKit#791 merges.
WEBKIT_VERSIONis its preview build, for CI on every platform. The merged commit replaces it before this leaves draft.Problem
panic(main thread): abort() called, exit 134, also insidetry/catch.Date.parse()anddb.prepare()ofbun:sqliteandnode:sqlitereach it.StringImpl::tryGetUTF8ForCharactersregrows its buffer withVector::resize(). The 1.5x that it asks for is over theVectorlimit, andCRASH()follows.Fix
test/js/bun/util/utf8-conversion-limit.test.ts. On main's pin the first (release) and the second (debug) get exit 134. With this pin the file passes in release and withbun bd test.Background
UTF8View(BunString.h) callstryGetUTF8()for every string that is not 8-bit ASCII.Date.parse()converts inside JavaScriptCore, so a fix in bun's bindings does not reach it.Downsides
new Bun.CookieMap("a=%41" + thatString), inStringBuilder::didOverflow(). A failed allocation of the first buffer or of the result also still crashes.Notes
Repro. Release builds, linux x64.
textis"\u65e5".repeat(477_218_588) + "\ud800a". "main" ise655c58032with the pin0c06faadf6. "this pin" is71d0d43997with the preview build.Date.parse(text)NaN(14.1 s, peak RSS 5,489 MB)new Database(":memory:").prepare("select '" + text + "'")(bun:sqlite)SQLiteError: statement too long(7.5 s, 4,581 MB)db.run()with the same statementSQLiteError: statement too longnew DatabaseSync(":memory:").prepare("select '" + text + "'")(node:sqlite)ERR_SQLITE_ERROR: string or blob too big(7.7 s, 4,580 MB)crypto.hkdfSync("sha256", "key", text, "info", 8)require.resolve.paths(text)new Bun.CookieMap("a=%41" + text)The last row converts now and then aborts in
StringBuilder::didOverflow()(StringBuilder.cpp:46), called fromdecodeURIComponentSIMDforCookieMap.cpp:117. That is a different limit.The tests.
a long 16-bit string with an unpaired surrogate converts instead of aborting. Release builds on a machine with 12 GiB or more. The child needs 5 GB and takes 11 s. On main: exit 134.the second buffer for an unpaired surrogate has the exact size, and can fail. Debug builds. WebKit's debug-onlymaxSingleAllocationSizeoption refuses every allocation over 4 MiB here. On main:ASSERTION FAILED: Requested size (4500000) exceeds max single allocation size set for testing (4194304), exit 134.an unpaired surrogate converts to U+FFFD on each side of the conversion's buffer sizes. Every build. It passes on main too: it holds the bytes of the second conversion, which is a different converter now.What the bump carries. The preview is the fork's main with oven-sh/WebKit#791 on top. The fork's main has two changes that the current pin does not have: oven-sh/WebKit#774 and oven-sh/WebKit#787.
Size. Release builds of
71d0d43997, linux x64: text is 89,103,803 bytes with the pin0c06faadf6and 89,098,939 bytes with the preview. That is the three changes together.Not in this PR.
.utf8()that assert on a failed conversion. sqlite, console, process: do not abort on a string too long to convert to UTF-8 #42868 moves the ones in sqlite, console and process, and edits the same test file.[policy-decision:webkit] gate passed · iteration 0 · 2 files touched
passes on PR (with fix)
diff hotspot
gate history · 1 passed · 0 rejected · iteration 0
evidence per changed file