Skip to content

Bump WebKit (oven-sh/WebKit#791 preview): a long 16-bit string with an unpaired surrogate no longer aborts the conversion to UTF-8 - #44949

Draft
robobun wants to merge 1 commit into
mainfrom
robobun/99d68896/webkit-utf8-conversion
Draft

robobun wants to merge 1 commit into
mainfrom
robobun/99d68896/webkit-utf8-conversion

Conversation

@robobun

@robobun robobun commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Draft until oven-sh/WebKit#791 merges. WEBKIT_VERSION is its preview build, for CI on every platform. The merged commit replaces it before this leaves draft.

Problem

  • A 16-bit string with an unpaired surrogate aborts the process when its UTF-8 form is about 1.43 GB or more: panic(main thread): abort() called, exit 134, also inside try/catch. Date.parse() and db.prepare() of bun:sqlite and node:sqlite reach it.
  • The cause is in WTF: after a failed strict conversion, StringImpl::tryGetUTF8ForCharacters regrows its buffer with Vector::resize(). The 1.5x that it asks for is over the Vector limit, and CRASH() follows.

Fix

Background

  • UTF8View (BunString.h) calls tryGetUTF8() for every string that is not 8-bit ASCII. Date.parse() converts inside JavaScriptCore, so a fix in bun's bindings does not reach it.
  • Only an unpaired surrogate makes the strict conversion fail. A second conversion then writes U+FFFD.

Downsides

Notes

Repro. Release builds, linux x64. text is "\u65e5".repeat(477_218_588) + "\ud800a". "main" is e655c58032 with the pin 0c06faadf6. "this pin" is 71d0d43997 with the preview build.

Call main this pin
Date.parse(text) exit 134 NaN (14.1 s, peak RSS 5,489 MB)
new Database(":memory:").prepare("select '" + text + "'") (bun:sqlite) exit 134 SQLiteError: statement too long (7.5 s, 4,581 MB)
db.run() with the same statement exit 134 SQLiteError: statement too long
new DatabaseSync(":memory:").prepare("select '" + text + "'") (node:sqlite) exit 134 ERR_SQLITE_ERROR: string or blob too big (7.7 s, 4,580 MB)
crypto.hkdfSync("sha256", "key", text, "info", 8) exit 134 returns 8 bytes
require.resolve.paths(text) exit 134 returns an array
new Bun.CookieMap("a=%41" + text) exit 134 exit 134

The last row converts now and then aborts in StringBuilder::didOverflow() (StringBuilder.cpp:46), called from decodeURIComponentSIMD for CookieMap.cpp:117. That is a different limit.

The tests.

  1. a long 16-bit string with an unpaired surrogate converts instead of aborting. Release builds on a machine with 12 GiB or more. The child needs 5 GB and takes 11 s. On main: exit 134.
  2. the second buffer for an unpaired surrogate has the exact size, and can fail. Debug builds. WebKit's debug-only maxSingleAllocationSize option refuses every allocation over 4 MiB here. On main: ASSERTION FAILED: Requested size (4500000) exceeds max single allocation size set for testing (4194304), exit 134.
  3. an unpaired surrogate converts to U+FFFD on each side of the conversion's buffer sizes. Every build. It passes on main too: it holds the bytes of the second conversion, which is a different converter now.

What the bump carries. The preview is the fork's main with oven-sh/WebKit#791 on top. The fork's main has two changes that the current pin does not have: oven-sh/WebKit#774 and oven-sh/WebKit#787.

Size. Release builds of 71d0d43997, linux x64: text is 89,103,803 bytes with the pin 0c06faadf6 and 89,098,939 bytes with the preview. That is the three changes together.

Not in this PR.


[policy-decision:webkit] gate passed · iteration 0 · 2 files touched

passes on PR (with fix)
Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/js/bun/util/utf8-conversion-limit.test.ts'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "test/js/bun/util/utf8-conversion-limit.test.ts"
bun test v1.4.4 (367d939d9)

test/js/bun/util/utf8-conversion-limit.test.ts:
(pass) a string whose UTF-8 form does not fit in a buffer is an error instead of an abort [13999.47ms]
(pass) a short Latin-1 string converts as before [103.44ms]
(pass) a short 16-bit string converts as before [14.87ms]
(skip) a long 16-bit string with an unpaired surrogate converts instead of aborting
(pass) the second buffer for an unpaired surrogate has the exact size, and can fail [1290.63ms]
(pass) an unpaired surrogate converts to U+FFFD on each side of the conversion's buffer sizes [455.32ms]

 5 pass
 1 skip
 0 fail
 5 expect() calls
Ran 6 tests across 1 file. [25.64s]
Exit: 0
diff hotspot
scripts/build/deps/webkit.ts                   |   2 +-
 test/js/bun/util/utf8-conversion-limit.test.ts | 164 ++++++++++++++++++++++++-
 2 files changed, 164 insertions(+), 2 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                            reads  edits  tests
scripts/build/deps/webkit.ts                        1      1     22
test/js/bun/util/utf8-conversion-limit.test.ts      2      0     22

The conversion of a 16-bit string to UTF-8 allocates the second buffer,
for a string with an unpaired surrogate, at its exact size. Before, it
grew the first buffer by 1.5x, and the process aborted when that was
more than a Vector can hold (a first buffer of 1,431,655,766 bytes or
more).

Tests in test/js/bun/util/utf8-conversion-limit.test.ts: the long
string through bun:sqlite and node:sqlite (release builds), the exact
size and the error for a refused second buffer (debug builds), and the
bytes of strings with an unpaired surrogate around the buffer sizes.
@robobun

robobun commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 4:45 PM PT - Oct 10th, 2026

❌ @robobun, your commit b08d964 has 1 failures in Build #124385 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 44949

That installs a local version of the PR into your bun-44949 executable, so you can run:

bun-44949 --bun

@robobun

robobun commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on main e655c58032 (release build, linux x64). Date.parse("\u65e5".repeat(477_218_588) + "\ud800a") exits 134 with panic(main thread): abort() called, also inside try/catch. The same string through db.prepare() of bun:sqlite and of node:sqlite does the same.

The fix is oven-sh/WebKit#791. This pull request stays a draft until that one merges.

CI on b08d9643: the new tests pass on all 11 test lanes. The alpine, macOS and Windows lanes run the long-string test, and the x64-asan lane runs the refused-allocation test. The one red job is test/js/bun/io/bun-write.test.js on x64-asan (AddressSanitizer: heap-use-after-free). The final build of #44895 has the same failure without this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant