-
Notifications
You must be signed in to change notification settings - Fork 5.1k
worker_threads: copy argv/execArgv for the worker thread instead of sharing the parent's StringImpls #37140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
worker_threads: copy argv/execArgv for the worker thread instead of sharing the parent's StringImpls #37140
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -64,7 +64,7 @@ | |
| use core::ptr::NonNull; | ||
| use core::sync::atomic::{AtomicBool, AtomicU32, Ordering}; | ||
|
|
||
| use bun_core::{String as BunString, WTFStringImpl}; | ||
| use bun_core::{String as BunString, WTFStringImpl, WTFStringImplExt as _}; | ||
| use bun_io::KeepAlive; | ||
| use bun_threading::{Futex, Mutex}; | ||
|
|
||
|
|
@@ -97,12 +97,16 @@ | |
| mini: bool, | ||
| eval_mode: bool, | ||
| store_fd: bool, | ||
| /// Borrowed from C++ `WorkerOptions` (kept alive by the owning `Worker`). | ||
| argv_ptr: *const WTFStringImpl, | ||
| argv_len: usize, | ||
| exec_argv_ptr: *const WTFStringImpl, | ||
| exec_argv_len: usize, | ||
| inherit_exec_argv: bool, | ||
| /// Owned UTF-8 copies of the `WorkerOptions` argv/execArgv strings, made | ||
| /// on the parent thread in [`Self::create`]. The worker thread must never | ||
| /// see the parent's `WTF::StringImpl`s: wrapping one in a worker-heap | ||
| /// `JSString` lets worker-side atomization insert the shared impl into the | ||
| /// worker's thread-local atom table, and the parent's final deref (GC | ||
| /// sweep of the `Worker` wrapper) then aborts in `AtomStringImpl::remove` | ||
| /// ("the atom is in the string table of an other thread"). | ||
| argv: Vec<Box<[u8]>>, | ||
| /// `None` when the worker inherits the parent's execArgv. | ||
| exec_argv: Option<Vec<Box<[u8]>>>, | ||
| /// Heap-owned by this struct; freed in `destroy()`. | ||
| unresolved_specifier: Box<[u8]>, | ||
| preloads: Vec<Box<[u8]>>, | ||
|
|
@@ -436,26 +440,17 @@ | |
| self.eval_mode | ||
| } | ||
|
|
||
| /// Borrowed from the C++ `WorkerOptions` (kept alive by the owning | ||
| /// `WebCore::Worker`). | ||
| /// Worker-owned UTF-8 copies of the `WorkerOptions` argv strings. | ||
| #[inline] | ||
| pub fn argv(&self) -> &[WTFStringImpl] { | ||
| // SAFETY: `argv_ptr[..argv_len]` is borrowed from C++ WorkerOptions | ||
| // (BACKREF — kept alive by the owning Worker for `self`'s lifetime). | ||
| // `(null, 0)` is tolerated by `ffi::slice`. | ||
| unsafe { bun_core::ffi::slice(self.argv_ptr, self.argv_len) } | ||
| pub fn argv(&self) -> &[Box<[u8]>] { | ||
| &self.argv | ||
| } | ||
|
|
||
| /// `None` when | ||
| /// `inherit_exec_argv` (the worker inherits the parent's execArgv), | ||
| /// otherwise `Some(slice)` (possibly empty) borrowed from C++ WorkerOptions. | ||
| /// `None` when the worker inherits the parent's execArgv, otherwise | ||
| /// `Some(slice)` (possibly empty) of worker-owned UTF-8 copies. | ||
| #[inline] | ||
| pub fn exec_argv(&self) -> Option<&[WTFStringImpl]> { | ||
| if self.inherit_exec_argv { | ||
| return None; | ||
| } | ||
| // SAFETY: see `argv()`. | ||
| Some(unsafe { bun_core::ffi::slice(self.exec_argv_ptr, self.exec_argv_len) }) | ||
| pub fn exec_argv(&self) -> Option<&[Box<[u8]>]> { | ||
| self.exec_argv.as_deref() | ||
| } | ||
|
|
||
| fn set_requested_terminate(&self) -> bool { | ||
|
|
@@ -541,6 +536,28 @@ | |
| // SAFETY: `parent` is live (see above); borrow ends at `;`. | ||
| let store_fd = unsafe { (*parent).transpiler.resolver.store_fd }; | ||
|
|
||
| // Copy argv/execArgv to worker-owned UTF-8 while still on the parent | ||
| // thread; see the `argv` field doc for why the `WTF::StringImpl`s must | ||
| // not cross into the worker thread. | ||
| let copy_args = |ptr: *const WTFStringImpl, len: usize| -> Vec<Box<[u8]>> { | ||
| // SAFETY: caller passed a valid (ptr, len) pair (or `(null, 0)`, | ||
| // tolerated by `ffi::slice`) of live `WTF::StringImpl*`s kept | ||
| // alive by `Worker::create` across this call. | ||
| unsafe { bun_core::ffi::slice(ptr, len) } | ||
| .iter() | ||
| .map(|&s| { | ||
| // SAFETY: each element is a live `WTF::StringImpl*` (see above). | ||
| unsafe { &*s }.to_utf8().slice().to_vec().into_boxed_slice() | ||
| }) | ||
|
Check warning on line 551 in src/jsc/web_worker.rs
|
||
| .collect() | ||
| }; | ||
|
Check failure on line 553 in src/jsc/web_worker.rs
|
||
|
Comment on lines
+546
to
+553
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Extended reasoning...What the bug isThe new .map(|&s| {
// SAFETY: each element is a live `WTF::StringImpl*` (see above).
unsafe { &*s }.to_utf8().slice().to_vec().into_boxed_slice()
})
How null reaches this code path
auto coerceToIsolatedString = [lexicalGlobalObject](JSValue v) -> String {
String original = v.isSymbol() ? asSymbol(v)->tryGetDescriptiveString().value_or(String()) : v.toWTFString(lexicalGlobalObject);
return original.isolatedCopy();
};For
Why this is a regressionThe pre-PR code guarded exactly this condition, and this diff removes both guards:
Before this PR, the null was stored on the Step-by-step proof
FixRestore the null skip in unsafe { bun_core::ffi::slice(ptr, len) }
.iter()
.filter_map(|&s| {
// A `WTF::String` may have a null impl (e.g. `Symbol()` in
// argv/execArgv via `coerceToIsolatedString`); skip it, matching
// the pre-existing consumer guards this replaces.
core::ptr::NonNull::new(s).map(|nn| unsafe { nn.as_ref() }.to_utf8().slice().to_vec().into_boxed_slice())
})
.collect()(or push an empty |
||
| let argv = copy_args(argv_ptr, argv_len); | ||
| let exec_argv = if inherit_exec_argv { | ||
| None | ||
| } else { | ||
| Some(copy_args(exec_argv_ptr, exec_argv_len)) | ||
| }; | ||
|
|
||
| let worker = bun_core::heap::into_raw(Box::new(WebWorker { | ||
| cpp_worker, | ||
| // `parent` is the calling thread's live VM; non-null by FFI contract. | ||
|
|
@@ -549,11 +566,8 @@ | |
| mini, | ||
| eval_mode, | ||
| store_fd, | ||
| argv_ptr, | ||
| argv_len, | ||
| exec_argv_ptr, | ||
| exec_argv_len, | ||
| inherit_exec_argv, | ||
| argv, | ||
| exec_argv, | ||
| unresolved_specifier: spec_slice.slice().to_vec().into_boxed_slice(), | ||
| preloads, | ||
| name: if name_str.is_empty() { | ||
|
|
@@ -861,14 +875,9 @@ | |
| // RunCommand param table. The param table lives in | ||
| // `bun_runtime::cli` (forward-dep), so dispatch through | ||
| // `RuntimeHooks::parse_worker_exec_argv_allow_addons`. Currently | ||
| // only honours `--no-addons`; the hook owns the temporary UTF-8 | ||
| // alloc + clap parse + `args.deinit()`. `None` on parse failure | ||
| // only honours `--no-addons`. `None` on parse failure | ||
| // (the parent's setting is kept). | ||
|
|
||
| // SAFETY: `exec_argv` borrows C++ `WorkerOptions` kept alive by the | ||
| // owning `WebCore::Worker` for `self`'s lifetime; the hook only | ||
| // reads the slice and owns its own temporary allocations. | ||
| let parsed = unsafe { (hooks.parse_worker_exec_argv_allow_addons)(exec_argv) }; | ||
| let parsed = (hooks.parse_worker_exec_argv_allow_addons)(exec_argv); | ||
| if let Some(allow_addons) = parsed { | ||
| let parent_allows = transform_options.allow_addons.unwrap_or(true); | ||
| transform_options.allow_addons = Some(parent_allows && allow_addons); | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 nit: the UTF-8 round-trip here replaces lone surrogates with U+FFFD —
to_utf8()on a 16-bit impl routes throughconvert_utf16_to_utf8_append→decode_utf16_with_fffd, sonew Worker(src, {argv:['\uD800foo']})now seesprocess.argv[2].charCodeAt(0) === 0xFFFDwhere it was0xD800before (the old direct-wrap path preserved code units exactly). REVIEW.md's boundary-representation section calls out WTF-8 for lone surrogates specifically. Exotic trigger, so not blocking; consider storingBox<[u16]>for 16-bit impls (or a WTF-8-preserving encode) as a follow-up.Extended reasoning...
What the bug is
The new
copy_argsclosure converts each argv/execArgvWTF::StringImplto owned bytes viaWTFStringImplExt::to_utf8(), and the worker later reconstructs a JSString viaBunString::borrow_utf8(arg).to_js(). For a 16-bit impl,to_utf8()(src/bun_core/string/wtf.rs:60) callsstrings::to_utf8_alloc(self.utf16_slice())→convert_utf16_to_utf8_append(src/bun_core/lib.rs:1603). When simdutf returnsStatus::SURROGATE, that falls back toappend_wtf8_from_utf16(line 1590), whose doc comment reads "Unpaired surrogates are replaced with U+FFFD" and whose body callsdecode_utf16_with_fffd— returning(0xFFFD, 1)for a lone lead or trail surrogate (lines 1503-1511). The replacement happens before encoding, so the stored bytes areEF BF BD(UTF-8 for U+FFFD), not the WTF-8ED A0 80for U+D800.borrow_utf8().to_js()on the worker side then decodes standard UTF-8 and the original code unit is unrecoverable.Why this is a regression
Before this PR,
node_process.rsdidBunString::init(wtf).to_js(), which wraps the parent'sStringImpl*directly in a worker-heap JSString — the exact code-unit sequence was preserved byte-for-byte. That direct sharing is precisely the atomization bug this PR fixes, so the fidelity loss is a direct consequence of the change, not a pre-existing behavior. Node.js also preserves lone surrogates in worker argv (options are structured-cloned, which round-trips WTF-16 exactly).Why nothing prevents it
The PR description says "Behavior parity checked … (unicode, empty strings)", but valid Unicode round-trips fine through this path — only unpaired surrogates hit the FFFD replacement branch, and no test in the diff exercises one. REVIEW.md's "Validate representation at every boundary" section explicitly names "WTF-8 helpers for lone surrogates (real Windows paths contain them)" as the class to guard.
Step-by-step proof
new Worker(src, { eval: true, argv: ['\uD800foo'] }).WebWorker::create→copy_argsiterates the argv slice; the element is a 16-bitStringImplwith code units[0xD800, 0x66, 0x6F, 0x6F].to_utf8()sees!is_8bit()→to_utf8_alloc([0xD800, 0x66, 0x6F, 0x6F]).SURROGATE;append_wtf8_from_utf16callsdecode_utf16_with_fffd([0xD800, ...])→(0xFFFD, 1), thenencode_wtf8_runewritesEF BF BD. Remaining ASCII appends as66 6F 6F. StoredBox<[u8]>=[0xEF, 0xBF, 0xBD, 0x66, 0x6F, 0x6F].create_argv→BunString::borrow_utf8(arg).to_js()decodes those bytes as UTF-8 → JS string"\uFFFDfoo".process.argv[2].charCodeAt(0)yields0xFFFD; on the previous release it yields0xD800.Impact
Narrow: a JS string containing a lone surrogate passed through worker
argv/execArgvis silently altered. The realistic case is a Windows path with an unpaired surrogate (real dirents can contain them) forwarded viaargv. Exotic enough not to block a fix for a CI-killing memory-safety abort, but it is a genuine Node-compat / data-fidelity regression introduced by this change.Suggested fix
Either keep 16-bit impls as
Box<[u16]>and go throughBunString::borrow_utf16()on the worker side, or use a WTF-8-preserving encode (encode the surrogate code point itself, not U+FFFD) paired with a WTF-8-awareto_jspath. Both avoid the atomization bug (worker owns the bytes) while preserving code-unit fidelity.