[robobun]

Problem

• An embedder can make an ArrayBuffer over memory that another process also writes. Bun's Bun.mmap does: its buffer is a file mapping. isShared() is false for it.
• Native code that copies a SharedArrayBuffer before it reads a byte twice reads such bytes in place. In Bun, on a mapped file that another process rewrites: panic: assertion failed: self.input[pos.cast()] == unit (Bun.YAML.parse) and out-of-bounds writes in the UTF-8 decoders.

Fix

• ArrayBuffer::createFromBytes takes ArrayBufferExternalWriters::Yes. The flag is on ArrayBufferContents, so a transfer moves it and a copy of the bytes does not have it.
• canBeWrittenConcurrently() is isShared() or that flag, on ArrayBufferContents, ArrayBuffer and JSArrayBufferView. A typed array with no ArrayBuffer answers from its mode.
• Verified: JSTests/stress/array-buffer-external-writers.js, through $vm.createArrayBufferWithExternalWriters and $vm.canBeWrittenConcurrently. Copy a Bun.mmap view where a shared buffer is copied (file mappings only) bun#46512 uses it on this branch's preview build.

Background

• JavaScript sees an ordinary ArrayBuffer over a mapping. Only native code has to know that the bytes change.
• ArrayBufferContents owns the bytes. transfer(), a structured clone that transfers and a transfer to a Worker move it.
• Considered a table of mapped address ranges in Bun (a lock at every place that asks) and a SharedArrayBuffer from Bun.mmap (users see it).

Downsides

• Every ArrayBuffer has one more byte to set and to move. Release Bun: new ArrayBuffer(64) 1,082.9 to 1,085.4 instructions, transfer() 2,074.4 to 2,085.5, slice() 1,288.7 to 1,291.8. The sizes stay 56 and 96 bytes.
• Nothing in JavaScriptCore asks the question yet. TypedArray.prototype.sort copies a shared array first and still sorts a mapped one in place, see Notes.

Notes

Why the flag is on the contents. m = Bun.mmap(f); t = m.buffer.transfer() detaches m and gives t the same mapping. A flag on the view or on the ArrayBuffer object would be lost there. swap() carries the flag, so the move constructor, move assignment and detach() do. tryAllocate() clears it, so copyTo() (a transfer of a pinned buffer) and slice() give memory without it. shareWith() copies it with the other members. The test checks each of these from JavaScript.

What the flag does not say. A typed array with no ArrayBuffer yet (FastTypedArray, OversizeTypedArray) answers false from its mode. Bun's thread pool can write such a view during an fs.read. That case needs a record of the writer and is not part of this change. The flag has no setter: it is a property of the memory, given where the buffer is made.

Measurements. Release builds of Bun at oven-sh/bun@38afee6c84 with WebKit d6375c7 and with this branch's preview build. instructions:u, (I(2N) - I(N)) / N, minimum of 5 runs, concurrent JIT and GC off. A second copy of the first binary gives 1,083.2, 2,076.1 and 1,285.2 for the three operations. sizeof(ArrayBufferContents) is 56 and sizeof(ArrayBuffer) is 96 before and after, with the debug ASAN and the LTO headers, and a static_assert holds the first.

A way to +0 for the moves. The two flags can share one byte, so that swap() moves one byte as it did. That changes the 12 places that read m_hasMaxByteLength. I kept the two bools to keep the difference to upstream small. Say so if you prefer the one byte.

sort(). TypedArray.prototype.sort copies a shared array at three places in JSGenericTypedArrayViewInlines.h, because the radix sort counts the digits of every element and then places every element by those counts. With another process that rewrites a mapped Uint32Array, sort() stops in radixSortWithCounters at std::span<unsigned int>::operator[]: Assertion '__idx < size()' failed. An earlier revision of this pull request made those three places ask canBeWrittenConcurrently(). I took that out: the copy doubles the memory of a sort of a mapped array, and WTF::Vector cannot hold a copy of 2 GiB or more, so such a sort would throw where it works today. That needs its own decision.

simdutf. The first revision also carried the bound for rewind_and_validate_with_errors. It is independent of this change and is now #878.

[github-actions]

Preview build of a1ba2bc: autobuild-preview-pr-872-a1ba2bc3