Conversation
Every S3 write entry point is documented as resolving with the number of
bytes written, and the buffered path (Uint8Array/Blob/string) already did
that. The streamed paths all resolved a hardcoded 0:
- upload_stream (Response with a ReadableStream body, Bun.file,
Bun.write(s3file, Bun.file)) via S3UploadStreamWrapper::resolve
- writable_stream (S3File.writer().end()) via wrapper_callback
- Bun.write(path, s3file) via on_file_stream_resolve_request_stream
MultiPartUpload never tracked a cumulative byte count, so there was
nothing to resolve with. Track it as uploaded, bumped on each part and on
the single-PUT response. NetworkSink mirrors that into a wrote field via
on_writable, because the JS sink wrapper can be collected while the upload
is in flight (nothing references it once end() hands back a Promise),
which detaches the task pointer before the completion callback runs.
end_from_js is also reordered so the end promise exists before the upload
is triggered.
The download path already had FileSink.written available; read it instead
of resolving 0 in on_file_stream_resolve_request_stream and the
synchronous-completion branches of pipe_readable_stream_to_blob.
WalkthroughChangesS3 multipart and single-request uploads now track transferred bytes and return those counts from stream and client promises. Blob file-stream completions also return written bytes. New tests cover buffered, streamed, file, multipart, and filesystem write paths. Transferred byte counts
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
Found 1 issue this PR may fix:
🤖 Generated with Claude Code |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline nits: I traced the end_from_js reordering in streams.rs — the removed if !self.ended { ... } block was already dead (the old top-of-function self.end(None) set ended = true before it), and creating end_promise before end() is required so a synchronously-firing wrapper_callback has a promise to resolve. Also confirmed the new (*file_sink).written / (*this.sink).written reads in Blob.rs happen while the sink's +1 ref is still held (released by the immediately-following deref / FileStreamWrapper::Drop).
Extended reasoning...
Native S3/stream code with unsafe pointer derefs and control-flow reordering — deferring to human review. The two inline nits (flush() sync-path 0 vs cumulative, redundant server.stop alongside using) and the three comment-cop flags on the new multi-line comments are already on the thread; this note only records what was checked and ruled out so it isn't re-traced.
… drop redundant stop
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/js/bun/s3/s3-write-return-bytes.test.ts`:
- Around line 70-90: Extend the S3 write tests to directly call the client-file
write API (the S3 client’s write method with a key and Bun.file source) and
assert it returns and receives PAYLOAD bytes. Update the writer contract test
around writer() to call flush() and assert its returned byte count
independently, while preserving the existing end() total-byte assertion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 28ea2155-c8e1-4d87-83c3-2a0955576c61
📒 Files selected for processing (5)
src/runtime/webcore/Blob.rssrc/runtime/webcore/s3/client.rssrc/runtime/webcore/s3/multipart.rssrc/runtime/webcore/streams.rstest/js/bun/s3/s3-write-return-bytes.test.ts
There was a problem hiding this comment.
Both earlier nits (redundant server.stop(true) alongside using, and the two sync flush_from_js return sites still resolving 0) are addressed in 3b3bab0, and this pass found nothing new. Deferring to a human because the change reorders NetworkSink::end_from_js and adds unsafe (*file_sink).written reads across four native files — worth a maintainer's eyes on the lifecycle.
What was reviewed:
MultiPartUpload.uploadedis bumped only on server ack (per-part etag and single-PUT success), never on enqueue, so the resolved count matches bytes the origin confirmed.end_from_jsreorder: traced thatend()never clearsself.taskand the oldif !self.endedblock was dead, so creating the promise beforeend()is behavior-preserving.S3UploadStreamWrapper::resolvereadstask_mut().uploadedwhile the wrapper still holds its +1 task ref (released inDrop); Blob.rs readswrittenbefore the matchingFileSink::deref.- All
NetworkSinkflush/end resolution sites now agree on cumulativewrote; no remaining hardcoded-0 sibling was found.
Extended reasoning...
Overview
Fixes #23407: streamed S3 write/download paths resolved with a hardcoded 0 instead of the byte count. The change adds an uploaded: u64 accumulator to MultiPartUpload (bumped on part-ack and single-PUT success), mirrors it into a new NetworkSink.wrote via on_writable, and threads that value through every promise-resolution site: wrapper_callback, S3UploadStreamWrapper::resolve, both sync branches of flush_from_js, and the detached-task fallthrough of end_from_js. Separately, the three ReadableStream → FileSink resolution sites in Blob.rs now read FileSink.written (a pre-existing Cell<usize>) before releasing the sink ref. end_from_js is refactored to create end_promise before calling end() and to delegate the EOF write to end() in all three branches, deleting a dead if !self.ended block. A new test file exercises six entry points against an in-process mock S3 origin.
Security risks
None identified. No parsing of untrusted input changes, no auth/crypto/permissions surface. The mock server in the test is local (port: 0).
Level of scrutiny
Medium-high. This is a focused bugfix, but it spans four native files with intrusive-refcount and raw-pointer lifecycle invariants (MultiPartUpload, NetworkSink, FileSink, S3UploadStreamWrapper), and the end_from_js reorder changes when the upload's EOF write fires relative to promise creation. I traced each new unsafe deref back to the +1 ref that keeps the pointee live and confirmed the reorder is behavior-preserving (the removed inline EOF block was already dead because the top-level end() had set ended = true), but a maintainer familiar with the S3 sink lifecycle should confirm the wrapper_callback path can never observe a stale sink.wrote (it is last written in on_writable, which fires before completion in every path I traced).
Other factors
All prior review feedback on this PR has been addressed and marked resolved: the comment-cop multi-line-comment flags were condensed, the redundant server.stop(true) was dropped, and both sync flush_from_js return sites now use self.wrote. The bug-hunting system found nothing on the current revision. Test coverage is good — six cases including a real 2-part multipart upload — and the PR description states 5/6 fail on the released build. Given the unsafe-adjacent surface and the lifecycle reorder, I'm deferring rather than approving.
…iter tests under ASAN The writer() tests trip LeakSanitizer on the release-asan lane via a pre-existing NetworkSink leak (fix open in #34999); skip them under ASAN until that lands.
There was a problem hiding this comment.
No bugs found, but this touches unsafe pointer lifecycles across four native files and reorders NetworkSink::end_from_js control flow, so it's worth a human look.
Reviewed: the new uploaded/wrote counters are bumped before on_writable/wrapper_callback read them on both single-PUT and multipart paths; the (*file_sink).written reads in Blob.rs happen before the matching deref; the end_from_js refactor drops a dead if !self.ended block (end() already set it) and creates end_promise before end() can trigger completion; all four flush_from_js resolution sites now agree on cumulative bytes. The ASAN skip for the two writer() tests is scoped and cites #34999.
Extended reasoning...
Overview
The PR replaces hardcoded js_number(0.0) promise resolutions with actual byte counts across five S3/Blob streaming paths. It adds a uploaded: u64 counter to MultiPartUpload (bumped on part-ack and single-PUT success), mirrors it into a new NetworkSink.wrote field via on_writable, and reads FileSink.written before dereferencing the sink in three Blob.rs completion sites. NetworkSink::end_from_js is reordered so end_promise exists before end() runs. A new test file exercises seven entry points against a mock origin.
Security risks
None identified. No untrusted-input parsing, credential handling, or bounds arithmetic changed — the counters are internal accounting derived from data the runtime already sent.
Level of scrutiny
Medium-high. The individual edits are small and the counter additions are mechanical, but this is production S3 upload code with intrusive refcounting, raw-pointer callback contexts, and unsafe derefs whose ordering matters. The end_from_js change is a real control-flow refactor: the old code called end() first (which set self.ended and sent EOF), making the subsequent if !self.ended block dead; the new code consolidates to three end() calls placed after promise creation. That's correct as far as I can trace, but it's the kind of ordering change a maintainer familiar with the S3 state machine should confirm — particularly whether any path can fire wrapper_callback before end_promise is installed (I found none; HTTP completion is always async through the event loop).
Other factors
All prior inline feedback (mine, comment-cop, CodeRabbit) is resolved in 3b3bab0 and 2d452c5. Two new tests are ASAN-skipped due to a pre-existing NetworkSink leak tracked in #34999, which is reasonable but means the writer() paths aren't leak-checked here. The last CI status comment in the timeline is for the pre-fix commit; I don't have a green confirmation for 2d452c5.
…ve bytes Resolving flush() with the cumulative count broke the flush loops in s3.test.ts, which rely on flush() returning the bytes flushed by that call and 0 when nothing is pending. flush() resolution now matches main exactly; end() still resolves with the total uploaded.
There was a problem hiding this comment.
No issues found after the latest revisions — my earlier inline comments have been addressed and the flush() semantics were correctly reverted to the per-call delta after CI showed the cumulative change broke s3.test.ts. Deferring to a human because this touches the S3 multipart state machine, NetworkSink, and Blob→FileSink streaming across four files, and includes a control-flow reorder in NetworkSink::end_from_js (promise now created before end() runs) that goes beyond the headline byte-count fix.
What was reviewed:
end_from_jsreorder: verified the deduplicatedend()call is behavior-equivalent to the old inline EOF logic, and creatingend_promisebeforeend()is safe (value captured before any synchronous resolve).sink.wrotetiming: traced thaton_writable(which mirrorstask.uploadedintosink.wrote) fires beforewrapper_callbackon both the single-PUT and multipart-commit paths, soend_promiseresolves with the full count.Blob.rs:FileSink.writtenis aCell<usize>and each read happens while the +1 ref is still held, beforederef.flush_from_jssync-return sites are back to0.0, matching main and the delta contract the MinIO/R2 flush loops ins3.test.tsdepend on.
Extended reasoning...
Overview
The PR fixes five entry points (S3Client.write with a stream/file source, Bun.write(s3file, Bun.file), S3File.writer().end(), multipart writer().end(), and Bun.write(path, s3file)) that were resolving with a hardcoded 0 instead of the documented byte count. It adds an uploaded: u64 counter to MultiPartUpload (bumped on each part ack and on the single-PUT success), mirrors it into NetworkSink.wrote via on_writable, and reads FileSink.written for the download-to-file path. NetworkSink::end_from_js is also reordered so the end_promise exists before end() triggers the upload, and its inline EOF logic is folded into a call to end(). A new test file exercises all six entry points plus a 2-part multipart upload against an in-process mock origin; the two writer() tests are ASAN-skipped (pre-existing NetworkSink leak, tracked in #34999).
Security risks
None identified. The change is purely about what number a promise resolves with; no new input parsing, no auth/crypto/permission surface. The mock origin is an in-process Bun.serve({port: 0}).
Level of scrutiny
Moderate-to-high. S3 upload/download is a user-facing production path, and the change spans four files with a cross-file invariant (sink.wrote must be set by on_writable before wrapper_callback reads it) plus a control-flow reorder in end_from_js. The PR already went through one break-and-revert cycle on flush() semantics (my earlier suggestion to make flush() cumulative broke the flush loops in s3.test.ts on CI; dd50270 reverted it). That history and the multi-file coupling make this worth a maintainer's eyes rather than a bot approval.
Other factors
All prior review threads (mine, comment-cop, CodeRabbit) are resolved. The bug-hunting system found nothing on the current head. Tests look hermetic (local mock origin, tempDir, using for cleanup, no external network). The ASAN skip is narrowly scoped to the two writer() tests and documents the tracker; the direct-write and download tests remain enabled under ASAN. I traced the on_writable → wrapper_callback ordering on both the single-file (single_send_upload_response calls on_writable before done()) and multipart (drain_enqueued_parts calls on_writable before done() when the queue empties) paths and it holds, so sink.wrote is populated before the end promise resolves.
There was a problem hiding this comment.
No issues found, but I'd like a human to confirm the end_from_js reorder — moving self.end(None) to after end_promise creation changes when the completion callback (which runs finalize() → detach_writable()) can fire relative to promise setup, in refcounted S3 lifecycle code.
What was reviewed:
MultiPartUpload.uploadedis bumped at both ack sites (part etag + single-PUT) beforeon_writablemirrors it intosink.wrote, sowrapper_callbackreads the right total.FileSink.writtenreads happen while the +1 ref is still held (beforederef).flush()was reverted to its per-call delta contract in dd50270; the s3.test.ts flush loops are unaffected.- Prior nits (redundant
server.stop(true), comment length) are addressed; no new findings this run.
Extended reasoning...
Overview
The PR replaces three hardcoded js_number(0.0) promise resolutions with actual byte counts across S3 streamed upload (S3UploadStreamWrapper::resolve, writable_stream's wrapper_callback), NetworkSink::end_from_js, and Blob.rs FileSink pipe completion. It adds a u64 uploaded counter on MultiPartUpload (bumped on part-etag ack and single-PUT success) and a u64 wrote mirror on NetworkSink (populated in on_writable). end_from_js is reordered so end_promise exists before end() runs, and dead post-end() if !self.ended code is removed. A new test file covers seven entry points against a local mock origin.
Security risks
None identified. No user input parsing, no auth/crypto changes, no new external calls. The mock origin is a local Bun.serve({port: 0}); proxy env is cleared so the S3 client actually hits it.
Level of scrutiny
Moderate-to-high. The source changes are small and mostly additive (new counter + reads), but they sit in memory-unsafe Rust with intrusive refcounts and raw callback contexts. The end_from_js reorder is the one real control-flow change: previously end() ran first, so a synchronous completion path (e.g. process_multi_part → done() when buffered is empty and queue is drained) would fire wrapper_callback before end_promise existed and then finalize() would detach task, dropping into the 0.0 fallthrough. The new order creates the promise first, captures its value, then calls end(). I traced this and believe it's correct — end() doesn't touch self.task so the branch selection is unchanged, and the captured value survives wrapper_callback resolving/clearing the Strong — but it's exactly the kind of lifecycle reordering in refcounted networking code that benefits from a maintainer's eye.
Other factors
The PR has been through several review rounds. My earlier suggestion to make flush() cumulative was tried and reverted (dd50270) because it broke the flush loops in s3.test.ts on CI — the current diff leaves flush()'s per-call-delta contract exactly as on main and only changes end(). The two writer() tests are skipped under ASAN due to a pre-existing NetworkSink leak tracked in #34999, so that path isn't sanitizer-validated by this PR's coverage. All comment-cop and prior claude[bot] nits are resolved.
…n.write(dest, response) streams to disk (oven-sh#39690) ### What this changes `fetch()` response bodies now follow two rules, each implemented in one place: **1. Backpressure — one high-water mark.** Body bytes that no consumer has taken yet live in exactly two places: the HTTP→JS hop buffer (`scheduled_response_buffer`, HTTP thread) and the body `ReadableStream`'s internal buffer (JS thread). Whichever side reaches `BODY_HIGH_WATER_MARK` (256 KiB) flips the fetch `Flowing → Paused`; whoever takes bytes out flips it back and schedules the resume. The transport applies `Paused` after its next read (h1: stop polling the socket; h2: withhold `WINDOW_UPDATE`; h3: `want_read(false)`). `.text()`/`.json()`/`.arrayBuffer()`/`.bytes()`/`.blob()`, `Bun.write(file, res)` switch to `BufferAll` (never pause, pre-reserve `Content-Length`); `Bun.readableStreamTo*(res.body)` is never paused either. **2. Abandonment — one path.** When nothing can ever read the rest of a body — the `Response` was garbage-collected with nothing waiting on it, the parked body stream was collected, `reader.cancel()` / `res.body.cancel()`, or a null-body status (204/205/304/HEAD) that still has content on the wire — `abandon_response_body()` marks the fetch `Abandoned`, aborts the transport (h1: close the socket; h2/h3: reset the one stream, session stays pooled), releases the event-loop ref and the native response. Safe inside a GC sweep (no JS is touched). `BodyReceiveMode` is `Flowing | Paused | BufferAll | Abandoned`. Gone: pausing the transport after **every** chunk once a stream is attached (oven-sh#29831), the separate 256 KiB rule for streams (oven-sh#39590) vs. first-packet pause for untouched responses, the `Ignore` mode's "resume and download the rest into the void" path, `is_buffering_body`, and fetch's use of the `response_body_streaming` signal. Also (from the earlier commits on this PR): `.body` / `.textStream()` on a body that already failed now hand out *the* body stream and mark it used (`Body.rs`), and a `Response` collected while `Bun.write(file, res)` waits for it no longer drops the body (fixes oven-sh#40278). ### `Bun.write(dest, response)` streams to disk `Bun.write(path, response | request | readableStream)` used to collect the whole body in memory (via `on_receive_value`) and write it at the end; a bare `ReadableStream` was stringified to `"[object ReadableStream]"`; a `new Response(jsStream)` never settled; and a `Response` collected mid-download left the write pending forever (oven-sh#40278). Now: - A body that is a stream, or whose producer can stream (fetch, `Bun.serve` request bodies, `HTMLRewriter`), is piped into a `FileSink`. Native `ByteStream` sources are wired straight to the sink (`wire_native_sink`, no per-chunk JS); the fetch backpressure above applies, so a download to disk holds at most the high-water mark. **128 MiB `fetch → Bun.write`: peak RSS +161 MB on 1.4.0 → +13 MB here** (debug build). - `FileSink` gains `truncate`/`mkdirp` options (so `Bun.write` replaces the file and creates parent dirs, `writer()` is unchanged) and a completion promise for a piped stream that resolves with the bytes written or rejects with the error that ended the stream *or* the write (sync write error, deferred flush error, `end()` flush error are all recorded; on a write error the source is cancelled so the download stops). - The JS pump (`readStreamIntoSink`) treated a rejected `write()` as success and carried on; it now fails the pump with that error. String chunks are counted by UTF-8 length. - A body that has already fully arrived (also behind an untouched `.body` stream) is written as a blob. A used/locked/disturbed body rejects with `ERR_BODY_ALREADY_USED` instead of writing an empty file. - Types: `Bun.write`/`BunFile.write` accept `ReadableStream` and `Request`. ### S3 - **Upload** (`Bun.write(s3file, response)`, `s3file.write(stream)`, `s3file.writer()`): the multipart uploader's sink already returns backpressure when its part queue is full, so with the fetch change the origin is paced end to end (test: a 16 MiB fetch body through a one-part queue stops the origin while the part PUTs are held). These now resolve with the bytes written instead of `0` (oven-sh#35671). - **Download** (`s3file.stream()`, `Bun.write(file, s3file)`, `new Response(s3file)`): had **no** receive backpressure — the S3 client wired `Signals` without `body_receive_mode` and its producer's `on_ready` was a no-op, so a slow or absent reader buffered the whole object (fake 64 MiB object, stalled reader: origin sent all 64 MiB on 1.4.0). Now `S3HttpDownloadStreamingTask` uses the same `BodyReceiveMode` and HWM rule; `S3DownloadStreamWrapper` does the JS-thread half, resumes on drain, parks an unread stream (loop released, wrapper collectable) and aborts on collection/cancel. Stalled reader → origin stops at ~8 MiB (socket buffers + HWM); `Bun.write(file, s3file)` resolves with the byte count (was `0`); a process holding an unread S3 stream exits. - The source-ref + parked bit + HWM decision that fetch and S3 share is one type, `byte_stream::ProducerHold`. - **Aborting the source of an S3 upload no longer commits a truncated object.** `ByteStream::on_cancel` left a wired native sink attached; the producer's later error was dropped as "already done" but flagged the last chunk, so the multipart sink saw EOF on its next drain and sent `CompleteMultipartUpload`. A cancelled stream now fails its native sink with an `AbortError` (pre-existing on main; test: abort mid-upload rejects and nothing is committed). ### What your server sees, before → after | Client code | Before (main) | After | |---|---|---| | `const r = await fetch(u); if (!r.ok) return;` — body **≤ 256 KiB**, `r` still reachable | Transfer stalls after the first packet; connection pinned until `r` is GC'd, then the rest is read and the connection pooled | Transfer completes immediately; connection back in the keep-alive pool immediately | | same, body **> 256 KiB** (or endless), `r` later GC'd | Stalls after first packet; on GC the **entire remaining body is downloaded and discarded** (a 1 GiB or infinite body is read in full), connection pooled | Stalls at ~256 KiB + kernel socket buffers; on GC: h1 connection closed (server sees `ECONNRESET`/`EPIPE`), h2/h3 `RST_STREAM(CANCEL)` on that stream only | | `for await (const c of r.body)` / `getReader()` loop that keeps up | Socket read-poll disabled after every read and re-enabled from the JS thread (2× `epoll_ctl` + timer reset + cross-thread wake per read) | No pausing at all while the reader keeps up; server sees a smoother, un-stuttered send | | reader stalls (slow consumer, `pipeTo` to a slow sink, `Bun.serve` proxy `return fetch(u)` to a slow client) | Paused after each chunk | Paused once 256 KiB is waiting; resumed when it drains. Server sees TCP/h2 flow control kick in the same way, slightly later | | `r.body` touched but never read, `r` kept | Paused at 256 KiB (oven-sh#39590) | Same | | `reader.cancel()` / `AbortSignal` | Connection closed / stream reset | Same | | `205` (or other null-body status) framed **with** content | Extra bytes drained, connection pooled | Connection closed (RFC 9110 forbids content here) | | `r.text()`, `Bun.write(path, r)`, `Bun.readableStreamToText(r.body)` | Never paused | Never paused | ### RSS / memory - **Streaming consumer that keeps up:** unchanged (bytes pass straight through), minus the per-read syscalls. - **Stalled stream / untouched-but-reachable `Response`:** bounded at ≤ 256 KiB per side + one socket read (worst case ~512 KiB + one read if the JS thread is blocked while the stream already holds data). Before: an untouched `Response` held one packet (but pinned a connection per response); a touched-but-unread stream held 256 KiB. - **Abandoned + collected long body:** before, bandwidth and CPU for the whole remaining body (nothing retained, but it was all received, decompressed and thrown away); now nothing further is received. - Holding thousands of unread `Response` objects alive now costs up to 256 KiB each instead of one packet each — in exchange their connections are returned instead of pinned. Read or drop responses you don't need. ### Other observable changes - Chunk sizes seen by `getReader()` on a fast link are closer to wire/read granularity (more, smaller chunks) because the transport is no longer stop-and-wait per chunk. Total bytes and ordering are unchanged. - A `Response` collected while a *short* body is still mid-flight is aborted rather than finished-and-pooled. In practice a sub-256 KiB body that is `Flowing` completes in the same few milliseconds, well before a GC finalizer runs; we chose one rule over a "drain if small" special case (this is also what undici and Chromium do). - Bodies error earlier: a truncated/invalid short body is now detected as it arrives rather than when a reader first attaches (hence the `Body.rs` change so `.body`/`bodyUsed` behave on an already-failed body). ### How this compares | | Backpressure | Unread body, handle dropped | |---|---|---| | **undici (Node `fetch`)** | Pull-based: socket `pause()`d whenever the body stream's queue is non-empty (effectively per-chunk stop-and-wait, single thread) | `FinalizationRegistry` on `Response` → `body.cancel()` → request aborted → **socket destroyed**. undici docs tell you to always consume or cancel the body for this reason | | **libcurl** | No internal buffering; write callback per chunk, `CURL_WRITEFUNC_PAUSE` stops reading | Removing/cleaning up an easy handle mid-body = premature end → **h1 connection closed** (not reusable), h2 stream reset | | **Chromium** | Network service fills a 512 KiB data pipe; stops reading when full | Body/Response dropped → loader cancelled → **h1 connection closed** unless the body already completed, h2 `RST_STREAM` | | **Go `net/http`** | Reads on demand from `resp.Body` | `Body.Close()` with unread bytes → **connection not reused** | | **Bun (this PR)** | 256 KiB high-water mark across the two internal buffers; HTTP thread keeps reading while under it | ≤ 256 KiB bodies complete on their own and pool the connection; longer ones are aborted on GC / cancel (h1 close, h2/h3 stream reset) | So Bun ends up on the Chromium model (bounded pipe + cancel on drop), with the extra property that small bodies never need the consumer to show up for the connection to be reused. ### Tests No sleeps or quiescence polling in the tests this PR adds: waits are promises from the origin/bucket (first blocked write, Nth close/request, first part), `proc.exited`, `fs.watch`, or a `WeakRef` going empty with one full GC per event-loop turn. `test/js/web/fetch/fetch-backpressure.test.ts` (blocks "a Response whose body nothing touches", "body stream nothing is reading", "does not hold the process", "buffered consumers are not throttled", "peer … while receive is paused"), `fetch-backpressure.test.ts` block "S3 receive backpressure" (stalled reader pauses, Bun.write(file, s3file) byte count, unread stream doesn't hold the process, collected stream aborts — all fail on 1.4.0), `test/js/bun/io/bun-write.test.js` (block "Bun.write(path, response) streams the body to the file": streaming proof via bytes-on-disk before the origin finishes, collected Response, touched body, JS-stream body, Request body, bare ReadableStream, `/dev/full` rejections, used-body rejection), `body-mixin-errors.test.ts` (failed-before-read `.body` / `.textStream()`), `body.test.ts` (205 with content), `regression/issue/33227`, `fetch-response-finalizer-sweep`, `fetch-stream-cancel-leak`, `fetch-abort-stream-body`, `fetch-http2-client`, `fetch-keepalive`, `fetch-tcp-keepalive`, `body-stream` (9086), `stream-fast-path`, `body-clone`, `proxy.test.ts`, `filesink`, `spawn-stdin-readable-stream`, `streams.test.js`, `serve.test.ts` pass on the debug build. Pre-existing on this machine's debug build and unchanged by this PR: the four h3 "stalled …" cases and `fetch.stream` "multiple parts" brush the 5 s default under full-file concurrency (pass in isolation), `abort-signal-leak` (2×2500 aborted fetches take ~8 s in debug), `fetch-abort-socket-close-race` TLS case. Fixes oven-sh#40278. Fixes oven-sh#13237. Closes oven-sh#40333. Closes oven-sh#32906. Closes oven-sh#31739. Closes oven-sh#31689. Closes oven-sh#38184. Closes oven-sh#35671. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 4 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/fetch/fetch-response-finalizer-sweep.test.ts, test/js/web/fetch/fetch-backpressure.test.ts, test/js/web/fetch/body.test.ts, test/js/bun/io/bun-write.test.js <!-- robobun:evidence:end --> --------- Co-authored-by: Jarred Sumner <jarred@jarredsumner.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2 participants
Problem
S3Client.write/Bun.write(s3file, ...)/S3File.writer().end()/Bun.write(path, s3file)are all typed and documented as resolving with the number of bytes written. The buffered-source path (Uint8Array,Blob,Responsewith a string body) already returns the true count, but every streamed path resolves0:Any
n === expectedverification, progress accounting, or copy audit that trusts the documented return sees a spurious zero-byte transfer precisely on the large/streaming path where it matters.Cause
Three hardcoded
JSValue::js_number(0.0)resolutions:S3UploadStreamWrapper::resolve(src/runtime/webcore/s3/client.rs), which backss3.writewith aReadableStream/Bun.file/S3-to-S3 source.wrapper_callbackinsidewritable_stream(client.rs), which backsS3File.writer().end().on_file_stream_resolve_request_streamand the synchronous-completion branches ofpipe_readable_stream_to_blob(src/runtime/webcore/Blob.rs), which backBun.write(path, s3file)and any otherReadableStream -> FileSinkpipe.MultiPartUploadhad no cumulative byte counter to resolve with; the per-partflushedpassed toon_writablewas never a running total.Fix
MultiPartUploadgains anuploaded: u64, bumped on each part acknowledgement and on the single-PUT success response.S3UploadStreamWrapper::resolvereads it from the owned task pointer.NetworkSinkgains awrote: u64, mirrored fromtask.uploadedon everyon_writablecallback, andwrapper_callbackresolvesend()with that. Reading throughsink.taskinstead is unreliable: onceawait writer.end()suspends, the JS sink wrapper is eligible for collection (nothing references it past the returned promise), and its finalizer runsdetach_writable(), clearingsink.taskbefore the HTTP completion callback fires.end_from_jsis also reordered soend_promiseexists beforeend()triggers the upload.writer().flush()is untouched: it keeps its existing per-call delta contract (bytes flushed by that call,0when nothing is pending), which the flush loops ins3.test.tsdepend on.FileSink.writtenalready tracks bytes landed on disk; read it when resolvingon_file_stream_resolve_request_streamand theFulfilled/fallthrough branches ofpipe_readable_stream_to_blob.Verification
test/js/bun/s3/s3-write-return-bytes.test.tscovers all six entry points (including directS3Client.write(key, Bun.file(...)), the #23407 repro) plus a 2-part multipart upload against an in-process mock origin. The streamed cases fail on the released build and all pass with this change; relateds3-*.test.tsandbun-write.test.jscases are unchanged. The twowriter()tests are skipped under ASAN:writer()leaks itsNetworkSinkon main (pre-existing, fix open in #34999) and the new coverage trips LeakSanitizer on the release-asan lane.Fixes #23407
[review] gate passed · iteration 5 · 5 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 1 passed · 1 rejected · iteration 5
evidence per changed file