Skip to content

Bun.serve websocket: deliver queued publish() and corked send() before a forced close or process exit - #43711

Open
robobun wants to merge 8 commits into
mainfrom
robobun/ccc42348/ws-flush-publish-before-forced-close
Open

robobun wants to merge 8 commits into
mainfrom
robobun/ccc42348/ws-flush-publish-before-forced-close

Conversation

@robobun

@robobun robobun commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • server.publish() of a message under 16 KB returns the byte count, but no subscriber receives it when server.stop(true), ws.terminate(), server.unref() or process.exit() follows in the same tick. ws.send() inside a handler is lost the same way.
  • us_socket_close closes the fd before WebSocketContext::onClose runs (packages/bun-uws/src/WebSocketContext.h:269). onClose frees the subscriber without a drain and discards the cork buffer.
  • At process exit the loop does not tick again, so nothing drains the TopicTree (App.h:533).

Fix

  • WebSocket::flushBeforeForcedClose() drains the socket's subscriber, then uncorks. WebSocket::close() (ws.terminate()) and the parser's forceClose call it before us_socket_close. An open handler that throws uses close(false), so the corked 101 stays unsent.
  • App::close() (server.stop(true)) drains the TopicTree and uncorks its corked websocket before it closes the groups.
  • New Loop::flushPendingWrites() runs the uWS pre handler. VirtualMachine::on_exit calls it after the exit listeners.
  • Each flush writes only what publish() and send() already reported as sent. Bun.serve websocket: deliver queued publish() messages when unsubscribing from last topic #32852 drains the same way. Verified: six new tests in test/js/bun/websocket/websocket-server.test.ts. Five fail on bun 1.4.2. All pass on a debug build. Self-reviewed: 3 concerns raised, 1 addressed (Notes).

Background

  • The TopicTree is the uWS pub/sub index. A small publish() is stored once, with an index per subscriber. A loop pre or post handler, or the socket's next send(), writes the batch.
  • A cork is a 16 KB per-loop buffer. uWS corks a socket while its data handler runs, so send() frames go out in one write when the handler returns.
  • ws.terminate(), server.stop(true) and parser errors close the TCP socket at once, with no Close frame.
Notes

Repro for the first case (from the report):

const rc = server.publish("all", "bye"); // rc = 3
server.stop(true);                       // subscribers receive nothing

Controls that always delivered: ws.send() from a timer, a publish of 16 KB or more (it bypasses the batch), ws.close() after the publish, setTimeout(() => server.stop(true), 0). The batch also writes itself out at the 32nd pending message, so 40 small publishes then stop(true) delivered exactly the first 32.

Test cases added:

  • send() and publish() then server.stop(true) inside a message handler. The handler's socket is corked, so its publishes drain into the cork buffer. It must receive sent, bye0, bye1, bye2. The other subscriber must receive bye0, bye1, bye2.
  • The same with ws.terminate() on every socket.
  • A child process that publishes from a timer, then calls server.unref() or process.exit(0). Skipped on Windows: the exit resets the connection there and the reset can discard what the client has not read, the same reason serve.test.ts skips its process-exit test.
  • A raw TCP client that writes a legal text frame and an orphan continuation frame in one write. The echo of the first frame must arrive before the 1006 close.
  • An open handler that sends a frame and then throws, after a synchronous server.upgrade(). The raw client must receive no response at all. This passes on bun 1.4.2. It fails on the first version of this PR, where close() flushed the corked 101 and the client saw open, the frame, then a 1006 close.

on_exit runs for a natural exit, process.exit(), and a worker that exits on its own. A worker that its parent terminated runs no script, and the flush skips it. on_exit uses uws::Loop::get() and not vm.uws_loop(): Bun.spawnSync swaps event_loop_handle to an isolated loop that has no uWS LoopData. The drain callback only corks, sends and uncorks, so the flush runs no JavaScript.

Self-review concerns:

  • Addressed: the two process-exit tests were racy on Windows. They now skip there.
  • Not changed: on wss://, stop(true) ends with a reset after the TLS close_notify (packages/bun-usockets/src/context.c:127). That is the existing behavior for every byte written before stop(true), ws.send() included.
  • Not changed: a publish() made from a close handler while stop(true) walks the sockets is still dropped for the sockets that close later in the same walk. A flush there needs a hook before each us_socket_close inside us_socket_group_close_all.

Also not changed: onEnd (the peer sent FIN) still frees the subscriber without a drain. On Linux the FIN is dispatched in a later loop iteration than the data before it (usockets does not register EPOLLRDHUP), so the loop's own drain runs first and a test for it passes with or without a flush there. kqueue can report EOF on the data's event (packages/bun-usockets/src/loop.c, the eof drain), so a half-closing client on macOS can still lose a same-tick publish. I could not produce a failing case on this machine, so that path is left for a separate change.

#43013 edits the same lines of WebSocket::close() and forceClose for a different bug (a TLS forced close that waits for the peer). The two changes compose: the flush goes before whichever us_socket_close call runs.

Suites run on the debug build: test/js/bun/websocket/, test/js/bun/http/serve.test.ts, bun-server.test.ts, serve-listen.test.ts, six files of test/js/web/websocket/, test/js/web/workers/worker.test.ts, test/js/node/process/process.test.js. cargo check --target x86_64-pc-windows-msvc -p bun_uws_sys -p bun_jsc passes.

Local failures that this diff does not cause:

  • websocket-server.test.ts: eight send()/sendText()/sendBinary() tests time out at 10 s on the debug build while the concurrent (benchmark) test runs for 29 s. Each passes alone.
  • test/js/first_party/ws/: debug panic assertion failed: !self.body_read_ref.get().has, the subject of node:http: release body_read_ref when a response upgrades to a WebSocket #43427.
  • serve.test.ts: two tests need a non-root user and open egress. process.test.js: one test needs USER in the environment.

no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/websocket/websocket-server.test.ts

@robobun

robobun commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:33 AM PT - Sep 22nd, 2026

✅ @robobun, your commit b0f092307bd88f15f5911d0711f2fc4996f22930 passed in Build #119624! 🎉


🧪   To try this PR locally:

bunx bun-pr 43711

That installs a local version of the PR into your bun-43711 executable, so you can run:

bun-43711 --bun

@robobun

robobun commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for review.

How I reproduced it: inside a websocket message handler, server.publish("room", ...) three times, then server.stop(true) (or ws.terminate() on every socket). On bun 1.4.2 both subscribers receive nothing, while publish() returned 4 each time. The same loss shows from a timer with server.unref() or process.exit(0) right after the publish, and for a ws.send() echo when the next frame in the same read is an orphan continuation frame.

test/js/bun/websocket/websocket-server.test.ts has six new tests. Five fail with USE_SYSTEM_BUN=1 bun test and pass with bun bd test. The sixth (an open handler that throws fails the handshake) passes on both: it guards the one path that must keep closing without the flush.

@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

Understand this PR’s impact

Explore downstream dependencies and potential security impact with Blast Radius.

View blast radius →

Warning

Review limit reached

  • Run on-demand review

This review includes 9 billable files and costs up to $2.25.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 6 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 8b538d7d-2e68-4540-95bf-c8ebb2bdbc61

📥 Commits

Reviewing files that changed from the base of the PR and between 93d815e and b0f0923.

📒 Files selected for processing (9)
  • packages/bun-uws/src/App.h
  • packages/bun-uws/src/Loop.h
  • packages/bun-uws/src/WebSocket.h
  • src/jsc/VirtualMachine.rs
  • src/runtime/server/ServerWebSocket.rs
  • src/uws_sys/Loop.rs
  • src/uws_sys/WebSocket.rs
  • src/uws_sys/libuwsockets.cpp
  • test/js/bun/websocket/websocket-server.test.ts

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 65aea6ec-ae06-422b-abad-df7a995b2a7c

📥 Commits

Reviewing files that changed from the base of the PR and between eae4888 and 93d815e.

📒 Files selected for processing (8)
  • packages/bun-uws/src/App.h
  • packages/bun-uws/src/Loop.h
  • packages/bun-uws/src/WebSocket.h
  • packages/bun-uws/src/WebSocketContext.h
  • src/jsc/VirtualMachine.rs
  • src/uws_sys/Loop.rs
  • src/uws_sys/libuwsockets.cpp
  • test/js/bun/websocket/websocket-server.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.


Walkthrough

The change adds pending-write flush APIs across uWS and its Rust/C bridge. Shutdown paths flush queued publications and corked frames before closing. WebSocket tests cover same-tick close, process exit, and protocol errors.

Changes

WebSocket write flushing

Layer / File(s) Summary
Loop flush API
packages/bun-uws/src/Loop.h, src/uws_sys/Loop.rs, src/uws_sys/libuwsockets.cpp
Adds synchronous pending-write flushing to uWS loops, the Posix and Windows Rust wrappers, and the C ABI.
Shutdown flush paths
packages/bun-uws/src/App.h, packages/bun-uws/src/WebSocket.h, packages/bun-uws/src/WebSocketContext.h, src/jsc/VirtualMachine.rs
Flushes queued publications and corked data before WebSocket, application, and eligible VM shutdown paths close sockets.
Shutdown delivery regression tests
test/js/bun/websocket/websocket-server.test.ts
Tests delivery before same-tick shutdown, process exit, and protocol failure. Tests also verify publish results and close behavior.

Suggested reviewers: cirospaciari, dylan-conway, jarred-sumner

Priority: ➖ Normal

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: delivering queued WebSocket publishes and corked sends before forced close or process exit.
Description check ✅ Passed The description fully explains the problem, fix, affected behavior, limitations, and verification. It does not use the template headings exactly, but it includes equivalent information for both reques…

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟣 packages/bun-uws/src/WebSocketContext.h — Clients that half-close the TCP connection (send FIN, keep reading) still lose the publish() batch and corked send() frames queued for them in the same tick, while every server-initiated forced close now delivers them. onEnd at packages/bun-uws/src/WebSocketContext.h:381-385 calls uncorkWithoutSending and us_socket_close with no flushBeforeForcedClose. Fix: call flushBeforeForcedClose() in onEnd before the close, the same as forceClose at WebSocketContext.h:78, so the flush covers every forced-close path.

    Why this was flagged

    A non-browser websocket client (node net-based clients, load-testing tools, proxies) shuts down its write side with FIN after sending its last frame and keeps reading. usockets dispatches onEnd (packages/bun-uws/src/WebSocketContext.h:381) on the still-readable socket; uncorkWithoutSending discards any frames a handler corked and us_socket_close runs onClose, which frees the subscriber at WebSocketContext.h:282 without draining. Messages the server published in the same tick that publish() reported as sent are never written although the socket can still deliver them. The base branch behaves the same, so this is not a regression, but the PR's own description names this path as a sibling of forceClose and WebSocket::close that was left without the flush; the dismissal that 'browser clients send a Close frame' does not cover the non-browser population that reaches onEnd. Remedy: invoke flushBeforeForcedClose() in onEnd before us_socket_close.

    Verification: pre-existing — acknowledged in diff: the PR description says "Also not changed: onEnd (the peer sent FIN) still discards the cork buffer and frees the subscriber without a drain"; that note is accurate for the publish batch. Triggering condition: a client writes a WebSocket frame followed by TCP FIN (e.g. a net client doing socket.end(frame)) or a peer's FIN lands in the same epoll batch…

Comment thread packages/bun-uws/src/WebSocket.h Outdated
Comment thread packages/bun-uws/src/App.h
WebSocket::close() now flushes the cork buffer, which after a synchronous
upgrade still holds the 101 response. The open-exception path closes
without the flush so the client does not see an open event.
Comment thread src/jsc/VirtualMachine.rs Outdated
Comment thread src/runtime/server/ServerWebSocket.rs Outdated
Comment thread src/uws_sys/Loop.rs Outdated
Comment thread src/uws_sys/WebSocket.rs Outdated
Comment thread src/uws_sys/WebSocket.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the latest push and found no new bugs; the throwing-open handler case from my earlier comment is now handled by close(false) / close_without_flush() with a covering test. Because this changes forced-close and process-exit ordering in the uWS socket lifecycle (C++ and Rust), a human look would still be worthwhile.

What was reviewed:

  • Loop::flushPendingWrites() invoked from on_exit: the only registered pre handler is the App's topicTree->drain(), and the cork drain calls uncork() only, so no user JS runs; uws_get_loop() / uws_get_loop_with_native() always create LoopData, so the per-thread loop is valid there.
  • App::close() cork-slot uncork: restricted to sockets in this app's own websocket groups, so the AsyncSocket<SSL> cast matches the app's SSL mode; runs before us_socket_group_close_all so the fd is still open.
  • flushBeforeForcedClose() in WebSocket::close() and parser forceClose: checks us_socket_is_closed first; both SSL and non-SSL uws_ws_close thread the flush flag.
Extended reasoning...

The change adds a flush step before forced websocket closes (ws.terminate(), parser protocol errors, server.stop(true)) and at process exit, touching packages/bun-uws/src/{App.h,Loop.h,WebSocket.h,WebSocketContext.h}, the C ABI in src/uws_sys/libuwsockets.cpp, the Rust extern signatures in src/uws_sys/{Loop.rs,WebSocket.rs}, the exit path in src/jsc/VirtualMachine.rs, and the handler-threw path in src/runtime/server/ServerWebSocket.rs. It touches no auth, crypto, or input-parsing surface; the sensitive part is socket/cork lifetime ordering in native code. The latest commit addressed the earlier inline finding about the throwing open handler flushing the corked 101, and six new tests cover the variants; the bug hunt ran dry with no findings. Deferring rather than approving because native socket-lifecycle code and a new exit-path hook are the kind of change a maintainer should sanity-check, and the pre-existing close-handler publish drop noted earlier remains intentionally unaddressed per the PR description.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review completed

Nothing new to post: everything this review found is already covered by existing comments on this pull request or didn't merit a separate one.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants