Skip to content

usockets: dispatch on_end before the poll change in the half-open EOF arm - #44093

Open
robobun wants to merge 2 commits into
mainfrom
robobun/3bf73240/usockets-eof-dispatch-first
Open

robobun wants to merge 2 commits into
mainfrom
robobun/3bf73240/usockets-eof-dispatch-first

Conversation

@robobun

@robobun robobun commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • In the half-open EOF arm of us_internal_dispatch_ready_poll (packages/bun-usockets/src/loop.c), the loop changes the poll to writable and then dispatches on_end. An owner that closes the socket there pays for a poll change it does not need.
  • Per connection that the peer closes first: Bun.serve makes 1 epoll_ctl(MOD), a node:net server makes 2.

Fix

  • Dispatch on_end first. Then change the poll only when the socket is still open and no error close follows: no events for a socket that on_end shut down, writable otherwise.
  • A socket that stays open ends with the same poll events as before.
  • Both counts go to 0.00 (release builds, main a7c73fd against this PR).
  • Verified: three new cases in node-net-allowHalfOpen.test.js, and the existing suites in Notes.

Behaviour change: none

Background

  • A socket with allow_half_open stays open after the peer's FIN. The loop stops the readable poll, keeps the writable poll, and the owner decides when to close.
  • Owners on this arm: uWS HTTP (Bun.serve, node:http), the native handles of node:net, node:tls and node:http2, and Bun.listen / Bun.connect with allowHalfOpen: true.
  • This is groundwork for Bun.listen / Bun.connect: send the whole end(data) chunk before the socket closes #41785, which moves default Bun sockets onto this arm. Considered keeping the order there: it costs one poll change per connection.

Downsides

  • Code size: us_internal_dispatch_ready_poll grows by 52 B and 15 instructions (clang -O3, no LTO). The release binary is 80827976 B before and after.
  • kqueue and libuv were not run locally. The macOS and Windows CI lanes passed on eaffc29.
Notes
  • Control: when the server closes first, epoll_ctl(MOD) is 0.00 per connection before and after.
  • The new cases: a node:net server sends an 8 MiB reply to a client that sent its request and its FIN at once, for allowHalfOpen true and false and for a reply from the 'data' or the 'end' listener. They pass on main too: the PR changes one syscall, not behaviour.
  • Counter: an LD_PRELOAD shim that counts libc calls (no strace on the machine). Per connection = (count at N=300 - count at N=100) / 200, two runs each, identical. The peer is a separate node process that reads the reply and then sends its FIN. Other calls do not change: epoll_ctl ADD 1 and DEL 1, send 1, recv 2, TCP_CORK 0.
  • node:net drops 2 poll changes, not 1: on main the loop arms writable, then the shutdown() that on_end triggers drops it again.
  • us_internal_socket_follow_adopted runs before the poll change because on_end can adopt the socket into another group.
  • Suites run on a debug build of this branch, Linux x64: test/js/bun/net/{socket,tcp-server,socket-syscall-fault}.test.ts, test/js/node/net/{node-net-allowHalfOpen.test.js,node-net.test.ts,node-net-server.test.ts}, test/js/node/tls/{node-tls-socket-allow-half-open-option,node-tls-raw-end,node-tls-server,node-tls-connect,tls-syscall-fault}.test.ts, test/js/node/http/{node-http-server-socket-end-drain,node-http-server-close-drain,node-http-backpressure,node-http-req-socket-pause,node-http}.test.ts, test/js/node/http2/node-http2.test.js, test/js/bun/http/{serve,node-http-halfclose-midupload}.test.ts, the nine test/js/bun/test/parallel/test-net-half-open-peer-reset-*.mjs, and test/js/node/test/parallel/{test-net-allow-half-open,test-net-socket-no-halfopen-enforcer,test-tls-socket-allow-half-open-option}.js.
  • Failures in those runs, all also present on a build of main on the same machine: socket.test.ts 1 (should not call drain before handshake, needs the public internet), node-net.test.ts 11 (10 from localhost resolution in the container, 1 leak test that reaches its 60 s limit on debug builds), serve.test.ts 2 (/bun:info loopback check and the root-range port test, the container runs as root).

… arm

The half-open arm armed the writable poll and then dispatched on_end.
An owner that closes the socket in on_end (uWS HTTP on an idle
connection, node:net with allowHalfOpen false) paid one poll change for
a socket it closed right after. Dispatch first, then change the poll
only when the socket is still open: no events for a socket that on_end
shut down, writable otherwise.
@robobun

robobun commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

Status: groundwork for #41785, no behaviour change.

How the numbers were measured: release builds of main a7c73fd and of this branch, a Bun.serve server and a node:net server under an LD_PRELOAD counter of libc calls, a separate node client that reads the reply and then sends its FIN. Per connection = (count at N=300 - count at N=100) / 200.

  • Bun.serve, client closes first: epoll_ctl(MOD) 1.00 to 0.00
  • node:net server, client closes first: epoll_ctl(MOD) 2.00 to 0.00
  • Bun.serve, server closes first: 0.00 to 0.00

@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 93220a98-53e1-44c3-a297-6387ecb269f1

📥 Commits

Reviewing files that changed from the base of the PR and between eaffc29 and 889ba59.

📒 Files selected for processing (1)
  • test/js/node/net/node-net-allowHalfOpen.test.js

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.


Walkthrough

For half-open sockets that receive EOF, the loop now dispatches on_end before changing the poll mask. It follows any socket adopted by the callback and updates polling only when the resulting socket is open and the event has no error. Tests cover TCP replies after client FIN.

Changes

Half-open EOF Polling

Layer / File(s) Summary
EOF callback, poll-mask update, and TCP tests
packages/bun-usockets/src/loop.c, test/js/node/net/node-net-allowHalfOpen.test.js
The loop dispatches on_end before updating the poll mask and follows the socket returned after possible adoption. It sets no events for a shut-down socket and writable events for other eligible sockets. Tests verify complete payload receipt and socket closure for three server configurations.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 889ba

The change preserves the expected reply behavior after a client closes its sending side, with no identified issue requiring a fix before merge.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: dispatching on_end before changing the poll state in the half-open EOF path.
Description check ✅ Passed The description explains the problem, fix, behavior impact, verification results, test coverage, limitations, and trade-offs. It does not use the template headings exactly, but it provides the require…

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I traced the final poll state for each thing on_end can do (nothing, us_socket_shutdown, us_socket_pause, a partial us_socket_write, adoption via us_socket_adopt, close) against the old WRITABLE-then-on_end order — the resulting events match in every case (shutdown ends at 0 both ways since us_internal_socket_raw_shutdown masks with READABLE; write and resume already gate READABLE on read_eof, which is set before dispatch), and us_poll_change on epoll, kqueue and libuv dedups equal events so nothing extra is issued. The error+read_fin case skipping the poll change is fine because line 956 raw-closes right after. Given this is the shared dispatch path for every half-open socket, a human look is still worthwhile.

Extended reasoning...

The change reorders on_end dispatch and the epoll/kqueue/libuv poll change in the half-open EOF arm of us_internal_dispatch_ready_poll in packages/bun-usockets/src/loop.c, gating the poll change on the socket surviving on_end and selecting 0 events for a shut-down socket. It touches no security-sensitive surface but is the shared event-loop path for Bun.serve, node:net/tls/http2 and allowHalfOpen sockets on all backends. The end poll state was checked to be identical to main for each on_end outcome and the returned handle is validated via us_internal_socket_follow_adopted and us_socket_is_closed before use, but the PR ships no automated test and the code is core native dispatch, so a human review is warranted rather than approval.

Comment thread packages/bun-usockets/src/loop.c

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the updated PR (including the new regression test in 889ba59) and found no bugs; since this reorders user-callback dispatch against the poll change in usockets' core EOF arm, a human look is still worthwhile.

What was reviewed:

  • Traced the three on_end outcomes at packages/bun-usockets/src/loop.c:936-940 against the old order: close (no poll change, error close still runs at line 956), shutdown (us_internal_socket_raw_shutdown masks to READABLE, then the new change sets 0 — same end state as before), keep-open (WRITABLE, as before).
  • Checked writes issued from on_end: us_internal_rearm_writable already omits READABLE when read_eof is set, so the final poll change is a no-op there; deferred TLS shutdown (ssl_shutdown_after_spill) leaves the poll type as SOCKET, so it still arms WRITABLE.
  • us_dispatch_end always returns the socket (uws_sys vtable trampoline), so the NULL-skip cannot leave the readable poll armed on an open socket.
  • The new test wires error/close on both sides to the awaited promises and asserts exact byte count with content comparison; it passes on main too (no behavior change claimed), so it guards the arm rather than proving the reorder.
Extended reasoning...

The change reorders on_end dispatch before us_poll_change in the allowHalfOpen EOF arm of us_internal_dispatch_ready_poll in packages/bun-usockets/src/loop.c, making the poll change conditional on the socket surviving on_end, not being error-flagged, and selecting 0 events when on_end shut the socket down; a describe.each test with three cases is added to test/js/node/net/node-net-allowHalfOpen.test.js. It touches no security-sensitive surface. The diff is small and I traced the resulting poll state as equivalent to the old order for close, shutdown, keep-open, write-from-on_end and deferred TLS shutdown paths, but this is the core socket event loop shared by epoll, kqueue and libuv and the author did not run kqueue or libuv locally, which is why a human look is preferred over approval. The prior run's inline nit asked for a test; the second commit added one, so there is no outstanding objection.

@robobun

robobun commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

@robobun wake up!!

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun wake up!!

@robobun

robobun commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

Here. Status of the stack:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants