Repository navigation
Conversation
|
Updated 3:07 AM PT - Oct 21st, 2026
❌ @robobun, your commit f899e03 has 1 failures in
🧪 To try this PR locally: bunx bun-pr 46073That installs a local version of the PR into your bun-46073 --bun |
|
Status: reproduced on main with Pull request: #46073 |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
WalkthroughThe TLS input path now retains unread data while an SNI certificate callback suspends the handshake, then replays it after resolution. TLS raw-tap dispatch occurs in the loop before SSL processing. Renegotiation-window tracking now uses seconds. ChangesAsync SNI input buffering
Renegotiation-window tracking
Suggested reviewers: Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to
|
…back is pending A server socket of the fd TLS engine whose SNICallback has not answered closed the connection when any byte was still unread behind the ClientHello. A TLS 1.3 client that sends early data puts a ChangeCipherSpec and application_data records there, so it got no ServerHello. us_internal_ssl_on_data now moves those bytes to the connection. It keeps at most 6 + 16384 bytes: the ChangeCipherSpec record and the early data that BoringSSL skips behind it. More still drops the connection. After the answer, us_socket_sni_resolve drives the handshake as before and then reads the kept bytes through us_internal_ssl_on_data, the reader that a recv() uses. A handshake that the answer ended does not read them. us_socket_sni_resolve also closes a socket whose handshake failed in the re-drive, as ssl_on_writable does: a Bun.listen socket stayed open until its peer left. The raw tap of an upgraded socket moves from us_internal_ssl_on_data to the read dispatch in loop.c, so the raw half is not shown the kept bytes a second time. us_ssl_rare_t keeps its 64 bytes: the renegotiation window start is stored in seconds.
66f2dc2 to
f899e03
Compare
Problem
tls.ServerwhoseSNICallbackanswers later closes a TLS 1.3 client that sends early data:tlsClientErrorECONNRESET, no ServerHello. Node completes the handshake. node:tls: ask for the server name on the stream TLS engine #45471 found this, not a user.SSL_readstops withSSL_ERROR_PENDING_CERTIFICATE, andus_internal_ssl_on_data(openssl.c:2711) closes on any unread byte. The records behind the ClientHello are such bytes.Fix
us_socket_sni_resolvedrives the handshake as before, reads the kept bytes throughus_internal_ssl_on_data, and closes a socket whose handshake failed.loop.c, so the raw half sees kept bytes once.node-tls-server.test.tsandsocket.test.ts(57 new tests, 51 fail on main). Alsotest/js/node/tls,test/js/bun/net, 640 vendored Node tests. Self-reviewed: 18 concerns raised, 16 addressed.Background
openssl.c) reads through one BIO per event loop, a window onto onerecv(). Nothing can wait in it.us_select_cert_cbanswers "retry" while theSNICallbackis pending, which suspends the handshake behind the ClientHello.SSL_do_handshake(misreports a kept close_notify), a read BIO perSSL(every socket pays), no reads while suspended (hides a FIN).Downsides
Notes
Where this comes from. #45471 makes the other server TLS engine (
SSLWrapper, for a stream that is nonet.Socket) ask theSNICallback. Its body says that the fd engine "drops at the first byte" and that this "is reported separately". There is no issue for it. It is not a regression: the suspended handshake arrived with #31155, and 1.4.0 to 1.4.3 close such a client.Who can hit it. All four conditions must hold.
tls.Server, anhttp2secure server, a server-sidenew TLSSocket(netSocket, { isServer: true, SNICallback }), or aBun.listenwith theserverNamehook.https.createServerdoes not ask theSNICallbackon Bun (0 calls in a probe, 1 on Node).SNICallbackanswers after its call returned.Max Early Data: 0(openssl sess_id). A ticket ofopenssl s_server -early_datasays 16384.Repro.
openssl s_server -tls1_3 -early_dataissues a ticket. Thenopenssl s_client -tls1_3 -servername tenant.test -sess_in <ticket> -early_data <file>(OpenSSL 3.5.7) connects to atls.createServerwhoseSNICallbackanswers 50 ms later.subject=... CN=agent3,Early data was rejected,served. The server printssecureConnection tenant.test.Early data was rejected. The server printstlsClientError ECONNRESET.http2.createSecureServeranswers the recorded flight with a ServerHello too.What the source diff holds (+73/-20 in
openssl.candloop.c, and a comment ininternal.h).ssl_hold_inputis new and cold. TheSSL_ERROR_PENDING_CERTIFICATEarm ofus_internal_ssl_on_datacalls it where the old line closed.SSL_ERROR_WANT_READshares that arm and cannot leave a byte unread:BIO_s_custom_readreports a retry only for an empty window.us_ssl_rare_t.sni_heldis one block per suspended connection.us_ssl_rare_freefrees a block that no answer took.us_ssl_rare_tstays 64 bytes, and a static assert holds that: the renegotiation window start is nowuint32_tseconds.time(NULL)has a resolution of one second, so the window is the same.us_socket_sni_resolvetakes the block, callsssl_update_handshakeas before (this sends the flight and arms the retry of a parked write again) and then calls the reader. A handshake that the re-drive ended, by a refusal or by the owner'sshutdown(), does not read the block.ssl_close_if_fatalafter the re-drive, as inssl_on_writable. On main, a handshake that fails in the step after the answer (no shared cipher, for example) leaves aBun.listensocket open until its peer leaves. With the kept bytes, more connections would reach that state.node:tlsdestroys such a socket itself: its events are the same on main and on this branch, on both doors.us_internal_ssl_on_datahas three callers: therecv()inloop.c,us_socket_tls_feed, and nowus_socket_sni_resolve. Only the first brings new bytes from the wire, so the tap is there.upgradeTLSarms the tap after itsus_socket_tls_feed.The bound. BoringSSL skips at most 16384 bytes of rejected early data (
kMaxEarlyDataSkippedinvendor/boringssl/ssl/tls_record.cc). It discards the ChangeCipherSpec before it counts. So the engine keeps 6 + 16384 bytes, and BoringSSL judges what the engine kept after the answer, as it does for a synchronousSNICallback. Measured withopenssl s_client -early_dataof OpenSSL 3.5.7, which sends this early data in one record, and a ticket that allows 16384 bytes:SNICallbackERR_SSL_TOO_MUCH_SKIPPED_EARLY_DATA, alert 10ECONNRESETbefore the answerERR_SSL_TOO_MUCH_SKIPPED_EARLY_DATA, alert 10SNICallbacktoo.ERR_SSL_DECRYPTION_FAILED_OR_BAD_RECORD_MACafter the answer.Other events while bytes are kept. A probe ran 7 sequences on
listen()and onemit("connection", netSocket), each with a ChangeCipherSpec kept and with nothing behind the ClientHello: the answer alone, ahandshakeTimeoutof 1 ms, a write on the server's socket,destroy(),server.close(), a second answer, and a GC.ERR_MULTIPLE_CALLBACK. A write before the answer differs because'connection'of a listening server hands out the TLS socket in Bun and the raw socket in Node. After a plaintext write on the raw socket, the server reportsERR_SSL_WRONG_VERSION_NUMBERin Bun andECONNRESETin Node.Tests. No test needs the
opensslCLI. The rows write records behind the ClientHello of a real client, or replay the first flight ofopenssl s_client -early_data(320 bytes: a ClientHello that offers early data, a ChangeCipherSpec, one record of early data).node-tls-server.test.ts, describe "what the peer sends before the SNICallback answered", on three doors:listen(),server.emit("connection", netSocket), and the same after the socket read the ClientHello (the bytes then arrive throughus_socket_tls_feed). 51 tests run, 45 fail on main. The 6 that pass on main expect a drop before the answer.ALPNCallbackand in theSNICallbackof another connection, and an answer from the'data'listener of another TLS socket. The recorded flight: the handshake is open at 16384 skipped bytes and over at 16385.TLSSocket, which reports on the socket itself.socket.test.ts, 5 rows forBun.listen: a kept ChangeCipherSpec, a handshake that fails on a kept record or in the step after the answer (the engine closes it), andshutdown()before the answer (one report, the record is not read).ssl_close_if_fataldeleted 2 / 0, kept bytes shown to the raw tap 2 / 0, a hold that overwrites 2 / 0, the loop-state restore inus_select_cert_cbdeleted 6 / 0. The reset of the read window after the replay decides no test. It keeps the window of the loop from pointing at freed memory.test/js/node/tls(28 files),test/js/bun/net(6 files), the 186 vendoredtest-tls-*and the 454 vendoredtest-net-*,test-https-*,test-http2-*. No test fails that passes on main. The failures on both are: 5 tests that need the public network,test-tls-client-allow-partial-trust-chain.js,test-https-timeout.js, and tests that pass the 5 s limit of a debug build in the build container (tls.DEFAULT_CIPHERS reaches a client,concurrent Workers all see the same CA certificate lists,an ALPNCallback over a Duplex can drop its connection,the final handshake flight and the first write leave in one send()).Measurements (two release builds with ThinLTO, linux x64: main 4354b80, and this change on it).
size: text 106747353 → 106747865 (+512), data and bss equal, file size equal.us_internal_dispatch_ready_poll4427 → 4540 bytes,us_internal_ssl_on_data2143 → 2116,us_socket_sni_resolve497 → 711,us_ssl_rare_free151 → 165,ssl_hold_input282 (new,.text.unlikely). 88 other sections ofloop.candopenssl.chave the same size.ssl_update_handshake,BIO_s_custom_read,BIO_s_custom_write,us_internal_ssl_writevandus_select_cert_cbhave 0 differing instructions.loop.candopenssl.c(qemu-x86_64 -one-insn-per-tb -d exec, per function, client and server in one process):us_internal_dispatch_ready_poll298 → 296.ssl_retry_parked_write.SNICallbackor a synchronous one (both sides): -5.us_ssl_rare_freeis +3 perSSL_free.SNICallbackand nothing kept:us_socket_sni_resolve42 → 74 per answer, no allocation.ssl_hold_input: 0 runs in these workloads. 1 run per connection that replays the recorded flight (1reallocof 37 bytes, 1free).qemu -strace):recvfrom601 → 603,sendto602 → 602,epoll_ctl11 → 11.us_dispatch_dataandus_dispatch_ssl_raw_taphave 0 calls underus_socket_sni_resolveover the describe.strace,perf,valgrindandbloatyare not in the build container.Self-review. 18 concerns, 16 addressed in the code, in the tests or in this text. Not done:
Still open, as on main.
handshakeTimeouton atls.Serverreports the connection and leaves it open, as in Node. So the kept bytes stay until the answer or the close of the socket.SNICallbacktoo (table above). Node serves it.no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/tls/node-tls-server.test.ts, test/js/bun/net/socket.test.ts