Skip to content

uws: a split request head leaves the parser before its dispatch - #44210

Merged
Jarred-Sumner merged 4 commits into
mainfrom
robobun/214a4031/http-parser-fallback-head-tunnel
Sep 29, 2026
Merged

Jarred-Sumner merged 4 commits into
mainfrom
robobun/214a4031/http-parser-fallback-head-tunnel

Conversation

@robobun

@robobun robobun commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Supersedes #42789.

Problem

  • node:http, request head in two reads, no body: a 'connect' or 'upgrade' listener that calls socket.end() gets the head again with the next read, FIRST-connect example.test:443 HTTP/1.1\r\nHost: example.test:443\r\n\r\nFIRST-SECOND (new since node:http: the server follows Node (request body, framing, response finish, lifecycle) #43557). Bytes past 17 KB behind the head are lost (1.3.6 to 1.4.2).
  • consumePostPadded (packages/bun-uws/src/HttpParser.h:1671) empties its reassembly buffer after the dispatch, and for one handler answer only.
  • Bun.serve: a handler that closes the socket frees that buffer under the request, heap-use-after-free (1.3.6 to 1.4.2).

Fix

  • The parse frame owns the buffer during the dispatch, so no answer of the handler leaves a dispatched head in the parser or frees it.
  • For node:http, head of such a request is the whole rest of the read, as for a head in one read.
  • Verified: test/js/node/http/node-http-connect.test.ts (18 of 20 new cases fail on main). Suites: Notes.
  • Self-reviewed: 5 concerns raised, 5 addressed.

Background

  • fallback is the parser's std::string for a request head that needs several reads.
  • head is the third argument of 'connect' and 'upgrade'.
  • Considered a new handler answer for a half-open tunnel (HttpContext.h:593): every other answer keeps the defect and the use-after-free.

Downsides

  • head of such a tunnel grows (20,000 bytes against 17,187) and the first 'data' chunk goes. The stream is the same.
  • closeIdleConnections() in the handler of such a request closes the connection, like a head in one read.
  • Release text: +1,536 bytes. A plain POST with such a head copies 127,962 bytes into the dropped head Buffer (before 17,180, whole head 127,905).
Notes

Supersedes #42789: its parser hunk (the buffer that the parse frame owns) is the first part of this change, and its three Bun.serve cases and two fixtures come along unchanged. The tunnel fuzzer reported the two node:http faces as ledger entries 79844 (the repeat) and 79846 (the loss).

Repro, no timers. Node v26.3.0 prints "FIRST-SECOND".

import http from "node:http";
import net from "node:net";
import { once } from "node:events";

const server = http.createServer((req, res) => res.end("plain"));
const got = [];
const ended = Promise.withResolvers();
server.on("connect", (req, socket, head) => {
  got.push(head);
  socket.on("data", d => got.push(d));
  socket.on("end", ended.resolve);
  socket.write("HTTP/1.1 200 Connection Established\r\n\r\n");
  socket.end(); // nothing more to send, it still reads
});
await once(server.listen(0, "127.0.0.1"), "listening");
const { port } = server.address();
const head = "CONNECT example.test:443 HTTP/1.1\r\nHost: example.test:443\r\n\r\n";
const c = net.connect({ port, host: "127.0.0.1", allowHalfOpen: true });
await once(c, "connect");
c.write(head.slice(0, 30));
// A whole request on another connection: the server has read the first part by then.
const barrier = net.connect(port, "127.0.0.1");
barrier.resume();
barrier.end("GET /barrier HTTP/1.1\r\nHost: x\r\nConnection: close\r\n\r\n");
await once(barrier, "close");
c.write(head.slice(30) + "FIRST-");
c.resume();
await once(c, "end");
c.end("SECOND");
await ended.promise;
console.log(JSON.stringify(Buffer.concat(got).toString("latin1")));
server.close();

How it goes wrong on main. The second read completes the head in fallback. socket.end() in the listener shuts the socket down, so the request handler in HttpContext.h answers nullptr. consumePostPadded returns at once. It does not empty fallback, and it does not deliver the part of the read that did not fit into it (the buffer takes maxHeaderSize plus 864 bytes). Since #43557 a node:http tunnel reads after socket.end(). The next read is appended to the old head, and the tunnel gets the whole buffer. Before #43557 nothing was read after socket.end().

Which line is the fix. std::string reassembled = std::move(fallback) ahead of the dispatch removes the repeat and the use-after-free. The head span at the dispatch site removes the loss. The tunnel arm then has nothing left to deliver for node:http.

A third symptom goes with them. Over https, a plain request with a split head whose listener ends the socket raised 'clientError' HPE_INVALID_EOF_STATE at the client's FIN (1.4.0 to 1.4.2 and main, not 1.3.14). node-http-server-abort-events.test.ts has the case.

Not fixed here. An Upgrade request that declares a body still gets no body and no tunnel bytes when its listener calls socket.end() (#44206).

What changes on a path that is right today

  • head for a request head that took several reads is the whole rest of the read, the value that Node gives. A listener that ends its side later got the rest as its first 'data' chunk.
  • What reads fallback sees such a head like a head from one read once it is dispatched: the idle test of closeIdleConnections(), the request timer, the clientError at the peer's FIN.

Measurements. Linux x64. main is 9f70da074, and the other build is this change on it. The rows marked (h) come from a build of HttpParser.h alone with the release flags (clang -O3), outside the repo.

What main this PR
Tunnel streams equal to Node v26.3.0, 112 cells (http, https x connect, upgrade x head in 1, 2, 3 reads x four ways to end in the listener x 6 bytes, 64 KiB, and the listener that ends last) 44 112
Bun.serve fixtures under ASAN, clean of 3 0 3
(h) operator new + delete for each later split head on one connection (2 and 3 reads, 1000 heads) 0 + 0 0 + 0
(h) The same for the first split head (2 reads, 3 reads) 2 + 1, 3 + 2 2 + 1, 3 + 2
(h) Capacity that the parser keeps after a split head 17,280 17,280
(h) sizeof(uWS::HttpRequest) 6472 6472
Release text 80,692,444 80,693,980
consumePostPadded<false>, <true> 5914, 2301 6813, 3196
fenceAndConsumePostPadded<true, true>, <false, true> 1826, 2915 1631, 2915
(h) Instructions for each request, head in one read (Bun.serve, node:http) 8171, 8285 8176, 8282
(h) Instructions for each request, head in two reads 15,368, 15,553 15,406, 15,563
Native onData calls for each split-head tunnel (100 tunnels, lldb) 3 2
head Buffer of a plain POST, 256 KiB of body with the head (lldb): split head, whole head 17,180, 127,905 127,962, 127,905
closeIdleConnections() in the handler, 6 orders: a split head answers like a whole head 3 6
The same 6 orders equal to Node: split head, whole head 3, 5 5, 5
  • The disassembly of fenceAndConsumePostPadded<false, true>, the parse of a head from one read, is the same in the two release builds (717 lines, addresses left out).
  • consumePostPadded gets a larger frame and other registers. That is the 5 instructions more and the 3 less for a head in one read. No operation is added there.
  • Of the 6 orders, res.end() then closeIdleConnections() in the same tick moves away from Node's answer for a split head. A whole head answers that way today (node:http: closeIdleConnections() in the same tick as res.end() closes the connection of that request #44207).
  • Loss and garbage in releases, measured with the official linux x64 builds: 1.3.6, 1.3.14 and 1.4.2 deliver 16,323, 16,323 and 17,187 of 20,006 tunnel bytes and print garbage for req.url in the Bun.serve fixture. 1.3.5 has no head.

What no test in the repo pins. fallback.clear() behind the move, and the block that the parser gets back: they change allocations only. A test needs a hook into the parser. Each other clause fails a new case when it is deleted (the restore of a head that is not complete: the cases with three reads).

Suites on the debug build: node-http-connect, node-http-server-close-drain, node-http-server-abort-events, node-http-transfer-encoding, node-http-maxHeaderSize, node-http-maxHeadersCount, node-http-parser, node-http-req-complete, node-http-req-socket-pause, node-http-server-timeouts, node-http-with-ws, node-http, ws, request-smuggling, websocket-server-upgrade-early-frames, websocket-server-upgrade-reentrant, serve-pending-promise-abort-leak, serve.

The machine was under load from other work. Tests that start a process or move 64 MB reached the 5 s limit there: 2 in node-http-connect, 1 in serve-pending-promise-abort-leak, 2 in node-http-with-ws, 1 in node-http. A debug build of main does the same in those four files (5, 1, 1 and 1, not the same tests). serve.test.ts root range port and /bun:info fail on this machine with every build. No other test failed.

Follow-ups

Not run: macOS, Windows.


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/http/node-http-server-close-drain.test.ts, test/js/node/http/node-http-connect.test.ts

A request head that took several reads is parsed out of the parser's
fallback buffer. consumePostPadded emptied that buffer only after the
dispatch, and only when the handler answered with the same socket.

The parse frame now owns the buffer for the dispatch. The parser gets the
block back when the handler answers with the same socket, and the bytes
back when the head is not complete yet. For node:http the head of such a
request is the whole rest of the read, as for a head that came in one read.

The Bun.serve cases and their two fixtures come from #42789.
@robobun

robobun commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

Status

Reproduced on main 9f70da074 (debug and release builds, Linux x64) with the script in the Notes of the description.

With this change the three cases give Node's output, and the fixtures are clean.

Pull request: #44210

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: da5ea261-e66e-40e7-acad-0474c3e0a499

📥 Commits

Reviewing files that changed from the base of the PR and between 56fe3e8 and 1c45e81.

📒 Files selected for processing (2)
  • packages/bun-uws/src/HttpParser.h
  • src/runtime/server/NodeHTTPResponse.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.


Walkthrough

The parser now reassembles partial request heads and handles bytes following a completed head for node:http. New Bun and Node HTTP tests cover split-head header access, CONNECT and Upgrade tunnel data, and connection shutdown.

Changes

HTTP request handling

Layer / File(s) Summary
Partial request-head reassembly and remainder handling
packages/bun-uws/src/HttpParser.h, src/runtime/server/NodeHTTPResponse.rs
The parser reassembles fallback bytes with the current read, updates node:http request-head and CONNECT remainder handling, and restores the buffer after parsing. The upgrade comment describes the lifetime of request header views.
Bun split-head shutdown regressions
test/js/bun/http/serve-pending-promise-abort-leak.test.ts, test/js/bun/http/serve-split-head-*-fixture.ts
Fixtures split request heads across writes. Tests check header access after server shutdown and request-body rejection after a connection-close response.
Node HTTP CONNECT and Upgrade tunnel tests
test/js/node/http/node-http-connect.test.ts
Tests cover HTTP and HTTPS tunnels with split heads, accompanying and later payload bytes, and different socket shutdown schedules.
Node HTTP server shutdown tests
test/js/node/http/node-http-server-abort-events.test.ts, test/js/node/http/node-http-server-close-drain.test.ts
Tests cover client FIN handling and idle-connection closure when request heads arrive in one read or two.

Suggested reviewers: jarred-sumner, dylan-conway

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 1c45e

The split-head adjustment keeps the CONNECT/Upgrade head span within the current read after parsing consumes the request head. No actionable merge-blocking risk is established.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description clearly explains the problem, fix, scope, limitations, and verification results. It does not use the exact template headings, but it provides the required information in equivalent sec…
Title check ✅ Passed The title is concise, specific, and accurately identifies the main change: fixing split request heads that leave the parser before dispatch.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:00 AM PT - Sep 29th, 2026

✅ @robobun, your commit fe5cd144f89d8ac5fa537bd44d249c3d7497a885 passed in Build #121562! 🎉


🧪   To try this PR locally:

bunx bun-pr 44210

That installs a local version of the PR into your bun-44210 executable, so you can run:

bun-44210 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also checked the new pointer arithmetic at packages/bun-uws/src/HttpParser.h:1437: the head can only end past the had bytes from earlier reads, so the remaining length is at most maxCopyDistance and the widened span stays inside the current read, including when the buffer hit maxFallbackSize. The three exits of the reassembly frame (handler answered another socket, head consumed, head still incomplete) each leave fallback in a defined state and touch no member after a dispatch that may have freed the parser, and the node:http-only skip of dataHandler in the CONNECT arm is covered by the widened head, so no bytes are dropped or doubled on that path.

Extended reasoning...

The change reworks ownership of the split-head reassembly buffer in the vendored uWebSockets parser (packages/bun-uws/src/HttpParser.h, consumePostPadded and fenceAndConsumePostPadded) so the parse frame holds the bytes across the request dispatch, widens node:http req.head for a split head to the whole rest of the read, and skips the CONNECT-arm dataHandler for node:http; the Rust change is a comment only. It touches a security-sensitive surface (HTTP request parsing of peer-controlled bytes, memory safety under handler-driven socket teardown) but adds no new trust boundary. Tests add two spawned ASAN fixtures for Bun.serve and node:http tunnel/abort/close-drain matrices. Not approved because it is a memory-safety fix in a core parser hot path with one inline finding on the widened head copy; a human maintainer should weigh the head-copy cost for plain node:http requests.

Comment thread packages/bun-uws/src/HttpParser.h
The second request can reach a connection that the server has closed.
On Windows and macOS the client then gets ECONNRESET ahead of 'close',
and events.once(socket, 'close') rejects on that 'error'.
Comment thread src/runtime/server/NodeHTTPResponse.rs Outdated
Comment thread src/runtime/server/NodeHTTPResponse.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the current revision and found no bugs; because this rewrites the ownership of the parser's reassembly buffer during the dispatch and adds backward pointer arithmetic on a span into the read buffer, a human look at packages/bun-uws/src/HttpParser.h is still worthwhile.

What was reviewed:

  • fallback state on every exit of the fallback arm: moved back after a complete head (cleared), restored intact on an incomplete head (431 check still runs on the restored length), left empty when returnedData != user (error, upgrade, or a closed socket), and no member of the parser is touched between the dispatch and that return.
  • The req->head.data() - length extension at HttpParser.h:1435: it is in bounds only if the head ends past the had bytes from earlier reads, which the pre-existing errorStatusCodeOrConsumedBytes -= had adjustment already relies on, and ConsumeMinimally breaks after the head so consumedBytes >= had holds for the data += consumedBytes - had step.
  • The skipped dataHandler for a node:http CONNECT with remaining bytes: those bytes are exactly the head span the dispatch just delivered, and the Bun.serve arm keeps the call; the next tunnel read reaches the isConnectRequest early-dispatch at line 1227 with an empty fallback, so the repeat cannot recur.
  • The rewritten NodeHTTPResponse.rs comment still matches UpgradeCTX::reset() nulling request so preserve_web_socket_headers_if_needed cannot deref a dead request.
Extended reasoning...

The PR changes the fallback-buffer arm of consumePostPadded in packages/bun-uws/src/HttpParser.h so a stack-local string owns the reassembled head during the dispatch, sets req.head for node:http to the uncopied tail of the read and extends it backwards inside fenceAndConsumePostPadded, and skips the CONNECT dataHandler for node:http; the Rust change is a comment only, and the rest is tests (two spawned ASan fixtures for Bun.serve, node:http CONNECT/upgrade split-head matrices, closeIdleConnections and https FIN cases). It touches the HTTP request parser that handles untrusted network bytes, a memory-safety-critical surface, though no new parsing of adversarial lengths is introduced. The hunt ran dry with no findings and my reading confirmed the fallback bookkeeping and the head-span invariant, but the change is a use-after-free fix in shared parser core with pointer arithmetic that depends on a non-local invariant, so it is not simple enough to approve without a human. The only prior inline note from this bot was a performance nit that the author resolved; no third-party objections are recorded.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@Jarred-Sumner
Jarred-Sumner merged commit 6fe95ef into main Sep 29, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants