Skip to content

Normalize the request-target path before matching Bun.serve routes - #33096

Closed
robobun wants to merge 7 commits into
mainfrom
farm/dc5e99a7/serve-route-target-normalization
Closed

robobun wants to merge 7 commits into
mainfrom
farm/dc5e99a7/serve-route-target-normalization

Conversation

@robobun

@robobun robobun commented Jun 29, 2026 •

Copy link
Copy Markdown
Collaborator

What

Bun.serve dispatched routes by matching the raw request-target, while request.url is built with the URL parser, which resolves dot-segments (including their %2e spellings), treats \ as /, and ends the path at #. A raw request-target can therefore reach a different handler than the one request.url names. Browsers and fetch() normalize before sending, so only a raw socket client can produce these spellings.

Bun.serve({
  port: 3000,
  routes: {
    "/admin/x": req => new Response(`admin route ${req.url}`),
    "/w/*": req => new Response(`wildcard route ${req.url}`),
  },
  fetch: req => new Response(`fallback ${req.url}`),
});
GET /admin/x HTTP/1.1          -> "/admin/x" route   req.url: http://host/admin/x
GET /admin/../admin/x HTTP/1.1 -> fetch() fallback   req.url: http://host/admin/x
GET /w/../admin/x HTTP/1.1     -> "/w/*" route       req.url: http://host/admin/x

Handlers attached to a route (and anything keyed on which route matched) can be skipped by the raw spelling, while the handler that does run, and everything it logs, observes the normalized path. The %2e/%2E spellings of . and .., backslashes (which the URL parser turns into /), and # behave the same way.

Cause

HttpRouter::route() receives the raw target (HttpContext.h for HTTP/1.x, Http3Context.h for HTTP/3), while request.url goes through WTF::URL, which performs WHATWG normalization.

Fix

Normalize the path inside HttpRouter::route() (packages/bun-uws/src/HttpRouter.h) the same way the URL parser does: resolve . and .. segments including their %2e spellings, treat \ as /, and end the path at # (both callers already strip the query before routing). Paths that need no normalization, which is the common case, are matched exactly as before with no copy. Registered route patterns are untouched, empty segments are preserved, and normalization never walks above the root.

Deliberately unchanged, both in the percent-encoding family rather than the path-structure family this PR fixes:

  • No percent-decoding is applied before matching: /p/%73ecret still matches /p/:v rather than a literal /p/secret route, the same semantics as Express.
  • No percent-encoding is applied either, so a raw byte in the set the URL parser escapes (for example a literal backtick, {, or a non-ASCII byte) is still matched as sent while request.url shows it percent-encoded. Neither direction can change path structure (no new separators, no dot-segments), and both deserve one consistent policy in a follow-up rather than being half-changed here.
  • node:http servers register only /*, so handler selection cannot change, and IncomingMessage.url still exposes the raw request-target.

Verification

The new test in test/js/bun/http/bun-serve-routes.test.ts writes raw request-targets over a plain socket and asserts, for 25 spellings, that the handler that runs is the one request.url implies, that :params still percent-decode, that an encoded / (%2f) still does not split a segment, and that empty segments round-trip (.. pops an empty segment produced by // or \\ rather than the segment before it).

On main, 11 of the 25 spellings are dispatched to a handler other than the one request.url names (for example GET /w/../admin/x is served by the /w/* handler with request.url ending in /admin/x); with this change the test passes. The normalization was also checked differentially against the URL parser (new URL(target, base).pathname) over 37,449 generated request-targets built from /, \, ., a, %2e, %2E, %2f, and # with no disagreement. serve.test.ts, bun-serve-static.test.ts, bun-serve-html.test.ts, and bun-server.test.ts show no new failures, and node:http's IncomingMessage.url still returns the raw request-target.

The route table matched the raw request-target while request.url is built
with the URL parser, which resolves dot-segments (including their %2e
spellings), treats a backslash as a slash, and ends the path at #. A raw
target like "GET /w/../admin/x" was dispatched to the "/w/*" route (or to
the fetch fallback) while the handler observed request.url ending in
"/admin/x", so per-route handlers and guards could be bypassed by the
spelling of the request-target.

Normalize the path inside uWS::HttpRouter::route() the same way the URL
parser does, so routing and request.url agree. Percent-decoding is not
applied. node:http is unaffected: it only registers "/*" and keeps exposing
the raw request-target on IncomingMessage.url.
@robobun

robobun commented Jun 29, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:23 PM PT - Jun 29th, 2026

❌ @robobun, your commit bfe2f3a has 1 failures in Build #66982 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 33096

That installs a local version of the PR into your bun-33096 executable, so you can run:

bun-33096 --bun

@mintlify

mintlify Bot commented Jun 29, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
bun 🟢 Ready View Preview Jun 29, 2026, 7:50 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@coderabbitai

coderabbitai Bot commented Jun 29, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

HttpRouter now normalizes request-target paths before route matching, and a new raw-socket test suite covers the resulting route selection and req.url values. Two documentation pages were also updated.

Changes

URL Normalization in HttpRouter

Layer / File(s) Summary
Normalization buffer, helpers, and route() integration
packages/bun-uws/src/HttpRouter.h
Adds stable backing storage for normalized URLs, implements request-target normalization helpers, and routes through the normalized path before segment matching.
Raw request-target routing tests
test/js/bun/http/bun-serve-routes.test.ts
Adds a server-backed test suite that sends raw HTTP request-targets over TCP and checks route selection plus handler-observed req.url values across normalization cases.

Documentation updates

Layer / File(s) Summary
Base64 example and Web APIs table
docs/guides/util/base64.mdx, docs/runtime/web-apis.mdx
Adds a btoa()/atob() example in the base64 warning section and reformats the supported Web APIs table without changing its entries.

Possibly related PRs

  • oven-sh/bun#33040: Overlaps with the base64 documentation example update in docs/guides/util/base64.mdx.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: normalizing request-target paths before Bun.serve route matching.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The PR description is mostly complete and includes both purpose and verification, even though it uses custom headings instead of the template's exact section names.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/bun-uws/src/HttpRouter.h`:
- Around line 174-177: The comment on the request-path normalization contract is
too long and needs to be reduced to three lines while preserving the key
behavior. Update the documentation near the routing logic in HttpRouter.h so it
still states that handlers see the URL-parser-normalized path, that routes must
match that normalized path rather than the raw request-target, and that
percent-decoding is not performed; keep the wording tighter without changing the
meaning.

In `@test/js/bun/http/bun-serve-routes.test.ts`:
- Around line 1037-1065: The route-normalization table in
bun-serve-routes.test.ts covers fragment-delimited paths but is missing the
query-delimiter variant. Add a case in the existing cases array near the
fragment example in the route-matching test so the same behavior is verified for
a target containing ? and confirms matching stops before the query string. Keep
it aligned with the existing matrix-driven style used in the test.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 923c0a74-033b-41b5-981a-c81bb27d6579

📥 Commits

Reviewing files that changed from the base of the PR and between fb24aac and 95871d7.

📒 Files selected for processing (2)
  • packages/bun-uws/src/HttpRouter.h
  • test/js/bun/http/bun-serve-routes.test.ts

Comment thread packages/bun-uws/src/HttpRouter.h Outdated
Comment thread test/js/bun/http/bun-serve-routes.test.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/bun-uws/src/HttpRouter.h (1)

345-368: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Guard route() against re-entrant use on the same router. normalizeUrl() writes into shared normalizedUrlBuffer, and currentUrl/routeParameters are mutable router state, so a handler that calls route() again can overwrite the outer request’s segments and params mid-dispatch.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/bun-uws/src/HttpRouter.h` around lines 345 - 368,
`HttpRouter::route` is not safe for re-entrant use because `normalizeUrl()`,
`setUrl()`, and `routeParameters.reset()` mutate shared router state that can be
clobbered if a handler calls `route()` again. Add a guard in `route()` to detect
and block nested dispatch on the same router instance (for example with a
re-entrancy flag scoped to the router), and make sure the guard is cleared on
every exit path before returning from `route()` or `executeHandlers()`.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@packages/bun-uws/src/HttpRouter.h`:
- Around line 345-368: `HttpRouter::route` is not safe for re-entrant use
because `normalizeUrl()`, `setUrl()`, and `routeParameters.reset()` mutate
shared router state that can be clobbered if a handler calls `route()` again.
Add a guard in `route()` to detect and block nested dispatch on the same router
instance (for example with a re-entrancy flag scoped to the router), and make
sure the guard is cleared on every exit path before returning from `route()` or
`executeHandlers()`.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: fb9ca620-8f27-49ac-b1e4-4cd280bca2dd

📥 Commits

Reviewing files that changed from the base of the PR and between 95871d7 and cf21cbd.

📒 Files selected for processing (4)
  • docs/guides/util/base64.mdx
  • docs/runtime/web-apis.mdx
  • packages/bun-uws/src/HttpRouter.h
  • test/js/bun/http/bun-serve-routes.test.ts

Comment thread packages/bun-uws/src/HttpRouter.h
Dotfile segments like /.well-known/x start with a dot but are not
dot-segments, so urlNeedsNormalization now classifies the whole segment
instead of only its first byte, keeping those paths on the zero-copy path.
@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

On the re-entrancy note from the automated review: route() is only entered from the HTTP parser callbacks (HttpContext.h onHttpRequest and Http3Context on_stream_headers), which the event loop invokes one at a time on the loop thread, and a handler cannot cause another request on the same router to be dispatched before it returns. currentUrl, urlSegmentVector, and routeParameters are the same per-router state already written by setUrl() on every route() call, so normalizedUrlBuffer does not add a new re-entrancy hazard; a guard flag would be unreachable code.

robobun added 2 commits June 29, 2026 20:20
std::string::rfind and std::string_view::find_first_of pull MSVC's
vectorized find helpers (__std_find_last_trivial_1, __std_find_end_2)
into the Windows x64 baseline binary, which the static instruction scan
rejects for containing AVX instructions. Plain loops over these short
paths avoid the dependency.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the .well-known predicate — that's now exact. I'm deferring to a human reviewer since this adds ~100 lines of path-normalization logic to the per-request hot path in HttpRouter::route() and changes routing semantics in a security-relevant way (route confusion / traversal-style spellings).

Extended reasoning...

Overview

This PR normalizes the HTTP request-target before Bun.serve route matching so that routing agrees with what handlers observe via request.url (which goes through the WHATWG URL parser). The substantive change is in packages/bun-uws/src/HttpRouter.h: a new normalizeUrl() helper plus urlNeedsNormalization(), isSingleDotSegment(), isDoubleDotSegment(), and isEncodedDotAt(), with a per-router std::string normalizedUrlBuffer for stable backing storage, wired into route() via setUrl(normalizeUrl(url)). A new 21-case test matrix in bun-serve-routes.test.ts writes raw request-targets over a TCP socket and asserts both the dispatched route and the handler-observed req.url pathname. The two docs files are autofix whitespace/table-padding reflows with no content change.

Security risks

The change is itself a security hardening — before it, GET /w/../admin/x would be dispatched to the /w/* handler while request.url reports /admin/x, letting a raw-socket client skip route-attached guards or land in a handler whose prefix it has escaped. However, path-normalization code in an HTTP router is a classic high-risk surface: off-by-one errors, mishandling of %2e spellings, backslash-as-separator, or buffer-lifetime bugs here would directly affect every request. The implementation looks careful (length-gated segment classifiers, never walks above root, returns the input view unchanged when nothing needs normalizing, buffer is per-router and route() is not re-entered on the loop thread), and the test matrix is thorough, but this is exactly the kind of change that deserves a second pair of human eyes on the C++.

Level of scrutiny

High. This is new logic in the per-request hot path of HttpRouter for both HTTP/1.x and HTTP/3, and it changes user-visible routing semantics — paths that previously fell through to fetch() or matched a wildcard by raw spelling will now match exact/parameterized routes. The PR description deliberately scopes out percent-decoding and node:http, which is reasonable, but the semantics shift and the performance profile (zero-copy fast path vs. buffer copy) both warrant maintainer review.

Other factors

My earlier inline note about urlNeedsNormalization() over-triggering on dotfile segments (/.well-known/...) was addressed in 3ccbf27 — the predicate now classifies the whole segment via isSingleDotSegment/isDoubleDotSegment, and corresponding test rows were added. CodeRabbit's two style nits (comment length, query-delimiter test case) were also addressed and resolved. The bug-hunting system found no issues. The one CI failure (test-net-connect-memleak.js) is unrelated to this change. Given all that, the code looks correct to me, but the combination of hot-path C++, security relevance, and routing-semantics change puts it outside what I'm comfortable approving without a human.

@robobun

robobun commented Jun 29, 2026 •

Copy link
Copy Markdown
Collaborator Author

CI status: every lane that exercises this change is green, and the remaining failures are unrelated and differ between runs.

Build 66918: test/js/node/test/parallel/test-net-connect-memleak.js on alpine x64 and x64-baseline (GC finalization timing; passes 3/3 locally against this branch), plus the known-flaky Windows update_interactive_install.test.ts (retried).
Build 66928 (retrigger): the same memleak flake on the alpine lanes, a Buildkite artifact download timeout on the macOS 26 aarch64 shard (no tests ran there), and a 90s timeout of test/regression/issue/20965.test.ts on macOS 14 aarch64. That fixture requests a dot-segment-free path (/big), so route selection there is byte-identical with this change, and the test passes locally against this branch.
Build 66982 (bfe2f3a, the self-review follow-up): the same two again, nothing new: the alpine memleak flake and the same macOS 26 aarch64 agent (darwin-aarch64-26-5-1-1) timing out on the artifact download before any test ran.

The only CI failure this PR did cause (the Windows x64 verify-baseline AVX scan) was fixed in 6c00ed4 by not pulling in the MSVC vectorized STL helpers.

Locally against this branch: the raw request-target matrix in bun-serve-routes.test.ts (25 spellings), plus serve.test.ts, bun-serve-static.test.ts, bun-serve-html.test.ts, and bun-server.test.ts show no failures beyond ones reproducible on main, and node:http still exposes the raw request-target.

…segments

Both route() call sites hand the router a target already cut at the first
question mark (HttpParser querySeparator, Http3Request), so the query
branches were dead; the comment now states that precondition. New test rows
cover empty segments: a dot-segment pops the empty segment produced by an
adjacent slash or backslash rather than the one before it.
@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up commit bfe2f3a from a self-review pass on the diff:

  • Removed the query handling inside normalizeUrl: both dispatch sites (HttpParser querySeparator for HTTP/1.x, Http3Request for HTTP/3) cut the target at the first ? before routing, so those branches were unreachable. The function comment now states that precondition instead of implying it handles queries.
  • Added empty-segment coverage to the test matrix (/w//../admin/x, //a/../b, two consecutive backslashes, /p/a/..). A dot-segment pops the empty segment produced by an adjacent separator, not the segment before it; these rows pin that, which the previous rows did not.
  • Corrected the PR description: percent-encoding is out of scope in both directions (no decoding of %XX before matching, and no encoding of raw bytes the URL parser would escape). Neither affects path structure; they deserve one consistent policy in a follow-up.

Also checked the normalization differentially against the URL parser (new URL(target, base).pathname) over 37,449 generated request-targets made of /, , ., a, %2e, %2E, %2f, and # with no disagreement.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found and the earlier nit is addressed, but this rewrites the per-request path normalization in the core HTTP router — security-sensitive enough that a human should sign off on the semantics.

Extended reasoning...

Overview

This PR adds WHATWG-style request-target normalization inside HttpRouter::route() (packages/bun-uws/src/HttpRouter.h) so that Bun.serve route matching uses the same path that handlers observe via request.url. About 100 lines of new C++ implement dot-segment resolution (including %2e spellings), \\ → /, and # termination, with a fast-path predicate that skips the copy when no normalization is needed. A 25-case raw-socket test matrix is added in bun-serve-routes.test.ts. The two doc changes are pure formatting/whitespace reflow with no content change.

Security risks

This is squarely security-sensitive: it changes which handler receives a given raw request-target. The fix closes a routing/request.url mismatch (e.g. GET /w/../admin/x previously hit the /w/* handler while req.url showed /admin/x), but any bug in the new normalizer could itself create a mismatch the other way, or diverge from WTF::URL in an edge case (empty segments, trailing dot-segments, mixed encoded/literal dots). The implementation deliberately does not percent-decode and never walks above root, both of which look correct, and the test table covers the tricky cases I'd worry about — but this is hand-rolled URL normalization on the hot path of every HTTP request.

Level of scrutiny

High. HttpRouter::route() runs for every request to every Bun.serve instance, and the change reimplements a subset of WHATWG URL path semantics rather than calling into the existing parser. The normalizedUrlBuffer member also adds per-router mutable state whose lifetime backs the string_views handed to route parameters; the author's re-entrancy analysis in the thread looks right, but it's the kind of invariant a maintainer should confirm.

Other factors

My one prior inline comment (the conservative /.well-known fast-path predicate) was addressed in 3ccbf27 with added test coverage. CodeRabbit's two style nits are also resolved. The author's CI summary attributes remaining failures to known flakes. No correctness bugs surfaced from the multi-agent review of the latest revision. Given all that, this looks ready for a human reviewer to evaluate the normalization semantics and the buffer-lifetime story rather than for further automated iteration.

@robobun

robobun commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as part of a cleanup of stale pull requests. This PR has had no new commits since 2026-06-29, it conflicts with main, and its last CI run failed. This is not a judgment on the fix itself. If the problem still reproduces on a current build, reopen this PR after a rebase or open a new one against main.

@robobun

robobun commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator Author

Replaced by #42724, which carries this change rebased onto main, with a setup-time gate, an HTTP/3 check and more tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant