Skip to content

node:url: match node's authority assembly in legacy url.format() - #28752

Open
robobun wants to merge 5 commits into
mainfrom
farm/5903c497/fix-url-format-auth-colons
Open

robobun wants to merge 5 commits into
mainfrom
farm/5903c497/fix-url-format-auth-colons

Conversation

@robobun

@robobun robobun commented Apr 1, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #28751

Legacy url.format(urlObject) assembles the authority differently from node in four independent ways, all inside Url.prototype.format.

Repro:

import * as nurl from "node:url";
console.log(
  JSON.stringify([
    nurl.format({ auth: "u", hostname: "h" }),
    nurl.format({ host: "h", pathname: "p" }),
    nurl.format({ protocol: "file:", pathname: "/x/y" }),
    nurl.format({ protocol: "file", pathname: "//srv/x" }),
    nurl.format({ auth: "u:p:q", host: "h" }),
    nurl.format({ protocol: "http", hostname: "[::1]", port: 8 }),
    nurl.format(nurl.parse("http://u:p:q@h/")), // documented round-trip
  ]),
);
node v26.3.0: ["u@h",   "hp",    "file:///x/y", "file:////srv/x", "u:p:q@h",     "http://[::1]:8",   "http://u:p:q@h/"]
bun 1.4.0:    ["//u@h", "//h/p", "file:/x/y",   "file://srv/x",   "//u:p%3Aq@h", "http://[[::1]]:8", "http://u:p%3Aq@h/"]

Cause:

  • The // is gated on (!protocol || slashedProtocol[protocol]) && host.length > 0. Node gates it on slashes || slashedProtocol.has(protocol), so a protocol-less object gains a separator it should not have.
  • Node gives a hostless file: URL an empty authority (host = "//"), bun gives it nothing. file:/x/y and file:////srv/x are not the same URL: the latter, written as file://srv/x, names a remote host srv.
  • auth is run through encodeURIComponent and then colons are restored with .replace(/%3A/i, ":"). The regex is missing the g flag, so only the first %3A comes back, breaking the documented url.format(url.parse(x)) === x round-trip (and .href, which is built via format).
  • A hostname containing : is wrapped in brackets unconditionally, so an already-bracketed [::1] becomes [[::1]].

Fix: mirror node's Url.prototype.format for each: gate the separator on slashes || slashedProtocol[protocol], keep file:'s empty authority, add the g flag, and skip re-bracketing a hostname that already passes isIpv6Hostname.

src/node-fallbacks/url.js (the bun build --target=browser polyfill) is a copy of the same function and carried all four bugs plus an analogous missing g flag on search.replace("#", "%23"). It gets the same fixes so bundled output no longer diverges from the runtime.

Verification:

  • test/js/node/url/url-format.test.js: four new cases, one per divergence. They fail under USE_SYSTEM_BUN=1 bun test and pass under bun bd test.
  • test/regression/issue/28751.test.ts covers the auth round-trip.
  • test/bundler/bundler_browser.test.ts: NodeUrlFormatAuthority, NodeUrlFormatAuthColons#28751, NodeUrlFormatSearchHash cover the polyfill copy.
  • Exhaustive differential against node v26.3.0 over 99,792 combinations of protocol × host × hostname × auth × pathname × slashes × port: 17,136 outputs differed before, 0 after, for both the runtime module and the polyfill.
  • test/js/node/url/ and the test/js/node/test/parallel/test-url-* ports are unchanged (test-url-parse-format.js, which round-trips format(parse(x)) over a large table, still passes).
Out of scope

url.format(whatwgURL, options) still ignores options and drops auth (already tracked as a TODO in test/js/node/url/url-format-whatwg.test.js, and separately in #32424). This PR only touches the legacy Url.prototype.format path. The file: fix does improve url.format(new URL("file:///x/y")), which previously returned file:/x/y.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization's overage spend limit has been reached.

Code review is billed via overage credits. To resume reviews, an organization admin can raise the monthly limit at claude.ai/admin-settings/claude-code.

Once credits are available, push a new commit or reopen this pull request to trigger a review.

@github-actions github-actions Bot added the claude label Apr 1, 2026
@robobun

robobun commented Apr 1, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:12 PM PT - Jul 6th, 2026

❌ @robobun, your commit b3c7d9c has some failures in Build #69032 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 28752

That installs a local version of the PR into your bun-28752 executable, so you can run:

bun-28752 --bun

@coderabbitai

coderabbitai Bot commented Apr 1, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Auth normalization in Url.prototype.format now unescapes all case-insensitive %3A sequences and query # characters are percent-encoded globally. A regression test was added to verify decoded colons in formatted auth and that search # characters become %23.

Changes

Cohort / File(s) Summary
URL formatting logic
src/js/node/url.ts
In Url.prototype.format, replace all case-insensitive %3A occurrences using a global regex (/%3A/gi) when normalizing auth; encode all # characters in search globally (search.replace(/#/g, "%23")).
Regression test
test/regression/issue/28751.test.ts
Added tests: one verifies url.parse() decodes percent-encoded colons in auth and url.format() outputs decoded user:pass in authority; another verifies Url with search containing # returns query with # percent-encoded as %23.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The code changes directly address issue #28751: fixing the regex in url.format() to use the global flag (/%3A/gi) and percent-encoding all # characters in search, with regression tests validating both fixes.
Out of Scope Changes check ✅ Passed All changes are scoped to the linked issue: fixing the colon-replacement regex in auth normalization and encoding # in search strings, with regression tests covering only these specific edge cases.
Title check ✅ Passed The title is concise and clearly related to the legacy url.format changes, though it summarizes them more broadly than the specific auth fix.
Description check ✅ Passed The description covers the PR's purpose, repro, fix, and verification, though it uses custom headings instead of the exact template sections.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/regression/issue/28751.test.ts`:
- Line 4: Replace the in-test dynamic require of the URL module with a
module-scope import: remove the line that declares const url = require("url")
inside the test and add a top-level import for the URL module (e.g., import * as
url from "url" or appropriate named/default import) so the test uses a static,
module-scope import; ensure all references to the symbol url in the test
continue to work after the replacement.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9ded39e8-e2b4-4e95-b0e9-d5d9d5c9b028

📥 Commits

Reviewing files that changed from the base of the PR and between 3ed4186 and bc5a39e9db23cb010a7959c767024562ebbad472.

📒 Files selected for processing (2)
  • src/js/node/url.ts
  • test/regression/issue/28751.test.ts

Comment thread test/regression/issue/28751.test.ts Outdated
@robobun
robobun force-pushed the farm/5903c497/fix-url-format-auth-colons branch from a214a91 to ff32881 Compare April 1, 2026 18:17
@km-anthropic

Copy link
Copy Markdown
Collaborator

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the g flag addition is correct and the regression test covers the fixed behavior.

Extended reasoning...

Overview

This PR modifies one line in src/js/node/url.ts (Url.prototype.format): the regex /%3A/i becomes /%3A/gi, ensuring all percent-encoded colons are decoded back when formatting the auth component. A matching regression test is added in test/regression/issue/28751.test.ts.

Security risks

None. The change affects URL formatting output only, making it more faithfully match Node.js behavior. There is no auth bypass or data exposure risk.

Level of scrutiny

This is a minimal, mechanical fix — a single character addition to a regex flag. The intent is unambiguous, the Node.js reference behavior is well-defined, and the test directly exercises the multi-colon case. Low scrutiny is appropriate.

Other factors

The bug hunter identified a pre-existing, analogous missing g flag on search.replace("#", "%23") at line 541 (not introduced by this PR). That is a separate issue for a follow-up. The CI failures shown in the build comment are unrelated (libuv/tinycc clang warnings on aarch64 and a build-cpp failure), not caused by these changes.

Comment thread src/js/node/url.ts
@robobun
robobun force-pushed the farm/5903c497/fix-url-format-auth-colons branch from e799d72 to 5646fa1 Compare April 1, 2026 19:26

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/regression/issue/28751.test.ts`:
- Around line 1-2: Add a single-line GitHub issue URL comment at the top of the
test file referencing the issue from the filename (use //
https://github.com/oven-sh/bun/issues/28751); place it before the existing
imports (before the lines importing expect/test and format/parse/Url) so the
regression test follows the repo convention.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1b7072ef-6f71-40cf-83ab-f0c4bd7cadc7

📥 Commits

Reviewing files that changed from the base of the PR and between e799d722c174ab758e3896d449744da85f1d2db3 and 5646fa17870657ec420c505097c363bf5525ad08.

📒 Files selected for processing (2)
  • src/js/node/url.ts
  • test/regression/issue/28751.test.ts

Comment thread test/regression/issue/28751.test.ts Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — both the auth colon and search hash fixes are correct and the regression tests cover both cases.

Extended reasoning...

Overview

Two one-line regex fixes in Url.prototype.format in src/js/node/url.ts: adding the g flag to /%3A/i (auth colon restoration) and replacing the string-based search.replace("#", "%23") with search.replace(/#/g, "%23"). A new regression test file covers both behaviors.

Security risks

None. Both changes only affect URL serialization output fidelity; there is no auth bypass, injection surface, or data exposure introduced.

Level of scrutiny

Low. Both changes are minimal, surgical, and mechanically correct — adding a missing g flag is a well-understood JavaScript regex fix with no side effects beyond replacing all matches instead of just the first.

Other factors

All prior review feedback (coderabbit’s module-scope import suggestion and my own pre-existing # encoding bug report) was addressed in follow-up commits. No bugs were found by the automated bug hunting system. The test cases are clean and directly validate the fixed behavior.

@robobun
robobun force-pushed the farm/5903c497/fix-url-format-auth-colons branch from 5646fa1 to aeadc9e Compare April 1, 2026 19:49

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — both missing g flags addressed and covered by regression tests.

Extended reasoning...

Overview

This PR touches two lines in src/js/node/url.ts (Url.prototype.format) and adds a new regression test file. Both changes are minimal, mechanical fixes that add missing g flags to .replace() calls.

Security risks

None. These are pure string-encoding corrections with no auth, crypto, or permission implications.

Level of scrutiny

Low. The root cause is well-understood (string .replace() without a regex g flag only replaces the first match), the fix is a one-character addition, and the behavior is directly verifiable against Node.js output. The test cases cover both the auth colon restoration and the search hash encoding.

Other factors

My prior review flagged the search.replace("#", "%23") companion bug (missing g flag, same issue class). The author fixed it in e799d72 and added a second test case. All reviewer feedback has been addressed and the PR is complete.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Closing this PR because it has been inactive for more than 90 days.

@github-actions github-actions Bot closed this Jul 1, 2026
@robobun robobun reopened this Jul 1, 2026
@robobun
robobun force-pushed the farm/5903c497/fix-url-format-auth-colons branch 2 times, most recently from ebaa154 to eccef71 Compare July 1, 2026 01:35

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — one-character regex flag fix that matches Node.js behavior, with regression tests.

Extended reasoning...

Overview

This PR changes a single character in src/js/node/url.ts — adding the g flag to /%3A/i → /%3A/gi in Url.prototype.format so that all encoded colons in the auth field are restored, not just the first. It also adds test/regression/issue/28751.test.ts with three focused cases (the exact issue repro, an ordinary user:pass sanity check, and a username-only-colon variant). This matches Node.js's implementation of the same line in lib/url.js.

Security risks

None. This is string formatting of the legacy url.format() output; the change strictly makes the output match Node.js more closely (fewer characters left percent-encoded in the userinfo component, which is what Node has always done). No auth, crypto, permissions, or untrusted-input parsing paths are affected.

Level of scrutiny

Low. The fix is mechanical and self-evidently correct — String.prototype.replace with a non-global regex replaces only the first match, and the intent of this line (restore the : separator after encodeURIComponent) clearly requires all occurrences. The search.replace(/#/g, "%23") sibling case I flagged in an earlier round is already fixed on main and confirmed present in the current file.

Other factors

All prior review feedback (module-scope import, issue URL comment, search/# global replace) has been addressed and the threads are resolved. The remaining inline note about src/node-fallbacks/url.js is explicitly a pre-existing issue in an untouched browser-polyfill copy and is non-blocking. The CI failures on the latest build are all scripts/build/ci.ts build-rust/build-cpp infrastructure failures across every platform, unrelated to a pure src/js/ built-in module change. The PR description documents USE_SYSTEM_BUN=1 fail / bun bd pass verification.

Comment thread src/js/node/url.ts

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — all prior feedback addressed; the browser polyfill copy is now fixed and covered by bundler tests.

Extended reasoning...

Overview

This PR fixes a Node.js-compat bug in the legacy Url.prototype.format() where .replace(/%3A/i, ':') only restored the first percent-encoded colon in auth credentials. The fix adds the g flag (/%3A/gi), matching current Node.js behavior (which uses replaceAll). The same one-character fix is applied to the sibling browser-polyfill copy in src/node-fallbacks/url.js, along with the analogous search.replace('#', '%23') → /#/g fix there. Tests are added in test/regression/issue/28751.test.ts (runtime) and test/bundler/bundler_browser.test.ts (browser-target polyfill).

Security risks

None. This is pure string-formatting output in the legacy node:url module — no auth, crypto, filesystem, or network paths touched. The change makes percent-decoding of auth colons more complete, not less; it does not introduce new decoding of untrusted input.

Level of scrutiny

Low. The source changes total three characters across two regex literals and one string→regex conversion, all in copy-pasted implementations of the same well-known Node.js function. The behavior is trivially verifiable by hand (encodeURIComponent('a:b:c') → 'a%3Ab%3Ac'; global replace restores all colons). The PR description confirms fail-before/pass-after via USE_SYSTEM_BUN=1.

Other factors

I reviewed this PR twice previously — first flagging the sibling search.replace('#', ...) bug in src/js/node/url.ts (fixed upstream during rebase), then flagging the untouched browser-polyfill copy at src/node-fallbacks/url.js. Both were addressed; all review threads are resolved. The regression test covers the multi-colon case, the ordinary user:pass case (no behavior change), and the username-only-colon case. The two new itBundled tests exercise the polyfill fix end-to-end via --target=browser. The musl build failures in CI are on scripts/build/ci.ts and unrelated to this change.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the polyfill copy and bundler tests address the last remaining sibling site.

Extended reasoning...

Overview

This PR adds the g flag to two regex-replace calls in Url.prototype.format: /%3A/i → /%3A/gi in both src/js/node/url.ts (runtime node:url) and src/node-fallbacks/url.js (browser-bundler polyfill), plus "#" → /#/g in the polyfill to match the runtime copy. It adds a three-case regression test (test/regression/issue/28751.test.ts) and two itBundled browser-target tests in test/bundler/bundler_browser.test.ts.

Security risks

None. This is output-formatting behavior in the legacy url.format() API — the change makes percent-decoding of colons in the auth field more complete (matching Node.js), not less. No auth, crypto, or permission logic is touched.

Level of scrutiny

Low. The core fix is a one-character regex flag addition that brings Bun's copy of the legacy Url.prototype.format into line with Node.js's reference implementation (which has always used /%3A/gi). The polyfill change is byte-identical. Both copies of the function are near-verbatim ports of Node's lib/url.js, so "match upstream" is the correct bar here and it's met.

Other factors

All prior review feedback is resolved: my two earlier comments (the sibling search.replace("#", ...) bug and the untouched polyfill copy) were both applied, and the CodeRabbit nits (module-scope import, issue-URL comment) are in. The regression test covers the reported repro plus the ordinary and username-only variants; the bundler tests exercise the polyfill path end-to-end. The one CI failure (test-net-connect-memleak.js) is unrelated to node:url. No CODEOWNERS cover these paths.

@robobun

robobun commented Jul 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

The code diff is green and the branch is rebased onto current main. The URL fix (runtime node:url plus the src/node-fallbacks browser polyfill) and its tests pass on every completed lane.

CI is red only on lanes unrelated to this change:

  • darwin aarch64 test-bun: buildkite-agent artifact download timed out after 120s for the darwin-aarch64-build-bun step, so the test job never ran (artifact-download infrastructure, not a test failure).
  • Windows: transpiler-cache (EBUSY on temp-dir removal) and bun install, which CI auto-retried under its flaky category.

An earlier run also flagged test/js/bun/cookie/cookie-map.test.ts; that was a stale-base artifact (the branch predated #33425, which updated the Expires assertions to IMF-fixdate) and is resolved by the rebase. None of these touch node:url or the bundler.

@robobun
robobun force-pushed the farm/5903c497/fix-url-format-auth-colons branch from e037770 to f1ae550 Compare July 6, 2026 09:47
@robobun robobun changed the title Fix url.format() only replacing first colon in auth credentials node:url: match node's authority assembly in legacy url.format() Jul 6, 2026
@robobun

robobun commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Widened the scope of this PR. The missing g flag on the auth colon turned out to be one of four independent divergences in the same ~40-line Url.prototype.format, so the rest are now fixed here too rather than in a second PR against the same function:

  • a protocol-less object gained a spurious // (format({ auth: "u", hostname: "h" }) → //u@h)
  • file: with no host lost its //, and a //-pathname was collapsed into a remote host (file:////srv/x → file://srv/x, host srv)
  • an already-bracketed IPv6 hostname was bracketed again ([[::1]])

Same treatment for the src/node-fallbacks/url.js copy. Exhaustive differential against node v26.3.0 over 99,792 input combinations now shows zero divergences for both copies (17,136 before). Prior LGTMs were against the one-line diff, so this needs a fresh look.

Comment thread src/js/node/url.ts
robobun and others added 3 commits July 6, 2026 13:50
Url.prototype.format runs auth through encodeURIComponent, then restores
colons with auth.replace(/%3A/i, :). The regex was missing the global
flag, so only the first %3A was decoded. Add the g flag so all colons
are preserved through the parse/format roundtrip.

Closes #28751
The browser-bundler url polyfill (src/node-fallbacks/url.js) carried the
same two missing-g-flag bugs that the runtime node:url had: auth colons
were restored with /%3A/i and search # with replace("#", ...), so only
the first occurrence was handled. Match the runtime copy by using /%3A/gi
and /#/g. Covered by browser-target bundler tests.
robobun added 2 commits July 6, 2026 13:50
Three more ways the legacy formatter diverged from node's
Url.prototype.format, all in the same authority block:

- a protocol-less object with a host gained a spurious "//", so
  format({ auth: "u", hostname: "h" }) returned "//u@h". Gate the
  separator on slashes || slashedProtocol[protocol], as node does.
- "file:" with no host lost its "//", turning "file:///x/y" into
  "file:/x/y" and collapsing a "//"-pathname into a remote host
  ("file:////srv/x" became "file://srv/x", host "srv").
- an already-bracketed IPv6 hostname was bracketed again, producing
  "http://[[::1]]:8".

The browser polyfill in src/node-fallbacks carries the same copy of
format() and gets the same three fixes.
The authority-assembly block read this.slashes in both the outer and
inner if conditions, which trips the no-duplicate-conditional-property-access
lint. Read it once into a local, matching the thisHost/thisHostname/thisQuery
pattern already used in this function. Behavior is unchanged.
@robobun
robobun force-pushed the farm/5903c497/fix-url-format-auth-colons branch from 78ab3a6 to b3c7d9c Compare July 6, 2026 13:51
@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Status after #34660 landed (checked on a build of current main, 165dc9f):

format(parse("http://us%3Aer:pass%3Aword@localhost/"))   main: http://us:er%3Apass%3Aword@localhost/   node: http://us:er:pass:word@localhost/
format({ auth: "u", hostname: "h" })                      main: //u@h                                   node: u@h
format({ host: "h", pathname: "p" })                      main: //h/p                                   node: hp
format({ protocol: "file:", pathname: "/x/y" })           main: file:/x/y                               node: file:///x/y
format({ protocol: "file", pathname: "//srv/x" })         main: file://srv/x                            node: file:////srv/x
format({ protocol: "http", hostname: "[::1]", port: 8 })  main: http://[[::1]]:8                        node: http://[::1]:8
format({ ..., search: "?a#b#c" })                         main: http://localhost/?a%23b#c               node: http://localhost/?a%23b%23c

Leaving this open for the polyfill fix. What remains is to rebase, dropping the src/js/node/url.ts, url-format.test.js and 28751.test.ts changes (already covered on main) and keeping src/node-fallbacks/url.js plus the bundler_browser.test.ts cases.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

url.format() in Bun does not decode auth credentials correctly after url.parse()

2 participants