Scoped IPv6 addresses in url

[jtremback]

Is this library meant to conform to https://tools.ietf.org/html/rfc6874 ?

That is, able to accept urls that contain IPv6 address scopes like this http://[fe80::a%25en1]

[SimonSapin]

It is meant to conform to https://url.spec.whatwg.org/, including https://url.spec.whatwg.org/#concept-ipv6-parser which does not appear to accept that input.

[raggi]

I would like to re-open this discussion if possible.

We (Fuchsia) make heavy use of IPv6 link-local in our development workflows. We have a body of existing software that takes IPv6 link-local as a dependency that does TUF over HTTP for both software distribution and various test/command and control operations.

We also regularly run this software on multi-homed devices that have more than one link-local ipv6 interface. Our existing body of software that implements this functionality is in C, C++ and Go, and we are moving a bulk of those particular software components to Rust. We are using this package in that reimplementation, and are very grateful for it. If we can reach consensus that supporting link-local ipv6 can/should be implemented in some way, we are very happy to contribute the relevant changes.

Unfortunately the current state of resolution of this feature is such that there is no valid way to correctly or functionally support link-local ipv6. On a multi-homed system it is incorrect and non-working to ignore the scope.

I have read some of the historical discussions that lead to the current position in WHATWG that appear to be heavily browser weighted. I believe that the browser world has a somewhat different set of constraints and potentially more specialized concerns than a general purpose URL parsing library.

There are good arguments regarding information leakage and potential attack vectors involving the feature set that are in some cases quite speculative, but in almost all cases are very focused on browser threat models. As it stands the WHATWG spec being non-functional for IPv6 link local does not seem to be a major problem for browsers, as link local ipv6 appears to be essentially unused there (implied by the fact that it simply doesn't work if it is implemented as specified by WHATWG). This decision in WHATWG appears to have been made in 2015, when ipv6 deployment was more limited than it is today. I think that general purpose URL parsing is a different problem space than browser specific URL parsing policies. I believe our use case to be correct and valid, and unsupported due to this special case choice.

---

The implementation that is provided by Go is for example a very sensible standpoint:

The parser parses as following the RFC URI specifications - that is, unescape the address, and parse the scope. This produces an address structure that includes the scope.

When producing an HTTP request, the scope is NOT included in the Host: header, so as to avoid leaking local link level information and names to other hosts. The scope is not useful / usable to the server anyway.

When establishing a Socket on which to send the HTTP request, the scope is used to correctly bind.

---

Some additional notes:

There was a suggestion in this comment from @sleevi here: https://www.w3.org/Bugs/Public/show_bug.cgi?id=27234#c2 that the scope is only really important for link local routing. I also believe that to be true, and for our use case, it would be acceptable (to us) if an http client implementation disallows scopes for addresses other than those in the link-local subnet.

We also wonder if it would be acceptable to implement support for this behind a feature flag of some form, along with associated documentation that explains the trade-offs/concerns. As mentioned above, we're happy to do any such work - we're just looking to reach consensus first.

[SimonSapin]

Since the goal of this library is to implement the WHATWG URL standard, there are basically two options:

• There is consensus among browser vendors to change the spec. https://github.com/whatwg/url/issues is the venue to discuss that.

• Or, this library has a long-term maintainer who is interested in extending its scope beyond browser implementations. This is not the case today. See also https://users.rust-lang.org/t/help-wanted-maintaining-rust-url/10707. Perhaps there could be a switch in ParseOptions to opt into an alternative parsing mode.

[ahl]

• Or, this library has a long-term maintainer who is interested in extending its scope beyond browser implementations. This is not the case today. See also https://users.rust-lang.org/t/help-wanted-maintaining-rust-url/10707. Perhaps there could be a switch in ParseOptions to opt into an alternative parsing mode.

I wanted to check if this might have occurred cc: @valenting

[valenting]

@ahl We do have some extra maintainers from the Deno project, but this is a very niche topic, so I doubt they are interested in pursuing it.

For any progress on this to be made, the URL standard would have to make some changes first, which seems unlikely. See whatwg/url#392.