net/http: fix, test, document requesting and serving %2F in paths

[bradfitz]

$ cat slash.go
package main

import (
        "log"
        "net/http"
        "net/http/httptest"
)

func main() {
        ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
                log.Printf("Got request for %q", r.RequestURI)
        }))
        defer ts.Close()
        req, _ := http.NewRequest("GET", ts.URL, nil)

        // Try setting Opaque to just a path.

        req.URL.Opaque = "/%2F/%2F/%2F/"
        log.Printf("1. Request URL String = %q", req.URL)
        _, err := http.DefaultClient.Do(req)
        if err != nil {
                log.Fatalf("Do: %v", err)
        }

        println("")

        // Try setting Opaque to just //host/path, so it'll String-ify
        // to a valid full URL (as required for App Engine's urlfetch)

        req.URL.Opaque = "//why-hello-there/%2F/%2F/%2F/"
        log.Printf("2. Request URL String = %q", req.URL)
        _, err = http.DefaultClient.Do(req)
        if err != nil {
                log.Fatalf("Do: %v", err)
        }
}


$ go run slash.go
1. Request URL String = "http:/%2F/%2F/%2F/"
Got request for "/%2F/%2F/%2F/"
                                                                                                                      
2. Request URL String = "http://why-hello-there/%2F/%2F/%2F/";
Got request for "//why-hello-there/%2F/%2F/%2F/"


The 1st works on for client & server with Go's *http.Transport.  (but stringifies
poorly, which means it doesn't work on App Engine).

The 2nd stringifies correctly (so it'll work with App Engine's urlfetch), but it doesn't
send the correct HTTP request with *http.Transport (it should include the scheme, if the
Opaque starts with "//")

[bradfitz]

Comment 1:

Maybe we should also change (note: I'm not saying "fix") App Engine's urlfetch.go to
stringify the URL differently if Opaque is set with no Host, getting the Host from
req.Host)
Currently urlfetch.go is just:
...
       Url:                           proto.String(req.URL.String()),
...

[bradfitz]

Comment 2:

Sent out https://golang.org/cl/7369045/ to update (*net.URL).RequestURI to do
the sane thing with the URL.Opaque begins with "//", in which case it generates an HTTP
request line like:
GET http://host/path HTTP/1.1
Rather than the invalid:
GET //host/path HTTP/1.1

[bradfitz]

Comment 3:

This issue was updated by revision fb21bca.

R=adg, rsc, campoy
CC=golang-dev
https://golang.org/cl/7369045

[bradfitz]

Comment 4:

Now implemented and tested.  Docs are in https://golang.org/cl/7375047

[bradfitz]

Comment 5:

This issue was closed by revision 0462aad.

Status changed to Fixed.