Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stalled requests due to delayed hyper server replies #432

Closed
m10e opened this issue Apr 6, 2015 · 3 comments
Closed

Stalled requests due to delayed hyper server replies #432

m10e opened this issue Apr 6, 2015 · 3 comments

Comments

@m10e
Copy link

m10e commented Apr 6, 2015

Bug encountered using:
rustc 1.0.0-beta (9854143cb 2015-04-02) (built 2015-04-02)
hyper, version 0.3.9
OS X 10.10.2, fully updated as of issue creation
Google Chrome, version 41.0.2272.118 (64-bit)
Wireshark, version 1.12.4 (from macports)

I looked at the hyper issue backlog and this might be related to issue #368, however I'm not sure if that issue implicates the hyper server as the problem, while this ticket may do that.

The http server in the reproduction code below serves "/" with a simple HTML file that references a lot of non-existing scripts (a1.js through a25.js). When the browser tries to fetch them, it gets 404s.

When loading 127.0.0.1:1234 in Chrome, the root HTML and most of the aXX.js requests finish, but 5 of them are always pending for a long time. They then finish one by one in exact 5 minute intervals.

Using wireshark, I saw that all the finished requests (whether 200s or 404s) were performed over the same TCP connection. All stalled/pending requests were using one TCP connection per request.

The GET requests from the browser were sent immediately upon the openings of the TCP connections, however the replies were delayed the aforementioned multiples of 5 minutes. The replies were timestamped correctly in the returned HTTP headers (i.e. exactly when they appeared in the TCP stream), so they were actually written to the TCP buffer when they appeared in the stream (hopefully ruling out immediate writes that were delayed due to lack of flushing or similar). It seems like the hyper server is delaying its writing of replies, but I don't know why.

I found some (possibly outdated) information on the Internet about Google Chrome having a max 6 connection per origin policy, so that may be related to the fact that the first opened TCP connection (the one requesting "/") is successfully reused for all requests except for the 5 of them that get their own connections (and delayed replies) in an attempt to parallellize the fetching.

I tested a bit with Firefox as well, but the results were inconclusive so I have nothing actionable to say about them.

If you want more details or if I've made some mistake, just shout. My reply may be slow, as I'm on Europe time. I'd love some help with this, as it's currently a show stopper for my (admittedly unimportant pet) project.

Thanks a lot for working on Rust and its ecosystem!

Here's the reproduction code. Sorry about it being ugly, my Rust is still bad. The unwrap()s are for brevity.

extern crate hyper;

use std::io::{Write};

use hyper::server::{Handler, Request, Response, Server};
use hyper::net::Fresh;
use hyper::status::StatusCode;
use hyper::uri::RequestUri;

fn serve_root(mut res: Response<Fresh>) {
    let data =    br#"<html>
  <head>
    <title>Test</title>
    <script src="/a1.js"></script>
    <script src="/a2.js"></script>
    <script src="/a3.js"></script>
    <script src="/a4.js"></script>
    <script src="/a5.js"></script>
    <script src="/a6.js"></script>
    <script src="/a7.js"></script>
    <script src="/a8.js"></script>
    <script src="/a9.js"></script>
    <script src="/a10.js"></script>
    <script src="/a11.js"></script>
    <script src="/a12.js"></script>
    <script src="/a13.js"></script>
    <script src="/a14.js"></script>
    <script src="/a15.js"></script>
    <script src="/a16.js"></script>
    <script src="/a17.js"></script>
    <script src="/a18.js"></script>
    <script src="/a19.js"></script>
    <script src="/a20.js"></script>
    <script src="/a21.js"></script>
    <script src="/a22.js"></script>
    <script src="/a23.js"></script>
    <script src="/a24.js"></script>
    <script src="/a25.js"></script>
  </head>
  <body>
    <p>Test.</p>
  </body>
</html>"#;
    *res.status_mut() = StatusCode::Ok;
    let mut res = res.start().unwrap();
    res.write_all(data).unwrap();
    res.end().unwrap();
}

fn serve_404(mut res: Response<Fresh>) {
    *res.status_mut() = StatusCode::NotFound;
    let mut res = res.start().unwrap();
    res.write_all(b"Not found.").unwrap();
    res.end().unwrap();
}

struct SimpleHandler;

impl Handler for SimpleHandler {
    fn handle(&self, req: Request, res: Response<Fresh>) {
        match req.uri {
            RequestUri::AbsolutePath(ap) => {
                println!("Begin: {}", ap);
                if ap == "/" {
                    serve_root(res);
                } else {
                    serve_404(res);
                }
                println!("End: {}", ap);
            },
            _ => panic!("Only AbsolutePath handled."),
        };
    }
}

fn main() {
    let sh = SimpleHandler;
    let address = "127.0.0.1:1234";
    let server = Server::http(sh);
    server.listen_threads(address, 1).unwrap();
}
@seanmonstar
Copy link
Member

Yep, this a duplicate of #368. You have 1 thread handling connections, and our current keep-alive loop will keep trying to read on the socket until a Connection: Close header is received, or EOF. It's common of browsers to open a connection to a server, and keep it alive for several minutes for any additional requests that may be needed. This means that 1 thread is busy trying to read on the 1 connection, until the browser closes it, and then the server can handle all the other requests that piled up.

@reem has been working on a Queue of sorts that will still have keep-alive support, but will try new incoming requests before continuing to read from a keep-alive connection.

@seanmonstar
Copy link
Member

Im going to close this as a dupe, but it's definitely a real bug that we wish to fix!

@dashed
Copy link

dashed commented Jun 16, 2016

I'm running into this issue for v0.9.8. For now, I disable keep-alive: i.e. server.keep_alive(None);.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@seanmonstar @dashed @m10e