-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Header order gets rearranged for duplicated header names #2572
Comments
Yes it does, this is a known implementation detail in hyper. Headers are stored in a hashmap, with duplicate values being placed in into the same index, just next to the first value. When iterating, since it doesn't have a unique entry, the value is iterated just after the initial pair. Spec-wise, this should be fine. The order of values for a specific name are always preserved in relation to each other (since for example, order of
How does it make it harder to understand? The semantic meaning is the same. If someone wanted to always see the exact bytes from the socket, they could be printed from the read callback of a |
Sure, it is fine HTTP wise, I'm not arguing about that. I do however have an application where I drop in hyper and want to make sure the replacement is as invisible as possible and this is another thing that makes it harder for me to do that. I'm saying that I want the API (be able) to deliver the headers in the same order as they come over the wire. Imagine users dumping the headers from a server in an automatic fashion to do something with them. After an upgrade of the client, the headers suddenly appears in a new order without the server doing anything different. This will surprise (some) users. (The difference will also put some friction for the curl test suite since the outputs aren't identical anymore, but that's a more minor concern.) When I speak of "harder to understand" I mean that curl is often used to debug HTTP and server issues. And in servers, there are often seveal different components or "layers" that can provide headers to the response. If there are three different components that does this: A, B and C then in a typical case they could output their headers in a serial fashion. First A's headers, then B's headers and last C's headers. Fire up curl against the server and see all the headers in the response arrive fine. Enter curl+hyper that now have shuffled the order of the headers. Now curl no longer shows things in the same order they flew over the wire. The headers that came in an order that made sense from the server, and that you as a server admin understand and can map back to A, B and C you can no longer do that because suddenly C's (duplicate) header appears among A's headers. Confusing. And to an end user it also looks rather arbitrary. |
So, this left me thinking about what curl actually wants from hyper, vs what hyper tries to provide for Rust. While it tries to implement transfer semantics of HTTP, it also provides in it's Rust API a conversion from the bytes to more structured data, such as a hashmap of headers. It doesn't seem like curl really wants to use that hashmap, as long as hyper manages internally things like content-length or transfer-encoding. What if... the C API provided an option to ask for the original header data, as a single unaltered buffer? It'd be cheap in hyper, just a ref count bump on that buffer, and in curl you wouldn't need to re-stitch together the headers into lines, or have a discrepancy of whether it was It could just be an option set on either struct hyper_buf *hyper_response_headers_raw(struct hyper_response *resp); |
That sounds like pure goodness to me! |
Alright, I opened #2575 to track add that (shouldn't be too hard). I should probably also point out that even when grabbing that buf, you don't need to handle content-length or transfer-encoding yourself, and if there are any other specific headers you want to grab and act on, you can still query for them specifically on the |
I'm having second thoughts on this route, since using this buffer would basically mean that I have to parse the headers on my own anyway which would go against one of the primary reasons for me to use hyper in the first place. |
I have a HTTP client test case that receives the following response:
The key take-way is that there are two
Location:
headers. My client downloads the response and then compares that it stored exactly what was sent. The order of the headers as delivered by hyper is however modified from the original order, which makes comparing really hard - and it makes it harder to understand what's going on over the wire when we can't show the same order.Hyper delivers the headers in this order:
The text was updated successfully, but these errors were encountered: