Allowing invalid headers in requests

[grnmeira]

I'm not a hundred percent sure how to approach this, but basically I've been having some issues when dealing with requests containing invalid headers.

I noticed this while using hudsucker, a HTTPS proxy lib that uses hyper. I noticed some specific requests that have trouble with how httparse deal with them. While debugging it's clear that the errors I get come from what httparse sees as invalid headers, and also I noticed that httparse has some options to recover from these invalid headers, apparently at first focused on responses, not requests.

So I'm looking for some directions here of what can be done, I'd be happy to help with pull requests if needed too. My first impressions tell me that:

• If I understand correctly, httparse seems to ignore that particular configuration because here it just uses the configuration partially and defaults the configuration before parsing the request's headers. Does it make sense for ParserConfig to have an extra "ignore_invalid_headers_in_requests" field?
• Giving the above, I believe hyper could expose that as a configuration as well for when the low level connection API is used and apply it here when necessary. Looks like ParseContext already has some configurations for responses, we could introduce something similar in this case for requests.

Any help is appreciated.

[nox]

Yes, it does make sense to add a config setting for ignoring invalid headers in requests. We focused on responses back then because that's the only location we found invalid headers, but given you found some in requests that makes sense now.

As for the actual implementation, you first want to start with httparse as you said, maybe take inspiration from the commit that introduces ignore_invalid_headers_in_responses?

[grnmeira]

Thanks @nox. I'll check the contribution docs and give it a shot.

[grnmeira]

PR here: seanmonstar/httparse#145

[grnmeira]

Now that the PR is merged, what are the forces that drive httparse releases at the moment? I have interest in introducing this option for hyper if the idea is welcome.

[seanmonstar]

Thanks for the reminder! The current master has a bunch of performance improvements, and there's one other PR that was going to finish them off. I pinged that one (seanmonstar/httparse#139) to see if it can be ready. Otherwise, we can release! (With all the performance gains, a write up and announcement would be a good idea too).