Skip to content

drop duplicate Host header on pooled HTTP/2 path (bump 0.7.0) - #32

Merged
liquidsec merged 2 commits into
devfrom
fix-http2-duplicate-host-header
May 20, 2026
Merged

drop duplicate Host header on pooled HTTP/2 path (bump 0.7.0)#32
liquidsec merged 2 commits into
devfrom
fix-http2-duplicate-host-header

Conversation

@liquidsec

@liquidsec liquidsec commented May 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

build_request unconditionally added a Host header derived from the URI for every request. For the low-level http1 dispatch paths (dispatch_direct, forward proxy) that's correct — hyper::client::conn::http1::SendRequest does not auto-populate Host and HTTP/1.1 requires it. For the pooled high-level client, hyper_util::client::legacy::Client already populates Host (HTTP/1.1) or :authority (HTTP/2) from the URI itself. Adding our own Host on top meant every HTTP/2 request went out with both :authority and host in the HPACK block, which some origin servers / WAFs reject as a protocol violation (RFC 9113 §8.3.1 implies :authority is the canonical source and most servers expect only one).

How it was found

api.hackertarget.com returned 400 Bad Request to every blasthttp request while curl --http2 to the same host returned 200 with identical headers. HPACK-decoding blasthttp's HEADERS frame showed:

:method: GET
:scheme: https
:authority: api.hackertarget.com
:path: /hostsearch/?q=tesla.com
host: api.hackertarget.com           <-- duplicate
accept-encoding: gzip, deflate, br
user-agent: curl/8.5.0

curl's HPACK had only :authority, no host.

Fix

build_request now takes a manual_host_header: bool flag.

  • dispatch_request (pooled path) → false (let hyper populate the canonical Host/:authority from the URI).
  • dispatch_direct (low-level http1, used by resolve_ip/request_target) → true (raw SendRequest needs the manual Host for HTTP/1.1).
  • dispatch_forward_proxy (low-level http1, forward proxy for HTTP targets) → true (same reason).

Caller-supplied custom Host headers continue to pass through unchanged on every path — virtualhost/host-header probes that override the URI-derived value still work.

Tests

  • Existing test_build_request_auto_host_from_uri / _custom_host_overrides_auto / _multiple_host_headers / _origin_form_strips_authority / _absolute_form_preserves_uri / _request_target_absolute_form updated to pass the new flag with the matching expectations for each path.
  • Added test_build_request_no_manual_host_for_pooled_path: verifies the pooled path emits no Host header from build_request (hyper does it).
  • Added test_build_request_custom_host_passes_through_pooled_path: verifies a caller-supplied Host on the pooled path is still preserved (so virtualhost probes work).
  • All 147 Rust tests + 111 Python tests pass.

Verification

Against api.hackertarget.com from a venv with the built wheel: status 200, valid CSV body, no behavior change to other hosts.

Version

Bump 0.6.1 → 0.7.0.

build_request unconditionally added a Host header derived from the URI.
For the low-level http1 dispatch paths (dispatch_raw, forward proxy)
that's correct -- hyper's raw SendRequest does not auto-populate Host
and HTTP/1.1 requires it. For the pooled high-level client, hyper
already populates Host (HTTP/1.1) or :authority (HTTP/2) from the URI
itself. Adding a Host on top means HTTP/2 requests go out with both
:authority and host in the HPACK block, which some origin servers and
WAFs reject as a protocol violation (RFC 9113 §8.3.1).

Discovered via api.hackertarget.com, which returned 400 Bad Request to
every blasthttp request while curl --http2 to the same host succeeded.
HPACK decoding of blasthttp's HEADERS frame showed both :authority and
host pointing at the same value.

build_request now takes a manual_host_header flag. dispatch_request
(pooled path) passes false; dispatch_raw and dispatch_forward_proxy
(low-level http1 paths) pass true.

Bump 0.6.1 -> 0.7.0.
@liquidsec
liquidsec requested a review from ausmaster May 20, 2026 17:24
@liquidsec
liquidsec merged commit 5775460 into dev May 20, 2026
15 checks passed
@liquidsec
liquidsec deleted the fix-http2-duplicate-host-header branch May 20, 2026 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants