Skip to content

Blasthttp streaming endpoint - #17

Merged
liquidsec merged 3 commits into
stablefrom
async-generator
May 1, 2026
Merged

Blasthttp streaming endpoint#17
liquidsec merged 3 commits into
stablefrom
async-generator

Conversation

@TheTechromancer

@TheTechromancer TheTechromancer commented Apr 27, 2026

Copy link
Copy Markdown

Summary

Adds request_batch_stream — an async-iterator variant of request_batch that yields BatchResults in completion order as each request finishes, instead of awaiting the entire batch and returning a list. A slow request no longer blocks faster peers behind it in the input list, and Python work can overlap with in-flight HTTP I/O.

image

The existing request_batch is unchanged.

async for r in client.request_batch_stream(configs, concurrency=100):
    if r.success:
        ...

What changed

Rust

  • New send_batch_stream in src/batch.rs returning impl Stream<Item = BatchResult>. Each request runs as its own tokio::spawn'd task and pipes results through an mpsc::unbounded channel — so HTTP work keeps progressing while the consumer is busy iterating the previous batch (see "Throughput" below).
  • New PyBatchResultIterator pyclass in src/python.rs with __aiter__/__anext__. __anext__ drains the underlying stream into a Vec<PyBatchResult> capped at 1000 items or 200ms to amortize the Python↔Rust boundary cost. Pattern mirrors blastdns::PyBatchIterator.
  • merge_limiters helper extracted from send_batch and reused by both batch functions so the shared-vs-per-call rate-limit selection logic stays in one place.

Python packaging

  • Switched to maturin's mixed Python+Rust layout: module-name = "blasthttp._native", python-source = "python".
  • New python/blasthttp/__init__.py re-exports the native types and defines a thin BlastHTTP wrapper that:
    • forwards everything to _native.BlastHTTP via __getattr__ (no per-method boilerplate),
    • implements request_batch_stream as an async def generator that unwraps each batch into individual BatchResult objects.
  • The h2 submodule is re-aliased so import blasthttp.h2 still works.

Benchmark

  • Added blasthttp-python-stream and blasthttp-python-stream-200k engines to scripts/benchmark.py for direct comparison against the batched API.

Behavior notes

  • request_batch is unchanged. BlastHTTP's public Python surface is otherwise the same — all native methods/getters are forwarded through __getattr__.
  • Concurrency is gated before spawn by a semaphore acquire on the driver, so at most concurrency requests are in-flight at any time. Rate-limit acquire happens before the semaphore, matching send_batch's dispatch pacing.
  • In-flight tasks are NOT cancelled if the consumer drops the iterator; they run to completion and their channel sends fail silently. Same model as send_batch.
  • Result delivery order is completion order, not dispatch order.

Test plan

  • cargo test --lib — 134 tests pass
  • cargo clippy --features python --all-targets -- -D warnings — clean
  • pytest tests/python/ — 30 tests pass (existing suite unchanged)
  • Mixed-latency live smoke test against httpbin.org/delay/N for N ∈ {3,0,2,0,1} confirmed completion-order delivery (0s results land at 0.35s/1.47s while 3s is still in flight)
  • Benchmark on the bundled local server, 20k requests, 100 workers — streaming within 5% of batched

@TheTechromancer
TheTechromancer marked this pull request as ready for review April 30, 2026 18:39
@liquidsec
liquidsec merged commit 81c38b4 into stable May 1, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants