D1: the async residual is CPython, not the SDK — evidence, uvloop extra, PERFORMANCE.md

[ilpanich]

Closes D1/J5 of claude_dev/improvement-after-run5-benchmark.md (ilpanich/axiam).

The finding

Benchmark run 5 put this SDK's check_access at p50 40.2 ms / p95 116.8 ms / 311 rps against Go, Java and Rust's ~10 ms / ~60 ms / 835–869 rps at the same 16-way concurrency. The plan's reading was that the async rewrite had left something slow in axiam_sdk. It had not.

The measurement that settles it is running the same client against a server that does no work at all — an out-of-process stub (a threaded stub in the client's own process competes for the GIL and is indistinguishable from client cost), 16 in-flight:

client	p50	p95	throughput	client CPU
raw httpx.AsyncClient, no cookie jar	44.0 ms	51.5 ms	347 rps	1 589 µs/call
raw httpx.AsyncClient + cookie jar	47.2 ms	58.4 ms	324 rps	2 121 µs/call
AsyncAxiamClient.check_access	47.3 ms	57.9 ms	324 rps	2 172 µs/call

1. The SDK is not the residual. ~50 µs/call over raw httpx with the same jar — about 2% of client CPU. A cProfile of the hot path puts every one of the top 28 cost centres in httpcore/anyio: connection_pool._assign_requests_to_connections alone calls connection.is_idle 611 times per request, and six module-import lookups happen per call inside anyio/httpcore themselves. No axiam_sdk frame appears.

2. ~310 rps is a per-process CPython ceiling, not an AXIAM number. The stub is not the bottleneck: three client processes driving it concurrently reached 320 + 308 + 301 = 929 rps aggregate, each capped at the same ~310. Run 5's 311 rps against the real server is that same ceiling.

3. The plan's p50 ≤ 20 ms target is arithmetically unreachable in one process. 16 in-flight × ~2 ms CPU each on a single event-loop thread has a floor near 32 ms regardless of how fast the server answers. The 44–47 ms above is almost entirely client-side queueing against a zero-latency server.

What changed

• [speed] extra installing uvloop. Measured on the same harness: client CPU 2 182 → 1 735 µs/call (−20%), p95 68 → 55 ms, throughput +7%. The SDK still does not install a loop policy — choosing the event loop belongs to the application, not to a library it happens to import.
• PERFORMANCE.md — the numbers, the method, the reproduction, and the two things that actually help: use uvloop, and scale with processes rather than with in-flight calls per process.
• README and CHANGELOG point at both.

No source changes; the SDK's behaviour is unchanged.

The four suspects, and what the evidence indicts

1. Per-request session/connector churn — absent. One httpx.AsyncClient, built lazily and held for the client's lifetime.
2. Sync-in-async on the hot path — the CSRF lock, JSON parsing and pydantic construction sit inside the ~50 µs the SDK adds and never reach the profile. An executor hand-off would cost more than it saves; a fast path skipping AccessResult would trade the typed contract for well under 1% of the request.
3. uvloop absent — real, measured, now shipped.
4. Per-call object churn / pydantic on check() — see (2). Not indicted.

The cookie jar is the one non-trivial SDK-adjacent cost (~530 µs/call, 25% of client CPU): http.cookiejar is pure Python and httpx re-runs policy and domain matching per request and response. It is not removable — the AXIAM session is cookies (CONTRACT.md §3) — and reimplementing cookie policy to save CPU is not a trade to make in an auth client.

Related

• Track E (benchmark harness) + the D-track harness audits axiam#279 — the harness-side half (the bench prefers uvloop and records event_loop / client_worker_threads, so a CPU figure always says which loop produced it).

---

Generated by Claude Code