docs(release): add perf metrics table for #28289 write-up

release_notes/v1.87.0rc1/index.md

@@ -209,11 +209,19 @@ Plus a Vertex / Anthropic `supports_output_config` flag flip on all `claude-opus
 
 ## Performance / Loadbalancing / Reliability improvements
 
-- **Anthropic `/v1/messages` streaming hot path** — cut per-request and per-chunk overhead on the proxy's Anthropic streaming path, with byte-identical wire output guaranteed by parity tests that diff the logged and billed payloads between the fast and legacy paths. Measured on a real 4-pod `m7i.xlarge` deployment (no HPA) streaming 256 `text_delta` chunks per request, against both Anthropic and Bedrock Invoke: **TTFT overhead ~90% lower** (p50 2220% → 165%, p95 3057% → 316%, p99 3111% → 328%) with **higher sustained throughput** (TPM +12% / +6% / +4% at p50 / p95 / p99) - [PR #28289](https://github.com/BerriAI/litellm/pull/28289)
+- **Anthropic `/v1/messages` streaming hot path** — cut per-request and per-chunk overhead on the proxy's Anthropic streaming path, with byte-identical wire output guaranteed by parity tests that diff the logged and billed payloads between the fast and legacy paths. Measured on a real 4-pod `m7i.xlarge` deployment (no HPA) streaming 256 `text_delta` chunks per request, against both Anthropic and Bedrock Invoke — **TTFT overhead ~90% lower** with **higher sustained throughput** (full numbers below) - [PR #28289](https://github.com/BerriAI/litellm/pull/28289)
     - Skip work that's a no-op in the default config: the per-chunk Datadog span when tracing is off, the per-chunk streaming hook when no callback / guardrail / cost-injection is active, and the agentic post-processing wrapper when no callback overrides its hook (it otherwise buffers every chunk and rebuilds the response from SSE just to call hooks that all return `(False, {})`).
     - Stop doing the same work twice per request: serialize the request body once and reuse it for the pre-call log and the wire, memoize the optional-params type-hint resolution (~80µs/request), and skip the redundant `strip_empty_text_blocks` scan when the async wrapper already sanitized.
     - Cheaper end-of-stream reconstruction: collapse the homogeneous run of `content_block_delta` text events into a single equivalent SSE event before `stream_chunk_builder`, removing O(output-token) `ModelResponseStream` constructions; tool-use / thinking / citations streams fall back to the unchanged legacy path.
     - Cheaper hot-path logging: gate debug f-string evaluation behind `isEnabledFor(DEBUG)`, hoist `cost_injection_active` out of the per-chunk loop, and drop one async-generator layer per chunk in `async_sse_data_generator`.
+
+*Anthropic `/v1/messages` streaming, 256 `text_delta` chunks/request — 4 pods on `m7i.xlarge` (4 vCPU / 16 GB), no HPA:*
+
+| Metric | Baseline (`v1.87.0-dev.1`) | Patched ([#28289](https://github.com/BerriAI/litellm/pull/28289)) | Change |
+| --- | --- | --- | --- |
+| TPM (p50 / p95 / p99) | 2634 / 2808 / 2867 | 2952 / 2968 / 2971 | +12% / +6% / +4% |
+| TTFT overhead % (p50 / p95 / p99) | 2220 / 3057 / 3111 | 165 / 316 / 328 | ~90% lower |
+
 - **Bedrock / SageMaker** — Switch to lazy loading for response streaming - [PR #28189](https://github.com/BerriAI/litellm/pull/28189)
 - **Granian ASGI** — Add Granian as a supported ASGI server for better throughput stability - [PR #26027](https://github.com/BerriAI/litellm/pull/26027)
 - **Prisma** — Expose Prisma idle/connect timeout + extra DB URL params so production deployments can tune connection pools - [PR #28395](https://github.com/BerriAI/litellm/pull/28395)