Skip to content

perf: streaming latency improvements — 4 targeted hot-path fixes - #22346

Merged
ishaan-jaff merged 4 commits into
mainfrom
perf/streaming-latency-improvements
Feb 28, 2026
Merged

perf: streaming latency improvements — 4 targeted hot-path fixes#22346
ishaan-jaff merged 4 commits into
mainfrom
perf/streaming-latency-improvements

perf: cache model-level guardrail lookup per request, not per chunk

5f2b7bc
Select commit
Loading
Failed to load commit list.
GitHub Advanced Security / CodeQL completed Feb 28, 2026 in 2s

1 configuration not found

Warning: Code scanning cannot determine the alerts introduced by this pull request, because 1 configuration present on refs/heads/main was not found:

Default setup

  • ❓  /language:python

View all branch alerts.