perf(stream): compile hot-path regexes once, bound token caches, fix quadratic buffering - #12179

perf(stream): compile hot-path regexes once, bound token caches, fix quadratic buffering#12179

diegosouzapw merged 8 commits into

cheaper-inference:release/v3.8.51cheaper-inference/OmniRoute:release/v3.8.51from

opensource-elearning:perf/v3.8.51-stream-memoryopensource-elearning/OmniRoute:perf/v3.8.51-stream-memoryCopy head branch name to clipboard