Skip to content

perf(core): add pure-ASCII fast path to text token estimation - #6551

Merged
wenshao merged 1 commit into
QwenLM:mainfrom
dexhunter:weco/tokenizer-ascii-fast-path
Jul 8, 2026
Merged

perf(core): add pure-ASCII fast path to text token estimation#6551
wenshao merged 1 commit into
QwenLM:mainfrom
dexhunter:weco/tokenizer-ascii-fast-path

perf(core): add pure-ASCII fast path to text token estimation

0155138
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
Classify PR
succeeded Jul 8, 2026 in 2s