[SMG] perf: optimize tokenizer for reduced CPU and memory overhead#14752

slin1237 merged 1 commit intomainsgl-project/sglang:mainfrom

tokenizer-smgsgl-project/sglang:tokenizer-smgCopy head branch name to clipboard