Consolidate tokenizer interface#2954

larryliu0820 wants to merge 1 commit intomainpytorch/executorch:mainfrom

export-D55944780pytorch/executorch:export-D55944780Copy head branch name to clipboard