Add skip_tokenizer_init args. - #959

Merged

merrymercy merged 14 commits into

sgl-project:mainfrom

gryffindor-rr:feat/tokenizer

Aug 9, 2024

Commits on Aug 7, 2024

Add skip_tokenizer_init args.
lzhang
committed
798b319
Fix wrong clause.
lzhang
committed
670a374
Add skip_tokenizer_init to detokenizer manager.
lzhang
committed
db6e26d

Commits on Aug 8, 2024

Merge branch 'main' of github.com:sgl-project/sglang into feat/tokenizer
lzhang
committed
b6ce08d
Add unittest to test skip_tokenizer_init.
lzhang
committed
3f62200

Commits on Aug 9, 2024

Merge branch 'main' of github.com:sgl-project/sglang into feat/tokenizer
lzhang
committed
a4d996a
Merge branch 'main' of github.com:sgl-project/sglang into feat/tokenizer
lzhang
committed
71625d9
Fix running error. modify according to review.
lzhang
committed
75074c4
Fix lint error.
lzhang
committed
710f136
Fix test.
lzhang
committed
277a836
Fix testing.
lzhang
committed
85505e6
Fix lint error of skip tokenizer test.
lzhang
committed
2556644
Fix lint.
lzhang
committed
dc18a29
Merge branch 'main' into feat/tokenizer
merrymercy
authored
a7fa37e