Skip to content

Commit

Permalink
Update bert_gen.py
Browse files Browse the repository at this point in the history
  • Loading branch information
Stardust-minus authored Sep 30, 2023
1 parent d5bea44 commit 11dde03
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions bert_gen.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import torch.multiprocessing as mp
from text.japanese_bert import tokenizer as jp_tokenizer
from text.chinese_bert import tokenizer
from text.japanese import text_normalize

def process_line(line):
rank = mp.current_process()._identity
Expand Down

0 comments on commit 11dde03

Please sign in to comment.