Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Tokenizer' object has no attribute '_token_pad_id' #516

Open
Jwinre opened this issue May 22, 2023 · 0 comments
Open

'Tokenizer' object has no attribute '_token_pad_id' #516

Jwinre opened this issue May 22, 2023 · 0 comments

Comments

@Jwinre
Copy link

Jwinre commented May 22, 2023

提问时请尽可能提供如下信息:

基本信息

  • 你使用的操作系统: windows 11
  • 你使用的Python版本: 3.6
  • 你使用的Tensorflow版本: 1.14
  • 你使用的Keras版本: 2.3.1
  • 你使用的bert4keras版本: 0.8.4
  • 你使用纯keras还是tf.keras:
  • 你加载的预训练模型:

核心代码

# 请在此处贴上你的核心代码。
# 请尽量只保留关键部分,不要无脑贴全部代码。

输出信息

Traceback (most recent call last):
File "D:/pycharm_pro/PyCharm 2023.1/plugins/python/helpers/pydev/pydevd.py", line 1496, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "D:\pycharm_pro\PyCharm 2023.1\plugins\python\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "E:\AAA\Bert\bert4keras-master\pretraining\data_utils.py", line 379, in
tokenizer, word_segment, sequence_length=sequence_length
File "E:\AAA\Bert\bert4keras-master\pretraining\data_utils.py", line 158, in init
super(TrainingDatasetRoBERTa, self).init(tokenizer, sequence_length)
File "E:\AAA\Bert\bert4keras-master\pretraining\data_utils.py", line 22, in init
self.token_pad_id = tokenizer._token_pad_id
AttributeError: 'Tokenizer' object has no attribute '_token_pad_id'


### 自我尝试
不管什么问题,请先尝试自行解决,“万般努力”之下仍然无法解决再来提问。此处请贴上你的努力过程。
已经确认词表有[PAD],换了几个模型和词表还是这样报错,求助大神们

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant