Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix]update vocab_size in init_config #3260

Merged
merged 5 commits into from
Sep 14, 2022
Merged

[BugFix]update vocab_size in init_config #3260

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3f185fb
update vocab_size in init_config
wj-Mcat Sep 14, 2022
4337678
Merge branch 'develop' into fix-resize-token
ZHUI Sep 14, 2022
611c755
make update_init_config more common
wj-Mcat Sep 14, 2022
ba8b9d6
Merge branch 'fix-resize-token' of github.com:wj-Mcat/PaddleNLP into …
wj-Mcat Sep 14, 2022
ca95199
Merge branch 'develop' into fix-resize-token
wj-Mcat Sep 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions paddlenlp/transformers/model_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -546,6 +546,20 @@ def resize_token_embeddings(self,
self.base_model.config['vocab_size'] = new_num_tokens
self.vocab_size = new_num_tokens

# update init_config
def update_init_config_vocab_size_field(sub_dict: dict):
if 'vocab_size' in sub_dict:
sub_dict['vocab_size'] = new_num_tokens
return
models = [
arg for arg in sub_dict.get('init_args', [])
if isinstance(arg, PretrainedModel)
]
if models:
update_init_config_vocab_size_field(models[0].init_config)

update_init_config_vocab_size_field(self.init_config)

wj-Mcat marked this conversation as resolved.
Show resolved Hide resolved
# TODO([email protected]): add tie_weight.
# TODO(westfish) Add tie_weight to tie the weights between the input embeddings and the output embeddings if needed.

Expand Down