Skip to content

Add an AlbertMaskedLM task model #718

@mattdangerw

Description

@mattdangerw

In #653 we added a masked language modeling task for RoBERTa. We can make a similar change for the Albert model.

  • Update AlbertTokenizer to expect a mask token.
  • Add a AlbertMaskedLMPreprocessor preprocessor layer and tests.
  • Add a AlbertMaskedLM task model and tests.
  • Update keras_nlp/models/__init__.py to export AlbertMaskedLM and AlbertMaskedLMPreprocessor.

Metadata

Metadata

Labels

stat:contributions welcomeAdd this label to feature request issues so they are separated out from bug reporting issuestype:featureNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions