Add a DistilBertMaskedLM task model

[mattdangerw]

In #653 we added a masked language modeling task for RoBERTa. We can make a similar change for the DistilBERT model.

• Update DistilBertTokenizer to expect a mask token.
• Add a DistilBertMaskedLMPreprocessor preprocessor layer and tests.
• Add a DistilBertMaskedLM task model and tests.
• Update keras_nlp/models/__init__.py to export DistilBertMaskedLM and DistilBertMaskedLMPreprocessor.

[mattdangerw]

@ADITYADAS1999 would you like to take this on?

[ADITYADAS1999]

@ADITYADAS1999 would you like to take this on?

I take this one. 🧑‍💻
/assign

[ADITYADAS1999]

• Update DistilBertTokenizer to expect a mask token.

Hey @mattdangerw can you describe this update little bit or provide some similar references.

[mattdangerw]

@ADITYADAS1999 sure just take a look the code changes (not docstring changes) to the tokenizer here https://github.com/keras-team/keras-nlp/pull/653/files#diff-5f511b47f35f0f4a756ed6748c1a6d0feda574b387b81a0be366f3e4e16ea29e

This is basically a two liner, we need to add a self.mask_token_id property to the tokenizer base class. The mask token for DistilBert is [MASK] I believe (though worth checking!).

[ADITYADAS1999]

@ADITYADAS1999 sure just take a look the code changes (not docstring changes) to the tokenizer here https://github.com/keras-team/keras-nlp/pull/653/files#diff-5f511b47f35f0f4a756ed6748c1a6d0feda574b387b81a0be366f3e4e16ea29e

This is basically a two liner, we need to add a self.mask_token_id property to the tokenizer base class. The mask token for DistilBert is [MASK] I believe (though worth checking!).

thanks @mattdangerw , this is very helpful. 👍🏻

[ADITYADAS1999]

PR open at #724

[jayam30]

Hey!
I would like to take this issue up,

[ADITYADAS1999]

Hey! I would like to take this issue up,

Hello @jayam30 this issue is already assigned.
You can check out contributor welcome issues here
https://github.com/keras-team/keras-nlp/issues

[mattdangerw]

Fixed in #724