Add a DebertaV3MaskedLM task model

[mattdangerw]

In #653 we added a masked language modeling task for RoBERTa. We can make a similar change for the DebertaV3 model.

• Update DebertaV3Tokenizer to expect a mask token.
• Add a DebertaV3MaskedLMPreprocessor preprocessor layer and tests.
• Add a DebertaV3MaskedLM task model and tests.
• Update keras_nlp/models/__init__.py to export DebertaV3MaskedLM and DebertaV3MaskedLMPreprocessor.

[Plutone11011]

Hi, I would like to contribute.

[Plutone11011]

@mattdangerw I have a question regarding testing. Currently SentencePiece tokenizer doesn't support the [MASK] special token as an argument to the trainer (https://github.com/google/sentencepiece/blob/master/doc/options.md).
If you train the tokenizer with a sentence that contains [MASK], it will treat it like a normal word token.

So, for testing, should I add a sentence with [MASK] and adjust token ids accordingly, or do you see a better approach?
I'm talking specifically about this part of the code, where we train SentencePiece on our sentences, to then build DebertaV3 preprocessor/tokenizer

vocab_data = tf.data.Dataset.from_tensor_slices(
            ["the quick brown fox", "the earth is round"]
        )
        sentencepiece.SentencePieceTrainer.train(
            sentence_iterator=vocab_data.as_numpy_iterator(),
            model_writer=bytes_io,
            vocab_size=10,
            model_type="WORD",
            pad_id=0,
            bos_id=1,
            eos_id=2,
            unk_id=3,
            pad_piece="[PAD]",
            bos_piece="[CLS]",
            eos_piece="[SEP]",
            unk_piece="[UNK]",
        )

[mattdangerw]

Great question! And I am not sure the answer.

Definitely deberta handled this somehow, as the ["MASK"] token is right there in the vocab. So one option would be to poke around https://github.com/microsoft/DeBERTa.

I'm guessing they might be leveraging this -> https://github.com/google/sentencepiece/blob/master/doc/special_symbols.md

Have you tried passing control_symbols= or user_defined_symbols= to the trainer? If either work it's probably fine, we are only using this for testing, but bonus points if we handle this the same was as deberta does upstream I guess.

[mattdangerw]

Fixed in #721