Fix integration tests of DeBERTa

[BigBird01]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to the it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

[BigBird01]

@LysandreJik I just fix the numeric part of the tests. Another issue is that I just made the change to the model state keys, i.e. change bert.encoder to deberta.encoder. However, I can only upload the model to DeBERTa/deberta-base, DeBERTa/deberta-large. Could you help to mv those two model to the namespace of microsoft? Or could you add me to the organization Microsoft?

[LysandreJik]

Hi! Sure, I can add you to the microsoft organization. What's your username on the hub? Thanks!

[LysandreJik]

I'm uploading the two models with the modified names bert -> deberta right now.

[BigBird01]

Hi! Sure, I can add you to the microsoft organization. What's your username on the hub? Thanks!

The name is DeBERTa

[LysandreJik]

Cool, I'm adding you! I've done a PR here #7229 that solves all the integration tests. Do you mind reviewing it before we merge it? I've added comments to explain why the changes were so.

[BigBird01]

Hi! Sure, I can add you to the microsoft organization. What's your username on the hub? Thanks!

The name is DeBERTa

Hi, @LysandreJik

Did you add me *DeBERTa to microsoft? I still can't see my account under Microsoft.
Seems the model you uploaded to Microsoft/deberta-base and Microsoft/deberta-large is not loadable due to a format issue.

[julien-c]

I've added you manually @BigBird01, but you should have been able to request to join from the website – was this not the case?

[LysandreJik]

@BigBird01, what's the issue you have? I can load both:

>>> from transformers import DebertaModel
>>> model = DebertaModel.from_pretrained("microsoft/deberta-base")
Downloading: 100%|██████████| 448/448 [00:00<00:00, 510kB/s]
Downloading: 100%|██████████| 559M/559M [00:50<00:00, 11.1MB/s]
Some weights of the model checkpoint at microsoft/deberta-base were not used when initializing DebertaModel: ['deberta.embeddings.position_embeddings.weight']
- This IS expected if you are initializing DebertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing DebertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
>>> model = DebertaModel.from_pretrained("microsoft/deberta-large")
Downloading: 100%|██████████| 449/449 [00:00<00:00, 578kB/s]
Downloading: 100%|██████████| 1.63G/1.63G [02:42<00:00, 9.98MB/s]
Some weights of the model checkpoint at microsoft/deberta-large were not used when initializing DebertaModel: ['deberta.embeddings.position_embeddings.weight']
- This IS expected if you are initializing DebertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing DebertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).