Diverse beam search 2

[patrickvonplaten]

What does this PR do?

Copy of #8627 because branch got messed up.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

[patrickvonplaten]

@ayushtiku5 I tweeted about it and tagged you on the tweet: https://twitter.com/PatrickPlaten/status/1336681238485229568 - hope that's fine for you :-)

[LSinev]

@ayushtiku5 can you please check if HammingDiversityLogitsProcessor and PrefixConstrainedLogitsProcessor can be speeded up with functions like torch.scatter, torch.gather, torch.masked_fill, torch.index_fill, torch.index_add, torch.index_copy? I believe there is room for improvement in
https://github.com/huggingface/transformers/blob/master/src/transformers/generation_logits_process.py#L406-L409 and https://github.com/huggingface/transformers/blob/master/src/transformers/generation_logits_process.py#L471 (in the way like #9557 and #9600) but I have no experience using these processors to create good enough examples for speed testing and corner cases research.