Fix GPT-NeoX-20B past handling, attention computation

[zphang]

What does this PR do?

• Fixes GPT-NeoX-20B handing of the past object to correctly be used in .generate
• Swaps attention computation for one more similar in the original training code, to hopefully avoid NaNs
• Update docstring, removed unnecessary dropout configs in config object

Fixes # (issue)

#17632
#17452 (Hopefully)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@sgugger

[sgugger]

Thanks for the fix, I can confirm that with this PR, I get the same generations in float32 and float16 (whereas before, I get either a crappy one or some Nans in float16) for EleutherAI/gpt-neox-20b.

The cleaning up in the config LGTM, thanks for making the docstring on par with the defaults, and the two attributes you remove are not used anywhere.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[sgugger]

There are a few equivalence tests failing with the PR, if you can dive into it. Let us know if you need any help!

[patrickvonplaten]

Out of curiosity why is the second statement needed here? The past[0] is not None part?

[patrickvonplaten]

Thanks for cleaning this up!

[patrickvonplaten]

Thanks for fixing @zphang !

[zphang]

I've run the tests locally and they pass, so I can't seem to reproduce the test errors. Can someone else give them a try?

[sgugger]

The tests pass on GPU but not on CPU on my side. So doing

CUDA_VISIBLE_DEVICES="" pytest tests/models/gpt_neox/test_modeling_gpt_neox.py

reproduces the failure.

[sgugger]

Thanks again! Nice to be able to use GPT-Neo-X in float16 for generations :-)