[Gemma v2] Enable gemma v2 on gaudi

[billishyahao]

What does this PR do?

This patch aims to enable gemma2 both training and inferencing on gaudi device. Also static shape support is enabled as well. The performance results are shown as below:

python run_generation.py --model_name_or_path ../../../gemma-2-9b/ --bf16 --use_kv_cache --reuse_cache --use_hpu_graphs --max_input_tokens 128 --max_new_tokens 128 --bf16 --batch_size 4

Input/outputs:
input 1: ('DeepSpeed is a machine learning framework',)
output 1: ('DeepSpeed is a machine learning framework that enables training of large-scale deep learning models on a single GPU or across multiple GPUs. It is designed to be easy to use and highly scalable, making it a powerful tool for researchers and practitioners working with large-scale deep learning models.\n\nDeepSpeed is built on top of PyTorch, a popular deep learning framework, and provides a set of tools and libraries that make it easy to train large-scale models. It includes features such as zero-shot inference, which allows models to be used for inference without the need for retraining, and distributed training, which enables models to be trained across multiple GPUs.\n\nDeepSpeed is also',)


Stats:
--------------------------------------------------------------------------------------------------------------
Throughput (including tokenization) = 347.9053767194258 tokens/second
Number of HPU graphs                = 20
Memory allocated                    = 20.74 GB
Max memory allocated                = 20.82 GB
Total memory available              = 94.62 GB
Graph compilation duration          = 5.919518291004351 seconds
--------------------------------------------------------------------------------------------------------------

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[billishyahao]

Fixed all comments @ssarkar2 feel free to review again.

[billishyahao]

Rebase the patch to address the conflict wth main.

[billishyahao]

@regisss Could you approve this ?

[billishyahao]

Rebase the patch to address the conflict wth main at 11/13

[Luca-Calabria]

Hi @billishyahao @libinta I would like to help merging this PR. It seems it needs just a rebase and if needed fixes after rebasing.

[emascarenhas]

@ssarkar2 , You approved this PR earlier, so assuming this can go ahead now that it is rebased with latest habana-main branch?

[emascarenhas]

@libinta , We could start run-test on this while we are waiting for Sayantan to confirm.

[nedo99]

@regisss could we start the tests so we can merge this PR?

[github-actions]

The code quality check failed, please run make style.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Luca-Calabria]

As you can see I created a new PR #1504 as fork of this one to add the fixes on style. This because we don't have write access to this repo.

[emascarenhas]

@Luca-Calabria , Please close this as it is no longer needed?