add qwen2 classification

[ranzhejiang]

• To meet the customer's demands, we need to add the classification function interface of the qwen2 on gaudi
• you can test it with following code

import torch
import habana_frameworks.torch.core as htcore
from optimum.habana.transformers.modeling_utils import adapt_transformers_to_gaudi
from transformers import AutoTokenizer, AutoModel, AutoModelForCausalLM, AutoModelForSequenceClassification, AutoModelForTokenClassification
adapt_transformers_to_gaudi()

tokenizer = AutoTokenizer.from_pretrained("/mnt/HF_models/qwen2-7b", trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained("/mnt/HF_models/qwen2-7b",num_labels=3, trust_remote_code=True).bfloat16().to('hpu')
print(model)

texts = [
"Today’s exercise makes your body feel very relaxed"
]

label_to_id = {
"positive": 0,
"negative": 1,
"neutral ": 2
}

inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")

with torch.no_grad():
outputs = model(**inputs)

logits = outputs.logits
predictions = torch.argmax(logits, dim=-1)

for text, prediction in zip(texts, predictions):
print(f"Text: {text} -> predicted label: {list(label_to_id.keys())[prediction.item()]}")

[Luca-Calabria]

LGTM!

[12010486]

Ok, thanks @Luca-Calabria. Waiting for @regisss then

[ranzhejiang]

@regisss Would you please review this PR? Thank you.

[regisss]

@ranzhejiang There are merge conflicts to solve following #1698.
Can you also make sure your code is up-to-date with Transformers v4.49 please?

Can you also add a test for it? It can be the code snippet you posted above 🙂

[ranzhejiang]

Ok, I will solve it later, and add this demo in example, thanks for review.

[Luca-Calabria]

@ranzhejiang @regisss any update here?

[regisss]

My comment above with

Can you also make sure your code is up-to-date with Transformers v4.49 please?
Can you also add a test for it? It can be the code snippet you posted above 🙂

should be addressed

[ranzhejiang]

Sorry for late reply owing to others. For first comment, I have made sure my code is up-to-date with Transformers v4.49, now no confilict, for second comment, I had not find proper tests for this text-classification task example before, so I will add this the code snippet into https://github.com/huggingface/optimum-habana/tree/main/examples/text-classification before 5.23
, then it will give reference example for our customers.

[IlyasMoutawwakil]

hi @ranzhejiang ! hope my question is not self evident, but why do we need to implement those custom task classes ? my understanding is that if they all use the same based model QwenModel, which we patch with QaudiQwenModel, the original transformers QwenModelForSequenceClassification will work just fine on gaudi with all optimization in GaudiQwenModel, as it's simply a linear layer on top. My question could be interpreted as : what's Gaudi-specific here ?

[IlyasMoutawwakil]

my point is that the line in adapt_transformers_to_gaudi: transformers.models.qwen2.modeling_qwen2.Qwen2Model = GaudiQwen2Model already makes these qwen2 models with task heads compatible/optimized for gaudi.

[ranzhejiang]

What you said make sense, it seems that this plan can not run work well for auto tp when I firstly give this pr, but now it can works well for single card, but I still need confirm what you said in 8-card env.

[ranzhejiang]

What you said makes sense, I have changed my code and not using GaudiQwen2Model for init, and the new PR is in #2062