Qwen2VL: skip base input_ids-inputs_embeds equivalence check#34535
Qwen2VL: skip base input_ids-inputs_embeds equivalence check#34535gante merged 1 commit intohuggingface:mainfrom
input_ids-inputs_embeds equivalence check#34535Conversation
| # 2.C - No easy fix, let's skip the check that compares the outputs from `input_ids` and `inputs_embeds` | ||
| has_complex_embeds_computation = any( | ||
| model_name in model_class.__name__.lower() for model_name in ["moshi"] | ||
| model_name in model_class.__name__.lower() for model_name in ["moshi", "qwen2vl"] |
There was a problem hiding this comment.
This flag skips the check that passing inputs_embeds = model.get_input_embeddings()(input_ids) should be equivalent to passing input_ids to generate.
This is not true in Qwen2VL, whose computation of inputs_embeds is more complex
There was a problem hiding this comment.
OK for me. Just curious: would this be addressed in the future to make it work
There was a problem hiding this comment.
Not unless we create a model function to embed inputs, similar to def get_input_embeddings(self).
(I think we should do it eventually, we're starting to have a few models with non-standard inputs embeddings creation!)
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…gingface#34535) it has complex inputs_embeds computation
3 participants
What does this PR do?
Computing
inputs_embedsinQwen2VLis more complex than simply embeddinginput_ids(seeQwen2VLForConditionalGeneration), so the basic check doesn't apply :)Fixes: