Remove the head masking block in some vision models#41620

ydshieh merged 2 commits intomainhuggingface/transformers:mainfrom

update_vithuggingface/transformers:update_vitCopy head branch name to clipboard