Skip to content

Conversation

@ydshieh
Copy link
Collaborator

@ydshieh ydshieh commented Oct 15, 2025

What does this PR do?

#36545 introduced a mistake regarding the headmask

before that PR, we had some

        # Mask heads if we want to
        if head_mask is not None:
            attention_probs = attention_probs * head_mask

but after the PR, it became

    # Mask heads if we want to
    if attention_mask is not None:
        attn_weights = attn_weights * attention_mask

which is apparently wrong.

Furthermore, we already remove the support of head masking.

This PRs simply remove the blocks.

@github-actions
Copy link
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: audio_spectrogram_transformer, deit, dinov2, dinov2_with_registers, dinov3_vit, dpt, ijepa, videomae, vit, vit_mae, vit_msn, vitpose_backbone, vivit, vjepa2, yolos

@ydshieh ydshieh requested a review from vasqu October 15, 2025 13:45
@ydshieh ydshieh changed the title Update vit Remove the head masking block in some vision models Oct 15, 2025
Copy link
Contributor

@vasqu vasqu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx, what nasty bug 😓

@ydshieh ydshieh merged commit da38291 into main Oct 15, 2025
18 checks passed
@ydshieh ydshieh deleted the update_vit branch October 15, 2025 13:51
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

ngazagna-qc pushed a commit to ngazagna-qc/transformers that referenced this pull request Oct 23, 2025
SangbumChoi pushed a commit to SangbumChoi/transformers that referenced this pull request Jan 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants