Skip to content

Allow Attention and Experts to be used as standalone modules #43622

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Cyrilvallez merged 9 commits into from
Jan 30, 2026
Merged

Allow Attention and Experts to be used as standalone modules #43622

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
606fe45
default
Cyrilvallez Jan 30, 2026
299fbab
fix all
Cyrilvallez Jan 30, 2026
920c3df
fix
Cyrilvallez Jan 30, 2026
cda074a
add warning
Cyrilvallez Jan 30, 2026
a8ae8ff
strict check
Cyrilvallez Jan 30, 2026
1f96197
fix
Cyrilvallez Jan 30, 2026
b4a3a75
add test
Cyrilvallez Jan 30, 2026
05405f7
fix
Cyrilvallez Jan 30, 2026
a00ab92
improve test
Cyrilvallez Jan 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/transformers/integrations/moe.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ def __init__(self, config, *args, **kwargs):
def forward(self, *args, **kwargs):
experts_forward = original_forward

if self.config._experts_implementation != "eager":
if self.config._experts_implementation not in (None, "eager"):
experts_forward = ALL_EXPERTS_FUNCTIONS[self.config._experts_implementation]

return experts_forward(self, *args, **kwargs)
Expand Down