🚨All attention refactor🚨#35235

Merged

Cyrilvallez merged 99 commits intomainfrom

all-attention-refactor

Dec 18, 2024

Commits on Dec 16, 2024

refactor LlamaAttention

ArthurZucker
authored and
Cyrilvallez
committed
minimal changes

ArthurZucker
authored and
Cyrilvallez
committed
fix llama

ArthurZucker
authored and
Cyrilvallez
committed
update

ArthurZucker
authored and
Cyrilvallez
committed
modular gemmas

ArthurZucker
authored and
Cyrilvallez
committed
modular nits

ArthurZucker
authored and
Cyrilvallez
committed
modular updates

ArthurZucker
authored and
Cyrilvallez
committed
nits

ArthurZucker
authored and
Cyrilvallez
committed
simplify

ArthurZucker
authored and
Cyrilvallez
committed
gpt2

ArthurZucker
authored and
Cyrilvallez
committed
more modualr and fixes

ArthurZucker
authored and
Cyrilvallez
committed
granite

ArthurZucker
authored and
Cyrilvallez
committed
modular modular modular

ArthurZucker
authored and
Cyrilvallez
committed
nits

ArthurZucker
authored and
Cyrilvallez
committed
update

ArthurZucker
authored and
Cyrilvallez
committed
qwen2 + starcoder2
Cyrilvallez
committed
mostly gemma2
Cyrilvallez
committed
Update image_processing_auto.py
Cyrilvallez
committed
fix
Cyrilvallez
committed
Update modular_starcoder2.py
Cyrilvallez
committed
fix
Cyrilvallez
committed
remove all copied from attentions
Cyrilvallez
committed
remove gcv
ArthurZucker
committed
make fix-copies
ArthurZucker
committed
oups
ArthurZucker
committed
oups2.0
ArthurZucker
committed
fix some modulars + all copied from
Cyrilvallez
committed
should be good now
ArthurZucker
committed
Merge branch 'all-attention-refactor' of github.com:huggingface/transformers into all-attention-refactor
Cyrilvallez
committed
Merge branch 'all-attention-refactor' of github.com:huggingface/transformers into all-attention-refactor
Cyrilvallez
committed
revert unwanted changes
Cyrilvallez
committed
Update modeling_decision_transformer.py
Cyrilvallez
committed
finish cleanup
Cyrilvallez
committed
Update modeling_olmo.py
Cyrilvallez
committed
consistency
Cyrilvallez
committed
re-add gradient checkpointing attribute
Cyrilvallez
committed
fix
Cyrilvallez
committed
style
Cyrilvallez
committed
make config necessary
Cyrilvallez
committed
bis
Cyrilvallez
committed
bis
Cyrilvallez
committed
Update modeling_my_new_model2.py
Cyrilvallez
committed
is_causal attr
Cyrilvallez
committed
fix
Cyrilvallez
committed
remove past kv return from decoder layer
Cyrilvallez
committed
fix
Cyrilvallez
committed
default rope config
Cyrilvallez
committed
correctly fix rope config
Cyrilvallez
committed
fix bias
Cyrilvallez
committed
fix gpt2 attention output
Cyrilvallez
committed
fix test
Cyrilvallez
committed
fix inits
Cyrilvallez
committed
fix default sdpa
Cyrilvallez
committed
fix default sdpa implementation
Cyrilvallez
committed
harmonize classes
Cyrilvallez
committed
fix mistral
Cyrilvallez
committed
fix sliding window models
Cyrilvallez
committed
mixtral
Cyrilvallez
committed
be more explicit
Cyrilvallez
committed
style
Cyrilvallez
committed
fix
Cyrilvallez
committed

Commits on Dec 17, 2024