-
Notifications
You must be signed in to change notification settings - Fork 31.9k
Adds support for OPT in Flax and TF. #17227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
99 commits
Select commit
Hold shift + click to select a range
3685706
update flax code
ArthurZucker 23b0842
update and clean
ArthurZucker bce44cd
retreive tf and flax bits
ArthurZucker 02709f4
add template tests and dummy objects
ArthurZucker 7a85ec4
Clean code and deleted decoder/encoder stuff for modelling
ArthurZucker 0efb414
removed encoder stuff, clean tests
ArthurZucker 21cef00
cleanup modeling OPT
ArthurZucker bf3404f
cleanup test
ArthurZucker d8d5c6f
clean test
ArthurZucker e1722aa
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker 1437496
Update src/transformers/models/opt/modeling_flax_opt.py
ArthurZucker f084d16
Update src/transformers/models/opt/modeling_tf_opt.py
ArthurZucker 2a421c0
update based on review
ArthurZucker 2057121
Merge branch 'add-opt-flax-tf' of github.com:ArthurZucker/transformer…
ArthurZucker 7dc05df
revert wrongly delted code
ArthurZucker c4616f9
clean decoder_input ids
ArthurZucker 3f55918
Update, most basic tests are passing; still need to handle head mask …
ArthurZucker df6a41c
Update modeling_flax_opt.py
ArthurZucker c9836a9
update flax tests
ArthurZucker 002aa02
update decoder forward
ArthurZucker b2cc3e6
update
ArthurZucker 99eb79d
update test and variable name
ArthurZucker 2fe8c84
update flax code with projection layers
ArthurZucker 7bb3fd1
update falx code
ArthurZucker 78d331e
Update code and fixed slow tests
ArthurZucker f16608c
Fixed test for modeling opt
ArthurZucker 9853263
update flax tests
ArthurZucker 47636cc
Update tests
ArthurZucker 43756d9
embed positions not yet properly handled in flax, a small detail
ArthurZucker cde30b3
update tests and code
ArthurZucker ee7ea83
update code, 1 test fails locally
ArthurZucker 93fbb50
renamed tests with flax and added jit generation test
ArthurZucker 2bf89b7
added jax jit test reduce tolerance for inference no head
ArthurZucker a1a1303
make style
ArthurZucker f7f3a81
make styel
ArthurZucker 9cd76d4
fixup
ArthurZucker 857cb14
deleted # Copied from transformers.models.bart.modeling_flax_bart whe…
ArthurZucker b67dd92
fix copies
ArthurZucker 22eff2d
fix copies
ArthurZucker 585a791
added jit test in test_logits
ArthurZucker 90a6248
update jit test
ArthurZucker ccd2b38
clean a bit
ArthurZucker 0ee898b
Merge branch 'main' into add-opt-flax-tf
ArthurZucker fc2f3b5
make style
ArthurZucker 003d431
fix doc a bit
ArthurZucker 92be5cd
fix TFOPTPretrainedModel type to TFOPTPreTrainedModel
ArthurZucker 89d6089
remove pretrained from doc
ArthurZucker e3748c6
update
ArthurZucker 39a5dc0
update
ArthurZucker 4778601
Should fix the generation test with jax.jit, thanks to Suraj
ArthurZucker f8f37c4
Update src/transformers/models/opt/modeling_flax_opt.py
ArthurZucker 4ed46db
Update src/transformers/models/opt/modeling_flax_opt.py
ArthurZucker 77ec74a
reformated
ArthurZucker d1ac7f3
update based on review
ArthurZucker 7fd102d
Fixed proj layer and tests
ArthurZucker e980cc0
clean a comment
ArthurZucker b23c2e5
style
ArthurZucker 00e0087
removed from_pt parameter as weights are on the hub for flax
ArthurZucker c312f12
Update tests/models/opt/test_modeling_flax_opt.py
ArthurZucker c2a46a0
Update tests/models/opt/test_modeling_flax_opt.py
ArthurZucker f3f687a
Update based on review
ArthurZucker 593fe0a
remove slow comments
ArthurZucker 353b5f9
intial commit
ArthurZucker 5d4b721
Update
ArthurZucker 2f01151
Updated code, model fromm PT load correctly, output is wrong
ArthurZucker 6ee842f
Update, tests and code, 8 tests left need to add jitted ones, logits …
ArthurZucker c2c730a
fixed __init__ bug
ArthurZucker 4544c86
fixed __init__
ArthurZucker c6b00dc
qulity
ArthurZucker 4e3acfd
sorted imports on init
ArthurZucker eae4fbb
Update tests
ArthurZucker f0e05fe
Update, 1 last test
ArthurZucker 1c1f85f
save_load now passes
ArthurZucker 1d282ef
padding is wrong but tests look good
ArthurZucker 66b0aa0
style
ArthurZucker 2e62da3
fix .mdx file
ArthurZucker c5bca23
update
ArthurZucker 5412146
update test using from pretrained
ArthurZucker d120a72
added more XLA test
ArthurZucker 4a2cac0
final tests should be solved
ArthurZucker e9438f7
style
ArthurZucker 12879ad
Cleanup
ArthurZucker 2bde027
style
ArthurZucker 8f72143
revert attention mask type
ArthurZucker 193c72f
update tol
ArthurZucker bd6eb00
quality
ArthurZucker 9f86150
revert previous cange
ArthurZucker 55b1309
Squashed commit of the following:
ArthurZucker 14784bb
Update based on reviews
ArthurZucker 545928a
update parent class of TFOPTModel
ArthurZucker dcafbcb
make style
ArthurZucker 13e4471
quality
ArthurZucker 9f1d0c2
fixed flax generation test
ArthurZucker 7a402d1
fix tf generationt test typo
ArthurZucker 2b512f4
clean flax and handled learned positional embedding
ArthurZucker 9017224
fixed TF tests
ArthurZucker 4a9fd9d
update flax code
ArthurZucker 7ad5265
make style
ArthurZucker 8186621
slow back in place
ArthurZucker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| *.py eol=lf | ||
| *.rst eol=lf | ||
| *.md eol=lf | ||
| *.md eol=lf | ||
| *.mdx eol=lf |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should not be here - maybe a problem with
git pullorgit merge?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes most probably, I merged with main, which gave me issues with the torch tests. Could not rebase here