-
Notifications
You must be signed in to change notification settings - Fork 34.1k
[CLAP] Add CLAP to the library #21370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 202 commits
Commits
Show all changes
227 commits
Select commit
Hold shift + click to select a range
c6b06b5
add model like clip
ArthurZucker 7547c82
update
ArthurZucker 23c56ac
text model ok
younesbelkada 405e653
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
younesbelkada 1771782
clap text works
younesbelkada 9b276aa
some refactor
younesbelkada 45c36ba
more refactor
younesbelkada abee382
more refactor
younesbelkada a7219ec
more refactor
younesbelkada c553315
correct fusion
younesbelkada 4360623
more refactor
younesbelkada e3aff6f
new modules
younesbelkada 00eb73b
add basic processor
ArthurZucker 45e7ce9
fixup
ArthurZucker b5c483f
remove whisper copioed from
ArthurZucker 4084369
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker fc0d323
audio logits match
younesbelkada 8a27723
add doc
ArthurZucker 27f133f
correct filters mel and add maxlength
ArthurZucker 5ddc2f3
style
ArthurZucker fbb6124
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 815c5ce
few fixes
younesbelkada 95c400b
forward passes
ArthurZucker a41ff1a
fixup
ArthurZucker 38f7fe8
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 275633c
fixup
ArthurZucker 1b3a820
some clean up
younesbelkada 8fed2d0
remove mels form the dictionnary
ArthurZucker 2684c70
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 6b9051c
pad after the repeat
ArthurZucker a468f8d
update padding when dsmaller
ArthurZucker 844540b
fix padding
ArthurZucker aeb340e
style
ArthurZucker 5eaa517
use swin patch merging
younesbelkada 1182287
use copied from swin
younesbelkada 1a4caac
processor with any tokenizer
ArthurZucker 5e604a8
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker e94c9dd
more copied from
younesbelkada e9ff994
some clean up
younesbelkada 7dec8bc
more refactor
younesbelkada c352948
fix mel when rand_trunc
ArthurZucker f2d6ffe
style
ArthurZucker e71749c
remove unused imports
ArthurZucker d44efd1
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker f1627c6
update processing
ArthurZucker 612e46f
remove image processing tests
ArthurZucker c3042e2
add testing fiel
ArthurZucker 68cbadb
fixmodeling issues
ArthurZucker c20c66e
replace with `is_longer`
younesbelkada 9c03ac7
clap in serialization
ArthurZucker 6717d72
more refactor
younesbelkada a38a016
`make fixup`
younesbelkada f958fa1
make fixup
younesbelkada c218e3c
fix feature extractor
ArthurZucker 65cfca4
update test feature extractor
ArthurZucker 6f759c5
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 50613fe
`make fixup`
younesbelkada b1f46dc
clean up config
younesbelkada 5115057
more clean up
younesbelkada 833c061
more cleanup
younesbelkada 2f99f82
Merge branch 'main' into add-clap-model
younesbelkada 8ee3051
update tests
ArthurZucker e472482
refactor tests and inits
ArthurZucker b0f63c2
removeCLAP vision config
ArthurZucker 21b2b94
remove CLAP from image procssing auto and dummy vision objects
ArthurZucker 506772e
update inits
ArthurZucker 6efadbf
style
ArthurZucker 972f45e
re order classes in modeling clap
ArthurZucker c7b6cb9
Use roberta tokenizer as the other weights are not open sourced
ArthurZucker 13ab798
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker 48b346a
small cleaup
ArthurZucker d299422
remove tokenization CLAP
ArthurZucker f627ab9
processor tokenizr is roberta
ArthurZucker fdcfed1
update feature extraction doc
ArthurZucker 3dc78e8
remove vclap from model zero shot
ArthurZucker 0fef96f
update f_min and f_max to frequency_xx
ArthurZucker 0515a18
some changes
younesbelkada 0dcff26
make fixup
younesbelkada 48d611b
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker fe43210
consistent behavior ebtween rand_crop and fusion
ArthurZucker e44c75e
add numpy resize and bilinear and documentation
ArthurZucker 43df206
move resizing to image utils
ArthurZucker 5379926
clean feature extraction
ArthurZucker 107c544
import resize from correct file
ArthurZucker 7eb278d
resize in image transforms
ArthurZucker a4c1940
update
ArthurZucker 9594cc5
style
ArthurZucker 51d2b64
style
ArthurZucker ce79e64
nit
ArthurZucker 1bfd7dd
remove unused arguments form the feature extractor
ArthurZucker 6794f16
style
ArthurZucker 1b7b39d
few fixes + make fixup
younesbelkada 552aee2
oops
younesbelkada 05674c7
fix more tests
younesbelkada b8a4e44
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 74d3c4f
add zero shot audio classification pipeline
ArthurZucker cad3f42
update zeroshot classification pipeline
ArthurZucker f0d2194
fixup
ArthurZucker 754bed1
fix copies
ArthurZucker 5f0e467
all CI tests pass
younesbelkada 47f714f
make fixup + fix docs
younesbelkada 7d3ede5
fix docs
younesbelkada 4e5db4b
fix docs
younesbelkada 5d23429
update tests pip;eline
ArthurZucker aaa68ce
update zero shot pipeline
ArthurZucker c8e9c73
update feature extraction clap
ArthurZucker c54fce3
update tokenization auto
ArthurZucker 95f1352
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 4debc3d
use nested simplify
ArthurZucker 0be1e66
update pipeline tests
ArthurZucker c2e207b
Apply suggestions from code review
younesbelkada e47c696
split in two lines
younesbelkada dbab96e
fixes
younesbelkada 9c83ff7
refactor
younesbelkada b5b929a
clean up
younesbelkada 5e3aa5f
add integration tests
younesbelkada 2ea06b8
update config docstring
ArthurZucker bc20ff4
style
ArthurZucker d5fa7b1
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 312dd92
update processor
ArthurZucker 3cd99bf
fix processor test
younesbelkada b6b1dd0
fix feat extractor tests
younesbelkada 1ce1a25
update docs
ArthurZucker 6b451f7
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker d3b2d94
Apply suggestions from code review
younesbelkada 7ed1747
fix readmes
younesbelkada 6b2a6db
fix tips
younesbelkada b6fa932
Update src/transformers/models/auto/configuration_auto.py
ArthurZucker 62ffdf7
update doc and remove todo -> properly explained
ArthurZucker 1593f3c
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 8868e3c
fix idx and typo
ArthurZucker 9b9bd4b
typoe
ArthurZucker 9460eaa
cleanup config
ArthurZucker 82f4ede
cleanup tests, styles and doc
ArthurZucker d90a9f2
ignore docstyle on image transform
ArthurZucker ff25a45
add conversion script
younesbelkada a6c3cc2
remove the `clap` indx in favor of `CLAP`
ArthurZucker 79e5845
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tra…
ArthurZucker 01bb870
update __init
ArthurZucker 1b2c514
nits
ArthurZucker c805bae
Update src/transformers/pipelines/__init__.py
ArthurZucker 640f6f9
fix bug
younesbelkada 2b8f466
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
younesbelkada 177a705
clarifiy config
younesbelkada 57fd58c
fix copy
younesbelkada 3a9f5ac
Merge branch 'main' into add-clap-model
ArthurZucker 0874aba
fix init
younesbelkada 31d3204
Apply suggestions from code review
younesbelkada 948eb0a
fix model output
younesbelkada f75c02a
fix comment
younesbelkada 7d7a000
make fixup
younesbelkada 63d073a
make fixup
younesbelkada 3bea427
rename to `Clap`
younesbelkada c20981d
replace to `Clap`
younesbelkada 6c31509
replace to `Clap`
younesbelkada fb24c4e
repo consistency
younesbelkada bd29fec
again repo-consistency
younesbelkada 5cf2fa8
make fixup
younesbelkada 90e4c03
Apply suggestions from code review
ArthurZucker 729fa51
add config
younesbelkada df70e2c
changes
younesbelkada 7ebeacf
update conversion
younesbelkada 3be208e
Apply suggestions from code review
younesbelkada 505c623
remove unused function
younesbelkada efb526e
update based on code reviews
ArthurZucker 7d1c2d3
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 2d57cfc
style
ArthurZucker 441e247
more comments
younesbelkada 1d82be5
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
younesbelkada 51b3dde
cleanup
younesbelkada 8d5e5ad
clean up
younesbelkada 38ce551
style
ArthurZucker 7b38c76
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker aa85941
apply suggestions
younesbelkada a45b221
Empty commit
ArthurZucker b9ee298
pipeline will be added in a different PR
ArthurZucker a47f063
update calls to audio utils functions
ArthurZucker 9eb1ab4
update pipeline init
ArthurZucker fd1957b
style
ArthurZucker 773b225
style
ArthurZucker c7a7cf6
styling again
ArthurZucker c970399
use pad
younesbelkada 21f60ce
fix repo-consistency
ArthurZucker 9ba34cd
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 9ca1780
update utils and add doc for audio utils
ArthurZucker 0e0990a
clean up resize by using torch. update inits accordingly
ArthurZucker 1c82889
style
ArthurZucker 626a664
CLap's tokenizer is RobertA
ArthurZucker a1471b6
add audio utils to internal toctreee
ArthurZucker 7ed4f0f
update totctree
ArthurZucker c07ab1e
style
ArthurZucker 291b8f8
update documentation and normalize naming accross audio utils and fea…
ArthurZucker 80a6212
style
ArthurZucker b29ee04
clean up
younesbelkada 56d2b00
update doc and typos
ArthurZucker 556c86f
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker a5d7cd1
fix doctest
younesbelkada d5376ea
update modelin code, got rid of a lot of reshaping
ArthurZucker f4b0441
style on added doc audio utils
ArthurZucker 48b98d0
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker ad82198
update modeling clap
ArthurZucker 6ed76fa
style
ArthurZucker af97ade
Apply suggestions from code review
ArthurZucker c5749da
docstringvariables with CLAP
ArthurZucker fd0bd47
rename key
younesbelkada 13f41eb
update modeling CLAP
ArthurZucker 7865442
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 2655e13
update audio utils docstring
ArthurZucker dbe50a7
update processing clap
ArthurZucker 34255aa
fix readmes
younesbelkada c4cdd97
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
younesbelkada de162eb
fix toctree
younesbelkada 83d0716
udpate configuration clap
ArthurZucker 0a25e75
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker 230b516
fix init
younesbelkada f379031
make fixup
younesbelkada fe1fbe3
fix
younesbelkada 75171e3
fix
younesbelkada 2fba86b
update naming
ArthurZucker d75a5f8
Merge branch 'add-clap-model' of https://github.com/ArthurZucker/tran…
ArthurZucker c221e1d
update
ArthurZucker 25610ce
update checkpoint path
ArthurZucker 6856ff0
Apply suggestions from code review
ArthurZucker 1ce6363
Major refactoring
ArthurZucker a8dc9a4
Update src/transformers/models/clap/configuration_clap.py
ArthurZucker 9b5b252
merge
ArthurZucker 66b7dfe
Merge branch 'main' of https://github.com/huggingface/transformers in…
ArthurZucker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.