Add support for Opt in tf and flax #17226
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- putting use cache to False
- remove commented block
- remove unecessary files
- remove a test file - added the logits test
Co-authored-by: Patrick von Platen <[email protected]>
- rm mask filling example on docstring
- remove useless args
- more tests should pass now - needs to clean more - documentation still needs to be done
- change attention architecture to BART-like - modify some tests - style fix
- remove opt for: - QA - cond generation - seq classif
TOkenizers are not implemented
Co-authored-by: Arthur <[email protected]>
Co-authored-by: Arthur <[email protected]>
Co-authored-by: Arthur <[email protected]>
``` ...).unsqueeze(``` by ``` >>>).unsqueeze(```
…izers for Training (huggingface#17154) * add torch SGD and Adagrad optimizer bits * refine naming Co-authored-by: Sylvain Gugger <[email protected]> Co-authored-by: Sylvain Gugger <[email protected]>
…e_dummy_inputs (huggingface#17105) * propagate attention_mask dtype * fixup&style
Co-authored-by: kuanwee.heng <[email protected]>
* Create RetriBERT tests folder * Add missing RetriBERT tokenizer test file * Apply style corrections * Add non-english filter * Update tests/retribert/test_tokenization_retribert.py Co-authored-by: SaulLu <[email protected]> * Update tests/retribert/test_tokenization_retribert.py Co-authored-by: SaulLu <[email protected]> * Move test files to new directory * Update import path for testing utils to new test file structure Co-authored-by: SaulLu <[email protected]>
…6907) * add seed worker and set_deterministic_seed_for_cuda function to enforce reproducability * change function name to enable determinism, add docstrings, reproducability support for tf * change function name to enable_determinism_for_distributed_training * revert changes in set_seed and call set_seed within enable_full_determinism * add one position argument for seed_worker function * add full_determinism flag in training args and call enable_full_determinism when it is true * add enable_full_determinism to documentation * apply make fixup after the last commit * Update src/transformers/training_args.py Co-authored-by: Sylvain Gugger <[email protected]> Co-authored-by: Sylvain Gugger <[email protected]>
…gface#17166) * Remove unneeded columns for IterableDataset * Add test * Update trainer tests * Edit docstring * Lint * Apply feedback * Apply feedback
* Fix markdown code block * Use consistent spelling for self-attention * Fix typos and phrasing * Fix code style
* Ensure tensors are at least 1d for pad and concat * Compatibility * Fix * Fix * Add test * Retrigger CI * Consistency with master * Retrigger CI
* [WIP] Add FLAVA model This PR aims to add [FLAVA](ihttps://arxiv.org/abs/2112.04482) model to the transformers repo. Following checklist delineates the list of things to be done for this PR to be complete: [x] Flava init [x] Flava base models [x] Flava layers [x] Flava Configs [x] Flava encoders [x] Flava pretraining models [ ] Flava classification/retrieval models (To be added in a separate PR) [x] Documentation updates [x] Imports updates [x] Argstring updates [x] Flava pretrained checkpoints [x] Flava tests [x] Flava processors [x] Sanity check [x] Lint
…16922) * adding philosophy.mdx translation to Spanish * adding philosophy.mdx translation to Spanish * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/philosophy.mdx Co-authored-by: Omar U. Espejel <[email protected]> * philosophy translation to Spanish * Update _toctree.yml * Update _toctree.yml * nits Co-authored-by: Omar U. Espejel <[email protected]>
* Spanish version of language_modeling.mdx doc file * modification to toctree.yml file * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/language_modeling.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Correct position of Guías conceptuales Co-authored-by: Omar U. Espejel <[email protected]>
…e#16882) * Spanish translation of fast_tokenizers.mdx * add fast_tokenizers to the spanish _toctree.yml * Update docs/source/es/fast_tokenizers.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/fast_tokenizers.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/fast_tokenizers.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/fast_tokenizers.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/fast_tokenizers.mdx Co-authored-by: Omar U. Espejel <[email protected]> * Update docs/source/es/fast_tokenizers.mdx Co-authored-by: Omar U. Espejel <[email protected]> Co-authored-by: Omar U. Espejel <[email protected]>
…xamples (huggingface#16685) * Change nits in Spanish for quicktour.mdx - Add tasks names in English too. - Fix small nits in Spanish * Translate index.mdx to Spanish * Translate body of index. * Translated the compatible models list (not the papers´ names). Since this should not be updated manually, I can come back to the original text. * Add models and a dataset for Spanish in the code exmaples * Replaced the English models to Spanish versions. * Add index to _toctree.yml and fix Spanish * Fix double ““ error * Change negative example in ASR example * make style * Debug style in quicktour.mdx
* Fix contents in index.mdx to match docs' sidebar * Eliminates api section from contents
Collaborator
Author
|
Closing for a new pull request where history is fixed |
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds support for OPT in Flax and TF.
Also clean Pytorch code a bit.
Who can review?
@LysandreJik, @patrickvonplaten, @patil-suraj, @sgugger