Add Tensorflow handling of ONNX conversion#13831

Merged

lewtun merged 846 commits intohuggingface:masterfrom

Albertobegue:onnx-conversion-with-tensorflow

Feb 10, 2022

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 27, 2022

Fix doc examples: cannot import name (#14698)

ydshieh
authored and
Alberto Bégué
committed
Fix: change tooslow to slow (#14734)

ydshieh
authored and
Alberto Bégué
committed
Small fixes for the doc (#14751)

sgugger
authored and
Alberto Bégué
committed
Update transformers metadata (#14724)

sgugger
authored and
Alberto Bégué
committed
Fix name

sgugger
authored and
Alberto Bégué
committed
Mention no images added to repository (#14738)

authored and
Alberto Bégué
committed
Avoid using tf.tile in embeddings for TF models (#14735)

ydshieh
authored and
Alberto Bégué
committed
Change how to load config of XLNetLMHeadModel (#14746)

josutk
authored and
Alberto Bégué
committed
Improve perceiver (#14750)

NielsRogge
authored and
Alberto Bégué
committed
Convert Trainer doc page to MarkDown (#14753)

sgugger
authored and
Alberto Bégué
committed
Update Table of Contents (#14755)

sgugger
authored and
Alberto Bégué
committed
Fixing tests for Perceiver (#14739)

Narsil
authored and
Alberto Bégué
committed
Make data shuffling in `run_clm_flax.py` respect global seed (#13410)

bminixhofer
authored and
Alberto Bégué
committed
Adding support for multiple mask tokens. (#14716)

authored and
Alberto Bégué
committed
Fix broken links to distillation on index page of documentation (#14722)

amitness
authored and
Alberto Bégué
committed
[doc] performance: groups of operations by compute-intensity (#14757)

stas00
authored and
Alberto Bégué
committed
Fix the doc_build_test job (#14774)

sgugger
authored and
Alberto Bégué
committed
Fix preprocess_function in run_summarization_flax.py (#14769)

ydshieh
authored and
Alberto Bégué
committed
Update t5.rst (#14776)

Xing Han Lu
authored and
Alberto Bégué
committed
TF model cards (#14720)

Rocketknight1
authored and
Alberto Bégué
committed
Update Perceiver code examples (#14783)

NielsRogge
authored and
Alberto Bégué
committed
Improve Perceiver docs (#14786)

authored and
Alberto Bégué
committed
Release: v4.14.0

LysandreJik
authored and
Alberto Bégué
committed
Docs for v4.14.0

LysandreJik
authored and
Alberto Bégué
committed
Move import (#14787)

sgugger
authored and
Alberto Bégué
committed
PoC for conserving old links (#14754)

authored and
Alberto Bégué
committed
Removes images to put them in a dataset (#14781)

LysandreJik
authored and
Alberto Bégué
committed
Post sphinx-clean up and contributing guide updates (#14790)

authored and
Alberto Bégué
committed
Fix the build documentation job (#14788)

sgugger
authored and
Alberto Bégué
committed
Update CONTRIBUTING.md (#14799)

kamalkraj
authored and
Alberto Bégué
committed
Update CONTRIBUTING.md (#14800)

kamalkraj
authored and
Alberto Bégué
committed
Train step fix (#14796)

Rocketknight1
authored and
Alberto Bégué
committed
Add Speaker Diarization and Verification heads (#14723)

anton-l
authored and
Alberto Bégué
committed
[Generate] Make generate multi-modal (#14784)

authored and
Alberto Bégué
committed
Add WavLM (#14354)

authored and
Alberto Bégué
committed
Remove datasets requirement (#14795)

LysandreJik
authored and
Alberto Bégué
committed
[WavLM] Correct position bias computation (#14805)

patrickvonplaten
authored and
Alberto Bégué
committed
Add test (#14810)

NielsRogge
authored and
Alberto Bégué
committed
[WavLM] Layerdrop is not allowed for first layer (#14811)

patrickvonplaten
authored and
Alberto Bégué
committed
[Generate] Correct input_ids detection (#14815)

patrickvonplaten
authored and
Alberto Bégué
committed
Implement head_mask for Flax BERT and other models copied from BERT (#14620)

stancld
authored and
Alberto Bégué
committed
Convert rst to mdx bert (#14806)

authored and
Alberto Bégué
committed
Wav2Vec2 meets phonemes (#14353)

authored and
Alberto Bégué
committed
[ImageGPT] Deprecate pixel_values input name to input_ids (#14801)

authored and
Alberto Bégué
committed
[Seq2SeqTrainer] Remove model input name hack (#14802)

patrickvonplaten
authored and
Alberto Bégué
committed
up (#14829)

patrickvonplaten
authored and
Alberto Bégué
committed
[WavLM] Fix slow tests (#14845)

patrickvonplaten
authored and
Alberto Bégué
committed
Add SD and SV heads for WavLM (#14847)

anton-l
authored and
Alberto Bégué
committed
Add an argument to set bucket_cap_mb for PyTorch DDP (#14756)

changlan
authored and
Alberto Bégué
committed
Update CONTRIBUTING.md (#14835)

kamalkraj
authored and
Alberto Bégué
committed
Fix dead link to benchmarks.ipynb (#14842)

DerekChia
authored and
Alberto Bégué
committed
[Perceiver] Skip multi-gpu tests for now (#14813)

patrickvonplaten
authored and
Alberto Bégué
committed
Add 'with torch.no_grad()' to integration test forward pass (#14821)

h-holm
authored and
Alberto Bégué
committed
Add 'with torch.no_grad()' to integration test forward pass (#14820)

h-holm
authored and
Alberto Bégué
committed
Add a main_input_name attribute to all models (#14803)

authored and
Alberto Bégué
committed
[doc] typo (#14849)

stas00
authored and
Alberto Bégué
committed
[logging] implement warning_advice / TRANSFORMERS_NO_ADVISORY_WARNINGS (#14669)

stas00
authored and
Alberto Bégué
committed
Make the onnx submodule init lazy (#14855)

sgugger
authored and
Alberto Bégué
committed
Convert docstrings of modeling files (#14850)

sgugger
authored and
Alberto Bégué
committed
[Bart] better error message (#14854)

patrickvonplaten
authored and
Alberto Bégué
committed
Only create the model card on process 0 (#14857)

sgugger
authored and
Alberto Bégué
committed
[ASR example] Improve example + add more examples (#14848)

patrickvonplaten
authored and
Alberto Bégué
committed
Fix the value error typo of AdamW's betas' valid values checking (#14780)

dourgey
authored and
Alberto Bégué
committed
Add custom `stopping_criteria` and `logits_processor` to `generate` (#14779)

authored and
Alberto Bégué
committed
Replace commit sha by commit url for update jobs (#14852)

authored and
Alberto Bégué
committed
[examples/summarization] deal with None in data records (#14816)

stas00
authored and
Alberto Bégué
committed
[doc porting] several docs (#14858)

authored and
Alberto Bégué
committed
Mass conversion of documentation from rst to Markdown (#14866)

sgugger
authored and
Alberto Bégué
committed
Skip failing test

sgugger
authored and
Alberto Bégué
committed
Fix FLAX_MULTIPLE_CHOICE_SAMPLE typo (#14871)

mishig25
authored and
Alberto Bégué
committed
Fixes in marian doc (#14872)

sgugger
authored and
Alberto Bégué
committed
Fix `FlaxMarianMTModel` return block. (#14873)

sgugger
authored and
Alberto Bégué
committed
Fix doc mistakes (#14874)

sgugger
authored and
Alberto Bégué
committed
Convert model files from rst to mdx (#14865)

authored and
Alberto Bégué
committed
update the arguments `add_prefix_space` and `trim_offsets` in `backend_tokenizer.post_processor` of `RobertaTokenizerFast` (#14752)

SaulLu
authored and
Alberto Bégué
committed
Feature/fix slow test in mluke (#14749)

ryokan0123
authored and
Alberto Bégué
committed
Updated deberta attention (#14625)

authored and
Alberto Bégué
committed
IterableDatasetShard should use per device batch size instead of real batch size (#14714)

SysuCharon
authored and
Alberto Bégué
committed
Fix typo in error message

sgugger
authored and
Alberto Bégué
committed
Fix Perceiver docs (#14879)

NielsRogge
authored and
Alberto Bégué
committed
Fix pytorch image classification example (#14883)

mariosasko
authored and
Alberto Bégué
committed
Onnx enable tasks for supported models (part 2) (#14700)

michaelbenayoun
authored and
Alberto Bégué
committed
Properly indent return block (#14887)

sgugger
authored and
Alberto Bégué
committed
Release: v4.15.0

patrickvonplaten
authored and
Alberto Bégué
committed
Docs for v4.16.0dev0

patrickvonplaten
authored and
Alberto Bégué
committed
Keras metric callback (#14867)

authored and
Alberto Bégué
committed
Convert rst files (#14888)

sgugger
authored and
Alberto Bégué
committed
Fix installation instructions for BART ONNX example (#14885)

lewtun
authored and
Alberto Bégué
committed
Fix doc examples: ... takes no keyword arguments (#14701)

authored and
Alberto Bégué
committed
Fix AttributeError from PreTrainedTokenizerFast.decoder (#14691)

aphedges
authored and
Alberto Bégué
committed
Add 'with torch.no_grad()' to integration test forward pass (#14808)

h-holm
authored and
Alberto Bégué
committed
Add ONNX support for MarianMT models (#14586)

authored and
Alberto Bégué
committed
add custom stopping criteria to human eval script (#14897)

lvwerra
authored and
Alberto Bégué
committed
Set `run_name` in MLflowCallback (#14894)

yangdong02
authored and
Alberto Bégué
committed
Add TFCLIPModel (#13967)

authored and
Alberto Bégué
committed
[AutoTokenizer] Fix incorrect from pretrained (#14900)

patrickvonplaten
authored and
Alberto Bégué
committed
Update diarization and WavLM tolerances (#14902)

anton-l
authored and
Alberto Bégué
committed
[doc] post-porting (#14890)

stas00
authored and
Alberto Bégué
committed
[Generate] Remove attention_mask and integrate model_main_input_name (#14856)

patrickvonplaten
authored and
Alberto Bégué
committed
Fix failing GPU trainer tests (#14903)

sgugger
authored and
Alberto Bégué
committed
Better logic for getting tokenizer config in AutoTokenizer (#14906)

sgugger
authored and
Alberto Bégué
committed
[doc] install - add jax (#14912)

stas00
authored and
Alberto Bégué
committed
[WavLM] fix wavlm docs (#14910)

patrickvonplaten
authored and
Alberto Bégué
committed
Fix Perceiver docs (#14917)

Sanster
authored and
Alberto Bégué
committed
ChunkPipeline (batch_size enabled on `zero-cls` and `qa` pipelines. (#14225)

authored and
Alberto Bégué
committed
Add `ElectraForCausalLM` -> Enable Electra encoder-decoder model (#14729)

stancld
authored and
Alberto Bégué
committed
fix to issue #14833 in data_collator - consider no labels (#14930)

Ayal Klein
authored and
Alberto Bégué
committed
Fix duplicate call to save_checkpoint when using deepspeed (#14946)

MihaiBalint
authored and
Alberto Bégué
committed
Doc styler v2 (#14950)

sgugger
authored and
Alberto Bégué
committed
Convert last rst file (#14952)

sgugger
authored and
Alberto Bégué
committed
[doc] consistent True/False/None default format (#14951)

authored and
Alberto Bégué
committed
[doc] :obj: hunt (#14954)

stas00
authored and
Alberto Bégué
committed
Doc styler examples (#14953)

sgugger
authored and
Alberto Bégué
committed
Style

sgugger
authored and
Alberto Bégué
committed
[doc] :class: hunt (#14955)

authored and
Alberto Bégué
committed
Add Speech Seq2Seq Training script (#14792)

patrickvonplaten
authored and
Alberto Bégué
committed
[WavLM] give model for precision (#14958)

patrickvonplaten
authored and
Alberto Bégué
committed
Update README.md (#14965)

patrickvonplaten
authored and
Alberto Bégué
committed
[Tests] Speed up tokenizer tests (#14964)

patrickvonplaten
authored and
Alberto Bégué
committed
[Wav2Vec2] Rename model's feature extractor to feature encoder (#14959)

patrickvonplaten
authored and
Alberto Bégué
committed
refactor: replace `assert` with `ValueError` (#14970)

jaketae
authored and
Alberto Bégué
committed
remove absl workaround as it's no longer needed (#14909)

stas00
authored and
Alberto Bégué
committed
Fixing a pathological case for slow tokenizers (#14981)

Narsil
authored and
Alberto Bégué
committed
[AutoProcessor] Correct AutoProcessor and automatically add processor… (#14881)

patrickvonplaten
authored and
Alberto Bégué
committed
[Generate] correct encoder_outputs are passed without attention_mask (#14980)

patrickvonplaten
authored and
Alberto Bégué
committed
Adding `num_return_sequences` support for text2text generation. (#14988)

authored and
Alberto Bégué
committed
Enabling `tokenizers` upgrade. (#14941)

Narsil
authored and
Alberto Bégué
committed
Allow training to resume even if RNG states are not properly loaded (#14994)

sgugger
authored and
Alberto Bégué
committed
Map model_type and doc pages names (#14944)

authored and
Alberto Bégué
committed
Fixing t2t pipelines lists outputs. (#15008)

Narsil
authored and
Alberto Bégué
committed
Improve truncation_side (#14947)

authored and
Alberto Bégué
committed
Large audio chunking for the existing ASR pipeline (#14896)

authored and
Alberto Bégué
committed
fix missing import (#15016)

ydshieh
authored and
Alberto Bégué
committed
[Tests] Correct Wav2Vec2 & WavLM tests (#15015)

patrickvonplaten
authored and
Alberto Bégué
committed
Update parallelism.mdx (#15013)

hyunwoongko
authored and
Alberto Bégué
committed
Fix Code block (#14983)

flozi00
authored and
Alberto Bégué
committed
Fix a little typo (#15002)

milyiyo
authored and
Alberto Bégué
committed
Add Flax RoFormer (#15005)

authored and
Alberto Bégué
committed
Hotfix `chunk_length_s` instead of `_ms`. (#15029)

Narsil
authored and
Alberto Bégué
committed
[doc] Update parallelism.mdx (#15018)

hyunwoongko
authored and
Alberto Bégué
committed
[megatron convert] PYTHONPATH requirements (#14956)

stas00
authored and
Alberto Bégué
committed
Fix doc example: mask_time_indices (numpy) has no attribute 'to' (#15033)

authored and
Alberto Bégué
committed
Adding QoL for `batch_size` arg (like others enabled everywhere). (#15027)

Narsil
authored and
Alberto Bégué
committed
[CLIP] Fix PT test (#15041)

patrickvonplaten
authored and
Alberto Bégué
committed
[SpeechEncoderDecoder] Fix from pretrained (#15043)

patrickvonplaten
authored and
Alberto Bégué
committed
[CLIP] Fix TF test (#15042)

patil-suraj
authored and
Alberto Bégué
committed
Add Flax image captioning example (#14864)

authored and
Alberto Bégué
committed
Enabling `TF` on `image-classification` pipeline. (#15030)

Narsil
authored and
Alberto Bégué
committed
wrapped forward passes in torch.no_grad() (#15037)

mattchurgin
authored and
Alberto Bégué
committed
Add detectron2 to Github actions (#15053)

NielsRogge
authored and
Alberto Bégué
committed
Remove old asserts. (#15012)

Narsil
authored and
Alberto Bégué
committed
Add 'with torch.no_grad()' to BertGeneration integration test forward passes (#14963)

itsTurner
authored and
Alberto Bégué
committed
Update run_speech_recognition_seq2seq.py (#14967)

flozi00
authored and
Alberto Bégué
committed
[VisionTextDualEncoder] Fix doc example

ydshieh
authored and
Alberto Bégué
committed
Resubmit changes after rebase to master (#14982)

kct22aws
authored and
Alberto Bégué
committed
[Fix doc examples] Add missing from_pretrained (#15044)

authored and
Alberto Bégué
committed
[VisionTextDualEncoder] Add token_type_ids param (#15073)

ydshieh
authored and
Alberto Bégué
committed
Fix convert for newer megatron-lm bert model (#14082)

yoquankara
authored and
Alberto Bégué
committed
[Wav2Vec2 Speech Event] Add speech event v2 (#15083)

patrickvonplaten
authored and
Alberto Bégué
committed
fix model table cell text alignment (#14999)

ydshieh
authored and
Alberto Bégué
committed
Update check_repo.py (#15014)

kamalkraj
authored and
Alberto Bégué
committed
Make OpenAIGPTTokenizer work with SpaCy 2.x and 3.x (#15019)

cody-moveworks
authored and
Alberto Bégué
committed
Change assignee for tokenizers (#15088)

LysandreJik
authored and
Alberto Bégué
committed
support the trocr small models (#14893)

authored and
Alberto Bégué
committed
fix doc example - AttributeError: type object 'RagModel' has no attribute 'from_question_encoder_generator_pretrained' (#15076)

ydshieh
authored and
Alberto Bégué
committed
Fix style

sgugger
authored and
Alberto Bégué
committed
Model summary horizontal banners (#15058)

mishig25
authored and
Alberto Bégué
committed
Use tqdm.auto in Pipeline docs (#14920)

bryant1410
authored and
Alberto Bégué
committed
[doc] normalize HF Transformers string (#15023)

stas00
authored and
Alberto Bégué
committed
Happy New Year! (#15094)

sgugger
authored and
Alberto Bégué
committed
[DOC] fix doc examples for bart-like models (#15093)

patil-suraj
authored and
Alberto Bégué
committed
[performance doc] Power and Cooling (#14935)

authored and
Alberto Bégué
committed
Add TFVisionEncoderDecoderModel (#14148)

authored and
Alberto Bégué
committed
Add test to check reported training loss (#15096)

sgugger
authored and
Alberto Bégué
committed
Take gradient accumulation into account when defining samplers (#15095)

sgugger
authored and
Alberto Bégué
committed
fix doc example - TypeError: forward() got an unexpected keyword argument 'input_ids' (#15092)

ydshieh
authored and
Alberto Bégué
committed
Fix cookiecutter (#15100)

NielsRogge
authored and
Alberto Bégué
committed
[Wav2Vec2ProcessorWithLM] improve decoder downlaod (#15040)

patrickvonplaten
authored and
Alberto Bégué
committed
Adds IBERT to models exportable with ONNX (#14868)

MaximovaIrina
authored and
Alberto Bégué
committed
change metric_key_prefix in seq2seq_trainer.py (#15099)

JejuWayfarer
authored and
Alberto Bégué
committed
Print out durations of all scheduled tests (#15102)

LysandreJik
authored and
Alberto Bégué
committed
Add Nystromformer (#14659)

authored and
Alberto Bégué
committed
Fix failing test (#15104)

LysandreJik
authored and
Alberto Bégué
committed
add spaces badges

AK391
authored and
Alberto Bégué
committed
Transformer-XL badge

AK391
authored and
Alberto Bégué
committed
Reformer Spaces badge

AK391
authored and
Alberto Bégué
committed
XLNet spaces badge

AK391
authored and
Alberto Bégué
committed
BERT spaces badge

AK391
authored and
Alberto Bégué
committed
ALBERT spaces badge

AK391
authored and
Alberto Bégué
committed
Roberta spaces badge

AK391
authored and
Alberto Bégué
committed
Distilbert spaces badge

AK391
authored and
Alberto Bégué
committed
ConvBERT spaces badge

AK391
authored and
Alberto Bégué
committed
XLM Spaces badge

AK391
authored and
Alberto Bégué
committed
XLM-Roberta Spaces badge

AK391
authored and
Alberto Bégué
committed
FlauBERT spaces badge

AK391
authored and
Alberto Bégué
committed
ELECTRA Spaces badge

AK391
authored and
Alberto Bégué
committed
Funnel Transformer spaces badge

AK391
authored and
Alberto Bégué
committed
Longformer Spaces badge

AK391
authored and
Alberto Bégué
committed
BART Spaces badge

AK391
authored and
Alberto Bégué
committed
Pegasus Spaces badge

AK391
authored and
Alberto Bégué
committed
MarianMT Spaces badge

AK391
authored and
Alberto Bégué
committed
T5 Spaces badge

AK391
authored and
Alberto Bégué
committed
MT5 Spaces badge

AK391
authored and
Alberto Bégué
committed
MBART spaces badge

AK391
authored and
Alberto Bégué
committed
ProphetNet spaces badge

AK391
authored and
Alberto Bégué
committed
DPR Spaces badge

AK391
authored and
Alberto Bégué
committed
XLM-ProphetNet Spaces badge

AK391
authored and
Alberto Bégué
committed
Doc styler tip (#15105)

sgugger
authored and
Alberto Bégué
committed
Update ONNX docs (#14904)

authored and
Alberto Bégué
committed
Fix saving FlaubertTokenizer configs (#14991)

vmaryasin
authored and
Alberto Bégué
committed
Update TF test_step to match train_step (#15111)

Rocketknight1
authored and
Alberto Bégué
committed
Fix typo in doc template

sgugger
authored and
Alberto Bégué
committed
Pipeline ASR with LM. (#15071)

authored and
Alberto Bégué
committed
use block_size instead of max_seq_length in tf run_clm example (#15036)

authored and
Alberto Bégué
committed
fix: switch from slow to generic tokenizer class (#15122)

lvwerra
authored and
Alberto Bégué
committed
Fix #14357 (#15001)

ydshieh
authored and
Alberto Bégué
committed
Fix link to deepspeed config

sgugger
authored and
Alberto Bégué
committed
Add ONNX configuration classes to docs (#15121)

lewtun
authored and
Alberto Bégué
committed
Add `with torch.no_grad()` to DistilBERT integration test forward pass (#14979)

jaketae
authored and
Alberto Bégué
committed
mBART support for run_summarization.py (#15125)

banda-larga
authored and
Alberto Bégué
committed
doc-builder -> doc-build (#15134)

authored and
Alberto Bégué
committed
Update src/transformers/onnx/convert.py

authored and
Alberto Bégué
committed
Merge branch 'onnx-conversion-with-tensorflow' of https://github.com/Albertobegue/transformers into onnx-conversion-with-tensorflow
Alberto Bégué
committed
Integrate tensorflow onnx conversion with the changes in master
Alberto Bégué
committed
Change documentation to mention conversion with Tensorflow
Alberto Bégué
committed

Commits on Jan 28, 2022

Define global variables even if tf or pt not available
Albertobegue
committed

Add Tensorflow handling of ONNX conversion#13831lewtun merged 846 commits intohuggingface:masterhuggingface/transformers:masterfrom Albertobegue:onnx-conversion-with-tensorflowCopy head branch name to clipboard

Commits

Commits on Jan 27, 2022

Add Tensorflow handling of ONNX conversion#13831
lewtun merged 846 commits intohuggingface:masterfrom
Albertobegue:onnx-conversion-with-tensorflow