Exporters onnx module by michaelbenayoun · Pull Request #403 · huggingface/optimum

michaelbenayoun · 2022-09-27T15:47:06Z

What does this PR do?

This PR adds the ONNX export to optimum, under optimum.exporters.onnx, which was previously supported in transformers. This also comes with a refactoring.

DummyInputGenerator classes have been added: their goal is to handle the generation of dummy inputs. This can be used in any sitution, and is not tied to the ONNX export at all.
All the base classes that were present in transformers are kept, but they now use DummyInputGenerators to generate dummy inputs. This means that there is no need to provide a tokenizer or a feature extractor to the OnnxConfig.
"Intermediate" configuration classes have been added to represent a middle-end between the base generic class, and the model specific onnx config class: EncoderOnnxConfig, DecoderOnnxConfig, Seq2SeqOnnxConfig, VisionOnnxConfig, TextAndVisionOnnxConfig, etc.
Massive class inheritance for model specific onnx config classes when possible

What's left?

DummyInputGenerators docstring
Automatic feature detection using the Hub (@julien-c) (Exporters automatic task detection #445)
Fixing some axis naming issue observed for some models
Making all the models pass the export test
Should we keep the attention_mask as input when not needed? Or make it an option?
Add support for audio tasks
Validate models with a different shape than the one used to trace
Allow 2 "versions" of tests:
- One which is fast, and lightweight for the CI to not take too long and fail from memory issues
- One which is slower, and performs the export on much bigger models

HuggingFaceDocBuilderDev · 2022-09-27T16:07:49Z

The documentation is not available anymore as the PR was closed or merged.

flozi00 · 2022-09-29T17:25:02Z

Any plans or advice about adding speech related tasks ?

michaelbenayoun · 2022-09-30T12:36:35Z

Hi @flozi00 ,
Yes the plan is to cover as much models as possible, so we will definitely support speech related tasks. It will happen in multiple PRs though.

regisss

Thanks for this huuuge work @michaelbenayoun! I have a few comments and I also spotted a few typos.

optimum/utils/input_generators.py

optimum/exporters/onnx/convert.py

optimum/exporters/onnx/features.py

optimum/exporters/onnx/model_configs.py

optimum/exporters/onnx/config.py

optimum/exporters/onnx/utils.py

regisss

Huge work @michaelbenayoun 🚀 🚀 🚀

tests/exporters/test_onnx_export.py

lewtun

Amazing work @michaelbenayoun - I really like the elegant approach to using inheritance with the ONNX configs!

I've left a few comments, but overall this looks really nice. The main question is whether we should use this feature as an opportunity to clean up some name choices we made in transformers

Also, I think this PR should copy-paste the serialization.mdx docs from transformers to optimum (with the appropriate changes of course)

lewtun · 2022-10-26T11:10:09Z

optimum/exporters/features.py

+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Model export features manager."""


I always found the word "features" a bit confusing in the original exporter - does it refer to feature vectors or something else?

Since we have the opportunity to do something new, would something like "architecture" or "head" be more appropriate?

👍 for architecture, or what about just pipeline type or task type?

More generally 100% agree that we could use this opportunity to improve the DX as much as we can

@osanseviero

I like:

TaskManager

ArchitectureManager

I fully agree that "features" sounds odd here.

My 2 cents: "task" sounds great, especially since people with little ML knowledge will understand it instantly whereas "head" will not mean much to them, "architecture" is as confusing as "features" IMO and "pipeline" can mean many different things.

either task or pipeline would be consistent with the rest of the Hub. If consistent with the Hub, then task fits better. If consistent with transformers, then pipeline.

I went for task!
Actually I think that pipeline is not suitted because we do not match the pipeline names. Like sequence-classification instead of text-classification.

Task sounds good (even if we can't 100% match the Hub definitions because of past key-value variants + the fact we're really mapping "task" to AutoModelForXxx).

optimum/exporters/features.py

lewtun · 2022-10-26T11:18:45Z

optimum/exporters/features.py

+    )
+
+ExportConfigConstructor = Callable[[PretrainedConfig], "ExportConfig"]
+FeatureNameToExportConfigDict = Dict[str, ExportConfigConstructor]


Is it necessary to include the type in this object? Wouldn't FeatureNameToExportConfig be better?

lewtun · 2022-10-26T11:40:00Z

optimum/exporters/onnx/config.py

+        if self.use_past:
+            common_inputs["attention_mask"][1] = "past_encoder_sequence + sequence"
+            common_inputs["decoder_input_ids"] = {0: "batch"}
+            # common_inputs["decoder_attention_mask"] = {0: "batch", 1: "past_decoder_sequence + sequence"}


No, still not sure about this.

optimum/exporters/onnx/convert.py

optimum/exporters/onnx/model_configs.py

lewtun · 2022-10-26T11:45:24Z

optimum/exporters/onnx/model_configs.py

+        }
+
+
+class AlbertOnnxConfig(BertOnnxConfig):


Very elegant!

optimum/exporters/onnx/model_configs.py

regisss

Huuuge PR @michaelbenayoun, thanks 🚀

regisss · 2022-10-26T20:43:38Z

optimum/exporters/features.py

+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Model export features manager."""


I fully agree that "features" sounds odd here.

My 2 cents: "task" sounds great, especially since people with little ML knowledge will understand it instantly whereas "head" will not mean much to them, "architecture" is as confusing as "features" IMO and "pipeline" can mean many different things.

optimum/exporters/features.py

optimum/exporters/onnx/__main__.py

optimum/exporters/onnx/model_configs.py

tests/exporters/test_onnx_export.py

optimum/exporters/features.py

optimum/exporters/onnx/config.py

michaelbenayoun added 7 commits September 23, 2022 15:34

[WIP] ONNX export in optimum => Bert works

7731177

[WIP] ONNX export in optimum => GPT-2 / GPT Neo

dd0ab63

[WIP] ONNX export in optimum => T5

15e9b8c

[WIP] ONNX export in optimum

0b896bf

[WIP] Add support for TensorFlow export

2023c69

[WIP] onnx export

180d7dd

Update file headers

e63a030

regisss reviewed Oct 3, 2022

View reviewed changes

michaelbenayoun added 5 commits October 10, 2022 11:43

[WIP] cache features.py

91dca2e

Refactor features.py

a967899

Bert like models ready

f0b2505

Apply suggestions

1c88966

Clean up

e90cb18

regisss approved these changes Oct 11, 2022

View reviewed changes

tests/exporters/test_onnx_export.py Outdated Show resolved Hide resolved

michaelbenayoun added 8 commits October 12, 2022 12:31

Backup input_generators.py version

b0bdd46

Most seq2seq models work

2cab267

Add vision models

c6f043f

Support for GPT-J

25b8c5c

CLIP is working

8bf6c80

Add support for the remaining models

1a85ef9

Make style

918947f

Fix workflow and remove commented code

9b8c3ee

michaelbenayoun requested a review from regisss October 26, 2022 09:37

lewtun reviewed Oct 26, 2022

View reviewed changes

lewtun mentioned this pull request Oct 26, 2022

Add support ORT whisper #420

Merged

3 tasks

michaelbenayoun added 2 commits October 26, 2022 15:44

nit

3b81d6c

Renamed dynamic axes

fd5654d

michaelbenayoun added 2 commits October 27, 2022 11:20

Fix BART

c01f5ec

Drop support for PyTorch 1.10 and lower

fd0a302

regisss reviewed Oct 27, 2022

View reviewed changes

michaelbenayoun added 4 commits October 27, 2022 15:06

Add re-order inputs method

25c253f

Small fixes

fbcbfb4

Changed FeaturesManager to TasksManager

0bb9af2

Apply suggestions

0e8f19c

regisss reviewed Oct 27, 2022

View reviewed changes

optimum/exporters/onnx/config.py Outdated Show resolved Hide resolved

michaelbenayoun added 11 commits October 28, 2022 10:55

Fix Bloom export

e5e0152

Fix mt5 export

55f683a

Change ATOL_FOR_VALIDATION for ResNet

89705c8

Update github actions for exporters to use pytest

40ad484

Styling

96f2e0a

Remove old comment

057958e

Fix github action for exporters

97e9f08

Fix tests that are not passing

217a7f0

Styling

e1862d1

Change Blenderbot model to use

3ffb7e5

Change Resnet ATOL value

868e1a2

michaelbenayoun merged commit 5972cc7 into huggingface:main Nov 2, 2022

fxmarty mentioned this pull request Dec 21, 2022

[BUG] Using the tensorrt slow down the inference #605

Closed

4 tasks

echarlaix mentioned this pull request Jun 18, 2025

remove timm from exporters extra #2299

Merged

		}


		class AlbertOnnxConfig(BertOnnxConfig):

Conversation

michaelbenayoun commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

HuggingFaceDocBuilderDev commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flozi00 commented Sep 29, 2022

Uh oh!

michaelbenayoun commented Sep 30, 2022

Uh oh!

regisss left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

regisss left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lewtun left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

regisss left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

michaelbenayoun commented Sep 27, 2022 •

edited

Loading

HuggingFaceDocBuilderDev commented Sep 27, 2022 •

edited

Loading

regisss left a comment •

edited

Loading