Skip to content

Add option to set ORT_DISABLE_ALL as optimization #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tanmayv25 merged 2 commits into from
Dec 18, 2023
Merged

Add option to set ORT_DISABLE_ALL as optimization #195

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
94c0359
Allow ORT_DISABLE_ALL value
casassg Jun 21, 2023
efa593b
update readme
casassg Dec 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ Details regarding when to use these options and what to expect from them can be
* `inter_op_thread_count`: Sets the number of threads used to parallelize the execution of the graph (across nodes). If sequential execution is enabled this value is ignored.
A value of 0 means ORT will pick a default which is number of cores.
* `execution_mode`: Controls whether operators in the graph are executed sequentially or in parallel. Usually when the model has many branches, setting this option to 1 .i.e. "parallel" will give you better performance. Default is 0 which is "sequential execution."
* `level`: Refers to the graph optimization level. By default all optimizations are enabled. Allowed values are -1 and 1. -1 refers to BASIC optimizations and 1 refers to basic plus extended optimizations like fusions. Please find the details [here](https://onnxruntime.ai/docs/performance/graph-optimizations.html)
* `level`: Refers to the graph optimization level. By default all optimizations are enabled. Allowed values are -1, 1 and 2. -1 refers to BASIC optimizations, 1 refers to basic plus extended optimizations like fusions and 2 refers to all optimizations being disabled. Please find the details [here](https://onnxruntime.ai/docs/performance/graph-optimizations.html).

```
optimization {
Expand Down
2 changes: 2 additions & 0 deletions src/onnxruntime.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,8 @@ ModelState::ModelState(TRITONBACKEND_Model* triton_model)
optimization_level = GraphOptimizationLevel::ORT_ENABLE_BASIC;
} else if (graph_level == 1) {
optimization_level = GraphOptimizationLevel::ORT_ENABLE_EXTENDED;
} else if (graph_level == 2) {
optimization_level = GraphOptimizationLevel::ORT_DISABLE_ALL;
}
}
}
Expand Down