[OpenVINO-EP] Enabling OpenVINO Runtime options for Perftest application #6654
Conversation
Enabling a flag to pass OpenVINO Runtime options as an string argument using a command line.
Enables OpenVINO EP runtime options into onnxruntime_perf_test. Now these options can be passed as an argument to the perf test CPP application using key-value pairs seperated by a space via a command line. Signed-off-by: MaajidKhan <[email protected]>
Signed-off-by: MaajidKhan <[email protected]>
Signed-off-by: MaajidKhan <[email protected]>
|
In the help section, I have added the runtime options available as a tabbed segment so that these options don't seem mixed with the other perf test options. Have added an example as well, so that it becomes easy for user to use these options instead of referring or going back to the documentation. |
|
/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux CPU Minimal Build E2E CI Pipeline,Linux Nuphar CI Pipeline,MacOS NoContribops CI Pipeline,Linux OpenVINO CI Pipeline,orttraining-distributed |
|
Azure Pipelines successfully started running 9 pipeline(s). |
|
/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline |
|
Azure Pipelines successfully started running 9 pipeline(s). |
|
@HectorSVC Looks like GPU CI pipeline is failing multiple times and I think it has nothing to do with OpenVINO code changes. |
|
/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline |
|
Azure Pipelines successfully started running 9 pipeline(s). |
|
/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux CPU Minimal Build E2E CI Pipeline,Linux Nuphar CI Pipeline,MacOS NoContribops CI Pipeline,Linux OpenVINO CI Pipeline,orttraining-distributed |
|
Azure Pipelines successfully started running 9 pipeline(s). |
| int cudnn_conv_algo{0}; | ||
| bool do_cuda_copy_in_separate_stream{false}; | ||
| bool set_denormal_as_zero{false}; | ||
| std::basic_string<ORTCHAR_T> ov_config_file; |
There was a problem hiding this comment.
config_string? it's not really a file.
| "\t-d [cudnn_conv_algorithm]: Specify CUDNN convolution algothrithms: 0(benchmark), 1(heuristic), 2(default). \n" | ||
| "\t-q: [CUDA only] use separate stream for copy. \n" | ||
| "\t-z: Set denormal as zero. When turning on this option reduces latency dramatically, a model may have denormals.\n" | ||
| "\t-i: [OpenVINO only] Specify OpenVINO runtime options as key value pairs. Different options available are: \n" |
There was a problem hiding this comment.
I was thinking -i could be used for any EP. wonder if there's a way to convey that it's generic,
while at the same time documenting the possible options for OpenVINO EP
There was a problem hiding this comment.
Yeah, I made -i flag EP generic now. It can be used by any EP's to pass their respective run time options as key-value pairs.
Here's how it looks now.
Signed-off-by: MaajidKhan <[email protected]>
|
/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux CPU Minimal Build E2E CI Pipeline,Linux Nuphar CI Pipeline,MacOS NoContribops CI Pipeline,Linux OpenVINO CI Pipeline,orttraining-distributed |
|
Azure Pipelines successfully started running 9 pipeline(s). |
|
/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline |
|
Azure Pipelines successfully started running 9 pipeline(s). |
3 participants
Description:
Enabling OpenVINO Runtime options for perftest application
Motivation and Context:
Enables OpenVINO EP runtime options into onnxruntime_perf_test.
Now these options can be passed as an single argument to the perf test CPP
application using key-value pairs separated by a space via command line.