Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inference add ONNXRuntime back-end #39988

Merged
merged 32 commits into from
Mar 10, 2022
Merged

Conversation

heliqi
Copy link
Contributor

@heliqi heliqi commented Feb 28, 2022

PR types

New features

PR changes

Others

Describe

paddle inference增加paddle2onnx、onnxruntime集成

  1. 新的onnx推理维不支持旧版的run接口(不会报错,调用返回false并提示),只实现了新的ZeroCopyRun接口
  2. 新增加了4个API接口(c++、c、python、go语言都同步有增加),用于开启onnxruntime推理、关闭onnxruntime推理、查询开启状态、开启onnxruntime优化。具体API接口为:
 // 开启onnxruntime
void AnalysisConfig::EnableONNXRuntime();
// 判断是否成功开启
bool AnalysisConfig::ONNXRuntimeEnabled();
 // 关闭onnxruntime
void AnalysisConfig::DisableONNXRuntime();
// 开启onnxruntime优化
void AnalysisConfig::EnableORTOptimization();

用户使用例子:

// 创建默认配置对象
paddle_infer::Config config;
// 设置预测模型路径
config.SetModel(FLAGS_model_file, FLAGS_params_file);
// 新增配置选项,启用 ONNX Runtime
config.EnableONNXRuntime();
// 开启onnxruntime优化(也可以不开启,使用onnxrutnime非必选项)
config.EnableORTOptimization();
// 根据 Config 创建预测对象
auto predictor = paddle_infer::CreatePredictor(config);
// 推理
predictor->Run();

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

Shixiaowei02
Shixiaowei02 previously approved these changes Mar 2, 2022
Copy link
Contributor

@jiweibo jiweibo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

缺失单测和CI,是否能修改PR-CI-Inference流水线,ort默认打开.

后面可以加上去,也在Paddle-Inference-Demo上加一个使用ort的示例

paddle/fluid/inference/api/onnxruntime_predictor.cc Outdated Show resolved Hide resolved
paddle/fluid/inference/api/onnxruntime_predictor.cc Outdated Show resolved Hide resolved
paddle/fluid/inference/api/onnxruntime_predictor.cc Outdated Show resolved Hide resolved
paddle/fluid/inference/api/onnxruntime_predictor.h Outdated Show resolved Hide resolved
paddle/fluid/inference/api/onnxruntime_predictor.h Outdated Show resolved Hide resolved
paddle/fluid/inference/api/onnxruntime_predictor.h Outdated Show resolved Hide resolved
paddle/fluid/pybind/inference_api.cc Outdated Show resolved Hide resolved
Copy link
Contributor

@qili93 qili93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for CMakeLists.txt

Copy link

@leiqing1 leiqing1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

麻烦更新下对应API文档。

@heliqi
Copy link
Contributor Author

heliqi commented Mar 10, 2022

麻烦更新下对应API文档。
API文档我等下提一个PR到对应的另一个repo

@Shixiaowei02
Copy link
Contributor

approve for CMakeLists.txt

Copy link
Contributor

@winter-wang winter-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@Superjomn Superjomn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API 名称修改,可以下个PR改下

Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@heliqi
Copy link
Contributor Author

heliqi commented Mar 10, 2022

麻烦更新下对应API文档。
API文档我等下提一个PR到对应的另一个repo
已提文档PR: PaddlePaddle/Paddle-Inference-Demo#222

@jiangjiajun jiangjiajun merged commit 431afc3 into PaddlePaddle:develop Mar 10, 2022
@jiangjiajun
Copy link
Collaborator

数据拷贝上仍然有优化的空间,下个PR解决 @heliqi

@heliqi
Copy link
Contributor Author

heliqi commented Mar 15, 2022

@jiangjiajun 优化PR: #40561

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.