Add FLAGS_allow_cinn_ops & FLAGS_deny_cinn_ops for controlling op types used in training with CINN. #36842
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
wzzju
changed the title
add+relu loss values (CINN).add+relu loss values (CINN Test).
thisjiang
reviewed
Oct 28, 2021
| cinn_losses = train(self.tmpdir, "paddle") | ||
| set_cinn_flag(False) | ||
| pd_losses = train(self.tmpdir, "cinn") | ||
| np.allclose(cinn_losses, pd_losses) |
There was a problem hiding this comment.
Suggested change
| np.allclose(cinn_losses, pd_losses) | |
| self.assertTrue(np.allclose(cinn_losses, pd_losses, atol=1e-5)) |
wzzju
changed the title
add+relu loss values (CINN Test).
CtfGo
reviewed
Nov 2, 2021
| if (!graphs_.count(graph_key)) { | ||
| graphs_[graph_key] = std::move(graph); | ||
| } else { | ||
| LOG(WARNING) |
There was a problem hiding this comment.
这种情况不确定，如果只是运用build_cinn_pass一次，应该不会出现，这里已改为使用PADDLE_ENFORCE。
| @@ -59,40 +61,60 @@ std::string CinnCompiler::AddGraph(std::unique_ptr<Graph> graph) { | |||
| ProgramDesc program; | |||
| GraphToProgram(*graph, &program); | |||
| program.Proto()->SerializeToString(&graph_key); | |||
There was a problem hiding this comment.
序列化的字符串直接作为key比较冗余，查找效率低、还占空间。是否以其hash值作为key，CinnCompiler额外提供接口可以由key获取其子图的序列化字符串？
There was a problem hiding this comment.
std::string作为std::unordered_map的key，查找时就是使用string的hash code查找的，你说不存储这个字符串，是想通过hash code回转得到愿字符串吗？这个恐怕是做不到的
There was a problem hiding this comment.
不是，查找时是string->hash吧，我意思是key存hash code，CinnCompiler提供接口由key获取graph->debug string替代ReadableProtoStr，不过这不重要。
Xreki
previously approved these changes
Nov 3, 2021
thisjiang
reviewed
Nov 3, 2021
| auto* cinn_compiler = CinnCompiler::GetInstance(); | ||
| const auto& compiling_graph = cinn_compiler->FindGraph(compilation_key); | ||
| // viz_graph("compiling_graph.dot", const_cast<Graph*>(&compiling_graph)); | ||
| viz_graph("compiling_graph.dot", const_cast<Graph*>(&compiling_graph)); |
There was a problem hiding this comment.
这个测试生成的文件是不是应该注释掉？或者在函数最后删掉？
There was a problem hiding this comment.
方便后续调试，先不删除，ci上build目录生成的文件会被自动清理的。
thisjiang
reviewed
Nov 3, 2021
| } | ||
| // if the op type is registered in CINN and deny_ops is not empty, return | ||
| // true only when it is not in deny_ops | ||
| auto deny_ops = StringSplit(FLAGS_deny_cinn_ops, kDelim); |
thisjiang
reviewed
Nov 3, 2021
| node->Name()) != nullptr; | ||
| // if the op type is registered in CINN and allow_ops is not empty, return | ||
| // true only when it is in allow_ops | ||
| auto allow_ops = StringSplit(FLAGS_allow_cinn_ops, kDelim); |
There was a problem hiding this comment.
这个可以放在外面吧？就不用每次都计算了
Suggested change
| auto allow_ops = StringSplit(FLAGS_allow_cinn_ops, kDelim); | |
| auto allow_ops = StringSplit(FLAGS_allow_cinn_ops, kDelim); | |
| auto teller = [&allow_ops](const Node* node) { ... }; |
lanxianghit
previously approved these changes
Nov 3, 2021
CtfGo
previously approved these changes
Nov 3, 2021
wzzju
dismissed stale reviews from CtfGo, thisjiang, and lanxianghit
via
0f86f5f
Xreki
previously approved these changes
Nov 3, 2021
lanxianghit
approved these changes
Nov 3, 2021
Xreki
approved these changes
Nov 3, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Describe
FLAGS_allow_cinn_ops和FLAGS_deny_cinn_ops两个flag，用于控制Paddle训练中使用CINN算子代替原生算子的种类。CinnCompiler类在多线程运行环境下的正确性（主要关于对cache_的操作）。CinnCompiler类增加VizGraph和ReadableKey方法，用于打印子图的dot信息以及获取具备可读性的CompilationKey。test_parallel_executor_run_cinn.py单测中增加对以下两种情况中add+relu模型loss值的比较：add+relu模型add+relu模型