Add eager layout autotune #45409

AnnaTrainingG · 2022-08-25T06:19:46Z

PR types

Others

PR changes

OPs

Describe

Add eager layout autotune
本PR 修改了Conv.py中实现，不涉及功能修改及接口变动
新动态图单测测试结果：

老动态图测试结果

自动代码生成效果 dygraph_functions.cc：

自动代码生成效果 eager_final_state_op_function_impl.h：

paddle-bot · 2022-08-25T06:19:55Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

zkh2016 · 2022-09-01T11:00:03Z

paddle/fluid/eager/eager_layout_transformer.h

+  void SetOutTensorLayout(
+      std::vector<paddle::experimental::Tensor*>* out_tensor) {
+    for (size_t i = 0; i < out_tensor->size(); i++) {
+      VLOG(4) << "out layout is"


这里打印的log会不会有点多？

当前版本为基础版本，为便于功能调试添加的打印，后续会统一删除

zyfncg · 2022-09-01T11:11:12Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

+LAYOUT_LOGIC_TEMPLATE=\
+"""
+  if (paddle::imperative::LayoutAutoTune::Instance().UseLayoutAutoTune()) {{
+    VLOG(5) << "Check and Prepare For LAYOUT";


这里可以把API名带上，LOG读起来会更直观些

好的，当前版本为基础版本，为便于功能调试添加的打印，后续会统一调整

zyfncg · 2022-09-01T11:21:04Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

+                layout_autotune_attr) == 0:
+            layout_logic_str = ""
+        else:
+            # after_call_str = f"return {forward_function_name}({layout_inputs_call_args_str});\n"


注释的代码可以去掉

zyfncg · 2022-09-01T11:23:13Z

paddle/fluid/eager/eager_layout_auto_tune.h

+// For agnostic op like add / relu
+inline std::shared_ptr<EagerLayoutTransformer> EagerLayoutAutotune(
+    const std::string& op_name,
+    const paddle::small_vector<std::vector<paddle::experimental::Tensor>,
+                               kSlotSmallVectorSize>& tensors_vector) {
+  VLOG(3) << " Optimze Layout agnostic op: " << op_name;


agnostic op 是指？

与layout类型无关的OP 如exp relu等

zyfncg · 2022-09-01T11:39:31Z

paddle/fluid/eager/eager_layout_transformer.h

+
+ protected:
+  std::string op_name_;
+  std::string final_layout_;


这里final_layout_为什么使用string而不是DataLayout类型？

当前为基础版本，为便于打印，后续将统一修改为DataLayout

zyfncg · 2022-09-01T11:44:03Z

paddle/fluid/eager/eager_layout_transformer.h

+  std::unordered_set<std::string> heavily_input_{"x", "y", "input"};
+};
+
+class EagerConcatOpTransformer


为什么这些Op的Transformer需要手写实现？这个开发成本似乎比较高

该类OP可通过特定转换完成功能支持，无需添加Transpose，后续同类OP较多时可复用

zyfncg · 2022-09-01T11:45:09Z

paddle/fluid/imperative/layout_autotune.h

+  DataLayout desired_layout_{DataLayout::UNDEFINED};
+
+  DataLayout default_layout_{DataLayout::UNDEFINED};


可能需要加注释解释下desired_layout_和default_layout_的功能含义

好的下个PR统一添加

zyfncg · 2022-09-01T11:47:36Z

paddle/fluid/imperative/layout_transformer.h

        if (var != nullptr && (paddle::imperative::GetDataLayout(var) ==
                               LayoutAutoTune::Instance().GetDesiredLayout())) {
          in_layout = LayoutAutoTune::Instance().GetDesiredLayout();
          break;
        }
      }
    }
-    SetVarsLayout(outs, in_layout);
+    VLOG(3) << "Optimze Layout agnostic op: " << type_ << " "
+            << paddle::framework::DataLayoutToString(in_layout);


DataLayout重载了<<，应该可以直接打印，不用再加 DataLayoutToString

好的下个PR统一调整打印

zyfncg · 2022-09-01T11:50:20Z

paddle/phi/api/lib/data_transform.cc

@@ -52,12 +52,16 @@ inline bool NeedTransformPlace(const paddle::platform::Place& input,
  return ret;
 }

-inline bool NeedTransformLayout(const DataLayout& input,
+inline bool NeedTransformLayout(const paddle::platform::Place& place,


phi下面需要使用phi::Place

place参数建议放在input和target之后

好的下个PR 修改

zhangting2020

LGTM

momozi1996

LGTM

JiabinYang

some comments

JiabinYang · 2022-09-05T02:43:30Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

-
+LAYOUT_LOGIC_TEMPLATE=\
+"""
+  if (paddle::imperative::LayoutAutoTune::Instance().UseLayoutAutoTune()) {{


How about merge this into egr::Controller, too many singleton instance looks weird..

此处是为统计新动态图与老动态的的判断条件

Controller里是有一个Tracer的我理解是不是可以共用

JiabinYang · 2022-09-05T02:44:35Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

@@ -992,6 +1008,9 @@ def GenerateForwardDefinitionAndDeclaration(self, is_inplaced):
        amp_tensors_vector_optional_list = []
        amp_autocast_list = []
        amp_autocast_optional_list = []
+        layout_autotune_list = []


Add some comment block to mark autotune code gen, or seal this into a function

好的，为避免代码冲突，后续提交PR统一修改

JiabinYang · 2022-09-05T02:45:49Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

@@ -1023,6 +1048,9 @@ def GenerateForwardDefinitionAndDeclaration(self, is_inplaced):
                        amp_autocast_list.append(
                            f"auto NEW_{name} = egr::EagerAmpAutoCast(\"{name}\", {name}, amp_dst_dtype, op_name);\n"
                        )
+                    layout_autotune_list.append(


Bad name, tmp var name should only have lower case letters.

这个是为了复用amp的参数列表，为避免修改他人代码，直接使用了NEW_，
当前PR为基础版本后续将提PR统计修改完善

JiabinYang · 2022-09-05T02:46:03Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

@@ -1037,6 +1065,9 @@ def GenerateForwardDefinitionAndDeclaration(self, is_inplaced):
                    amp_autocast_optional_list.append(
                        f"auto NEW_{name} = egr::EagerAmpAutoCasts(\"{name}\", {name}, amp_dst_dtype, op_name);\n"
                    )
+                    layout_autotune_optional_list.append(


这个是为了复用amp的参数列表，为避免修改他人代码，直接使用了NEW_，
当前PR为基础版本后续将提PR统计修改完善

JiabinYang · 2022-09-05T02:46:11Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

@@ -1047,10 +1078,59 @@ def GenerateForwardDefinitionAndDeclaration(self, is_inplaced):
                    amp_autocast_list.append(
                        f"auto NEW_{name} = egr::EagerAmpAutoCasts(\"{name}\", {name}, amp_dst_dtype, op_name);\n"
                    )
+                    layout_autotune_list.append(
+                        f"auto NEW_{name} = transformer->TransInTensor(\"{name}\", {name});\n"


这个是为了复用amp的参数列表，为避免修改他人代码，直接使用了NEW_，
当前PR为基础版本后续将提PR统计修改完善

JiabinYang · 2022-09-05T02:47:52Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py


            inputs_args_definition_list[pos] = arg_str
            inputs_args_declaration_list[pos] = arg_str

+        # for layout autotune attr


Is there any possible to make this a independent code block, as we've already had too many functional code here.

目前没有其他方案

我的意思把这个部分的代码统一整合到一个方法中，因为目前已经有很多字功能代码在code_gen里了我们后期也需要整理下

JiabinYang

LGTM for now, but plz fix mentioned problem in later PR ASAP

cherry-pick from #45826 LayoutAutotune 支持 inplace 类型的OP 根据 Add eager layout autotune #45409 修改意见调整UseAutotune 将LayoutAutotune判断放到controller中，与AMP 判断保持一致

AnnaTrainingG force-pushed the DataLayout branch 3 times, most recently from c71bffc to f6774d7 Compare August 31, 2022 02:55

AnnaTrainingG added 11 commits August 31, 2022 09:18

update

210e92a

update check error

1600b1e

Ctest error

cb09034

update api after rebase

3a7a058

Add layout in eager layout

a3495dd

Set ctest for eager

b01cb28

NEED Transpose

7610599

update test_dataloader_dataset

280d505

-1000

b6f633b

update

47a92d7

update for compile

75657ab

AnnaTrainingG force-pushed the DataLayout branch from 8397eac to 75657ab Compare September 1, 2022 01:33

AnnaTrainingG added 2 commits September 1, 2022 01:45

update

9251ad2

-1000

f085c9c

AnnaTrainingG requested review from zhangting2020, veyron95 and zyfncg September 1, 2022 05:52

zkh2016 approved these changes Sep 1, 2022

View reviewed changes

zkh2016 reviewed Sep 1, 2022

View reviewed changes

zyfncg reviewed Sep 1, 2022

View reviewed changes

zhangting2020 approved these changes Sep 2, 2022

View reviewed changes

momozi1996 approved these changes Sep 5, 2022

View reviewed changes

Ligoml approved these changes Sep 5, 2022

View reviewed changes

JiabinYang reviewed Sep 5, 2022

View reviewed changes

JiabinYang approved these changes Sep 5, 2022

View reviewed changes

lanxianghit approved these changes Sep 5, 2022

View reviewed changes

AnnaTrainingG merged commit d7d9807 into PaddlePaddle:develop Sep 5, 2022

AnnaTrainingG mentioned this pull request Sep 7, 2022

Update layoutautotune for inplace #45826

Merged

AnnaTrainingG mentioned this pull request Sep 19, 2022

[Cherry-pick]Update layoutautotune for inplace (#45826) #46226

Merged

		DataLayout desired_layout_{DataLayout::UNDEFINED};

		DataLayout default_layout_{DataLayout::UNDEFINED};

Add eager layout autotune #45409

Add eager layout autotune #45409

Conversation

AnnaTrainingG commented Aug 25, 2022 • edited Loading

PR types

PR changes

Describe

paddle-bot bot commented Aug 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AnnaTrainingG Sep 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhangting2020 left a comment

Choose a reason for hiding this comment

momozi1996 left a comment

Choose a reason for hiding this comment

JiabinYang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AnnaTrainingG Sep 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JiabinYang left a comment

Choose a reason for hiding this comment

AnnaTrainingG commented Aug 25, 2022 •

edited

Loading

AnnaTrainingG Sep 1, 2022 •

edited

Loading

AnnaTrainingG Sep 5, 2022 •

edited

Loading