Move some activation to phi #40727

phlrain · 2022-03-19T08:00:21Z

PR types

Breaking changes

PR changes

OPs

Describe

move activation to phi

… move_temporal_shift_to_phi

… move_some_activaion_to_phi

paddle-bot-old · 2022-03-19T08:00:25Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… move_some_activaion_to_phi

YuanRisheng · 2022-03-21T02:52:13Z

paddle/fluid/operators/activation_op.h

-            typename dX>
-  void operator()(Device d, X x, Out out, dOut dout, dX dx) const {
-    dx.device(d) = static_cast<T>(0.5) * dout / out;
+struct SqrtGradGradFunctor : public BaseActivationFunctor<T> {


二阶导是后边的pr迁移吗

是发生了奇怪结果错误，代码有了，准备分另外一个pr来搞

YuanRisheng · 2022-03-21T02:54:00Z

paddle/fluid/operators/activation_op.h

-    functor(place, eigen_in, eigen_out, eigen_p, eps);
-  }
-};
-
 template <typename DeviceContext, typename T>
 class LogitGradKernel : public framework::OpKernel<T> {


Kernel已迁，这个可以删掉

YuanRisheng · 2022-03-21T02:55:38Z

paddle/fluid/operators/activation_op.cc

+// REGISTER_OPERATOR(
+//     expm1, ops::ActivationOp, ops::Expm1OpMaker,
+//     ops::ActivationOpInferVarType,
+//     ops::ActivationGradOpMaker<ops::Expm1GradFunctor<float>::FwdDeps(),
+//                                paddle::framework::OpDesc>,
+//     ops::ActivationGradOpMaker<ops::Expm1GradFunctor<float>::FwdDeps(),
+//                                paddle::imperative::OpBase>,
+//     std::conditional<ops::CanInplaceAct<ops::Expm1GradFunctor<float>>(),
+//                      ops::ActFwdInplaceInferer, void>::type);


无用注释删掉

这里是不是没删

YuanRisheng · 2022-03-21T02:57:59Z

paddle/fluid/operators/activation_op.kps

-};
-
-template <typename T>
-struct CudaMishGradFunctor : public BaseActivationFunctor<T> {


这些cuda functor也需要使用USE_PHI_FUNCTOR引过来，这个文件里还有xpu的注册需要用到这些functor

YuanRisheng · 2022-03-21T03:03:22Z

paddle/phi/kernels/funcs/activation_functor.h

+// template <typename T>
+// struct SoftsignGradFunctor : public BaseActivationFunctor<T> {
+//   template <typename Device, typename X, typename Out, typename dOut,
+//             typename dX>
+//   void operator()(Device d, X x, Out out, dOut dout, dX dx) {
+//     dx.device(d) =
+//         dout * (static_cast<T>(1) / (static_cast<T>(1) + x.abs()).square());
+//   }
+
+//   static constexpr ActBwdOpFwdDeps FwdDeps() { return ActBwdOpFwdDeps::kDepX;
+//   }
+// };


无用注释删掉

YuanRisheng · 2022-03-21T03:07:00Z

paddle/phi/kernels/impl/activation_impl.h

+    // functor(*place, To32BitIndex(x), To32BitIndex(out));
+    functor(*place, x, out);


这里修改的原因是？

误操作，我回复过去

YuanRisheng · 2022-03-21T03:13:43Z

paddle/fluid/operators/activation_op.cc

@@ -1675,7 +1679,6 @@ REGISTER_OPERATOR(
    ops::ActivationOpDoubleGrad<ops::SqrtGradGradFunctor<float>::FwdDeps()>,
    ops::ActivationDoubleGradOpInplaceInferer);

-REGISTER_ACTIVATION_CPU_KERNEL(sqrt, Sqrt, SqrtFunctor, SqrtGradFunctor);


sqrt和square有SelcetedRows类型的输入，在selectedrows未迁移的情况下删掉这里会不会出现问题

我恢复下

DannyIsFunny · 2022-03-22T11:01:23Z

修复infrt CI 失败的问题：
#40826

… move_some_activaion_to_phi

chenwhql · 2022-03-26T05:37:49Z

paddle/fluid/operators/activation_op.cc

+// REGISTER_OPERATOR(
+//     expm1, ops::ActivationOp, ops::Expm1OpMaker,
+//     ops::ActivationOpInferVarType,
+//     ops::ActivationGradOpMaker<ops::Expm1GradFunctor<float>::FwdDeps(),
+//                                paddle::framework::OpDesc>,
+//     ops::ActivationGradOpMaker<ops::Expm1GradFunctor<float>::FwdDeps(),
+//                                paddle::imperative::OpBase>,
+//     std::conditional<ops::CanInplaceAct<ops::Expm1GradFunctor<float>>(),
+//                      ops::ActFwdInplaceInferer, void>::type);


这里是不是没删

chenwhql · 2022-03-26T05:39:51Z

paddle/phi/kernels/clip_by_norm_kernel.h

+template <typename T, typename Context>
+void ClipByNormSparseKernel(const Context& ctx,
+                            const SelectedRows& x,
+                            float max_norm,


selected rows移动到对应目录

clip by norm有其他的注册问题，单独提pr处理

chenwhql · 2022-03-26T05:42:25Z

paddle/phi/kernels/gpu/clip_by_norm_kernel.cu

+
+}  // namespace phi
+
+// PD_REGISTER_KERNEL(


注释移除吗

clip by norm有其他的注册问题，单独提pr处理

chenwhql · 2022-03-26T05:44:00Z

paddle/phi/ops/compat/activation_sig.cc

+    return KernelSignature("sqrt", {"X"}, {}, {"Out"});
+  }
+
+  return KernelSignature("unregistered", {}, {}, {});


如果迁移完整的话，应该不需要unregister

chenwhql · 2022-03-26T05:44:54Z

paddle/phi/ops/compat/clip_by_norm_sig.cc

+  if (ctx.IsDenseTensorInput("X")) {
+    return KernelSignature("clip_by_norm", {"X"}, {"max_norm"}, {"Out"});
+  } else {
+    return KernelSignature("clip_by_norm_sparse", {"X"}, {"max_norm"}, {"Out"});


selected rows和已有的命名风格保持一致

clip by norm有其他的注册问题，单独提pr处理

… move_some_activaion_to_phi

XiaoguangHu01

LGTM

This reverts commit e77a947.

This reverts commit 05f3d48.

phlrain added 14 commits March 8, 2022 00:38

update

3fc0d19

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

4e23ac6

… move_temporal_shift_to_phi

add forward case

c552d1a

update

def3363

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

c7c81fe

… move_temporal_shift_to_phi

update; test=develop

6c7a03b

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

4be77e5

… move_temporal_shift_to_phi

add some grad kernel; test=develop

21beb08

move gpu kernel; test=develop

77812c0

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8b8c770

… move_temporal_shift_to_phi

update

7834cf1

update;

66b6a5b

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

7a23e76

… move_some_activaion_to_phi

update test;

aa9c1bb

phlrain added 5 commits March 19, 2022 09:00

fix selected rows bug;

685432f

add mix vector include ;

e4ffcc8

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

377d2a0

… move_some_activaion_to_phi

add mixed vector depen; test=develop

aa5c3b0

add logit grad signature;

aca7bb9

YuanRisheng reviewed Mar 21, 2022

View reviewed changes

phlrain added 7 commits March 23, 2022 08:29

polish code

c5c8aa0

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

0f829f4

… move_some_activaion_to_phi

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

0b23b8e

… move_some_activaion_to_phi

fix bug;

a19cbd4

add namespace for abs

f7855a5

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

de125ec

… move_some_activaion_to_phi

revert code

b2c1c68

phlrain added 7 commits March 25, 2022 02:41

format

88597a5

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

609bdee

… move_some_activaion_to_phi

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8b13e86

… move_some_activaion_to_phi

fix bug

9b735e6

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

d8bb56e

… move_some_activaion_to_phi

remove cmake dep

80c2d39

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

0306098

… move_some_activaion_to_phi

PaddlePaddle locked and limited conversation to collaborators Mar 25, 2022

PaddlePaddle unlocked this conversation Mar 25, 2022

chenwhql reviewed Mar 26, 2022

View reviewed changes

phlrain added 11 commits March 26, 2022 14:25

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

95e1451

… move_some_activaion_to_phi

add square sqrt selected rows support

e084fb3

update

14910f8

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

4f1dcbe

… move_some_activaion_to_phi

remove clip norm

28cbd53

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

3917bc1

… move_some_activaion_to_phi

add standalone executor sqrt dep

e8e2d71

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

2b354db

… move_some_activaion_to_phi

standalone exec denp sqrt

7904904

remove sqrt op in cmkaelist

9f61ab9

open some case

ece6173

YuanRisheng approved these changes Mar 28, 2022

View reviewed changes

chenwhql approved these changes Mar 28, 2022

View reviewed changes

raindrops2sea approved these changes Mar 28, 2022

View reviewed changes

XiaoguangHu01 approved these changes Mar 28, 2022

View reviewed changes

phlrain merged commit e77a947 into PaddlePaddle:develop Mar 28, 2022

tianshuo78520a mentioned this pull request Mar 29, 2022

Revert "Move some activation to phi" #41056

Merged

phlrain pushed a commit that referenced this pull request Mar 29, 2022

Revert "Move some activation to phi (#40727)" (#41056)

05f3d48

This reverts commit e77a947.

phlrain added a commit that referenced this pull request Mar 29, 2022

Revert "Revert "Move some activation to phi (#40727)" (#41056)"

0747378

This reverts commit 05f3d48.

phlrain added a commit that referenced this pull request Mar 30, 2022

Revert "Revert "Move some activation to phi (#40727)" (#41056)" (#41095)

91bb52c

This reverts commit 05f3d48.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move some activation to phi #40727

Move some activation to phi #40727

phlrain commented Mar 19, 2022

paddle-bot-old bot commented Mar 19, 2022

YuanRisheng Mar 21, 2022

phlrain Mar 22, 2022

YuanRisheng Mar 21, 2022

YuanRisheng Mar 21, 2022

chenwhql Mar 26, 2022

phlrain Mar 27, 2022

YuanRisheng Mar 21, 2022

phlrain Mar 27, 2022

YuanRisheng Mar 21, 2022

phlrain Mar 27, 2022

YuanRisheng Mar 21, 2022

phlrain Mar 22, 2022

phlrain Mar 27, 2022

YuanRisheng Mar 21, 2022

phlrain Mar 22, 2022

DannyIsFunny commented Mar 22, 2022

chenwhql Mar 26, 2022

chenwhql Mar 26, 2022

phlrain Mar 28, 2022

chenwhql Mar 26, 2022

phlrain Mar 28, 2022

chenwhql Mar 26, 2022

phlrain Mar 27, 2022

chenwhql Mar 26, 2022

phlrain Mar 28, 2022

XiaoguangHu01 left a comment

		// functor(*place, To32BitIndex(x), To32BitIndex(out));
		functor(*place, x, out);

Move some activation to phi #40727

Move some activation to phi #40727

Conversation

phlrain commented Mar 19, 2022

PR types

PR changes

Describe

paddle-bot-old bot commented Mar 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DannyIsFunny commented Mar 22, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment