Add probability distribution transformation APIs #40536

cxxly · 2022-03-14T13:10:14Z

PR types

New features

PR changes

APIs

Describe

Adds 13 transformation APIs and 2 distribution APIs :

new transformation APIs:

Transform
AbsTransform
AffineTransform
ChainTransform
ExpTransform
IndependentTransform
PowerTransform
ReshapeTransform
SigmoidTransform
SoftmaxTransform
StackTransform
StickBreakingTransform
TanhTransform

new distribution APIs:

Independent
TransformedDistribution

Examples:

import paddle

x = paddle.to_tensor([1., 2.])
affine = paddle.distribution.AffineTransform(paddle.to_tensor(0.), paddle.to_tensor(1.))

print(affine.forward(x))
# Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True,
#        [1., 2.])
print(affine.inverse(affine.forward(x)))
# Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True,
#        [1., 2.])
print(affine.forward_log_det_jacobian(x))
# Tensor(shape=[1], dtype=float32, place=Place(gpu:0), stop_gradient=True,
#        [0.])

paddle-bot-old · 2022-03-14T13:10:17Z

✅ This PR's description meets the template requirements!
Please wait for other CI results.

paddle-bot-old · 2022-03-14T13:10:19Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

iclementine · 2022-03-30T03:11:43Z

python/paddle/distribution/transform.py

+    """
+    BIJECTION = 'bijection'  # bijective(injective and surjective)
+    INJECTION = 'injection'  # injective-only
+    SURJECTION = 'surjection'  # surjective-inly


TCChenlong

LGTM
TODO：Add Chinese documentation

jeff41404 · 2022-03-30T03:50:02Z

python/paddle/distribution/transform.py

+            print(affine.forward(x))
+            # Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True,
+            #        [1., 2.])
+            print(affine.inverse(power.forward(x)))


"power" means paddle.distribution.PowerTransform？

已更新，粘贴错误，粘贴了一个旧的代码示例

jeff41404 · 2022-03-30T03:54:13Z

Examples in Describe above: print(affine.inverse(power.forward(x))), "power" means paddle.distribution.PowerTransform? or should be affine?

cxxly · 2022-03-30T04:04:30Z

import paddle

x = paddle.to_tensor([1., 2.])
affine = paddle.distribution.AffineTransform(paddle.to_tensor(0.), paddle.to_tensor(1.))

print(affine.forward(x))
# Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True,
#        [1., 2.])
print(affine.inverse(affine.forward(x)))
# Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True,
#        [1., 2.])
print(affine.forward_log_det_jacobian(x))
# Tensor(shape=[1], dtype=float32, place=Place(gpu:0), stop_gradient=True,
#        [0.])

Examples in Describe above: print(affine.inverse(power.forward(x))), "power" means paddle.distribution.PowerTransform? or should be affine?

是 ''affine''，粘贴错误，文档和PR描述均已更新

jeff41404 · 2022-03-30T04:08:23Z

python/paddle/distribution/distribution.py

@@ -96,7 +96,7 @@ def prob(self, value):
        Args:
            value (Tensor): value which will be evaluated
        """
-        raise NotImplementedError
+        return self.log_prob(value).exp()



shall we add some abstract methods in base class, e.g. mean, variance and rsample

已添加，这样能保持不同子类持有的方法一致；存量 normal, uniform, categorical目前是NotImplementedError，按照设计文档计划后续统一更新

…ckage

… package

jeff41404

LGTM

XiaoguangHu01

LGTM

XiaoguangHu01 · 2022-03-31T01:49:43Z

python/paddle/distribution/__init__.py

+from paddle.distribution.kl import kl_divergence, register_kl
+from paddle.distribution.multinomial import Multinomial
+from paddle.distribution.normal import Normal
+from paddle.distribution.transform import *  # noqa: F403


这里需要import *的原因是什么呢？

distribution/transform.py 文件中定义了需要公开API的__ALL__列表，在distribution/__init__中用import * 全部导出，并添加到__init__.py 的__all__列表中，可以通过paddle.distribution.xxx访问，访问路径和竞品保持一致

cxxly force-pushed the distribution-transform branch from e7850b7 to 141c642 Compare March 28, 2022 10:25

cxxly force-pushed the distribution-transform branch from 141c642 to ae14d10 Compare March 28, 2022 11:22

cxxly force-pushed the distribution-transform branch from ae14d10 to 540de48 Compare March 28, 2022 12:38

cxxly force-pushed the distribution-transform branch from 540de48 to b14dae1 Compare March 28, 2022 17:03

cxxly force-pushed the distribution-transform branch from f37b54c to 9fcd9e8 Compare March 29, 2022 05:36

cxxly force-pushed the distribution-transform branch from 9fcd9e8 to 983c1fb Compare March 29, 2022 06:36

cxxly changed the title ~~Add probability distribution transformation APIs for Paddle~~ Add probability distribution transformation APIs Mar 29, 2022

cxxly force-pushed the distribution-transform branch from 983c1fb to b96f2e1 Compare March 29, 2022 16:08

cxxly force-pushed the distribution-transform branch from b96f2e1 to 3421235 Compare March 30, 2022 01:45

cxxly force-pushed the distribution-transform branch from 3421235 to 6985966 Compare March 30, 2022 02:56

iclementine reviewed Mar 30, 2022

View reviewed changes

iclementine previously approved these changes Mar 30, 2022

View reviewed changes

TCChenlong previously approved these changes Mar 30, 2022

View reviewed changes

jeff41404 reviewed Mar 30, 2022

View reviewed changes

cxxly dismissed TCChenlong’s stale review via 176b413 March 30, 2022 04:39

cxxly dismissed iclementine’s stale review via 176b413 March 30, 2022 04:39

cxxly force-pushed the distribution-transform branch from 6985966 to 176b413 Compare March 30, 2022 04:39

cxxly force-pushed the distribution-transform branch from 176b413 to 4af8502 Compare March 30, 2022 06:11

cxxly added 7 commits March 30, 2022 09:25

add random varaiable transformations API for paddle's distribution pa…

567b9ea

…ckage

add TransformedDistribution API for paddle's probability distribution…

bdee8c6

… package

add random variable transformation unitests for static graph

09354bf

replace math.prod which not support python3.7 with functools.reduce

1e3dd2c

add Independent and TransformedDistribution distribution

6e86fc5

add unittests for constraint

ffefb43

fix typo and AffineTransform sample code error

8a71e25

cxxly force-pushed the distribution-transform branch from 4af8502 to 723a279 Compare March 30, 2022 09:26

add mean,variance,rsample abstract method for Distribution

c8ef422

cxxly force-pushed the distribution-transform branch from 723a279 to c8ef422 Compare March 30, 2022 14:24

jeff41404 approved these changes Mar 31, 2022

View reviewed changes

XiaoguangHu01 approved these changes Mar 31, 2022

View reviewed changes

iclementine merged commit 6735a37 into PaddlePaddle:develop Mar 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add probability distribution transformation APIs #40536

Add probability distribution transformation APIs #40536

cxxly commented Mar 14, 2022 •

edited

Loading

paddle-bot-old bot commented Mar 14, 2022 •

edited

Loading

paddle-bot-old bot commented Mar 14, 2022

iclementine Mar 30, 2022

cxxly Mar 30, 2022 •

edited

Loading

TCChenlong left a comment

jeff41404 Mar 30, 2022

cxxly Mar 30, 2022

jeff41404 commented Mar 30, 2022

cxxly commented Mar 30, 2022

jeff41404 Mar 30, 2022

cxxly Mar 30, 2022

jeff41404 left a comment

XiaoguangHu01 left a comment

XiaoguangHu01 Mar 31, 2022

cxxly Mar 31, 2022 •

edited

Loading

Add probability distribution transformation APIs #40536

Add probability distribution transformation APIs #40536

Conversation

cxxly commented Mar 14, 2022 • edited Loading

PR types

PR changes

Describe

paddle-bot-old bot commented Mar 14, 2022 • edited Loading

paddle-bot-old bot commented Mar 14, 2022

iclementine Mar 30, 2022

Choose a reason for hiding this comment

cxxly Mar 30, 2022 • edited Loading

Choose a reason for hiding this comment

TCChenlong left a comment

Choose a reason for hiding this comment

jeff41404 Mar 30, 2022

Choose a reason for hiding this comment

cxxly Mar 30, 2022

Choose a reason for hiding this comment

jeff41404 commented Mar 30, 2022

cxxly commented Mar 30, 2022

jeff41404 Mar 30, 2022

Choose a reason for hiding this comment

cxxly Mar 30, 2022

Choose a reason for hiding this comment

jeff41404 left a comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

XiaoguangHu01 Mar 31, 2022

Choose a reason for hiding this comment

cxxly Mar 31, 2022 • edited Loading

Choose a reason for hiding this comment

cxxly commented Mar 14, 2022 •

edited

Loading

paddle-bot-old bot commented Mar 14, 2022 •

edited

Loading

cxxly Mar 30, 2022 •

edited

Loading

cxxly Mar 31, 2022 •

edited

Loading