Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[xdoctest] reformat example code with google style in No.36-43 #56440

Merged
merged 9 commits into from
Aug 31, 2023

Conversation

yuchen202
Copy link
Contributor

PR types

Others

PR changes

Others

Description

修改如下文件的示例代码,使其通过 xdoctest 检查:

Related links

@paddle-bot
Copy link

paddle-bot bot commented Aug 18, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added contributor External developers status: proposed labels Aug 18, 2023
@luotao1 luotao1 added the HappyOpenSource Pro 进阶版快乐开源活动,更具挑战性的任务 label Aug 21, 2023
@yuchen202 yuchen202 mentioned this pull request Aug 21, 2023
@megemini
Copy link
Contributor

把标题改一下~

另外,这个 PR 里面文件太多了 ~ 🤣

看看能不能拆一下,或者先改一个 ~

@yuchen202 yuchen202 changed the title Test 修改示例代码,使其通过xdoctest检查 Aug 23, 2023
@megemini
Copy link
Contributor

标题还是要改一下,参考:
#55834

标题统一格式,好方便以后管理、搜索之类的 ~

@yuchen202 yuchen202 changed the title 修改示例代码,使其通过xdoctest检查 [xdoctest] reformat example code with google style in No.36-43 Aug 23, 2023
@yuchen202 yuchen202 closed this Aug 23, 2023
@yuchen202 yuchen202 reopened this Aug 25, 2023
Copy link
Contributor

@sunzhongkai588 sunzhongkai588 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • 这几个代码示例没过,还是得修改一下
图片
  • 另外,还有一些代码的输出结果不要用 #,请修改

注意下PR描述里要加上 @megemini @SigureMo @sunzhongkai588 。 改的很不错了,大部份代码示例都通过,加油~

>>> num_classes = 20
>>> batch_size = 10
>>> num_samples = 6
>>> label = paddle.randint(low=0, high=num_classes, shape=[batch_size], dtype='int64')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • paddle.randint 也是带有随机性的,加一个随机种子吧
Suggested change
>>> label = paddle.randint(low=0, high=num_classes, shape=[batch_size], dtype='int64')
>>> paddle.seed(2023)
>>> label = paddle.randint(low=0, high=num_classes, shape=[batch_size], dtype='int64')
  • 下面的输出,也记得按照seed的结果同步修改

Comment on lines +2107 to +2108
>>> # required: distributed
>>> # Multi GPU, test_class_center_sample.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好像依赖外部文件 test_class_center_sample.py,这段代码skip吧

Suggested change
>>> # required: distributed
>>> # Multi GPU, test_class_center_sample.py
>>> # doctest: +SKIP('Depends on external files.')
>>> # required: distributed
>>> # Multi GPU, test_class_center_sample.py

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里改用 # doctest: +REQUIRES(env:DISTRIBUTED) 即可,因为我们 CI 没有分布式环境,使用该 requires 就是自动跳过

Comment on lines 2093 to 2102
>>> print(label)
>>> print(remapped_label)
>>> print(sampled_class_index)
>>> # the output is
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[10, 3 , 11, 14, 15, 15, 7 , 12, 5 , 13])
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[3, 0, 4, 7, 8, 8, 2, 5, 1, 6])
Tensor(shape=[9], dtype=int64, place=Place(cpu), stop_gradient=True,
[3 , 5 , 7 , 10, 11, 12, 13, 14, 15])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个形式看起来不太整洁,要不就一行print一行输出(输出的结果务必按照seed结果同步修改),如下:

Suggested change
>>> print(label)
>>> print(remapped_label)
>>> print(sampled_class_index)
>>> # the output is
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[10, 3 , 11, 14, 15, 15, 7 , 12, 5 , 13])
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[3, 0, 4, 7, 8, 8, 2, 5, 1, 6])
Tensor(shape=[9], dtype=int64, place=Place(cpu), stop_gradient=True,
[3 , 5 , 7 , 10, 11, 12, 13, 14, 15])
>>> print(label)
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[10, 3 , 11, 14, 15, 15, 7 , 12, 5 , 13])
>>> print(remapped_label)
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[3, 0, 4, 7, 8, 8, 2, 5, 1, 6])
>>> print(sampled_class_index)
Tensor(shape=[9], dtype=int64, place=Place(cpu), stop_gradient=True,
[3 , 5 , 7 , 10, 11, 12, 13, 14, 15])

>>> import paddle
>>> import paddle.nn.functional as F

>>> logits = paddle.randn([4, 6])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

randn有随机性,加seed

Suggested change
>>> logits = paddle.randn([4, 6])
>>> paddle.seed(2023)
>>> logits = paddle.randn([4, 6])

下面的输出也按照seed结果同步修改

>>> temperature = 0.01
>>> gumbel_softmax = F.gumbel_softmax(logits, temperature)
>>> print(gumbel_softmax)
>>> #out's value is as follows:
Copy link
Contributor

@sunzhongkai588 sunzhongkai588 Aug 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这行有点多余,删了吧

Suggested change
>>> #out's value is as follows:

Comment on lines 1088 to 1090
Tensor(shape=[2, 3], dtype=float32, place=Place(cpu), stop_gradient=True,
[[0., 0., 6.],
[0., 0., 0.]])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Tensor(shape=[2, 3], dtype=float32, place=Place(cpu), stop_gradient=True,
[[0., 0., 6.],
[0., 0., 0.]])
Tensor(shape=[2, 3], dtype=float32, place=Place(cpu), stop_gradient=True,
[[0., 0., 6.],
[0., 0., 0.]])

Comment on lines 1827 to 1839
>>> x = paddle.randn((3, 2), dtype="float32")
>>> # x: [[-0.32342386 -1.200079 ]
>>> # [ 0.7979031 -0.90978354]
>>> # [ 0.40597573 1.8095392 ]]
>>> weight = paddle.full(shape=[2, 4], fill_value="0.5", dtype="float32", name="weight")
>>> # weight: [[0.5 0.5 0.5 0.5]
>>> # [0.5 0.5 0.5 0.5]]
>>> bias = paddle.ones(shape=[4], dtype="float32", name="bias")
>>> # bias: [1. 1. 1. 1.]
>>> y = paddle.nn.functional.linear(x, weight, bias)
>>> # y: [[0.23824859 0.23824859 0.23824859 0.23824859]
>>> # [0.9440598 0.9440598 0.9440598 0.9440598 ]
>>> # [2.1077576 2.1077576 2.1077576 2.1077576 ]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如无特殊情况,请勿使用 # 的注释方式提供输出值

比如 x , 若要显示输出值,则

>>> print(x)
Tensor(shape=[3, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
       [[ 0.06132207,  1.11349595],
        [ 0.41906244, -0.24858207],
        [-1.85169315, -1.50370061]])

weight, bias, y 同理。另外,在x代码前记得加上seed

Comment on lines +303 to +304
>>> final_sequences = paddle.nn.functional.gather_tree(ids, parents)
>>> [[[2, 2], [1, 6]], [[3, 3], [6, 1]], [[0, 1], [9, 0]]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同理,如无特殊情况,请显性的提供输出值,如

Suggested change
>>> final_sequences = paddle.nn.functional.gather_tree(ids, parents)
>>> [[[2, 2], [1, 6]], [[3, 3], [6, 1]], [[0, 1], [9, 0]]]
>>> final_sequences = paddle.nn.functional.gather_tree(ids, parents)
>>> print(final_sequences)
Tensor(shape=[3, 2, 2], dtype=int64, place=Place(cpu), stop_gradient=True,
[[[2, 2],
[1, 6]],
[[3, 3],
[6, 1]],
[[0, 1],
[9, 0]]])

Comment on lines 171 to 178
>>> # linear.weight: [[-0.46245047 0.05260676]
>>> # [ 0.38054508 0.29169726]]
>>> # linear.bias: [-0.2734719 0.23939109]

>>> res = linear(data)
>>> # res: [[[-0.3553773 0.5836951]]
>>> # [[-0.3553773 0.5836951]]
>>> # [[-0.3553773 0.5836951]]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同理,如无特殊情况,请勿使用 # 的注释方式提供输出值

  • 另外,paddle.nn.initializer.Uniform有随机性,请在代码开始出加seed
Suggested change
>>> # linear.weight: [[-0.46245047 0.05260676]
>>> # [ 0.38054508 0.29169726]]
>>> # linear.bias: [-0.2734719 0.23939109]
>>> res = linear(data)
>>> # res: [[[-0.3553773 0.5836951]]
>>> # [[-0.3553773 0.5836951]]
>>> # [[-0.3553773 0.5836951]]]
>>> print(linear.weight)
Parameter containing:
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False,
[[ 0.36583614, 0.02014720],
[-0.24039063, 0.40525323]])
>>> print(linear.bias)
Parameter containing:
Tensor(shape=[2], dtype=float32, place=Place(cpu), stop_gradient=False,
[-0.07599911, -0.09358713])
>>> res = linear(data)
>>> print(res)
Tensor(shape=[3, 1, 2], dtype=float32, place=Place(cpu), stop_gradient=False,
[[[0.04944640, 0.33181331]],
[[0.04944640, 0.33181331]],
[[0.04944640, 0.33181331]]])

Comment on lines 227 to 234
>>> # inear.weight: [[ 0.06910077 -0.18103665]
>>> # [-0.02546741 -1.0402188 ]]
>>> # linear.bias: [-0.5012929 0.12418364]

>>> res = linear(data)
>>> # res: [[[-0.4576595 -1.0970719]]
>>> # [[-0.4576595 -1.0970719]]
>>> # [[-0.4576595 -1.0970719]]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同理,如无特殊情况,请勿使用 # 的注释方式提供输出值,修改方式和上述的 Uniform 一样

Comment on lines 1260 to 1264
>>> for i in range(2):
>>> for j in range(3):
>>> print(x[i,j,:,:])
>>> print(y_train[i,j,:,:]) # may all 0
>>> print(y_test[i,j,:,:])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>>> for i in range(2):
...     for j in range(3):
...         print(x[i,j,:,:])
...         print(y_train[i,j,:,:]) # may all 0
...         print(y_test[i,j,:,:])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请问这个print的结果要放吗

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不用了,for 里面这么多 print ,用注释也行 ~ 看的明白为主 ~

Comment on lines 1825 to 1828
>>> import paddle

>>> x = paddle.randn((3, 2), dtype="float32")
>>> print(x)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加个 seed ~

# y.shape = [2,3,4,5]
>>> x = paddle.randn([2,3*2*2,12])
>>> y = F.fold(x, output_sizes=[4, 5], kernel_sizes=2)
>>> # y.shape = [2,3,4,5]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>>> print(y.shape)
[2, 3, 4, 5]

Comment on lines 184 to 190
>>> # required: skiptest
>>> import paddle

q = paddle.rand((1, 128, 2, 16), dtype=paddle.float16)
>>> q = paddle.rand((1, 128, 2, 16), dtype=paddle.float16)

output = paddle.nn.functional.flash_attention(q, q, q, 0.9, False, False)
print(output)
>>> output = paddle.nn.functional.flash_attention(q, q, q, 0.9, False, False)
>>> print(output)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. 这段代码没必要 skip,把旧的 skiptest 去掉。
  2. 应该没必要使用 float16, dtype 也可以去掉。
  3. flash_attention 没有暴露出去,可以用 paddle.nn.functional.flash_attention.flash_attention 。
  4. 这个 output 有点多,可以考虑:
  • 加 seed
  • 改小点 q
  • 打印 output 的其他信息
  • 直接不用 print 了

Comment on lines 342 to 348
>>> # required: skiptest
>>> import paddle

q = paddle.rand((1, 128, 2, 16), dtype=paddle.float16)
>>> q = paddle.rand((1, 128, 2, 16), dtype=paddle.float16)

output = paddle.nn.functional.flash_attn_unpadded(q, q, q, 0.9, False, False)
print(output)
>>> output = paddle.nn.functional.flash_attn_unpadded(q, q, q, 0.9, False, False)
>>> print(output)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上

Comment on lines 161 to 171
>>> import paddle

>>> data = paddle.ones(shape=[3, 1, 2], dtype='float32')
>>> weight_attr = paddle.framework.ParamAttr(
... name="linear_weight",
... initializer=paddle.nn.initializer.Uniform(low=-0.5, high=0.5))
>>> bias_attr = paddle.framework.ParamAttr(
... name="linear_bias",
... initializer=paddle.nn.initializer.Uniform(low=-0.5, high=0.5))
>>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr)
>>> print(linear.weight)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uniform 是随机的,需要加个 seed ~

Comment on lines 217 to 227
>>> import paddle

>>> data = paddle.ones(shape=[3, 1, 2], dtype='float32')
>>> weight_attr = paddle.framework.ParamAttr(
... name="linear_weight",
... initializer=paddle.nn.initializer.XavierNormal())
>>> bias_attr = paddle.framework.ParamAttr(
... name="linear_bias",
... initializer=paddle.nn.initializer.XavierNormal())
>>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr)
>>> print(linear.weight)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上

Comment on lines 278 to 287
>>> import paddle

>>> data = paddle.ones(shape=[3, 1, 2], dtype='float32')
>>> weight_attr = paddle.framework.ParamAttr(
... name="linear_weight",
... initializer=paddle.nn.initializer.XavierUniform())
>>> bias_attr = paddle.framework.ParamAttr(
... name="linear_bias",
... initializer=paddle.nn.initializer.XavierUniform())
>>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上 ~ 另外,输出 print 一下 ~

 @megemini @SigureMo @sunzhongkai588
flash_attention.pyflash_attention 没有暴露出去,可以用 paddle.nn.functional.flash_attention.flash_attention 。这个真不会改
Comment on lines 2189 to 2197
>>> print(label)
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[10, 3 , 11, 14, 15, 15, 7 , 12, 5 , 13])
>>> print(remapped_label)
Tensor(shape=[10], dtype=int64, place=Place(cpu), stop_gradient=True,
[3, 0, 4, 7, 8, 8, 2, 5, 1, 6])
>>> print(sampled_class_index)
Tensor(shape=[9], dtype=int64, place=Place(cpu), stop_gradient=True,
[3 , 5 , 7 , 10, 11, 12, 13, 14, 15])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的输出结果都错了,再核对一下
image

>>> print(bias)
Tensor(shape=[4], dtype=float32, place=Place(cpu), stop_gradient=True,
[1., 1., 1., 1.])
>>> y = paddle.nn.functional.linear(x, weight, bias)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

下面得加输出代码 print(y)

>>> import paddle
>>> import paddle.nn.functional as F

>>> input_tensor = paddle.to_tensor([[[[-2.0, 3.0, -4.0, 5.0],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

前面应该加seed,之前review的时候也提到了,rrelu是有随机性的

# [[ 1.0481861 -2.1206741]]]
>>> import paddle

>>> data = paddle.ones(shape=[3, 1, 2], dtype='float32')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同理,前面加seed,并检查一下输出

Comment on lines 1933 to 1936
Tensor(shape=[3, 4], dtype=float32, place=Place(cpu), stop_gradient=True,
[[1.31539762, 1.31539762, 1.31539762, 1.31539762],
[0.18737471, 0.18737471, 0.18737471, 0.18737471],
[1.23271096, 1.23271096, 1.23271096, 1.23271096]])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

y的输出再校对一下,好像是错的
image

我测出来也应该是

Tensor(shape=[3, 4], dtype=float32, place=Place(cpu), stop_gradient=True,
       [[ 1.58740902,  1.58740902,  1.58740902,  1.58740902],
        [ 1.08524013,  1.08524013,  1.08524013,  1.08524013],
        [-0.67769694, -0.67769694, -0.67769694, -0.67769694]])

希望可以合入了
Comment on lines 1259 to 1260
>>> y_test = paddle.nn.functional.dropout2d(x, training=False) #test
... for i in range(2):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for 前面应该是 >>>

2023-08-30 10:01:40 >>> 1 sample codes ran timeout
2023-08-30 10:01:40 paddle.nn.functional.dropout2d:1 - more than 10.0s

错误分析:

  1. 因为这里用的是 ...(既可以是语句,也可以是输出),所以 xdoctest 在 parse docstring 的时候并没有报错(SyntaxError)。
  2. 当 xdoctest 实际运行检查的时候,抛出异常(SyntaxError)。
  3. 由于目前的 sampcd_processor 用的是 多进程/线程 的方式,所以在 子进程/线程 中抛出异常会卡住。
  4. 由于目前的 sampcd_processor 使用了 TIMEOUT 机制,从而在上述异常卡住运行,并超过限制时间的时候,会抛出异常。
  5. 最后,此代码被判断为 TIMEOUT,与 CI 的结果一致。

如果要捕获 run 时候的异常,得要重新设计,感觉没啥必要,毕竟有 TIMEOUT 这个机制做保护,错误信息目前也能从日志中看到 ~

to @megemini 记录一下,后续注意~

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

所以改一下是吗

将...改为>>>
Copy link
Contributor

@sunzhongkai588 sunzhongkai588 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for docs

@luotao1 luotao1 merged commit 0a15b0d into PaddlePaddle:develop Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor External developers HappyOpenSource Pro 进阶版快乐开源活动,更具挑战性的任务
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants