Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Random op] remove FLAGS_use_curand of all Random OP's CUDA implementation #41308

Merged
merged 1 commit into from
Apr 7, 2022

Conversation

zhwesky2010
Copy link
Contributor

@zhwesky2010 zhwesky2010 commented Apr 2, 2022

PR types

Performance optimization

PR changes

OPs

Describe

随机数系列OP 进行了整体升级,采用性能优的CUDA并行式 随机数算法curandStatePhilox4_32_10_t,并添加了不兼容保护开关FLAGS_use_curand,共涉及到以下PR:

  1. [Random op]optimize CUDA implementation of dropout OP #40874
  2. [Random op]optimize CUDA implementation of multinomial OP #40752
  3. [Random op]optimize CUDA implementation of randperm OP #40464
  4. [Random op]optimize CUDA implementaion of randint OP #39952
  5. [Random op]optimize CUDA implementaion of bernoulli OP #39732
  6. [Random op]optimize CUDA implementaion of uniform/gaussian OP #38611
  7. [Random op]add new API/OP:paddle.Tensor.exponential_ #38256
  8. [Random op]add new API/OP: paddle.poisson #38117

该PR是上面所有PR的最终升级PR,将移除不兼容保护开关FLAGS_use_curand,对性能有2~145倍提升,具体性能加速数据如下:

infoflow 2022-04-06 15-45-22

注:该PR仅改变上述API/OP的随机数采样方式,仅改变采样值,概率密度函数不变,理论上对模型不产生收敛影响。

@paddle-bot-old
Copy link

paddle-bot-old bot commented Apr 2, 2022

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@zhwesky2010 zhwesky2010 changed the title 1 remove FLAGS_use_curand and change all random op CUDA implementation remove FLAGS_use_curand and change all random op CUDA implementation Apr 2, 2022
@zhwesky2010 zhwesky2010 changed the title remove FLAGS_use_curand and change all random op CUDA implementation [Random op] remove FLAGS_use_curand for all random op's CUDA implementation Apr 2, 2022
@zhwesky2010 zhwesky2010 changed the title [Random op] remove FLAGS_use_curand for all random op's CUDA implementation [Random op] remove FLAGS_use_curand for all Random OP's CUDA implementation Apr 2, 2022
@zhwesky2010 zhwesky2010 changed the title [Random op] remove FLAGS_use_curand for all Random OP's CUDA implementation [Random op] remove FLAGS_use_curand of all Random OP's CUDA implementation Apr 6, 2022
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@jeff41404 jeff41404 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhwesky2010 zhwesky2010 merged commit 9714878 into PaddlePaddle:develop Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants