Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Prim] Optimize the use of the reshape operator in eager composite backward #69515

Merged
merged 1 commit into from
Nov 20, 2024
Merged

[Prim] Optimize the use of the reshape operator in eager composite backward #69515

merged 1 commit into from
Nov 20, 2024

Conversation

HydrogenSulfate
Copy link
Contributor

@HydrogenSulfate HydrogenSulfate commented Nov 19, 2024 •
edited
Loading

PR Category

Performance Optimization

PR Types

Improvements

Description

Pcard-75624

优化了动态图组合算子中reshape<T>的调用条件，减少在广播情况下的不必要调用。以divide_grad为例，过程如下：

输入x：[1, 3]
输入y: [3, 1]
输出z=x/y: [3, 3]

dx_raw = [3,3]

修改前：dx_raw --sum(keepdim=False)-->[3, ]--reshape-->[1, 3](dx)
修改后：dx_raw --sum(keepdim=True )-->[1, 3](dx)，减少了一个reshape<T>算子的调用

timeline对比：
节省了一个reshape dygraph的调用
image

image

@paddle-bot Paddle-bot
Copy link

paddle-bot bot commented Nov 19, 2024

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@HydrogenSulfate HydrogenSulfate merged commit 88a7f2c into PaddlePaddle:develop Nov 20, 2024
27 of 28 checks passed
@HydrogenSulfate HydrogenSulfate deleted the optimize_reshape_in_eager_composite branch November 20, 2024 03:02
linkk08 pushed a commit to linkk08/Paddle that referenced this pull request Nov 21, 2024
@HydrogenSulfate @linkk08
Optimize the use of the reshape operator in dynamic graph combination…
9e49354 
… operations to minimize unnecessary calls. (PaddlePaddle#69515)
This was referenced Nov 25, 2024
Merged
Merged
github-merge-queue bot pushed a commit to deepmodeling/deepmd-kit that referenced this pull request Dec 17, 2024
@HydrogenSulfate @pre-commit-ci
pd: support dpa1 (#4414)
e8167ce 
Summary of this PR:

1. upload DPA-1 related code
2. merge much develop code
3. add all eager composite operators except `softmax_grad`,
`p_norm_grad`, `split_grad`, and `concat_grad` to the composite operator
blacklist(<https://github.com/deepmodeling/deepmd-kit/pull/4414/files#diff-e678abb052b278f8a479f8d13b839a9ec0effd9923478a850bc13758f918e1e9R134-R148>)
to significantly improve model execution speed (reducing the time taken
from 100% more than PyTorch to about 10% to 15% more).


related PR: lanpa/tensorboardX#728


### Training curve:


![training_curves_comparison_eager_opt](https://github.com/user-attachments/assets/3b71fc99-5abf-4353-a61a-38737d3c7f2c)

### Accuracy test(left: paddle, right: torch):


![image](https://github.com/user-attachments/assets/a42b4bfd-c0f8-4eb8-85eb-ff1adf981dbb)


Ralated optimization of Paddle framework:
- [x] PaddlePaddle/Paddle#69349
- [x] PaddlePaddle/Paddle#69333
- [x] PaddlePaddle/Paddle#69479
- [x] PaddlePaddle/Paddle#69515
- [x] PaddlePaddle/Paddle#69487
- [x] PaddlePaddle/Paddle#69661
- [x] PaddlePaddle/Paddle#69660
- [x] PaddlePaddle/Paddle#69596
- [x] PaddlePaddle/Paddle#69556

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced several new classes for molecular descriptors, including
`DescrptDPA1`, `DescrptBlockSeAtten`, and `LayerNorm`, enhancing the
modeling capabilities for molecular simulations.
- Added new JSON configuration files for model parameters and multitask
models related to water simulations.
- Implemented new test classes for validating the functionality of the
`DPAtomicModel` and various descriptor classes.
- Added new test classes for evaluating denoising models, including
`TestDenoiseModelDPA1` and `TestDenoiseModelDPA2`.
- Enhanced the `ModelWrapper` class to clarify the handling of model
parameters and state management.

- **Bug Fixes**
- Improved internal logic for handling model state saving and loading,
ensuring consistency in outputs.

- **Documentation**
- Enhanced type hints and return annotations across various classes and
methods for better clarity.

- **Tests**
- Expanded the testing framework with new test cases for denoising
models and descriptor functionalities, ensuring robust validation of
features.
- Activated previously skipped tests for energy models, improving test
coverage.
- Enhanced multitask training tests with new configuration handling and
test classes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
github-merge-queue bot pushed a commit to deepmodeling/deepmd-kit that referenced this pull request Dec 25, 2024
@HydrogenSulfate @pre-commit-ci
pd: support dpa2 (#4418)
beeb3d9 
Support DPA-2 in paddle backend. This PR will be updated after #4414 is
merged.

### Training curve:


![training_curves_comparison_dpa2](https://github.com/user-attachments/assets/29bdeffa-cf2d-4586-afcf-7df0569997c3)



### Accuracy test(left: paddle, right: torch):


![image](https://github.com/user-attachments/assets/5bff55f3-1c39-4b95-93f0-68783e794716)


Ralated optimization of Paddle framework:
- [x] PaddlePaddle/Paddle#69349
- [x] PaddlePaddle/Paddle#69333
- [x] PaddlePaddle/Paddle#69479
- [x] PaddlePaddle/Paddle#69515
- [x] PaddlePaddle/Paddle#69487
- [x] PaddlePaddle/Paddle#69661
- [x] PaddlePaddle/Paddle#69660
- [x] PaddlePaddle/Paddle#69596
- [x] PaddlePaddle/Paddle#69556

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced new classes for molecular descriptors: `DescrptDPA2`,
`DescrptBlockRepformers`, `DescrptSeTTebd`, and `DescrptBlockSeTTebd`.
- Added new functions for tensor operations and descriptor management,
enhancing the capabilities of the module.
- Updated JSON configurations for multitask models to refine selection
criteria and data paths.

- **Bug Fixes**
- Improved error handling and parameter validation across various
descriptor classes.

- **Documentation**
- Enhanced test coverage for new descriptor functionalities and
configurations.

- **Tests**
- Added new test classes to validate the functionality of `DescrptDPA2`
and multitask training scenarios.
- Expanded test capabilities for descriptor classes based on installed
dependencies.
- Updated existing tests to support new configurations and
functionalities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cubehan3 cubehan3 cubehan3 approved these changes

@xiaoguoguo626807 xiaoguoguo626807 Awaiting requested review from xiaoguoguo626807 xiaoguoguo626807 is a code owner

@JiabinYang JiabinYang Awaiting requested review from JiabinYang JiabinYang is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@HydrogenSulfate @cubehan3