[cherry-pick] Support fp16 in HybridParallel and fix bugs in HybridOptimizer #36703
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add basic Cost Model, it uses executor to run program and profile it to get op time. This is an early basic version, we will add more functions in the future.
* Optimization of pool2d grad, first commit. * remove useless print codes * refine codes * refine codes * seal more operation into template specialization * fix template struct error in MaxPool2dGrad. * Fix header including error * refine code with comment * Seal the param-preparation codes into function for common use. * Seal the param-preparation codes into function for common use. * Seal the param-preparation into funciton and make it common for other kernels * polish code and erase useless template speicalization * Rerun triger * rerun trigger
…35510) * Create stateful OneDNNAXPYHandler object. This makes it possible to call it multiple times without recreating the oneDNN primitives every time. * Prepare SGDOpKernel to reuse its implementation from OneDNN kernel. * OneDNN SGD kernel. * Update call to use new OneDNNAXPYHandler object api. * Setup seed in proper place. * Enable OneDNN kernel only for single case. * For dense param and sparse grad. * Small refactor. * Enable oneDNN by op attr or by cmd line flag. * Use int64_t type for number of elements. * Support dense param and grad from OneDNN kernel. * Enable SGD OneDNN kernel when use MP BF16 optimizer. * Force non-copyable/movable OneDNNAXPYHandler. * Reuse OneDNNAXPYHandler for spare tensors in SUM op. * Fix SFINAE rules. * Remove recording event inside AXPY. * Get rid of internal primitive caching. * Stop use PP cache mechanims to store mem and primitive obj. * Handler obj store and reuse needed desc & prim * Do not derive from MKLDNNHandlerT
* support extern third_party lapack on Linux/Windows/Mac * fix ci
* Modify H2D and D2H as kQueue::Sync * fix interface error
* refine gc for new_executor, test=develop * refine, test=develop * refine, test=develop * merge, test=develop
* fix feed, test=develop * delete one test case, test=develop
* support nnadapter and ascend310 * modify code * add anchor_generator convert test * add gelu convert test * add conv2d convert test * modify anchor_operator convert test * modify conv2d test * modify con2d convert test * modify conv2d convert test * modify conv2d convert test * modify conv2d test * fix WITH_PYTHON compile error * modify test file * modify test file * modify test file * modify test file * modify test file * modify test file * modify test file * modify test file Co-authored-by: xiaoxiaohehe001 <[email protected]> Co-authored-by: jiweibo <[email protected]>
* support ernie-int8 test and prune op attribute test * remove using and use namespace * remove macro and use shell instead * Revert "remove macro and use shell instead" This reverts commit 615964b. * fix grammar error * fix shell error
Add new API : paddle.linalg.det & paddle.linalg.slogdet API Alias：paddle.det& paddle.slogdet
* init functional jacobian api * finish test with dtype float32 * add float64 test case * polish code * use atol=1e-5 with dtype float64 * fix for ci * set timeout for test_jacobian * polish API docstring * modify docstring
* refactored reshape multiop kernel and added flatten1/2 kernels * added formatting for flatten tests * CI fix * disabled reshape_kernel ops after succesful CI run * minor fix
* A leap of try for cudaLaunchCooperativeKernel * fix bugs * Totally replace the lar cuda kernel * Fix bugs * fix code according to comments * fix codes according to review comments * adding some function overload * relocate the power operation.
* fix extra op for expand, expand_as, tile, unstack * fix unique unstack dim 0 * Update expand_v2_op.cc * fix unique_op format
* gloo hdfs set check & gloo connect retry * add vlog * print gloo connect addr & add vlog * . * modify vlof * modify vlog * modify vlog
* Add Basic CINN Runner Class * Add CinnCacheKey * Add Cache logic and improve CinnCacheKey * Modify as reviewer commented * Implement hash_combine to fix MAC build.
* Initial Commit
* add unittest and add error information
* modify doc
* fix some error
* fix some word
* fix bug cudaDeviceProp* and modify error explanation
* fix cudaDeviceProp* error and unnitest samples
* fix hip error and PADDLE_WITH_HIP
* update style
* fix error is_compiled_with_cuda
* fix paddle.device.cuda.get_device_properties
* fix error for multi thread safe
* update style
* merge conflict
* modify after mentor review
* update style
* delete word
* fix unittest error for windows
* support string input and modify some code
* modify doc to support string input
* fix error for express information
* fix error for express information
* fix unnitest for windows
* fix device.startswith('gpu:')
* format error and doc
* fix after review
* format code
* fix error for doc compile
* fix error for doc compile
* fix error for doc compile
* fix error for doc compile
* fix error for doc compile
* fix py2 error
* fix wrong words and doc
* fix _gpuDeviceProperties
…main (#36121) * read envs in flags_map * add flags to undefok
* fix dygraph double grad dtype error when calling for high differential senario * reinvoke ci * add test for partial_engine.cc
remove recent linalg api in paddle.init; add args 'name' in some new linalg api interface same change in develop branch to #36112
* [HIP] fix op not support AMD GPU bug, the flag PADDLE_WITH_ROCM is invalid * [HIP] fix op not support AMD GPU bug, the flag PADDLE_WITH_ROCM is invalid * [HIP] fix op not support AMD GPU bug * [hybrid] seed and dropout op support force-cpu * [hybrid] seed and dropout op support force-cpu * [hybrid] seed and dropout op support force-cpu * [hybrid] seed and dropout op support force-cpu * [hybrid] seed and dropout op support force-cpu * [hybrid] fix seed ci failed issue * add AsExtra for force_cpu of seed op
* ps gpu dump * remove log
* Add paddle.linalg.eig op * remove comments * remove comments * extend batch_size to the origin * add real times complex functor & destroy the backward complex output bug * terminate output diff when input real tensors * correct tiny doc errors * move functions from eig_helper to svd_helper and remove eig_helper * remove tensor.Resize * remove no longer used code * use existing lapack functions * reply review comments 21/27 * remove .cu as this op is only executed on CPU * remove const_cast & add const in argument list for read-only references * fix sample code error in CI * remove template typename Tbase and more * remove eig exposure in paddle.* * add 'name=None' in eig python implementation * handle the unittest * try to solve the unittest * solve CI coverage * remove no longer used code * polish API doc and more * reply review comments * polish unittest, commit plan B * polish unittest
Add sparse_attention OPs, python api will be added in next pr
* add roi_align in vision/ops.py
|
Thanks for your contribution! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Describe
[cherrypick] Support fp16 in HybridParallel and fix bugs in HybridOptimizer