[AutoDiff] Canonicalize linear maps to be maximally indirect. #26709
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Canonicalize JVPs/VJPs to return maximally abstracted linear map functions with `@in_guaranteed` parameters and an `@out` result. This is a necessary step towards re-enabling LoadableByAddress: linear map type is no longer computed based on the original function type. See TF-11 for more info regarding LoadableByAddress. See TF-625 for more info regarding maximally abstracted linear maps.
dan-zheng
force-pushed
the
ad-all-indirect
branch
3 times, most recently
from
a690449 to
bf6b528
Compare
- Change differential generation to use only tangent buffers.
- Change pullback visitors to use only adjoint buffers.
- Change adjoint of active value propagation to use only adjoint buffers.
- Mark all tangent/adjoint value helpers as `[[deprecated]]`.
- They are not deleted because helpers may become useful after
SIL opaque values are introduced.
dan-zheng
force-pushed
the
ad-all-indirect
branch
from
bf6b528 to
328676e
Compare
Will re-add in follow-up for separation of concerns.
dan-zheng
force-pushed
the
ad-all-indirect
branch
2 times, most recently
from
9dbb1d6 to
361ae93
Compare
Contributor
Author
|
@swift-ci Please test tensorflow |
dan-zheng
force-pushed
the
ad-all-indirect
branch
from
361ae93 to
e38afb9
Compare
Contributor
Author
|
@swift-ci Please test tensorflow |
Contributor
Author
|
It would be good to test the performance impact of this patch. Using adjoint buffers instead of adjoint values may incur a performance penalty. |
rxwei
reviewed
Sep 21, 2019
Remove unused `AdjointValue` utilities for materialization and addition. `AdjointValue` itself is preserved to implement symbolic zero buffer optimization.
rxwei
reviewed
Sep 21, 2019
Address review feedback. Standardize variable naming.
`tuple_extract` is no longer testable after Differentiation has been moved before OwnershipModelEliminator. Old reproducers for `tuple_extract` now generate `destructure_tuple`. Deleting untestable code is prudent.
Avoid unnecessary local tangent struct struct allocation. Generate `struct_element_addr` into existing struct tangent buffer.
dan-zheng
force-pushed
the
ad-all-indirect
branch
from
cce599d to
a6988c8
Compare
Todo: add test. Currently, test crashes. This commit will be overwritten and rebased when done.
There are no known tests for `tuple_extract` visitors. Re-adding `tuple_extract` visitors may be necessary when differentiation supports inout parameters. This Gist shows cases where SILGen produces `struct_extract`: https://gist.github.com/dan-zheng/1343673d2d4d20d403306283b42d522b
Some cleanup will be refactored to a separate patch.
dan-zheng
force-pushed
the
tensorflow
branch
from
6dcf239 to
04dca63
Compare
Contributor
Author
|
A less invasive approach for re-enabling LoadableByAddress was found: #27923 Maximal indirection will not be pursued further: the differentiation transform should not be forced to generate maximally indirect code to workaround issues caused by LoadableByAddress. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Canonicalize JVPs/VJPs to return maximally abstracted linear map functions
with
@in_guaranteedparameters and an@outresult.This is a necessary step towards re-enabling LoadableByAddress:
linear map type is no longer computed based on the original function type.
Summary of changes:
handle maximally indirect linear maps.
See TF-11 for more info regarding LoadableByAddress.
See TF-625 for more info regarding maximally abstracted linear maps.
Initial patch by @rxwei - commits lost because resolving conflicts got too complicated, sorry.