[AutoDiff] Initial Forward Mode AD Support#26057
[AutoDiff] Initial Forward Mode AD Support#26057bartchr808 merged 51 commits intoswiftlang:tensorflowfrom
Conversation
04be590 to
0ca031b
Compare
40b3680 to
652a6d7
Compare
9835cb0 to
966c470
Compare
3c01ab8 to
5883668
Compare
|
@swift-ci please test tensorflow |
1 similar comment
|
@swift-ci please test tensorflow |
051a924 to
4e60732
Compare
4e60732 to
1883ac1
Compare
|
For time going to run tests now to get this merged in as soon as possible assuming there are only NFC changes that need to be made |
|
@swift-ci please test tensorflow |
1 similar comment
|
@swift-ci please test tensorflow |
6007902 to
129a3d5
Compare
|
@swift-ci please test tensorflow |
1 similar comment
|
@swift-ci please test tensorflow |
|
|
||
| ForwardModeTests.test("Protocols") { | ||
| let inst = Linear(m: 5, b: -2) | ||
| let (y1, diff1) = valueWithDifferential(at: 5) { x in inst.foo(x: x) } |
There was a problem hiding this comment.
This is not testing differentiation through protocols at all because you calling a method with concrete Self type Linear.
To differentiate through a protocol requirement, define a generic function that takes an instance bound to a generic parameter that conforms to the protocol, and differentiate that.
func genericFoo<T: DiffReq>(_ t: T, _ x: Float) -> Float {
t.foo(x)
}
...
valueWithDifferential(at: 5) { x in genericFoo(inst, x) }There was a problem hiding this comment.
discussed offline, will address this in the next PR which will support everything else
|
@swift-ci please test tensorflow |
2 similar comments
|
@swift-ci please test tensorflow |
|
@swift-ci please test tensorflow |
rxwei
left a comment
There was a problem hiding this comment.
Congrats on going through all the hurdles to finish this important step!
This PR extends on the previous work here on the
JVPEmitter(#25954) to make it be able to generate SIL code for thedifferentialfunction, as well as more correctly implementing the body of the JVP.'generates a valid JVP and differential. I have yet to add support for control flow and generics. This PR also renames
PullbackInfotoLinearMapinfo since it has a lot of tangent mapping helpers thatJVPEmittercan use when creating the differential.This is still early, so the JVP/Differential emission only runs with the flag
-Xllvm -run-jvp-generationwhen running Swift. With it, you can differentiate basic Float functions with+ - / *with novars like: