Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions docs/src/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

There are two ways to compute gradients with Mooncake.jl:

- through the standardized [DifferentiationInterface.jl](https://github.com/JuliaDiff/DifferentiationInterface.jl) API
- through the standardised [DifferentiationInterface.jl](https://github.com/JuliaDiff/DifferentiationInterface.jl) API
- through the native Mooncake.jl API

We recommend the former to start with, especially if you want to experiment with other automatic differentiation packages.
Expand Down Expand Up @@ -121,5 +121,30 @@ DI.jacobian(h, prep, backend, x)

## Mooncake.jl API

!!! warning
Work in progress.
### Mooncake.jl Functions

Mooncake.jl provides the following core differentiation functions:

- **Forward mode**: `Mooncake.value_and_derivative!!` - computes function value and the Frechet derivative
- **Reverse mode**: `Mooncake.value_and_gradient!!` - computes function value and gradient (when output is scalar)
- **Reverse mode**: `Mooncake.value_and_pullback!!` - computes function value and pullback (general case)

### Terminology Comparison with DifferentiationInterface.jl

Mooncake.jl uses discusses Frechet derivatives and their adjoints, as described in detail in [Algorithmic Differentiation](@ref). This differs from the conventions used by [DifferentiationInterface.jl](https://github.com/JuliaDiff/DifferentiationInterface.jl) and some other AD packages.

**General cases:**

- **Frechet derivative**: In forward mode, Mooncake computes the Frechet derivative `D f[x]`, which maps tangent vectors to tangent vectors. This corresponds to what DifferentiationInterface refers to as a "pushforward", and is implemented in `Mooncake.value_and_derivative!!`.

- **Adjoint of derivative and pullback**: In reverse mode, Mooncake computes the adjoint `D f[x]*` of the Frechet derivative, which maps cotangent vectors backwards through the computation. This corresponds to what DifferentiationInterface calls a "pullback" and is implemented in `Mooncake.value_and_pullback!!`.

**Special cases (scalar input/output):**

- **Derivative**: When the input is scalar, the Frechet derivative `f'(x) = D f[x](v)` with `v = 1` gives the ordinary derivative. This corresponds to `DI.derivative`, while Mooncake lacks an equivalent API and handles this as a special case of `Mooncake.value_and_derivative!!`.

- **Gradient**: When the output is scalar, the adjoint of the derivative applied to `1` gives the gradient `∇f`. This corresponds to `DI.gradient` and is implemented in `Mooncake.value_and_gradient!!`.

!!! info
For a detailed mathematical treatment of these concepts, see [Algorithmic Differentiation](@ref), particularly the sections on [Derivatives](@ref).

Loading