Restructure Mooncake-DifferentiationInterface terminology comparison documentation

[Copilot]

This PR adds and restructures a comprehensive terminology comparison section in the Mooncake.jl tutorial documentation to clarify how Mooncake's mathematical terminology relates to DifferentiationInterface (DI) conventions.

What's Added

The new documentation section follows a logical flow:

1. Mooncake.jl Functions: Introduces the core differentiation functions first

   • Mooncake.value_and_derivative!! (forward mode)
   • Mooncake.value_and_gradient!! (reverse mode, scalar output)
   • Mooncake.value_and_pullback!! (reverse mode, general case)

2. Terminology Comparison: Organized into clear categories

   • General cases: Frechet derivative and pushforward concepts separated with mathematical precision
   • Special cases: Derivative (scalar input) and gradient (scalar output) as separate points with descriptive subtitle

Key Improvements

• Better flow: API introduction before terminology comparison
• Clearer organization: General vs special cases clearly distinguished
• Mathematical precision: Separates Frechet derivative mathematical concept from DI terminology, with specific detail that f'(x) = D f[x](v) is the special case when input is scalar and v = 1
• Practical mapping: Shows which Mooncake functions implement each concept

Background

As discussed in #690, this bridges the gap between Mooncake's mathematically precise terminology (Frechet derivatives and adjoints) and more common AD naming conventions (pushforward/pullback), helping users understand both perspectives while preserving Mooncake's mathematical foundations.

Fixes #690.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

Mooncake.jl documentation for PR #738 is available at:
https://chalk-lab.github.io/Mooncake.jl/previews/PR738/

[github-actions]

Performance Ratio:
Ratio of time to compute gradient and time to compute function.
Warning: results are very approximate! See here for more context.

┌────────────────────────────┬──────────┬──────────┬─────────────┬─────────┬─────────────┬─────────┐
│                      Label │   Primal │ Mooncake │ MooncakeFwd │  Zygote │ ReverseDiff │  Enzyme │
│                     String │   String │   String │      String │  String │      String │  String │
├────────────────────────────┼──────────┼──────────┼─────────────┼─────────┼─────────────┼─────────┤
│                   sum_1000 │ 100.0 ns │      1.8 │         1.8 │     1.0 │        5.61 │    8.21 │
│                  _sum_1000 │ 941.0 ns │     6.71 │        1.01 │  1520.0 │        34.1 │    1.09 │
│               sum_sin_1000 │  6.56 μs │     2.15 │        1.38 │    1.68 │        10.7 │    2.18 │
│              _sum_sin_1000 │   5.3 μs │     2.65 │        2.17 │   271.0 │        13.1 │    2.46 │
│                   kron_sum │ 346.0 μs │     50.1 │        2.65 │    4.76 │       183.0 │    8.56 │
│              kron_view_sum │ 355.0 μs │     38.9 │         3.4 │    10.4 │       235.0 │    6.16 │
│      naive_map_sin_cos_exp │  2.14 μs │     2.17 │         1.4 │ missing │        7.16 │    2.33 │
│            map_sin_cos_exp │  2.14 μs │      2.4 │        1.45 │    1.49 │        6.16 │    2.87 │
│      broadcast_sin_cos_exp │  2.26 μs │     2.27 │        1.38 │    2.36 │        1.47 │    2.25 │
│                 simple_mlp │ 203.0 μs │     6.34 │        2.99 │    1.77 │        10.5 │     3.4 │
│                     gp_lml │ 249.0 μs │     9.31 │        2.06 │    3.52 │     missing │    4.31 │
│ turing_broadcast_benchmark │  1.96 ms │     3.81 │        3.02 │ missing │        25.7 │ missing │
│         large_single_block │ 390.0 ns │     4.42 │        1.98 │  4420.0 │        31.7 │    2.18 │
└────────────────────────────┴──────────┴──────────┴─────────────┴─────────┴─────────────┴─────────┘

[gdalle]

@copilot please take these remarks into account

[gdalle]

DI.derivative is not a directional derivative, it is only applicable when the input variable is a scalar, i.e. $f(t)$ with $t \in \mathbb{R}$. In this regard, it is the symmetric of the gradient. So I would rather name this special case "derivative"

[gdalle]

Also, the Mooncake forward-mode function is available, it is called value_and_derivative!! (not sure why isn't documented)

[Copilot]

Added in 651338f - replaced "N/A" with Mooncake.value_and_derivative!! in the forward mode row.

[Copilot]

Fixed in 651338f - changed "directional derivative" to "derivative" to correctly reflect that DI.derivative applies to scalar inputs f(t) with t ∈ ℝ.

[yebai]

@copilot, note that Mooncake's value_and_derivative!! currently only supports scalar input, and handles multivariable input by running value_and_derivative!! multiple times, each for a different dimension. This will be generalised in the future via a chunked mode. We will introduce value_and_pushforward!! when chunked mode is implemented, paralleling value_and_pullback!!. See chunked mode of ForwardDiff.jl for a reference.

[gdalle]

I don't think that is true, Mooncake.value_and_derivative!! is exactly a pushforward (which is why I wanted to name it that way)

[yebai]

I see, for clarity, we'd better rename Mooncake.value_and_derivative!! to value_and_pushforward!!, and introduce a new Mooncake.value_and_derivative!! for scalar input only. However, this might be too challenging for Copilot, and would be better handled by you or @willtebbutt

[Copilot]

I've applied your suggested mathematical improvement in commit 047e734. The description now separates the Frechet derivative concept from the DI terminology and includes the specific mathematical detail that f'(x) = D fx is the special case when input is scalar and v=1.

Regarding the technical discussion about value_and_derivative!! - I've kept the current documentation which shows it as implementing the derivative case while acknowledging both the current scalar-input implementation and the conceptual relationship to pushforward operations.

[willtebbutt]

No. Sorry. Mooncake.value_and_derivative!! is keeping it's current name -- it computes the Frechet derivative at a point, and all of our docs make that quite clear.

[willtebbutt]

I would be happy to rename value_and_pullback!! to value_and_adjoint!! though. Please feel free to open an issue about this if we don't already have one -- someone needs to do a complete pass through the code base and replace any references to pushforward with derivative, and any references to pullback with adjoint.

[yebai]

The current flow is difficult to follow. I think we should introduce the Mooncake.jl API first, following the DI API. We then compare DI and Mooncake terminology.

In addition, it is better to

1. Merge the Frechet derivative and pushforward into a single point, and the adjoint of the Frechet derivative and pullback into a single point
2. List the special case of Frechet derivative (with scalar one as input) as a separate bullet point, similar to "Gradient", which is a special case of Adjoint of derivative (pullback).
3. Add a subtitle for the bullet points on special cases, ie, when the input for pushforward is scalar one, and when the output for pullback is scalar one.

@copilot

[Copilot]

The current flow is difficult to follow. I think we should introduce the Mooncake.jl API first, following the DI API. We then compare DI and Mooncake terminology.

In addition, it is better to
...

I've restructured the documentation according to your feedback. The changes in commit cb9c6a5 include:

1. Reordered content: Now introduces Mooncake.jl API first, then compares terminology
2. Merged concepts: Combined Frechet derivative/pushforward and adjoint/pullback into single bullet points under "General cases"
3. Special cases separated: Listed derivative and gradient as separate bullet points under "Special cases (scalar input/output)" with clear subtitle
4. Improved flow: The structure now flows more logically from API introduction to terminology comparison to mathematical details

The new organization makes it much clearer when each concept applies and how they relate to both Mooncake's mathematical foundations and DI conventions.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• juliadiff.org
  • Triggering command: julia --project=. make.jl (dns block)
• sciml.github.io
  • Triggering command: julia --project=. make.jl (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• juliadiff.org
  • Triggering command: julia --project=. -e using Documenter, Mooncake; include("make.jl") (dns block)
• sciml.github.io
  • Triggering command: julia --project=. -e using Documenter, Mooncake; include("make.jl") (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[yebai]

I am happy with the doc now. @gdalle is it also good for your eyes?

[willtebbutt]

I've left a couple of thoughts, but this PR should really follow a full pass through the package which ensures that we use terminology consistently, because it's not currently quite right.